Welcome to WordPress. This is your first post. Edit or delete it, then start writing!
My website redesign was supposed to be a time-intensive and completely ineffectual effort to increase my readership. But mere days after, I landed one of the most steely-eyed, critical voices in the scientific discourse around the replication crisis. Scientists, as they exist in society’s imagination, should have an Asperger’s caliber disinterest in breaking errors gently, any otherwise attending to the feelings of others. Andrew Gelman is a more active, thoughtful, thorough, and terrifying bad methods sniper in scientist-to-scientist discourse today. Yikes. He found my blog, which sent him down a little rabbit hole. I seem to have come through it OK. Better than the other Seth he mentions!:
My own role in the social science’s current replication discourse is as a person with very interesting opinions that no one but me really cares about. Until today! Here is what I have to offer:
- Never too smart to be very wrong, about famous scientists who died believing rot
- White hat p-hacking, a primer, about reconciling experiment registration with exploratory data analysis.
- and The unexpected importance of publishing unreplicable research, about the hidden costs of always demanding the highest quality in empirical work.
It took five years, but it’s out, and I’m thrilled:
You can get an accessible version here
I’m happy to answer questions.
These are a few of my favorite doodles from the “A Mammals Notebook” collection of Erik Satie’s whimsical (i.e. silly) writings and drawings and ditties. Who knew he also wrote and joked and drew? I scanned many many more:
It’s a cold Black Friday morning, minutes before a major retailer with highly anticipated products opens its doors for the day. There are hundreds of people, but no one outside. Everyone is sitting peacefully in their car, warm and comfortable, and in the last seconds before the doors open, the very first arrivals, the rabidly devoted fans who drove in at 2AM, peacefully start to walk to their rightful place before the door as it is about to open, with numbers 2 through 10 through 200 filing wordlessly and without doubt into their proper places behind. There was no wonder or doubt so no comment, that hundreds of people spent only as much time in the cold as it took to get to the double doors, and everyone retained their rightful place in the tacit parking lot queue.
This utopian fiction isn’t fictional for being utopian, just for being big. This kind of system is a perfectly accurate description of events for a crowd of 1 or 2 or 3 or 4 people. I was there. It only starts becoming fanciful at 5 or 10 or more. Naturally, the first person to arrive knows that they are first, the second that they are second and the tenth that they are tenth. But knowing your place isn’t enough, you have to realize that everyone else knows their place.
I broke the utopia the first time I showed up at Wilson Tire to get my winter tires changed out. I had heard that they don’t take appointments and that I should arrive early, even before they open, to get served without being so far back in the day’s queue that I literally wait all day. So I drive up and park among the other five or so other cars and get out to wait by the benches by the door. I was worried that there were so many other cars, but with no one at the benches by the door I figured that they must be employees or something and I rushed to take my position at the front of the queue. This was at about 6:40, a little more than a quarter of an hour before the doors were to open. New Hampshire is usually still cold when you’re switching to or from your summer tires, and I never acclimated, so I was suffering through the cold, and suddenly I wasn’t alone. Four other people got out of their cars to join me, and as new people arrived the cold crowd outside the front door got bigger, with a sort of rough line forming up. I slowly realized that I was a defector, and that I had best imagine myself as behind the people who I had forced out of their cars. That cohort of us milled on the concrete landing, some starting to stand up in an actual queue, me staying more relaxed in my bench, trusting that others would be smart enough to know I was near the front, even though I hadn’t been smart enough waiting from my car to realize that they were. The later arrivals, who saw the milling but had no sense of its order, avoided the mess by queueing up on the asphalt, further away. Immediately before the doors opened one guy with incredible hubris got out of his car at the last minute and cut in front of all of us to be the first served. I was steamed, as much at him as at the docility of the others in my early bird cohort for not saying anything. But I tend to be more of a litigator than most.
It was only after several minutes that I realized that he must have been the first, he’d probably arrived at 6, and his confidence in boldly taking his rightful place was built on the recognition that the other early arrivals would realize he’d been first. But I never did, at least not in time, with the result that I made 5 people get out of their cars who, until my arrival, had peacefully and stably trusted each other to stay warm in their cars, and queue up physically when the time was right. But if I hadn’t done it, someone else would have. It’s much harder for the late arrivals than the early ones.
Number 1 doesn’t just know they’re first, they also know that 2 knows that they are first. 2 knows who 1 is and knows that 1 knows they know. They know that 3 won’t know who is 1 and who is 2, but they realize that 3 will be able to trust 1 and 2 to know each other. 4 might also realize that they are driving into a queue, but 5 and 6 just see a bunch of cars with no order. They know that they are number of 5 or 6, but they think that they’re they’re the last beans in a pile, rather than the last in an increasingly tacit queue. And even if they realize they’re in a queue, 6 might not trust that 5 realizes.
For a car queue to form, it’s not enough to know that you are 10, you have to realize that 9 know they are 9 (and who 10 is), 8 knows they are 8 (and who 9 is), 7 knows they are 7, and so on down to 1, 2, and 3, who know each other, and who know that they know each other knows. When everyone has the capacity to realize they are in a queue, they can queue in the warmth of their cars. But where are minds top out, and common knowledge of the queue breaks down, defection begins.
I’ve now had my tires changed out a few times at Wilson. Number 1 is never the first out of their car. It’s always number 4 or 5 or, in my case, 6. They tend to sit on a bench a few feet from the door. They are followed within a minute or so by the person who came right before them, who wants to signal their priority. And once two people are out, the cascase begins, and everyone else gets out, with the earliest birds standing by the door instead of sitting on the bench, so as to secure their proper place (and signal that they’re securing it). This stays stable, with the sitters knowing their place relative to the standers, and the standers knowing it too. But persons 9 and 10 come upon a disorderly sight, a confusing mix of sitters looking relaxed even though they were the nervous defectors, and standers trying to be in line without looking like they’re in line. 9 adapts to this unsteady sight by standing further away from the door on the asphalt, and 10 lines up behind 9. When the doors open, 9 and 10 watch with apparent wonder as the gaggle by the door fails to devolve into jockeying and each person wordlessly finds their proper place. Of course, it shouldn’t be any surprise: as long as everyone knows their own number, there is enough information for everyone to find their place and even enough information for everyone to keep everyone else accountable
This inevitable degradataion of newcomer’s mental models from queue to pile, from ordered to disordered, creates growing insecurity that people adapt to by moving from an imaginary line to a physical one. In a physical line, you don’t even have to know which number you are, you just have to know where the end of the line is, and so it can scales to hundreds or thousands. On the way to the physical line, a variety of alternative institutions—the very comfortable car queue, the cold but somewhat trustful bench queue, the eventually self-organizing aggregation by the door—ascended and then degraded as common knowledge of them degraded.
A line seems like a simple and straightforward thing. But what for me was moving to the bench to start a line looked others like someone trying to cut in a line that had already existed. If we were all capable of thinking harder and deeper, so capable that we could count on each other to always be doing so wordlessly, then Black Friday shoppers or summer blockbuster campers could enjoy a much more comfortable, satisfying, and civilized norm. But quiet pressures like human reasoning limits, and the degradation of common knowledge they trigger, cause cultural processes to select for institutions that are easy to think about. If you look around, you’ll see lots of situations where cognitive simplicity has won out over social efficiency or fairness, and absent a lot of awkward conversation, it’s perfectly natural to expect it.
I may be making this all up, but it’s very easy to test. All I’d have to do is get up at 5:30AM for the next 30 days and drive over with a clipboard record the following events:
Time and number of each arrival
Time at which each person exits their car
Order of entry into the tire shop.
Everything I’m saying make pretty clear predictions. With few people in the parking lot, people will stay in their cars. With many, they will start to get out and queue up. The first person out of the car will usually be the fourth or fifth arrival. The first arrival may tend to be the last out of the car.
This could be done from afar, at any tire shop, two times a year in any region with winters. An especially nice property of the domain is that this queueing problem, being something people only deal with a few times a year, is sort of a one-shot game, in that every morning brings entirely new people with more or (probably) less experience at navigating the trust and reasoning issues that make parking lot queues so fraught. But it’s hard to get out of bed, which I guess explains why there are so many theorists.
“Pavlov’s CAT!!!! GET IT?”
Here are 20 more or less professional cartoonists who had precisely that original thought. Guess how many of them managed to make it funny. I’m posting this because I’m surprised at how much failure there is on this. Is the idea of Pavlov’s Cat inherently, objectively unfunny?
Bonus, slightly fewer people made it to Schrödinger’s dog. Somehow, a few of these are kind of funny. Why is Schrodinger’s Dog less hackneyed than Pavlov’s Cat? What does it mean about humor or semantics. Also notice the different roles of the dog compared to those of the cat.
And what does not original plus not original equal? Still not original.
OK, fine, the last one is funny.
Advice for a young investigator
by Santiago Ramon y Cajal (1897)
Here is a good bit:
Once a hypothesis is clearly formulated, it must be submitted
to the ratification of testing. For this, we must choose
experiments or observations that are precise, complete, and
conclusive. One of the characteristic attributes of a great
intellect is the ability to design appropriate experiments.
They immediately ªnd ways of solving problems that
average scholars only clarify with long and exhausting
If the hypothesis does not fit the data, it must be rejected
mercilessly and another explanation beyond reproach
drawn up. Let us subject ourselves to harsh self-criticism
that is based on a distrust of ourselves. During the course
of proof, we must be just as diligent in seeking data contrary
to our hypothesis as we are in ferreting out data that may
support it. Let us avoid excessive attachment to our own
ideas, which we need to treat as prosecutor, not defense
attorney. Even though a tumor is ours, it must be removed.
It is far better to correct ourselves than to endure correction
by others. Personally, I do not feel the slightest embarrassment
in giving up my ideas because I believe that to fall and
to rise alone demonstrates strength, whereas to fall and wait
for a helping hand indicates weakness.
Furthermore, we must admit our own absurdities whenever
someone points them out, and we should act accordingly.
Proving that we are driven only by a love of truth, we
shall win for our views the consideration and esteem of our
Excessive self-esteem and pride deprive us of the supreme
pleasure of sculpting our own lives; of the incomparable
gratification of having improved and conquered ourselves;
of refining and perfecting our cerebral machinery—the legacy
of heredity. If conceit is ever excusable, it is when the
will remodels or re-creates us, acting as it were as a supreme
If our pride resists improvement, let us bear in mind that,
whether we like it or not, none of our tricks can slow the
triumph of truth, which will probably happen during our
lifetime. And the livelier the protestations of self-esteem
have been, the more lamentable the situation will be. Some
disagreeable character, perhaps even with bad intentions,
will undoubtedly arrive on the scene and point out our
inconsistency to us. And he will inevitably become enraged
if we readily correct ourselves because we will have deprived
him of an easy victory at our expense. However, we
should reply to him that the duty of the scientist is to adapt
continuously to new scientific methods, not become paralyzed
by mistakes; that cerebral vigor lies in mobilizing
oneself, not in reaching a state of ossiªcation; and that in
man’s intellectual life, as in the mental life of animals, the
harmful thing is not change, but regression and atavism.
Change automatically suggests vigor, plasticity, and youth.
In contrast, rigidity is synonymous with rest, cerebral lassitude,
and paralysis of thought; in other words, fatal inertia—certain
harbinger of decrepitude and death. With
winning sincerity, a certain scientist once remarked: “I
change because I study.” It would be even more self-effacing
and modest to point out: “I change because others study,
and I am fortunate to renew myself.” (pp 122–123)
Of course, he also said things like this:
To sum things up: As a general rule, we advise the man inclined toward science to seek in the one whom his heart has chosen a compatible psychological profile rather than beauty and wealth. In other words, he should seek feelings, tastes, and tendencies that are to a certain extent complementary to his own. He will not simply choose a woman, but a woman who belongs to him, whose best dowry will be a sensitive compliance with his wishes, and a warm and full-hearted acceptance of her husband’s view of life.
Unlike teachers of history and literature, unaccustomed to assigning writing that mixes nuggets of wisdom and bald sexism. I’m thinking of being explicit with my students that they have several options, to
read and think in a manner divorced from emotion, to take the good and leave the bad, or to dismiss it all as rot. That’s got problems, but so does everything else I can think of. Working on it.
It’s been years and this never left my head. The line is from a scene with a judge for a far-future transhumanist syndicate based on the teachings of Confucius.
The House of the Venerable and Inscrutable Colonel was what they called it when they were speaking Chinese. Venerable because of his goatee, white as the dogwood blossom, a badge of unimpeachable credibility in Confucian eyes. Inscrutable because he had gone to his grave without divulging the Secret of the Eleven Herbs and Spices. p. 92
I want all of my interactions with students to be about the transmission of wondrous ideas. All the other bullshit should be defined out of my life as an educator.
But life happens, and students can flake on you and on their classmates, and if you don’t discourage it, it gets worse. So now the transmission of wonder is being crowded out by discussion about your late policy. And late policies are a trap.
For a softy like me, any policy that is strong enough to actually discourage tardy work is too harsh to be credible. To say NO LATE WORK WILL BE ACCEPTED is all well and good until you hit the exceptions: personal tragedies you don’t want to know about, the student who thoughtfully gave you three weeks advance notice by email, your own possible mistakes. Suddenly you’re penalizing thoughtfulness, incentivizing students to dishonestly inflate their excuse into an unspeakable tragedy, and setting yourself up to be the stern looker-past-of-quivering-chins. And what’s the alternative? 10% off for each day late? I don’t want to be rooting through month-past late-night emails from stressed students, looking up old deadlines, counting hours since submission, or calculating 10% decrements for this person and 30% for that one, especially not when such soft alternatives actually incentivize students to do the math and decide that 10% is worth another 24 hours. Plus, with all of these schemes, you’re pretending you care about a 10:02 submission on a 10:00 deadline—or even worse, you’re forgetting reality and convincing yourself that you actually do care.
My late policy should be flagrantly generous and utterly fearsome. It should be easy to compute and clear and reasonable. It should most certainly not increase the amount of late work, especially because that increases the work on me. It should be so fair that no one who challenges it has a leg to stand on, and so tough that all students are very strongly incentivized to get their work in on time. It should softly encourage students to be good to themselves, while allowing students flexibility in their lives, while not being so arbitrarily flexible that you’re always being challenged and prodded for more flexibility.
What I wanted was a low effort, utterly fair policy that nevertheless had my students in constant anxiety for every unexcused minute that they were late.
Is that even possible? Meet the Gamble Protocol. It’s based around one idea: because humans are risk averse, you can define systems that students simultaneously experience as rationally generous and emotionally terrifying. All you have to do is create a very friendly policy with small, steadily increasing probabilities of awful outcomes.
The Gamble Protocol is a lot like the well-known “10% off for every day late.” In fact, in the limit of infinite assignments, they’re statistically indistinguishable. Under the Protocol, a student who gets an assignment in before the deadline has a 100% chance of fair assessment of their work. After the deadline, they have a steadily increasing chance of getting 0% credit for all of their hard work. No partial credit: either a fair grade or nothing at all. On average, a student who submits 100 perfect assignments at 90% probability gets an A-, not because all submissions got 90%, but because ten got 0%. A bonus, for my purposes, is that I teach a lot of statistical reasoning, so the Protocol has extra legitimacy as an exercise in experiential learning.
After experimenting a bit, and feeling out my own feelings, I settled on the following: for each assignment, I draw a single number that applies to everyone (rather than recalculating for every late student). I draw it whenever I like, and I always tell students what number got drawn, and how many students got caught. The full details go in the syllabus:
Deadline. If the schedule says something is due for a class, it is due the night before that class at 10:00PM. There is no partial credit for unexcused lateness; late assignments are worth 0%. However, assignments submitted after the deadline will get a backup deadline subject to the Gamble Protocol.
The Gamble Protocol. I will randomly generate a backup deadline between 0 and 36 hours after the main deadline, following a specific pattern. Under this scheme:
- an assignment that is less than 2 hours late (before midnight), has a 99% chance of earning credit,
- an assignment turned in before 2:00AM has a 98% chance of earning credit,
- an assignment turned in 12 hours late, by 10AM, has a 90% chance of earning credit,
- that jumps suddenly down to 80% between 12–14 hours, getting worse faster,
- an assignment turned in 24 hours late, before the next 10:00PM, has a 60% chance of earning credit,
- and an assignment turned in more than 36 hours late is guaranteed to earn zero credit.
I will not calculate the backup deadline until well after its assignment was due.
Calculating is easy. For each assignment,
- you can put the following numbers in a hat and draw:
0 2 4 5 6 7 8 9 10 11 12 12 12 12 12 12
12 12 12 12 14 15 16 17 18 19 20 21 22 23 14 15
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
32 33 34 35 24 25 26 27 28 29 30 31 32 33 34 35
24 25 26 27 28 29 30 31 32 33 34 35 24 25 26 27
28 29 30 31 32 33 34 35 24 25 26 27 28 29 30 31
32 33 34 35
- or you can open any online R console and paste this code:
deadline <- c( 0,2, c(4,5,6,7,8,9,10,11), rep(12, 10), rep(14:23,2), rep(24:35,5) ) sample(deadline)
I'm keeping data from classes that did and did not use this policy to see if it reduces late work. I still haven't chugged any of it, but I will if requested. For future classes, I was thinking of extending from 36 hours to a few days, so that it really is directly equivalent to 10% for a day's tardiness.
- Download this file, containing the homepage and puppy image in a folder
- Move the file where you want it installed and unzip it
- Drag the Google.html file to your browser
- Copy the address of the file from your location bar, and have it handy
- Copy that file name into your browser’s box for replacing or overriding the new tab page. Or, if you are on another browser, wherever in their options where new tab pages get customized.
- If you want to change the appearance of this page in any way, you can edit the Google.html file as you like. The easiest thing to do is search/replace text that you want to be different.