| Holds | File | Rows | Columns |
|---|---|---|---|
| the answers | TwoRed_fuel_clean.csv | 294 | 16 |
| the evidence | TwoRed_log_errata.csv | 65 | 8 |
14 What the record cannot say
14.1 The numbers you came for
This book has spent thirteen chapters producing figures. 39.25 miles per gallon. Three hundred miles in a day. 49 states and 11 provinces and territories. Every one of them’s derived, checkable, and sitting in a sentence built to hold it.
Those were the easy part.
What it took to be allowed to print them is the part nobody sees, and it’s a good deal larger than the figures. This chapter is about that, because a number you can’t audit is just a number somebody told you.
14.2 Where do you put the Florida confirmation?
Here’s the question that organizes everything else.
The state count has an obvious home. It goes in a headline, in a summary line, in the second paragraph of a talk. Everybody knows where a total lives.
Now: I drove a few miles into the Florida panhandle, stopped at a crossroads, asked a passing driver whether this was Florida, was told yes, and turned around. Where does that go?
It isn’t a fuel purchase, so the fuel log hasn’t got a row for it. It isn’t the end of a day, so the trip log hasn’t either. It’s the thing that makes the total honest, and there is no column anywhere in this record designed to hold it.
That’s not an oversight in my form design. It’s the general case. We build a place for the answer because we know what the answer will look like. We don’t build a place for the proof, because proof takes whatever shape the doubt happens to take, and there’s no knowing that in advance. We’re goal directed. We count the states. We are not procedure oriented, and the procedure is where the trust is.
14.3 What that looks like when you measure it
I can show you the asymmetry, because both files are sitting in the same folder.
The fuel log has 16 columns and not one of them is free text. Date, city, state, brand, price, odometer, gallons, cost. Every column knows what it’s for, every value’s got a type, and a new row slots in without anybody thinking about it. That’s what a table looks like when you’d worked out the question before you started collecting.
Now the errata file, which holds the corrections and the reasons for them.
status column is normally an enum of about five.
| Column | Distinct values | Rows |
|---|---|---|
| basis | 61 | 65 |
| status | 48 | 65 |
| field | 24 | 65 |
Look at the middle row. status holds 48 distinct values in 65 rows. In any ordinary schema that column’s a short list: open, confirmed, resolved, rejected, withdrawn. Here it holds sentences.
RESOLVED, and my earlier ‘cost is wrong’ diagnosis was REFUTED by the card.
WITHDRAWN, my classifier failed, not the data.
Nobody designs a status column like that. It happens because the alternative is losing the reason, and the reason turned out to matter more than the correction.
The basis column is worse, or better, depending on what you want from it: 61 distinct values in 65 rows, a median of 117 characters, and 51 of the 65 running past eighty. The longest’s 775 characters. It stopped being a field and became an argument, written out fresh each time, because every correction had to be justified in whatever terms that particular doubt required.
14.4 What actually settled things
And here’s what the arguments leaned on.
| what settled it | entries |
|---|---|
| the scans | 33 |
| a card statement | 9 |
| arithmetic | 9 |
| geography | 4 |
| Kim’s memory | 4 |
| an argument written out | 3 |
| spelling | 2 |
| the odometer | 1 |
Not one of these is the fuel log itself. The fuel log is what’s being corrected. Everything that does the correcting comes from outside it, and most of it from a source that had nothing to do with cars: a scanned sheet of paper, a credit card statement filed for tax, a road name someone happened to remember.
Chapter 9 is the reason the top row is so large. Before the scanning experiment there was no way to check a typed number against anything, and after it there were 33 corrections that came straight off the paper.
14.5 Florida, all the way down
So: the state that has no row anywhere, and what it took.
The first evidence was testimony, and testimony is unfalsifiable. I drove in, I asked a man, he said yes. I believe it because I was there. You have no way to check it and no way it could ever fail, which means it isn’t evidence at all, it’s a request.
The second was better and I didn’t produce it on purpose. I mentioned that a friend in Alaska lives on Styx River Road, that there’s only one other road in the United States with that name, and that I remember seeing it in Alabama just after leaving Florida. Huh? Another one??? How would I have known that except by a visit?
That’s a different kind of claim, and what makes it different’s that it’s useless. Nobody memorises an obscure county road name in order to prove something twelve years later. A fact you had no reason to want can’t have been manufactured, and unlike the testimony it can be looked up: there is a Styx River in Baldwin County, Alabama, and the Perdido it runs into is the Florida line.
The third is a photograph, and I’d forgotten I had it.
The coordinates put that corner 1.1 miles inside Florida, about 5 miles from the Atmore fill that brackets it. It’s checkable on a street map by anybody who doubts me, which is the entire point, and it’s the opposite of just believe me.
And the arithmetic agrees by staying silent. The leg those miles hang off, New Orleans to Atmore, reads 210 odometer miles against 170 straight, a ratio of 1.23. The 2014 trip’s own median is 1.23, with a middle half from 1.11 to 1.43. An eleven-mile detour sits inside ordinary routing and cannot be seen. The instrument isn’t failing there. It’s being asked a question below its resolution, and those are different things.
14.6 The one place the record says Florida, and it’s wrong
Now the trap, because it’s the best argument in this chapter for being procedure oriented.
There is a Florida entry in the financial records: a 7-Eleven in Orlando, on the 2015 card summary.
It isn’t TwoRed. Of sixteen fuel charges on the card that year, eight match the log to the cent and eight don’t. Seven of the eight are Hawaii. One is Orlando. The card covers the fleet; the log covers one car.
So the single place a statement says the word Florida is the one place it would mislead you completely. A goal-directed reader searching for Florida finds that row and stops. The only thing that catches it is a procedure: check the geography, check the year, check which car, check whether the charge matches the log to the cent. The right answer and the wrong one look identical until you do the work.
14.7 What proof costs
I want to be honest about the price, because it’s the reason this gets skipped.
Vermont is in this book because of a lunch. I stayed at a friend’s house a few miles over the line, having followed him home from his office in New Hampshire, and there were no filling stations on that road and no stopping anyway without losing him. No fuel, no overnight in the log, no row.
What settled it was a line on a card statement from May 2014: a restaurant in Quechee, Vermont, and the bill for two. Finding that meant going back through more than a decade of financial records looking for one transaction, in one state, in one month, from eleven years ago.
That is not an easy find. It took longer than any analysis in this book, and it wasn’t interesting work. It produced one line of text, and that line is worth more than any paragraph I could write about my own memory, because it was filed by somebody else, for their own reasons, and they had no idea what it would end up proving.
When you find it, it adds trust. That was the primary issue all along.
14.8 Three things I can’t settle, and one I got wrong
A chapter about trust should end with the things that don’t resolve.
The first is my own mistake. In August I wrote that the South Dakota leg was the most indirect stretch in sixteen years. It isn’t. Re-run against the completed geocoding, it ranks 60 of 270 legs, at the 78th percentile, and it isn’t first at any threshold you care to set. I’d found a striking number and I’d printed it without asking where it sat.
The conclusion survives on better evidence. The odometer reads 255 miles where the straight line is 139. Route it through Chadron, Nebraska, which is 12 miles from the state line, and the straight-line path comes to about 202. That leaves roughly 53 miles unaccounted for on a day with a dash to a border in it. Not a record, just room.
The second is a result I retired and shouldn’t have. The record used to show fuel economy falling with ambient temperature, and as the sample grew the effect faded, so I wrote it off and noted approvingly that the record had got good enough to kill a finding.
| Tanks | MPG per degree F | p | With tank length held | p |
|---|---|---|---|---|
| 94 | -0.126 | 0.003 | -0.112 | 0.004 |
| 125 | -0.101 | 0.007 | -0.090 | 0.006 |
| 152 | -0.082 | 0.010 | -0.069 | 0.015 |
That retirement does not reproduce. On today’s corrected data the effect is significant at every sample size I can construct. What’s true’s subtler and less satisfying: the coefficient decays by about a third as the record grows, and refuses to die. I’m not going to claim it now either, because ambient temperature travels with season, region and route, and this record cannot separate them. What I’ll claim is that I retired it too early, on a version of the data that no longer exists.
The third genuinely cannot be settled, and I can show you why rather than just asserting it. There may be a southern fuel-economy deficit. Here is what it is:
| Which states count as southern | Tanks | MPG difference | p |
|---|---|---|---|
| the six states this book groups as southern | 10 | -2.05 | 0.18 |
| a wider reading, nine states | 19 | -0.49 | 0.66 |
The answer moves by a factor of four depending on a boundary nobody ever wrote down, and neither version reaches significance. And the sample is closed: Creamsicle made 0 southern fills in three years, so no future data will settle this. It will be 10 tanks forever.
So the honest output is the fact that it cannot be settled, and not a coefficient. A number printed here would be read as a finding by everybody who saw it, and it would be my definition of southern doing most of the work.
14.9 The lesson, which isn’t about cars
Every dataset has a schema for its answers and no schema for its evidence.
The answers get columns, types and a place in the headline. The evidence gets whatever you can improvise: a free-text field that swells into paragraphs, a folder of scans, a photograph you forgot you took, a lunch receipt from 2014. That improvisation is where the trust lives, and because it hasn’t got a designed home it’s the first thing to be skipped and the first thing to be lost.
The temptation is always to say just believe me. It’s a strong temptation precisely because validating a claim’s often genuinely hard, and sometimes there’s no clean way to do it at all. Ignore it anyway. Go and look for the receipt. Most of the time you won’t find one, and the honest move then is to say so and mark the claim as testimony.
But sometimes you find the lunch.