| Substitution | Rows fixed, of 23 |
|---|---|
| 8 → 0, a dotted zero read as an eight | 6 |
| 0 → 8, the same test backwards | 1 |
| 6 controls (1, 3, 4, 5, 7, 9 → 0) | 1 |
9 Sixteen years of paper
9.1 An electronic shoebox
Everything in this book comes off paper.
20 scanned files, 67 pages, running from July 2010 to May 2017, and every one of them is a sheet I filled in by hand at a filling station or a motel desk. There’s no app in this story and there’s no telemetry. There’s a printed form with two tables on it, a pen, and sixteen years of remembering to use them.
I scan paper. All paper. It’s an electronic shoebox, and the only thing that makes it a dataset rather than a shoebox is that somebody eventually typed it up.
Here is one of the sheets.
Two tables, one page. That’s the design decision underneath half of what this book has found. The fuel halves got typed up years ago, into the spreadsheet everything here is built on. Most of the travel halves never did, which is why the trip log ran to seven rows for six years, until somebody went back to the scans and read the bottom of every page.
And there are two hands in this record, not one. Nancy traveled on many of these trips, and when she was aboard she often kept the log. Our writing couldn’t be less alike. I’m right-handed; she writes with her left, and she’s spent a career as a physician, which does to handwriting what a career as a physician is famous for doing. For a chapter about reading numbers off paper years later, that matters, and it’s why I’m wary of any explanation of an error that starts with the words my handwriting.
9.2 What it’s actually like to keep
I want to say something about the conditions, because they explain the errors better than any theory of error I could offer.
While this is a story about two cars, it is as much a story about data. Specifically the kind of data that’s added now and then. Often under difficult conditions, like when you have cold, shivering hands. Or times when you’re in a rush. It’s easy to write down a wrong number.
Transcription, too, often comes when you probably don’t want to be typing a lot of numbers, some of which are hard to read. It is those distractions that can break the flow and cause another form of data error. The bottom line is that these types of data are difficult to get correct.
There are two different failures in that, and they happen years apart.
The first is the field error. It’s cold, or it’s raining, or the pump’s running and I want to be back in the car, and I write a figure down wrong. That error happens once, at the pump, and the paper is the only witness to it.
The second is the transcription error. Some evening a long time later, I’m typing numbers off a stack of sheets, and the numbers are in my own handwriting, and some of them are hard to read. That error happens to a number that was correct when it was written.
They’ve got different causes and they leave different traces, and one of them turns out to be catchable in a way I’d never have expected.
9.3 The zero with a dot in it
My first thought about this was wrong, and the way it’s wrong is worth keeping.
I assumed the trouble was my own handwriting. I’ve been around computing all my working life, and people in that world often write a zero with a slash through it so it can’t be mistaken for a letter O. A slashed zero, glanced at in a hurry, does look a great deal like an 8. It made a tidy story and it isn’t what happened.
Look at the sheet in Figure 9.1, though. Those zeros are plain. I do slash them sometimes. I didn’t slash them here, and the pages this record’s made of are full of ordinary round zeros.
The dotted zero was on the receipts.
A pump prints you a slip. The type is small, the ribbon or the thermal head’s often tired, and on a good many of those printers the zero’s got a dot in the middle of it so it can’t be confused with an O. That’s the same idea as the slash and it fails the same way. Now read that slip in a car at the end of the day, with the interior light doing what interior lights do, and copy the figures onto a form.
That’s where the 0 and the 8 change places. Not in my writing. In the reading of somebody else’s printing, in bad light, by a man in a hurry to get to a motel.
So the error should still have a direction, and it should be the same one: a dotted zero can be taken for an eight; an eight is never taken for a zero. That’s testable, and it turns out the sheet’s got an instrument built into it.
Every fuel row carries a price per gallon, a number of gallons, and a cost. Three numbers that have to multiply. I didn’t record all three so they’d be checkable. I recorded them because the form had three columns.
So: take the source workbook, the file nobody is allowed to edit. Of 292 usable rows, 23 fail that multiplication by a cent or more. Then take each failing row, change a single digit, and see whether it comes back into line.
The permitted direction fixes 6. Its mirror fixes 1. Six unrelated digits fix 1 between them.
And the 6 don’t look like arithmetic luck. They look like the mechanism:
| the spreadsheet says | the receipt meant |
|---|---|
| 19.28 | 19.20 |
| 32.48 | 32.40 |
| 29.68 | 29.60 |
| 16.86 | 16.06 |
| 29.86 | 29.06 |
| 7.385 | 7.305 |
Every one of them is an 8 where a 0 belongs.
I want to be careful about how much that proves. 23 rows is not many, a digit-substitution search will always repair something by chance, and the mirror test fixing 1 row is exactly the reminder that it can. What the test establishes isn’t that every one of these is a misread zero. It’s that the errors have got a grain, and the grain runs the way a dotted zero says it should.
And here’s the part I find hard to get over. The document that would settle any one of these is the receipt, and the receipts are mostly gone. They were slips of thermal paper in a car. The sheet in Figure 9.1 survives because I filed it. The piece of printing that caused the error went in a bin in Ohio in 2012, and it isn’t coming back.
The signature in this data isn’t mine. It belongs to the pumps. Somebody at a fuel-dispenser company decided to put a dot in a zero so it wouldn’t read as a letter, and that decision’s sitting in a spreadsheet about a car fifteen years later, in the only form it could possibly have taken.
9.4 Running it again, and what actually drifted
There is a second way to ask the same question, and it is the way the audit asked it first. Instead of counting rows repaired, count substitutions attempted. If the slashed zero is a real mechanism, then swapping a 0 and an 8 should repair rows at a better rate than swapping any pair of digits with no physical story behind it.
| Pair | Substitutions | Repairs | Rate |
|---|---|---|---|
| 0 and 8 | 45 | 7 | 15.6% |
| 1 and 7 | 44 | 0 | 0.0% |
| 3 and 5 | 44 | 0 | 0.0% |
| 4 and 9 | 78 | 0 | 0.0% |
The pair with a physical story repairs 15.6% of the substitutions it is offered. The three without one repair nothing at all, across 166 attempts.
When the audit first ran this it got 2 repairs in 15 substitutions, a rate of 13.3%, with one stray hit among the controls. Run over every failing row rather than a handful, the rate holds at 15.6% and the controls go to zero. The result got cleaner, not shakier, which is not what usually happens when you widen a test.
So the finding stands. What did not survive was a sentence about it.
Somewhere between the audit and the writing of this chapter, that result was summarized in a working note as four fixes out of forty-four failing rows. Those numbers are not wrong so much as unrecoverable: they describe a different framing over a different set of rows, at a moment in the work nobody wrote down precisely enough to return to. I spent an hour trying to reproduce them and could not.
The test reproduced. The paraphrase of the test did not. And the paraphrase was the version that had made it into the plan for this chapter, which is to say it was the version that was about to be printed.
That is worth more than the finding it nearly replaced. A number that has been summarized once is not that number any more. It is a claim about a number, and it degrades quietly, and the only defense is to keep the thing that produced it and run it again. Every figure in this chapter is computed while these pages are built, off a workbook nobody is permitted to edit. If any of it stops being true, the sentence changes rather than the sentence surviving.
9.5 Where the corrections live
Which is how the errata came to be built the way they are.
The source is never edited. The 2014 workbook still says what it’s always said, including the wrong things. Corrections live beside it in their own file, each keyed to a row rather than to a place-name, each carrying what was logged, what it should be, and on what evidence.
| State | Count |
|---|---|
| settled | 39 |
| hypothesis | 10 |
| documented | 9 |
| open | 4 |
| withdrawn | 3 |
65 entries. 39 settled, 10 still hypotheses, 4 genuinely open, 9 documenting something that turned out not to be an error, and 3 withdrawn because the analysis was wrong rather than the data.
I’d draw your attention to that last number. Three of the sixty-five aren’t corrections to the data at all. They’re corrections to the corrector.
After all of it, the corrected dataset stands at 294 rows, of which 281 balance the pump identity to under a cent. The 13 that don’t each carry a documented reason.
9.6 The redundancy nobody designed
Here’s the thing I keep coming back to.
Every recoverable error in this record was caught because a number could be checked against another number. The price against the gallons against the cost. The odometer against the great-circle distance between two towns. The fuel log against a credit card statement filed in a different year for a completely different purpose.
None of that redundancy was put there on purpose. The form had three columns because a form has columns. I wrote down the town because I wanted to remember where I’d been. The card statement existed because that’s how the fuel got paid for.
And my own advice, written down decades after I started doing this, is exactly the lesson: when possible, enter fields that will let you cross-check values. Not just the date. Add the day of the week.
I gave that advice as though it were something I’d worked out and might pass on. It turns out the pump had been doing it for me since 2010.
9.7 The part we didn’t know
Something happened while this book was being written that turned out to matter more than any of the corrections in it.
I noticed I’d never entered some of the log data at all. Not lost, not wrong. Just never typed in. And around the same time I was describing this project to a friend who keeps a fuel log of his own, and between us we asked a question neither of us had thought to ask before: could a handwritten log simply be scanned and read?
You have to understand why that question hadn’t come up. Transcription is friction. It’s an evening of typing numbers you can barely read, and it produces nothing you didn’t already have on paper. Unless there’s a payoff worth the effort, the effort doesn’t get made, and that isn’t laziness, it’s an honest piece of arithmetic done by somebody who already knows roughly what the numbers say. It’s exactly why the fuel halves of these sheets were typed up years ago and most of the travel halves never were. The fuel numbers were going to be used. The travel numbers were just going to sit there.
So we decided I’d test it, on my own logs. Could handwritten data be read accurately?
It worked.
And it paid twice, and the second payment is the one we hadn’t seen coming.
The first is the obvious one. The typed spreadsheet held 7 trip-log rows for six years of driving. It now holds 139 legs across 13 trips, 137 of them with both odometer readings, 41,163 miles. None of that was lost and none of it was recovered. It had been on the paper the whole time, in a drawer, while everybody concerned assumed the record was in the computer. Every chapter in the middle of this book is made out of it.
The second is the one that changed how I think about this. Once the scans existed they could be read against the typed fuel log, line by line. And that’s where the errors turned up. All 65 of the errata in the table above came out of that comparison, or out of the audit it made possible. The typing had been done years earlier and nobody had ever gone back to check it against the source, because going back to the source meant reading the source, and reading the source was the friction.
There’s a small joke sitting in the middle of this. The one part of the record that had been typed up carries, in its date column, an entry reading 5/8/0201. The paper hasn’t got that error. The computer has.
9.8 What that’s a lesson about
Here’s what my friend and I came away with, and I don’t think it’s only about car logs.
Data that has been recorded has no value while it sits in a log. It’s got potential value, and sentimental value perhaps. But it can’t answer a question, or contradict you, or turn out to contain a mountain you thought you’d lost, until it’s somewhere a question can be put to it.
Moving it into an analysis environment is what gives it value. That’s the step, and it’s the step that doesn’t happen, because it’s the step with all the friction in it.
So reduce the friction and the move happens. Scanning did that here. A stack of paper that would have cost me weeks of typing became a dataset in a few sessions, and it brought its own audit along with it, because the images stay readable after the numbers come out and you can always go back to them.
I’ve kept these logs for sixteen years and I’d have told you, up until this year, that I knew what was in them. I didn’t. The trip log was twenty times bigger than the version anyone could query. Sixty-five errors sat in the fuel log unnoticed. A summit I couldn’t prove I’d reached was recorded three separate ways.
None of it was hidden. It was just still on paper. And we had no idea, until we tried, how cheaply that could stop being true.