Ten Chairs

In the winter of 1879 Francis Galton began handing out a questionnaire, and the first question was about breakfast.

He was fifty-seven and had spent his life measuring things nobody had thought to measure. He had measured weather, fingerprints, and the usefulness of prayer. Now he wanted to measure the mind’s eye, so he wrote a question plain enough that anyone could answer it:

Before addressing yourself to any of the Questions on the opposite page, think of some definite object — suppose it is your breakfast-table as you sat down to it this morning — and consider carefully the picture that rises before your mind’s eye.

Then he asked about the picture. Is it dim or fairly clear? Are all the objects sharp at the same time, or only the one you are looking at? Are the colors of the china and the toast and the mustard quite distinct and natural?

He sent it first to scientists, on the sensible theory that scientists would be his most careful observers. What came back baffled him.

To my astonishment, I found that the great majority of the men of science to whom I first applied protested that mental imagery was unknown to them, and they looked on me as fanciful and fantastic in supposing that the words “mental imagery” really expressed what I believed everybody supposed them to mean.

One of them wrote back to explain that Galton had made some elementary mistake. “It is only by a figure of speech that I can describe my recollection of a scene as a mental image which I can see with my mind’s eye,” he said. “I do not see it any more than a man sees the thousand lines of Sophocles which under due pressure he is ready to repeat. The memory possesses it.”

So Galton tried people at dinner parties instead, and there he got the opposite answer, with some impatience attached. Of course they saw the table. They described the toast, and the color of the tablecloth, and they were puzzled that he seemed to doubt them. The harder he pressed, the more obvious it became that they meant exactly what they said.

He eventually gathered enough replies to line up two first cousins, both grandsons of the same distinguished scientist, at opposite ends of the scale. The first wrote that he could see all four sides of a room at once, and then all four sides of two rooms, three rooms, four rooms, so clearly that if you asked him what sat in any particular corner of any of them, or asked him to count the chairs, he could do it without hesitating.

The second cousin, asked about the same breakfast table, wrote this:

I could not possibly count the chairs, but I happen to know that there are ten.

Part of what Galton concluded was simply wrong, and it should be said here rather than later. He decided that men of science are poor visualizers as a class, and that has not held up: his sample came to him by volunteering, his question all but told people which answer would be interesting, and later researchers measuring the same thing with instruments found no such pattern by occupation. But the other half of what he found was real, and it waited a long time to be believed. In 2015 Adam Zeman, Michaela Dewar and Sergio Della Sala described people who form no voluntary visual images at all and gave the condition a name, aphantasia, now put at roughly one to four per cent of people. Galton had walked into an enormous difference between one human being and the next, and it took a hundred and thirty-five years to be taken seriously.

And both cousins knew there were ten chairs.

What you already believe

Start from the thing you would defend without thinking about it.

The world is full of claims you cannot check. People tell you what causes what, what always happens, what kind of person does what kind of thing. Those claims are ambitious and they are often wrong, and you have learned to hold them loosely. But underneath all of that there is a floor you can stand on, and the floor is made of particular facts.

There is a cup on this table. It is red. The door is shut. It is raining. The man at the counter is wearing a green coat. These are not theories. They are not interpretations. They are the plainest, smallest, least ambitious things a person can say, and their smallness is exactly what makes them safe. A general claim sticks its neck out across every case that ever was or will be. A particular claim sticks its neck out about one cup, on one table, right now.

This is not a naive belief. It is the working assumption of nearly everything we have built for finding out the truth. A courtroom will not let a witness tell the jury what usually happens or what kind of man the defendant is; it wants to know what he saw, at that hour, in that room. Science does the same at a higher price: whatever the theory says, somebody eventually records a particular reading off a particular instrument on a particular afternoon. And when an argument is going badly and you want to reset it, you reach for the move everyone reaches for. Forget all that. Just tell me what you actually saw.

There is real insight in this, and Locke put the modest version of it as well as anyone. He thought the senses gave us a genuine third kind of knowing, weaker than proof but not therefore worthless, and he had no patience for people who wanted more certainty than the situation allowed. “He that, in the ordinary affairs of life, would admit of nothing but direct plain demonstration,” he wrote, “would be sure of nothing in this world, but of perishing quickly.” You cannot demand a proof before you will eat your dinner.

So the picture goes like this. Sensation delivers the particular, and the particular is the raw material. Reasoning takes the raw material and builds. The lower you go, the less of you there is in it, until at the very bottom you hit the thing you did not make: the plain fact, sitting there, evident.

Two words in that picture are doing more work than they look like they are doing. Evident. And fact.

Both halves, and the seam between them

The claim I want to make is that the evident particular fact is two different things held together, and the join has been sanded smooth.

Take the half that is evident. There is something in your experience right now that you are not inferring, not guessing, not building. Call it bare red, bare warm, bare loud. You have it immediately, and whatever else turns out to be true, you are not going to be talked out of it. This half is genuinely a floor.

Now try to state it. Not what it is a sign of. Not what is causing it. Just the thing itself, on its own, with nothing added. You will find that you cannot get a sentence out that stays inside it. Every sentence you try to write reaches past it and grabs a cup, or a table, or a lamp, or a room. The half of your experience that is truly evident turns out to be something you cannot report, cannot swear to, and cannot hand to anybody else.

And take the other half, the half that actually is a fact. There is a red cup on this table. That one you can state. You can point at it, testify to it, build an argument on it, and be right or wrong about it. But you did not receive it. You issued it. It is a verdict, and like most verdicts it went by so fast that you never saw it happen.

Descartes noticed the seam three hundred and eighty years ago, sitting at a window. Below him people were going past in the street, and he wrote down what he was actually being given:

And yet what do I see from the window but hats and coats which may cover automatic machines? Yet I judge these to be men.

He was not being paranoid. He was being careful about the ledger. What arrives is hats and coats. Men is something he adds, correctly, instantly, and without noticing.

So the thing you reach for as the floor of your knowledge is the most heavily worked-over item in your head, and the reason it feels like the floor is that the work left no footprints.

That claim needs its careful version stated here and not saved for the end, because a serious school of thought denies the whole story. J. J. Gibson spent his career arguing that perception is nothing like the assembly line I have just described. On his account there is no private appearance sitting there waiting to be replaced by a judgment, because vision does not start from momentary snapshots on a retina. A moving animal is bathed in a flowing field of light, and the flow itself specifies the layout of the world, the edges and surfaces and distances, with no inference required. Alva Noë and others have pushed harder: the “appearance” I say you lose was never an item you possessed. If Gibson is right there is no staple, because there were never two pieces.

So take the weaker claim, which holds either way. The particular fact you can report is not the earliest thing in the process, and which fact you end up with is settled by what the fact is for. Gibson does not deny that. His own central idea, the affordance, is exactly the claim that perception is organized around use. On the mechanism the two camps are still fighting. On the part the reader will find hardest to swallow, they agree.

And under both of them sits something lower than any theory: a large gap between what you notice by default and what you can be taught to notice, with years of work in between. That gap is an observation, not a doctrine, and it is enough to run on.

The immediate thing took the longest

Descartes, later in his life, went back and cut the seam open properly. Answering a critic, he divided what we lazily call a sense perception into three stages.

The first is purely physical: light hits the eye, something moves in the nerve. The second is the bare felt result — the color, the warmth, the pain. And the third is everything else:

Finally, the third contains all those judgments which, on the occasion of motions occurring in the corporeal organ, we have from our earliest years been accustomed to pass about things external to us.

That there is a stick out there. That it is three feet away. That it is straight. All of it, Descartes says, belongs to the understanding and not to sense at all. Then he explains why nobody can feel the difference:

Those which have been made from our earliest years in exactly the same manner as at present, about the things that have been wont to affect our senses, as similarly the conclusions of our reasonings, are referred by us to sense. And the reason for this is just that in these matters custom makes us reason and judge so quickly, or rather we recall the judgments previously made about similar things; and thus we fail to distinguish the difference between these operations and a simple sense preception.

The reason the particular fact feels immediate is not that it arrived first. It is that it has been arriving the same way since before you could talk. What we call immediate is not the least processed thing in the mind. It is the most practiced.

Aristotle had already put the same point in a form that should have been a warning. In the Ethics he is trying to explain why a very young man can be a first-rate geometer and cannot be a man of practical wisdom, and his answer is about particulars:

The cause is that such wisdom is concerned not only with universals but with particulars, which become familiar from experience, but a young man has no experience, for it is length of time that gives experience.

Geometry you can have at nineteen. Handling the case in front of you takes decades. And Aristotle is explicit that the faculty involved is a kind of perception, not a kind of reasoning: “a perception akin to that by which we perceive that the particular figure before us is a triangle.” Even this is a triangle lands on the hard-won side of his ledger.

We began with a picture in which the particular is where you start and the general is where you arrive. In the tradition’s actual account, the general is the cheap early achievement and the particular is the long apprenticeship. The starting point is a finish line.

What the senses are never wrong about

That pushes the seam open. Now put weight on it.

Aristotle’s theory of perception contains a promise that sounds like exactly what the floor-of-knowledge picture needs. There is something each sense cannot be wrong about:

I call by the name of special object of this or that sense that which cannot be perceived by any other sense than that one and in respect of which no error is possible; in this sense colour is the special object of sight, sound of hearing, flavour of taste.

No error is possible. That is a very strong guarantee, and it is not hedged.

Then see what it covers. Colour. Sound. Flavour. Not cup, not table, not three feet away, not one thing rather than two. Size and shape and number and motion are not the special object of any single sense, because you can see a shape and also feel it, and that is exactly why they fall outside the guarantee.

Aquinas draws the line out loud, and gives the standard example:

Sense is not deceived in its proper object, as sight in regard to colour, save accidentally through some hindrance occurring to the organ… Sense, however, may be deceived as regards common sensibles, as size or figure; when, for example, it judges the sun to be only a foot in diameter, whereas in reality it exceeds the earth in size.

The foot-wide sun is not his estimate. It is the old Heraclitean claim that Aristotle cites as the textbook case of an appearance that does not go away once you know better. That is the useful thing about it. Everyone in that argument knew perfectly well that the sun is enormous, and the sun still looked a foot wide, every single day, forever.

James measured the modern version of the same failure and found it just as bad. “The various answers to the familiar question, How large is the moon?” he wrote, “answers which vary from a cartwheel to a wafer, illustrate this most strikingly.” The real figure is about half a degree of arc, which is an aspirin tablet held at arm’s length. Nobody’s guess is anywhere near it, including yours, and you can settle the matter tonight with an aspirin.

So the guarantee is real and it is worthless for our purposes. The part of seeing that cannot be wrong is a part so thin that it contains no facts. Everything that makes a fact a fact — that this is one object, that it is over there, that it has that shape, that it is a cup — lives on the fallible side. The infallible witness can say exactly one syllable, and the syllable is not a sentence.

Which raises a question the picture we started with never had to ask.

Who is actually testifying

Plato got there first, and he got there by fussing over a preposition.

In the Theaetetus, Socrates has spent most of the dialogue building up the theory that knowledge just is perception, giving it every advantage, letting Protagoras speak in his own defense. Then he asks Theaetetus a small question. When you see a color, do you see it with your eyes, or through them?

Theaetetus, who is quick, says: through. And Socrates tells him why the word matters:

No one can suppose that in each of us, as in a sort of Trojan horse, there are perched a number of unconnected senses, which do not all meet in some one nature, the mind, or whatever we please to call it, of which they are the instruments, and with which through them we perceive objects of sense.

Then he shows what the senses cannot do on their own. You see a color and you hear a sound, and you know that they are two, and that each is one, and that they are different from each other. Which organ told you that? Not the eye, which knows nothing of sound. Not the ear. Not some third organ, because there isn’t one. Something else is holding the two reports side by side and comparing them, and it is not a sense.

And once you see that, you cannot stop seeing it, because is one, is two, is the same, is different are in every particular fact you have ever stated. Take them out of there is a red cup on this table and there is nothing left but a smear of red. Socrates lands it flatly:

Then knowledge does not consist in impressions of sense, but in reasoning about them; in that only, and not in the mere impression, truth and being can be attained.

Kant, two thousand years later, said the same thing in a single sentence, and his version is the one that does damage:

It is, therefore, quite correct to say that the senses do not err, not because they always judge correctly, but because they do not judge at all.

The senses have never been wrong, and the reason is not that they are good. It is that they have never said anything. Error and truth only exist where something is being asserted, and the eye asserts nothing. So every mistake you have ever charged to your eyes was your own ruling, entered under your eyes’ name. The phrase I saw it with my own eyes has always been a signature, not a source.

You can catch the ruling in the act with one drawing. In 1899 the psychologist Joseph Jastrow published a sketch that reads either as a duck facing left or a rabbit facing right. Wittgenstein borrowed it and noticed something about how we talk in front of it. Shown a plain rabbit drawing, you say “It’s a rabbit,” not “Now it’s a rabbit,” and Wittgenstein says that first answer is “the report of a perception.” But when the duck flips into a rabbit, what you say is now it’s a rabbit, and that sentence is not a report of anything: “The expression of a change of aspect is the expression of a new perception and at the same time of the perception’s being unchanged.” Nothing about the ink altered. Nothing about your eye altered. The only thing that moved was the verdict, and the word now is the sound of it moving.

Kant also drew the line the subtopic is really about, and he drew it in ordinary language. In the Prolegomena he distinguishes a judgment of perception from a judgment of experience, and his example is deliberately humble. “When the sun shines on the stone, it grows warm” is a judgment of perception; it records what happened in me, and Kant says it holds “only for us.” But “the sun warms the stone” is a different animal, because it has smuggled in the idea of cause, and cause was never anywhere in the sunshine or in the stone.

The second sentence is not the first sentence said with more confidence. It is a different kind of act, and it is the only one of the two that can go into evidence. Which means that when you take the stand and give the jury the plainest, most modest, least theoretical sentence you can manage, you are already doing the ambitious thing. There is no humbler option available. The humble option cannot be said out loud.

What it cost

So far this could sound like good news with a philosophical flavor. The mind does more than we noticed; the mind is impressive; fine. Now the bill.

William James, who spent more pages on this than anyone, found that the mind does not merely add a judgment on top of the appearance. It replaces it, and then loses the receipt.

Look at anything familiar. A door, a plate, a coin. It throws a different shape onto your eye from every angle, endlessly, as you move. Square tabletops show up as two sharp corners and two blunt ones. Round plates show up as ellipses. Out of that flood, James says, we pick one:

Out of all the visual magnitudes of each known object we have selected one as the real one to think of, and degraded all the others to serve as its signs.

And we do not pick the average, and we do not pick at random. We pick the one you get when the thing is square in front of you at the distance where you would hold it to examine it. That is the shape you will draw if asked to draw it, the shape you will call its real shape, the shape you will swear to. “Here we believe we see the object as it is,” James wrote of that position; “elsewhere, only as it seems.”

Then comes the part that costs something. The demoted appearances do not sit politely in the background where you could check on them. They become unavailable.

As I look along the dining-table I overlook the fact that the farther plates and glasses feel so much smaller than my own, for I know that they are all equal size; and the feeling of them, which is a present sensation, is eclipsed in the glare of the knowledge, which is a merely imagined one.

The far plate really is smaller in your eye. It is smaller right now. And your knowledge that the plates match is bright enough to put out a light that is actually on. Thomas Reid had described the same disappearance in a line James liked enough to quote:

They no sooner appear than, quick as lightning, the thing signified succeeds and engrosses all our regard. They have no name in language; and although we are conscious of them when they pass through the mind, yet their passage is so quick and so familiar that it is absolutely unheeded; nor do they leave any footsteps of themselves, either in the memory or imagination.

No footsteps. That is why the fact feels like a floor. Not because nothing was done to it, but because whatever was done cleaned up after itself.

And there is a sting in how the choice gets made. James is blunt about the criterion, and it is not evidence:

When two sensorial space-impressions, believed to come from the same object, differ, then the one most interesting, practically or æsthetically, is judged to be the true one.

Interesting. Practically. The real shape of a thing is the shape it has where your hands would hold it. The real size is the size at the distance where you would inspect it. Your particular facts have been cut to fit your reach. This is not a scandal about the senses; it is a working system doing its job, and it is the reason that the thing itself and the thing as it is useful to me are so hard to pull apart. Helmholtz, who took the trouble to look, put the rule plainly: “We only attend with any ease and exactness to our sensations in so far forth as they can be utilized for the knowledge of outward things; and we are accustomed to neglect all those portions of them which have no significance as regards the external world.”

He then listed what the neglect hides. The blind spot in your retina, which nobody felt for the whole of human history until Mariotte worked out in 1668 that it had to be there and went looking. The floating specks that have been in your eye all along and that people notice in middle age and assume are new. Double images, which are what almost everything outside your point of focus is doing right now. Helmholtz’s point is not that these are illusions. It is that they are facts about your present experience that you cannot reach without a trick, an instrument, or a theory telling you where to look.

Which means the thing can be bought back, and there is a profession that buys it. Ruskin, teaching drawing, said the aim was to recover what he called the innocence of the eye: a sort of childish perception of stains of color merely as such, without consciousness of what they mean. James reports the same instruction as the universal complaint of every drawing teacher to every student: “Don’t draw the thing as it is, but as it looks!” Beginners cannot do it. They draw what they know, which is why beginners’ tables have four right angles. The whole training is the slow purchase of an ability you had at two.

The modern demonstrations are cleaner than anything Ruskin had. Edward Adelson’s checker-shadow figure, published in 1995, puts two squares of exactly the same gray in front of you and makes one look obviously dark and the other obviously light, because a painted shadow falls across the board. Cover the rest of the image with your hands and the two squares snap to identical. Uncover it and they separate again, and you cannot hold them together while you can see the board. The judgment is not a belief you could talk yourself out of. It is a fixture.

Once in a while someone loses the machinery, and then you can see how much of it there was. In 1883 Charcot described a Viennese merchant who read Homer at sight and did his accounts by running his mind’s eye down columns of figures he could see. Then one morning it was gone. Not his sight: he saw everything perfectly, distinctly, in full color. What had gone was his stock of remembered appearances. Asked about the main square of a city he had visited for years, he said, “I know that it is there, but it is impossible to imagine it, and I can tell you nothing about it.” Asked to draw a minaret, he reasoned that it was a square tower and drew four lines. He once spoke to his own reflection, taking it for a stranger.

He could see. He could not picture. And he went on doing business, because he learned to route everything through words instead, which is roughly what Galton’s men of science had been doing all along.

Which leaves one more thing to price.

James asked what it would take to learn that black differs from white. His answer is a thought experiment. Suppose a hundred creatures, each given memory and the power to compare, each shown the same lot of sensations but in a different order, some seeing a thing once and some a thousand times. Then let the show stop, and leave them in a void with nothing but their memories.

Inevitably in their long leisure they will begin to play with the items of their experience and rearrange them, make classificatory series of them, place gray between white and black, orange between red and yellow, and trace all other degrees of resemblance and difference. And this new construction will be absolutely identical in all the hundred creatures.

Identical, no matter what order the world came in. Because, as James says, “To learn whether black and white differ, I need not consult the world of experience at all; the mere ideas suffice.”

Comparison is free. Classification is free. The general truth is the cheap one — it needs no world, no distance, no shape, no hands, and it comes out the same for every creature that can hold two things at once. The particular fact is the expensive one. It needs the whole apparatus: the selection, the demotion, the years of practice, and the silent replacement of the appearance by the verdict.

We have been calling one of these raw and the other refined. We had them backwards.

The breakfast table, again

Go back to Galton’s two cousins.

The first could count chairs in rooms he was not standing in. The second could not count the chairs in a room he had eaten breakfast in that morning. And both of them knew there were ten.

The fact came out identical, and it came out identical because a fact is the output of the machinery and carries no record of the machinery. Two men were running processes so different that neither could imagine the other’s, and the difference stayed invisible from the outside, and invisible from the inside too, for a hundred years, until somebody thought to ask about toast.

Galton’s own guess about why his scientists had so little imagery was wrong in its particulars and interesting in its shape. He suspected that “an over-ready perception of sharp mental pictures is antagonistic to the acquirement of habits of highly-generalized and abstract thought.” Wrong as a fact about scientists. But something near it is true about all of us and has nothing to do with profession: the better you get at operating on facts, the less of the appearance you have left to operate on. The efficiency is the loss. Reid’s lightning does not slow down with practice; it gets faster, and the footsteps get fainter.

None of this makes the cup less red or the table less there. Locke was right that you cannot demand a proof before dinner, and the system works, and the plates really are the same size. What it takes away is smaller and harder to replace. It takes away the move you have been making your whole life, the one where the argument gets heated and you step back to the plain ground and say look, all I’m telling you is what I saw.

That was never the plain ground. It was the most elaborate thing you had.

The plain ground is a smear of red you cannot put into a sentence, and ten chairs you did not count.