Gibson, 3D Modelling, Surfaces and Volumes

As in my previous post, I’m at page 40 of Gibson’s book.

Gibson’s view is that we need to understand our environment to understand vision. The environment in which we (and other animals) live is not arbitrary, it has its own rules and structure. Gibson therefore devotes a large part of his book to describing our environment, which he insists differs from the abstract worlds of physics and geometry. The environment is what we perceive from the physical world, it has its own logics.

It’s very entertaining to read Gibson’s description of our environment. It starts from scratch, as if the world around us had to be explained to an alien being. The result appears either as the work of a madman or of a genius.

Interestingly, Gibson’s description of the environment bears uncanny resemblances with how we would specify a virtual environment in a 3D modelling language. There is a ground plane, light sources, surfaces, vertices, edges, interiors and exteriors, textures. Perhaps we shouldn’t be surprised that a psychologist who seeks to describe how we visually experience the real world and a programmer who wants to create a language for simulating this very experience end up with similar primitives. There are important differences, of course. Gibson rejects Cartesian coordinates, emphasizes the irregularity and complexity of the world, and introduces many more entities and concepts.

Yet if Gibson had to design a 3D modelling language, it would almost certainly be based on shell models rather than solid geometry. He is obsessed with surfaces. Surfaces are of uttermost importance, it is where the action is, he writes. Granted, virtually all the sensory information we gather from objects (visual, olfactory and tactile) come from their surface. But if the goal is really to describe how we experience the world, then perhaps there should be a greater focus on volumes.

If I cut an apple in two, two new surfaces appear to me, but I am not surprised at all. Unless there is something weird (e.g., the apple is rotten, or there’s a worm inside), I don’t get much extra information from cutting it, i.e., there is no surprise. This means I was experiencing the apple as a solid, full object in the first place. Throughout our lives we have accumulated an extensive experience in cutting things such as meat or fruits, and breaking things such as rocks or sticks. While some objects are certainly hollow, many are full and solid. We know that, and this is how we experience most objects even if we can’t directly see inside.

As some philosopher puts it (I can’t recall whom), when I look at a tomato, I experience the redness of the tomato on its entire surface, not only on the part that’s facing me. Most likely, we also experience the redness inside the tomato.

When we hold an object, we also perceive its mass. It is conveyed through the object’s surface, but it depends on what’s inside. The mass of an object (or similarly, its density) is a crucial information in many tasks. It decides on how much force we will apply to an object when we grab it. It is also a crucial piece of information when we need to throw an object such as a rock, an example Gibson mentions himself.

Either Gibson was not aware of this, or more likely, his model of the environment is not meant to be fully phenomenal, i.e., it is not meant to directly capture our subjective experience of the world. After all, we certainly don’t experience things as abstract as optic ambient arrays. It’s already challenging to understand what these things mean. Rather, Gibsons’ constructs seem to lie somewhere between the phenomenal world and the physical world. They form a layer of abstraction on top of the phenomenal world, that can help explain the phenomenal world.

Posted in Uncategorized | Leave a comment

Manipulatory vision

Suppose, just for the fun of it, that a visual system is the set of things that are attached to an individual and that regularly participate in that individual’s visual information gathering activities.

The attachment requirement is just for preventing the definition from being too broad, and from including things such as ambient light. Therefore, a pair of eyeglasses could be part of somebody’s visual system, as well as a headlamp, but not a ceiling light.

If we follow that definition, a visual system would include eyes and neurons of course, but also a whole range of muscles including the ocular muscles, as well as the muscles a person uses to move her head to look around, and the muscles she uses to walk around. Gibson stresses this point very clearly in his ecological approach to visual perception.

But it bothers me that Gibson disregards the role of manipulation in perception. I’m only at page 40, but it’s telling that in the book’s introduction he lists four types of vision: snapshot vision (as in perceptual experiments involving brief presentations of visual stimuli), aperture vision (as in experiments involving a head rest), ambient vision (where someone looks around) and ambulatory vision. Why is manipulatory vision not mentioned?

There are simple types of manipulatory vision actions that are almost entirely equivalent to ambulatory vision. Grabbing an object and moving it closer to one’s eyes is almost the same as to walking closer to that object. Grabbing an object and turning it around is very similar to walking around that object. There are often reasons to choose one method over another, for example when an object is too heavy or when it is displayed in a museum, but in the end, it is not consistent to include locomotion in the visual system but not manipulation.

Manipulatory vision is remarkably rich. I can open a box or crack open a hollow object to see its content, I can pull a curtain to see what’s inside a room, or I can wipe a dusty picture. A thief who has just stolen gems or a fisherman who caught many fishes may also spill their catch on a horizontal surface and scatter the objects to assess their overall value. I can put two objects side by side to compare their sizes, and even reorder many objects according to their size or their type to get a grasp of the distribution of their properties. Obviously, manipulatory vision is very relevant to infovis, one could even argue that all interactive infovis is manipulatory vision.

What all previous examples have in common is that visual information gathering is the ultimate purpose of the task, not manipulation per se. If I only care about what’s in a box, I can open the box then close it, and the box returns to its initial state. The only difference is that I know what’s inside.

Motor control is not all about manipulatory vision. If I move a bulky object that obstructs my passage, or open my umbrella when it starts raining, or cut an onion to prepare food, my goal is to alter my environment for other reasons than information gathering. Gibson provides examples of more permanents changes, such as building paths or shelters. Visual feedback is necessary for motor control, but the ultimate purpose in these cases is to alter the environment, not to gather visual information. Of course the two can be combined.

Human activities such as writing letters or painting portraits are also ways of changing the environment. Computer tools are now helping us with such productivity and authoring tasks, and HCI is very much concerned with these. Perhaps this could be a way of distinguishing HCI from infovis, for those who insist on doing this.

When Gibson discusses manipulation, he describes it mostly as a way of modifying the environment. This is surprising for a book on visual perception. Manipulatory vision seems to be of considerable importance, especially in infovis, where the concept has been rediscovered and termed “interactive information visualization”. It is time infovis researchers and designers recognize that interactive information visualization has its roots in manipulatory vision, an ability that humans and likely other animals have evolved a long time ago.

Posted in Uncategorized | Leave a comment

Inventing new terms serve scientists more than science

In a blog post Michael Billig says:

Throughout the social sciences, we can find academics parading their big nouns and their noun-stuffed noun-phrases. By giving something an official name, especially a multi-noun name which can be shortened to an acronym, you can present yourself as having discovered something real—something to impress the inspectors from the Research Excellence Framework.

Michael Billig focuses mostly on social sciences but a lot of what he says applies to HCI and infovis. He published a book Learn to Write Badly: How to Succeed in the Social Sciences which seems cool just by the title. Scott McLemee wrote an interesting review of this book where he quotes William James:

In the 1890s, William James complained that trendy psychological jargon of his day, such as “apperception,” served little purpose beyond, as Billig puts it, “enabl[ing] professors to be professorial” so as “to impress the impressionable.”

It is better not to introduce new terms unless there is a good reason, and perhaps even avoid using trendy terms that can be expressed clearly with everyday words.

On the other hand, in this interview Michael Billig gives an interesting answer to the the question “What one piece of advice would you give to social science scholars?“:

This is a more complicated, less innocent question than it might seem. If I was advising young scholars about how to have a successful career, I would advise them to join networks, to use the long words favoured by those networks and to promote their work within and beyond those networks. But if was to advise young scholars how to be genuinely scholarly, I would tell them the opposite: they should try to stand apart from established networks and to try to translate the currently favoured big words into as simple a language as possible. I would warn them that, in the current climate of instant publication and constant academic self-promotion, this scholarly way is not the way to conventional success.

I wonder if there are other similar books or articles on this topic, perhaps less focused on social sciences.

Posted in Uncategorized | Leave a comment

Literature survey decision charts for students

Many students who start with research hate to read papers. They realize there are LOTS of papers around. But reading is great! If you’re just starting, my advice is, read as much as you can right now, but don’t take notes and allow yourself to forget about what you read. It would take time to explain why but trust me.

My second advice is, do not read every paper carefully and entirely. Dedicate the time T = I * C * R * I. Interestingness, Clarity, Relevance, Impact. The TICRI rule, if you want. Who wants to read a paper that is BUIN, Boring, Unclear, Irrelevant, and No one knows about ? Those papers should be treated as if they didn’t exist.

The two decision charts below capture the way I deal with related work during a project. They don’t address online search, note-taking, bibliography management, or writing literature surveys. Only the decision process when finding a paper, e.g., should I read it? Cite it? How?

This decision chart explains how to deal with references in general:


This decision chart specifically explains how to assess the relevance of a paper:


Perhaps a last advice, take a paper’s weaknesses as good news. Many students are annoyed by high-impact but somehow underwhelming or unconvincing papers. But those are great! They show that you have a role to play. Cite them to motivate your work.


Posted in Uncategorized | Leave a comment

The smaller the sample, the better

The recent Radiolab episode 9-volt nirvana investigates whether putting electricity through the brain can enhance learning. The technique (called tDCS, transcranial direct current stimulation) has apparently generated lots of interest, both among scientists and hobbyists. People build machines and post videos on youtube. This is crazy.

The Radiolab team thought the same, so they went asking a university professor, a specialist on the topic. The professor replied that the technique does appear to be relatively effective, but then carefully stressed the word “relatively”. He and the hosts went on explaining that a lot of the previous studies did find a positive effect, but they typically involved a small number of subjects. They concluded that “those sample sizes are not enough to make a very big claim”.

This strikes me as an example of a familiar statistical reasoning fallacy. I will call it “the ugly small sample fallacy” for the lack of another name, although other names must exist and I’m far from being the first one to point this out [reference to be added soon].

The ugly small sample fallacy goes like this: if a study reports some statistically significant findings but there were not many subjects, it is much less impressive than if it had reported the same findings with many more subjects. Impressive can mean at least two things: the effect is reliable and the effect is large. The reasoning is wrong in both cases.

The ugly small sample fallacy version 1

As we know, the reliability of an effect is assessed by statistical procedures, typically by computing and reporting statistical significance. Obviously these procedures already take into account sample size. That “a lot of the studies do find a positive effect” as mentioned in the podcast interview suggests that the effect was statistically significant for all these studies. This does not tell the whole story about reliability, but for sure sample size becomes irrelevant at this point. When you get p < .05, it does not matter whether you obtained it from 3 subjects or 1,000 subjects: the strength of evidence is the same. Granted, statistical significance is not known for being the best way of conveying strength of evidence, but that’s what people use so I’ll stick to it here.

The ugly small sample fallacy version 2

The second version of the fallacy is more interesting, and more wrong. The specialist on tDCS seems to imply that the low number of subjects overall is a good sign that the technique is only “relatively effective”, with a stress on relatively. It seems reasonable to interpret this as the effect on learning being somehow small. So this amounts to say that if statistical significance is obtained from a small sample, the effect must be small. The opposite is true. As we all know, statistical significance is much harder to obtain on a small sample than on a large sample. When you manage to get p < .05 on a small sample, you can be sure that either you were testing quite a respectable effect, or you were really lucky. Conversely, statistical significance can be obtained with ridiculously small effects provided the sample is large enough. This is why large-sample studies can be the subject of harsh criticism.

Yet we all feel that something must be wrong with small samples. What it means for a sample to be small varies across disciplines, the experiment designs used, and people’s personal tastes. The “small” studies on tDCS typically involved 20 subjects, sometimes 40 or 50. In HCI it’s quite different. Still, any reviewer or researcher will start to freak out below a certain sample size. Reviewers especially are easily put off by small samples (defined based on their own personal taste), generally more so than the researchers themselves (who appear to have different tastes despite being the same people). Either way, saying “I can’t trust the results of a x-participant study” is committing the ugly small sample fallacy version 1. I don’t know how common is version 2. Sometimes reviewers seem worried for yet another reason. I will call this the ugly small sample fallacy version 3.

The ugly small sample fallacy version 3

It goes like this: “x participants is not representative enough, with so few participants you can’t generalize results to the whole population”. This is a fallacy because what makes a sample representative is not the number of participants. It is the way they have been selected, and how well this selection procedure matches the population the researcher claims to be studying.

To summarize, we shouldn’t be too mean with small samples just because they’re small. Contrary to the common belief, if a study is statistically significant, the smaller the sample, the more impressive the study is.

Of course this is only the case when interpreting statistically significant results from a study that has been already carried out. Things are very different when you’re preparing a study, a topic on which I may post about later on. Things are also very different when you’re interpreting statistically non-significant results (something your textbook has strongly forbidden you to do anyway). And I should point out that very small samples (perhaps 10 or less) are seriously problematic to analyze if the investigator knows nothing about the population distribution, but this is also a different topic.

Posted in Uncategorized | Leave a comment

You got p = .06? This serves you well!

This morning I was reading another article pointing out the absurdity of dichotomous statistical thinking in empirical research. This is where you can find this nice quote: “Surely, God loves the .06 nearly as much as the .05”. I thought again about the day when a colleague reviewing a paper asked me whether drawing conclusions from p = .06 was wrong, and I had something urgent and said mechanically yes. Later this bothered me, but I now understand why this was the right advice.

Somehow there is something wrong when you object to the rules of a game only when they are not in your favor. People never complain when they get p < .05. They passed the test, they sure did a good job and their unquestionably reliable findings deserve to be published (unlike those other losers, ha!). But when they get p = .06 the reasoning often goes quite differently. Come on, it’s so close! Why should p = .06 be treated so differently from p = .05? How can that make sense? Indeed it does not, but these are the rules by which you accepted to play.

In the first chapter of his book “Statistics as Principled Argument”, the statistics methodologist Robert Abelson states six laws. One of them is “Never flout a convention only once“. I suppose he refers to the same sort of self-serving reasoning, and there is definitely some analogy with the universally criticized practice of data dredging.

There are solutions to this, but somehow people seem as good at ignoring them as waiters are at ignoring customers waving at them in the most touristy bars of Paris. Perhaps it is laziness, perhaps fear, maybe a mix of both, since quitting well-established statistical analysis rituals does take a tiny bit of effort and courage. Whether people really believe or pretend to believe that dichotomous decision rules with an arbitrary cut-off should keep their central place in science, then fine, but then perhaps they should fully embrace these rules and their unpredictable consequences.

This is why when an unlucky scientist just got p = .06 and bursts into tears, I sometimes want to say “this serves you well!“.


Posted in Uncategorized | Leave a comment

So, I’ve decided to start a blog

Well, it’s really more public notes than a blog. More on this silly decision later.

Posted in Uncategorized | Leave a comment