Andy Clark on Perceiving as Predicting

Andy Clark’s work in the philosophy of mind has long had a strong influence on my thinking. I’ll have a bit to say about how his extended mind thesis figures into philosophical practice in my upcoming talk for the Media Ecology Assocation, but in this post I want to explore his work on predictive processing and perception, as he’s converging on similar conclusions to my own about the nature of perception, understanding, imagination, and action—namely, that they all arrive together in the co-construction of experience.

His tools and models for making this claim are different from my own, and so what most interests me in this context is how his (more advanced) resources—including a computational theory of the brain, coupled with extended and embodied notions of cognition, hierarchical predictive processing models, and Bayesian accounts of inference—match up with what I’ll shorthand as the transcendental–phenomenological resources of philosophy that I’ve been using in my recent posts. Clark is not strictly speaking a cognitive scientist, but he’s definitely closer to the “neuro” in “neurophenomenology” than I am. What’s at stake for me here is the following question, How accurate and useful are these transcendental–phenomenological resources in the face of cognitive science?

By “transcendental–phenomenological” I mean loosely post-Kantian critical and transcendental philosophy, but I won’t foreclose the inclusion of the esteemed efforts of philosophers dating as far back as Plato and Aristotle, when construed along the lines of their extensive epistemic efforts. In short, my question concerns the ability of philosophy to yield anything like the kind of self-knowledge it advocates as delivering. If this question is answered in terms of the empirical details revealed about the causal physiological systems that underpin our biology, then the answer is clearly no, these resources don’t tell us anything about neurons, synapses, dendrites, axons, action potentials, and so on, but at the level of describing experience as the coordination of perception, understanding, imagination, and action, then it seems the answer is yes, the philosophical resources are up to the task, particularly when we treat them as fallible, experimental, and open to ongoing refinement and evolution.

That such resources are successful at all is also what draws me to the work of people like Mark Siderits and Jay Garfield, who add a Buddhist and contemplative dimension to the neurophenomenological discussion. Unfortunately, I am basically out of my lane when talking about either Buddhism or cognitive science, but I am reasonably confident that I can at least interpret in broad strokes the conclusions of these fields, especially in cases where the conclusions converge with those of philosophy. To be sure, I’m not claiming that convergence is the norm, or even that philosophers, scientists, and contemplatives are easily parsed in all cases (indeed, in these areas it is common to find a practitioner versed in more than one of these disciplines). What I am saying is that convergence, when it does occur, is a meaningful point of triangulation. It suggests we’re on the right track in some nontrivial way.

But let me get back to Clark. In the talk I include above, he offers the following summation of his view of the role of prediction in perception: “Instead of trying to build up a picture of how the world is on the basis of lots of little incoming cues, kind of getting them together to form a bigger and bigger picture of what’s out there, you try to work out what the incoming cues are on the basis of your existing picture of what’s out there. In other words, the brain is trying if you like to guess the incoming sensory data on the basis of what it knows about what the world is likely to be” (14:34–14:56). In challenging what Clark calls the “lego-type” view of perception being built up from discrete bits of atomistic data, we get a view of perception that works from the top down. In other words, on this account, perception tries to match incoming sensory signals with a multilayered top-down prediction about what the signals likely indicate.

These predictions are arranged hierarchically in terms of certainty gradients that draw on stored knowledge to make inferences about possible future occurrences within a specific scenario. On this view, perceivers are also imaginers; they make in-the-moment inferences about present objects of perception and use their inferential capacities to make predictions about what’s going to happen next. Again, these imaginative capacities are rooted in stored knowledge—as captured regularities within recognizable scenarios—and are used as the basis for understanding what we perceive in experience. Both learning and action, argues Clark, are rooted in the active involvement of cognition in prediction tasks, tasks that are simultaneous with the act of perception itself. This is the basis for Clark’s view that perception, understanding, and imagination are all participant at the ground floor of our experience of the world, including in our visual and motor understanding.

When Clark states that predictions are based on the stored knowledge of comparable past events, he is as far as I can tell offering a contemporary version of Kant’s description of the creation of empirical or a posteriori concepts, which involves comparison, reflection, and abstraction. I quote Kant (quick and dirty from Wikipedia):

In order to make our mental images into concepts, one must thus be able to compare, reflect, and abstract, for these three logical operations of the understanding are essential and general conditions of generating any concept whatever. For example, I see a fir, a willow, and a linden. In firstly comparing these objects, I notice that they are different from one another in respect of trunk, branches, leaves, and the like; further, however, I reflect only on what they have in common, the trunk, the branches, the leaves themselves, and abstract from their size, shape, and so forth; thus I gain a concept of a tree.

Thus for Clark as for Kant all perception is to some degree conceptual (Clark’s exact words are “percepts are always at least weakly conceptualized”). Interestingly, not only is Clark’s view consistent with the empirical side of Kant’s philosophy, it is also consistent in some ways with the transcendental or a priori side of his philosophy, insofar as Kant also deduced the necessity of both a regulative concept of world and the importance of imagination in making perception meaningful. Recall Clark’s words I quoted earlier, “the brain is trying if you like to guess the incoming sensory data on the basis of what it knows about what the world is likely to be.” The concept of world in both cases serves a regulative as opposed to a veridical function in cognition that in turn affords a kind of baseline in perception upon which the empirical manifold can be arrayed, analysed, understood, and compared.

It’s a functional notion of world, not a metaphysical one. This to me is quite similar to Clark’s assertion, at least formally, that perception is not in the first instance a building up of atomistic sense impressions, but is rather a top-down field of understanding within which phenomena are arrayed. Again, my point here is not that Clark makes no advance over Kant—his contributions to embodied and extended models of cognition surely do that, and his integration of neurophysiological data is surely beyond anything that Kant could have conceived—the point is much more modestly that Kant, and even his predecessors, have aged much better than some of my more pessimistic interlocutors and peers are willing to concede. I think this is good news, and it should serve as good motivation for philosophers to continue on in their work, especially insofar as philosophy’s relevance in today’s society is an ongoing question for many people, in and out of the university.


    • if you get a chance check out the Ward, especially ( the rest is important as he talks thru McDowell) around 24 mins in where he contrasts M-Ponty vs Kant in terms of contingent forms of bodily activity vs transcendental ego as the bundlers of signals.


      • I’ll check it out. Also, I’m on board with all the many people who came after Kant to show the various contingencies of bodily activity (and language and history and so on) that play into the construction of experience.


      • cool, also check yer email he kindly shared his outline/notes with us


      • This is an interesting talk. I jotted down 2–3 pages of notes and will write something up soon. Montero has some interesting answers to these questions when it comes to proprioception being guided by certain aesthetic (I’d say conceptual) ideals that encourage the re-modulation of action in this or that direction (i.e., towards this or that goal—e.g., more fluidity, less power, more grace). And I think this idea of sensory signals being synthesized at the level of motor intentionality is right (and this takes us back to Aristotle, who had aēsthesis as a form of sensory synthesis directly tied to the contingencies of bodily movement, rather than at some higher level like the understanding or the transcendental ego, to use Kant’s terms). Though, predictably, and as Ward mentions above, I find Kant’s description that concepts and intuitions are conversant (and are similarly expressible at the level of explicit language) precisely because they are varieties of the same type of synthesis, rather than two different kinds—one nonconceptual and bodily, the other conceptual and intellectual. Why the need for two types, I wonder? Along these lines I feel that (a) we end up with an unneeded dualism or bifurcation and (b) that we’re forced, as I think Ward does a good job of acknowledging, to apply a bit of sleight of hand in thinking that the body “just knows” how to get around in space. When I have time in the next few days I’ll write this up, and it will probably make it into the paper as I feel like we’re actually making some progress.


      • thanks Adam as always I’m grateful for all the hard work (and patience) you put into this line of inquiry, be glad to see what you make of it all, d.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s