23 MOTOR REPRESENTATION AND KNOWLEDGE OF SKILLED ACTION

Corrado Sinigaglia and Stephen A. Butterfill

23.1  Introduction

Most of the chapters in this volume concern the role of skill in doing. Our focus is on its role in observation. Why consider observation? Much evidence suggests that skill is every bit as important for observing as for doing. It is no exaggeration to say that your skills are a foundation of your ability to observe and acquire knowledge about others’ actions.

Start with an illustration. A  commentator and a former player are at a basketball match observing a player taking a free shot, potentially winning the match. Just as the ball leaves the player’s hands, the lights go out and the stadium is in complete darkness. So information about the early part of the player’s kinematics is available to the two observers whereas they have little or no information about the ball’s trajectory. Each observer, the commentator and former player, is asked to predict whether the ball went into the basket. Who would you bet is more likely to be right? Betting on the commentator might seem safe: she has years of experience watching shots taken from this perspective, while the former player, although skilled, has comparatively rarely observed the action from this perspective. Yet it is the former player, not the commentator, who is more likely to make the correct prediction.1 As this illustrates, being skilled in performing actions of a certain type can enable you to acquire observational knowledge about others’ actions of that type. But why? Why are those more skilled in performing certain actions (sometimes, at least) better able to acquire knowledge when observing those actions?

Our chapter aims to answer this question. Before facing it directly, in Section 23.2, we shall first review further evidence for the premise that those more skilled in performing certain actions are, at least sometimes, better able to acquire knowledge when observing those actions. We shall then elaborate a conjecture which, if correct, would answer the question. Our conjecture is that performing and observing actions involves a common element, namely motor representations of outcomes to which the actions are directed. It is this common element that explains why skills matter not only for performing actions but also for gaining knowledge in observing actions. While a body of evidence supports this conjecture (see Section 23.3), it faces an objection. The double life motor representations lead appears to require that they have two, incompatible directions of fit. After answering this objection (in Section 23.4), we find ourselves confronted with a further challenge. If our conjecture is right, whether we know something about the goals of an action sometimes depends on how we represent that action motorically. That is, motor representations can have content-respecting infuences on knowledge states. The challenge, which we call the Interface Problem, is how there could be such influence. At present many candidate answers are available, as we shall see in Section 23.5. In our view, neither any available evidence nor narrowly theoretical considerations are likely to yield decisive reasons in favour of any one answer. A deeper understanding of how expertise matters for gaining knowledge of observed actions ultimately requires new discoveries on how motor representations and knowledge states interface.

23.2  That skills matter for observational knowledge

The premise of our question is that those more skilled in performing certain actions are, at least sometimes, better able to acquire knowledge when observing those actions. This is illustrated by Aglioti et al.’s (2008) basketball study, as we have seen. But this is much too bold a premise to accept on the basis of a single study. What further evidence supports it?

The method of Aglioti et al.’s (2008) study relied on correlation as there was no intervention on the subjects’ level of skill. By contrast, Urgesi et al. (2012) included a training component. They studied volleyball fans and players, replicating and extending Aglioti et al.’s (2008) findings. They then studied two groups of nonplayers, giving one observational training and the other training in playing volleyball. They found that those trained in playing volleyball also acquired observational skills not found in those given observational training only. Specifically, only those trained became able to use bodily movement in predicting whether a shot would be in or out.2

Acquiring a skill through training usually involves observing actions, even if only one’s own. Could it be observational experiences associated with training, rather than the acquired skills to act, which explain improved observational abilities? To answer this question, Casile and Giese (2006) investigated the effects of training blindfolded subjects to acquire a new skill. (The blindfold ensured they could not visually observe the actions they were being trained to perform.) When walking, people typically swing their arms in phase with their legs. It is surprisingly difficult to adopt different phase relations between arms and legs, although this can be learnt (Chapman et al. 1970). Importantly, it can also be difficult to identify walking actions involving unusual phase relations between arms and legs through observation. Exploiting these facts, Casile and Giese (2006) trained subjects in performing a silly walk while blindfolded. Before training, and again after training, each subject was tested on her ability to visually identify walks that differed in the phase relation between arms and legs. The found not only that training improved accuracy (see Casile and Giese 2006: figure 2 on p. 70), but also that how well a subject could perform a novel walk after training was correlated with how accurately she could visually identify it after (but not before) training (pp. 71-2). This is evidence that the effects of having skills to act on your abilities to observe actions does not, or does not always, depend on gaining sensory experience in observing actions acquired as a side-effect of training.

Further evidence for the effect of skills to act on acquiring knowledge of others’ actions through observation comes from research with human infants. When observing a hand that is approaching some objects and about to grasp one of them, infants will, like adults, often look to the target of the action in advance of the hand arriving there (Falck-Ytter et al. 2006). As in adults, this proactive gaze indicates that the infants are tracking the goals of the observed actions.3 Critically, though, the occurrence of this proactive gaze in infants is related to their acquisition of the skills needed for performing the actions. For those infants who are as yet less good at reaching and grasping, their eyes do not arrive on an object to be grasped in advance of the hand grasping it (compare Kanakogi and Itakura 2011). Further, if we consider proactive gaze for different kinds of observed actions (such as putting objects into containers or various kinds of grasping actions), we find that infants’ gaze to the target of an action becomes more proactive as they become better able to perform the particular kind of action observed (Cannon et al. 2012). Ambrosini et al. (2013) extended this finding by exploiting the fact that grasping skill can be measured by the minimum number of digits used to grasp from ten to two. Remarkably, they found that this fine-grained measure of skill in performing actions correlated with infants’ abilities to track the goals of observed actions.4

What happens if we intervene on infants’ skills? Sommerville et al. (2005) put *sticky mittens’ on three-month-old infants and allowed them to play with objects. This allowed them not exactly to grasp objects but to manipulate them with their hands, which likely boosted their skills. Following the training (but, in other studies, not mere observation: see Sommerville et al. 2008; Gerson and Woodward 2014; Bakker et al. 2015), infants manifested abilities to track the goals of simple object-directed actions which untrained infants appeared to lack.5 This indicates that, as with adults, boosting an infant’s skill in performing an action can have a corresponding effect on the infant’s abilities to track the goals of actions of that type. The link between skills and action observation is present even as humans are first acquiring skills to manipulate worldly objects.

In this section we have seen evidence from a range of studies covering a diversity of skilled actions which supports our premise that those more skilled in performing certain actions are, at least sometimes, better able to acquire knowledge when observing those actions (for a longer review and some bolder claims, see Shiffrar and Heinen 2011).

This premise requires qualification. The cases of skill we have considered all involve very small-scale bodily actions: they are skills such as throwing a ball, walking, and grasping an object. These examples of skill contrast with, say, the skill that a successful politician exercises in winning an election for the fifth time despite having caused several disasters and being transparently corrupt. We should not generalise from one kind of skill to another without evidence.

23.3  The effects of skill depend on having capacities to represent actions motorically

Why should having a skill required for the performance of an action ever enhance your ability to observe and acquire knowledge about actions of that type? It is, sometimes or always, because there is an element common to having skill in performing an action and being able to acquire knowledge about that action through observation: both involve abilities to represent outcomes motorically. Or so we conjecture.

In this section our aim is to elucidate and support this conjecture. Let us start by further specifying the conjecture as the conjunction of three subclaims:

1. exercising skills in performing very small-scale bodily actions involves representing certain outcomes motorically;
2. motor representations of these very outcomes sometimes occur when you are merely observing the actions; and
3. the occurrence of these motor representations can enhance your ability to acquire knowledge about the actions when observing them.

In what follows we review evidence for each of the three subclaims in turn.

Skill in performing an action often depends in many subtle, ordinarily unnoticed ways on anticipating its consequences. For instance, to open a sliding drawer with any kind of finesse, you need to make anticipatory postural adjustments in order to maintain your balance, and these adjustments need to be tightly coordinated with the timing of your action. The difficulty, and the importance, of such anticipatory adjustments can be made vivid by considering developmental changes in infants’ abilities to perform actions such as reaches. Infants’ initial anticipatory postural adjustments are not well timed and only gradually develop adult-like temporal precision over a period of around seven months (Witherington et al. 2002; see also von Hofsten 1991). These and other minute observations of the development of abilities to perform very small-scale actions reveal that, often enough, the early parts of a skilled action anticipate the future parts in ways that cannot be determined from environmental constraints alone.

Adult-like skill in performing an action also involves anticipatory adjustments related to goals of later parts of the action (rather than merely to side-effects of action such as changes in posture). To illustrate, consider arranging some new books (the old-fashioned, paper kind) on your shelves. You take them out of a box on the table and place them on different shelves depending on where they belong in your collection. Some go on to high shelves, some are placed low down. Although mundane, this is a highly skilled activity. One aspect of the skill concerns how you grasp each book. Typically, the higher the shelf for which a book is destined, the lower on its spine you will grasp it (Cohen and Rosenbaum 2004). The lower grip is initially more awkward but makes things easier when you come to finally placing the book.6 As this illustrates, some skilled actions unfold in ways that anticipate the goals of later parts of the action (see Kawato 1999 for another example).

Given that skilled performance of very small-scale actions requires anticipation, including anticipation reflecting the goal’s future actions, how is such anticipation achieved? Many researchers hold that control of very small-scale actions involves motor representations.7 These representations specify outcomes that are quite abstract relative to bodily configurations and joint displacements. They may represent the movement of a book from one place to another, for example; or they may represent the securing of an object in a way that is neutral as between manual grasping, oral grasping and grasping with a tool (e.g., Rizzolatti et  al. 2001; Umiltà et al. 2008; Cattaneo et al. 2009). By representing outcomes, motor representations enable some forms of anticipatory control in performing very small-scale bodily actions.

A further reason for postulating motor representations of outcomes arises from the need to understand how action control can be computationally tractable given the many degrees of freedom, even after taking bodily synergies into account, afforded by the body’s joints (Tessitore et  al. 2013). Representing actions in terms of outcomes and the means by which they are achieved contributes to making anticipatory control tractable.

So much for the first of the three subclaims of our conjecture (item 1 in the list above). What about the second subclaim? This is the idea that motor representations lead a double life, for they sometimes occur not only when you are performing an action but also when you are merely observing another performing that action. Perhaps the most direct evidence in humans for this subclaim comes from measuring motor evoked potentials on the muscles of an observer. Several investigations have revealed that when motor activity is amplified in the brain of an observer using transcranial magnetic stimulation, there are minute indicators of muscle activation in the observer in the right muscles at the right time (e.g., Fadiga et al. 1995, 2002; Cattaneo et  al. 2009; Cavallo et  al. 2011). This is one indication that observing an action can trigger motor representations that would occur in you if you were not merely observing but actually performing the action. Further evidence for the occurrence of motor representations in action observation has been obtained from many scenarios using a wide variety of neurophysiological and behavioural measures in both human and monkey subjects (Rizzolatti and Sinigaglia 2016 provides reviews; see also Rizzolatti and Sinigaglia 2008, 2010).

Importantly for our purposes, what is represented motorically in action observation includes outcomes to which actions are directed. We know this thanks to experiments that (a)  vary features of the action such as kinematics and context while keeping the outcome constant (e.g., Rizzolatti et  al. 1988; Umiltà et  al. 2008); and (b)  vary the outcome while meticulously exploiting video editing to keep all other features of the action as constant as possible (e.g., Fogassi et  al. 2005). The findings from these experiments suggest that there are motor representations whose occurrence in an observer varies specifically with the outcome of the action observed (see Pavese, Chapter 18 in this volume).

Further, the degree to which motor representations are involved in observation depends on the level of your skill in performing the action (see Cross, Chapter 22 in this volume). Or at least this is suggested by comparing pianists with nonpianists (Haslinger et al. 2005), and ballet with capoeira dancers (Calvo-​Merino et al. 2005). In each case, observers skilled in performing the particular kind of action showed stronger brain activation in areas that would be involved in performing that action. This indicates that they represent the observed actions motorically in ways that the unskilled do not.

But what are those motor representations of outcomes doing in you when you are merely observing another’s actions? The answer, we suggest, is that they are enhancing your ability to acquire knowledge about the observed actions. Or, if not the whole answer, this is at least one thing those motor representations are doing. Hence, the third subclaim of our conjecture (item 3 in the above list). But why accept this?

Evidence for this subclaim comes from experiments in which subjects’ abilities to represent actions motorically are momentarily impaired. This can be done indirectly by constraining body parts, which is thought to impair abilities to represent motorically actions involving those body parts. Or it can be done more directly using transcranial magnetic stimulation (TMS) pulses. However it is done, the findings are broadly similar. The proactive eye movements which indicate goal tracking are less likely to occur (Ambrosini et al. 2012; Costantini et al. 2014), and your judgements about the goals of actions are slower (D’Ausilio et al. 2009) or less accurate (Urgesi et al. 2007; van Kemenade et al. 2012; Michael et al. 2014) when your ability to represent actions motorically is momentarily impaired.

If the effect of performance skills on action observation is sometimes or always mediated by motor representations as we are suggesting, then we might guess that how accurate your observations are would depend on how closely you and the agent observed are matched in skill. You would be best at using observations to predict outcomes of actions performed by someone very like you in skill. Accordingly, Knoblich and Flach (2001) and Knoblich et al. (2002) tested how matches in skill between observer and agent influence predictions. They had subjects watch videos of actions and predict their outcomes, where the agent in the video could be either a stranger or the subject herself (thus ensuring an ideal match in skill between observer and agent). They found that, for throwing darts and also for handwriting, subjects could best predict the outcomes of their own actions, even when not informed that the observed actions were their own.8

The subclaim that motor representations can enhance your ability to acquire observational knowledge of actions needs further specification. For it should not be assumed that the influence of motor representations on knowledge states is a matter of one-off triggering: it may be more plausible to suppose that motor representations dynamically influence processes by which knowledge states are arrived at and maintained (as Fridland 2016 suggests in a parallel case). And, further, the influence of motor representations on these epistemic processes should probably not be understood as a matter of determining their outcomes in the sense that you could predict exactly which knowledge states will be arrived at from knowledge of what is represented motorically (compare Burnston 2017). Finally, note that we are here offering no suggestions about how motor representations may enhance abilities to acquire knowledge: the view defended so far is silent on processes.

So why are those more skilled in performing certain actions (sometimes, at least) better able to acquire knowledge when observing those actions? In this section we have elaborated and defended our conjecture that it is sometimes or always because both involve abilities to represent outcomes motorically (see Shiffrar 2010 for a broader review). Your skills matter for action observation because you exercise these skills in observing much as you do in performing an action.

While a body of evidence appears to support this conjecture (as we have just seen), there is an objection to its coherence. The objection concerns the idea that motor representations both control actions (so must have a world-to-mind direction of fit) and are involved in tracking the goals of others’ actions (which requires a mind-to-world direction of fit).

23.4  An objection: motor representation and direction of fit

Our conjecture entails that there is a single kind of representation, motor representation, which features both in performing actions and in gaining observational knowledge of them. Let us take a further step and suppose that this is no accident but, rather, reflects the functions motor representations serve. It appears to follow that representations of this kind have different directions of fit. Apparently, some are world-to-mind insofar as they are supposed to lead to performing actions, while others are mind-to-world insofar as they are supposed to enable predictions of others’ actions.

This objection arises from the idea that motor representations lead a double life. On the view introduced earlier, what is represented motorically in action observation includes outcomes to which actions are directed. Further, these representations have the function of enhancing your ability to acquire knowledge about the observed actions. It seems, therefore, that these representations must be mind-to-world.That is, their function depends on the motor representation representing only outcomes that are actually goals of the observed action. Because the idea that motor representations lead a double life involves saying that there is just one kind of representation here and in action control, it appears to entail that different instances of a single attitude can have different directions of fit: some are world-to-mind, others are mind-to-world. But this is arguably incoherent. So the objection.

One response to the objection would be to embrace the idea that a single representation can have multiple directions of fit, as others have done (see, for example, Millikan 1995; Shea 2018: Section 7.2). As not everyone accepts this,9 we propose a reply to the objection that does not depend on contradicting it.10

As a preliminary to replying to this objection, consider an analogy. There is a rotary dial on your oven that enables you to initiate and control the oven’s activity. We might think of the dial as having an oven-to-instrument direction of fit: the oven temperature is supposed to adjust to the setting on the dial. But now suppose, further, that there is an indicator light on your oven that is illuminated unless the oven has reached the temperature specified by the dial. This enables you to use the dial to discover the temperature of the oven: if the light is on, you turn the dial down until just the point where the light goes off. Now the setting on the dial tells you the temperature of the oven. So, we might think of the dial as having an instrument-to-oven direction of fit.

This analogy will guide our response to the objection. The key point can be put like this. There is a core system featuring the dial, thermostat, heating element and oven. Relative to this core system, the dial always has an oven-to-instrument direction of fit. However, there is a larger system which embeds the core system and exploits it for novel ends. This larger system includes you and your capacity to temporarily prevent significant changes in the temperature of the oven (perhaps by moving the dial between settings too quickly for the heating element to respond). Relative to this larger system the dial has an instrument-to-oven direction of fit. So, to understand the dial's functions, we do need two directions of fit, oven-to-instrument and instrumentto-oven. But this is not quite to say that the dial has both directions of fit. For something has a direction of fit only relative to a particular system. Which direction of fit we see depends on which system we are considering. Understanding the dial does not require supposing that anything has two directions of fit relative to a single system.

Our response to the objection is similar. If we consider planning-like motor processes only (the core system), then each motor representation’s function is linked to initiating and controlling action. From this perspective, only a world-to-mind direction of fit is in view. But these planning-like motor processes can occur in the context of a larger system, one that involves something that somehow prevents performance of action. The functions of this larger system concern predicting which outcomes actions will be directed to. If we consider this larger system, it is natural to describe the motor representations as having a mind-toworld direction of fit. So, as in our analogy, which direction of fit we see depends on which system we are considering. We need never have two directions of fit in view simultaneously. Our reply to the objection, then, is that our conjecture involves no incoherence when properly understood.

So much for the objection. From here on we will assume that it is at least coherent to conjecture, as we have, that having skills enhances your observational abilities wholly or in part because it enables you to better represent the goals of others’ actions motorically. But the truth of this conjecture would raise a theoretical challenge. The challenge concerns how motor representations could have content-respecting infuences11 on knowledge states. We call meeting this challenge the Interface Problem.

23.5  How do motor representations influence knowledge states?

Suppose, as we have been considering, that having skills enhances your observational abilities because it enables you to better represent the goals of others’ actions motorically. Then we are sometimes in this situation: in observing an action, we represent a certain outcome motorically; and partly in virtue of this motor representation, we come to know that this outcome or a matching12 outcome is a goal of the observed action. It is not just that motor representations influence knowledge states: whether we know something can depend, in some way, on what we represent motorically. That is, motor representations can have content-respecting infuences on knowledge states. How is this possible?

There have been millennia of discussion about knowledge and, more recently, quite a bit of research on motor representation. By contrast, there is comparatively little written on how the two might be connected. For this reason, it seems to us worthwhile to consider a range of candidate answers at this stage. As the question is ultimately a scientific one, our hope is that some or all of the rough candidate answers reviewed in this section can be turned into hypotheses generating readily testable predictions.

One candidate answer—“Identity', as we shall call it—involves identifying motor representations with knowledge states. Perhaps, for example, motor representations that occur in action observation are one kind of knowledge state. This candidate answer implies that there is no issue concerning how motor representations can have content-respecting infuences on knowledge states; or, if there is, it is just a special case of the more general issue of how knowledge states can have content-respecting infuences on each other.

An alternative candidate answer is that motor representations are connected to knowledge states via inferential processes. There is no mystery about how one knowledge state can have content-respecting infuences on another knowledge state: inference is paradigmatically a process by which mental states have content-respecting infuences. It might, in principle, be that motor representations can similarly lead to knowledge states via a process of inference combining the two kinds of state. One attraction of this candidate answer (call it ‘Inference’) is that, if correct, it would appear to solve the Interface Problem without conceptual novelty or reliance on untested conjectures.

How can we determine the correctness of these views, Identity and Inference? Suppose you represent the goal of some observed action motorically. Then you should be in a position to know what the goal of the action is. How could you not? Given Identity, motor representations that occur in action observation are one kind of knowledge state, so failure to know would presumably involve having inconsistent beliefs or some other knowledge-preventing failure of rationality. And given Inference, only this or failure to make a simple inference could preclude you from knowing. So, if it turned out that humans can represent the goals of observed actions motorically while lacking corresponding knowledge despite there being no inconsistency or failure to make a simple inference, then we would have evidence against both Identity and Inference.

Is this prediction testable? Consider that there are parallels to Identity and Inference for a question about how motor representations relate to intentions rather than knowledge states. One consideration against these parallel views arises from ways motor representations and intentions can fail to match, as illustrated by Anarchic Hand Syndrome (compare Mylopoulos and Pacherie 2017:  323). Subjects with Anarchic Hand Syndrome, which usually follows lesions of the anterior part of the corpus callosum and of the supplementary motor area, perform actions incompatible with their avowed intentions. These patients may also refer to their anarchic hand as having a mind of its own (Della Sala et  al. 1994). Consider a subject with Anarchic Hand Syndrome who intends not to drink some hot tea until it cools. As one hand attempts to pick up the cup and bring it to her mouth, she needs to intervene with her other hand to put the cup back onto the table (Della Sala et al. 1991: 1114). There is clearly some mismatch between her intentions and her motor representation. But there is no reason independent of accepting Identity to suppose that the mismatch involves anything like the kind of irrationality that occurs when one knowingly has incompatible intentions. And since Anarchic Hand Syndrome is not specifically linked to failures in reasoning, nor is there reason independently of accepting Inference to conjecture that the mismatch is due to failure to make a simple inference. Our suggestion is that parallel considerations could be used to show that any discoverable mismatches between knowledge (rather than intention) and motor representation provide evidence against Identity and Inference. If so, the striking prediction is readily testable.

A third candidate answer, due to Mylopoulos and Pacherie (2017), involves the notion of an executable action concept.13 This is a concept that ‘could guide the formation of a volition, itself the proximal cause of a corresponding movement’ (Mylopoulos and Pacherie 2017: 324). To illustrate, in humans MANUAL REACH would typically be an executable action concept whereas WAG TAIL would not typically be. Further, they propose that concepts are executable action concepts in virtue of an association between the concept and a motor schema14 such that when a thought involving the concept occurs, the associated motor representation is activated.

We might suppose that the converse also occurs. That is, when a motor representation occurs in action observation, any associated executable action concepts are somehow activated, biasing the observer to think about the corresponding action.

Could this idea explain, in principle, how motor representations have content-respecting influences on knowledge states without there being any inference or translation process? Suppose that the concept MANUAL REACH is associated with a motor schema for manual reaching. Then when observing someone reach for a cup, this outcome is represented motorically. Such a motor representation involves both the specific outcome involving this particular cup and a motor schema. Because the motor schema is associated with the concept, activation of the motor schema increases the probability that the concept will be activated too.

Our fourth (and last) candidate answer invokes experience. When observing an action you may have an experience that provides you with reasons for a judgement about the goals of that action. Call this an experience revelatory of action. Now we know that motor representations can influence perceptual processes (Bortoletto et al. 2011), and there is even some evidence that motor expertise may influence whether an experience reveals an action (Funk et al. 2005). To make a leap, we might guess that which outcomes are represented motorically can influence which goals are revealed in experiences revelatory of action: that is to say, motor representations can shape experiences revelatory of action. Perhaps this is how motor representations can have content-respecting influences on knowledge states.

This candidate answer, call it Experience, suggests a rough parallel between two interface problems. One concerns representations that feature in perceptual processes, the other motor representations. In both cases, the question is how these representations can have contentrespecting influences on knowledge states. And in both cases the answer is that the representations shape experiences, which in turn provide reasons for judgements. There is a reason why some animals have experiences:  it provides a link between representations with different kinds of formats in cases where binding things together with inferential processes would be suboptimal (perhaps because it would break otherwise useful encapsulation, for example).

Of course, the parallel needs careful development. Whereas there are perceptual modalities, it is perhaps unlikely that there is also a motor modality. So the closest parallel for motor representations’ influence on knowledge states may be with amodal perceptual representations such as object indexes (as Sinigaglia and Butterfill 2015: 12 suggest). It would also be possible to develop Experience while rejecting the parallel with a perceptual interface problem entirely (see Sinigaglia and Butterfill 2016: 156-8).

However exactly it is developed, Experience faces a methodological objection. As a candidate answer to a question about one problem (How can motor representations have content-respecting infuences on knowledge states?) it raises two questions that appear no less puzzling. After all, it seems no easier to understand how motor representations can have content-respecting infuences on experiences, nor how experiences can have contentrespecting influences on knowledge. Our view is that such objections carry little weight when we are so far from understanding how to solve the Interface Problem. Any candidate answer that can be turned into a hypothesis capable of generating readily testable predictions is worth considering.

But could Experience really be turned into a testable hypothesis? We know that motor representations can influence judgements about the trajectories of bodies in motion (Shiffrar and Freyd 1990; Blake and Shiffrar 2007). But how? The Visual Hypothesis says these judgements are based are visual experiences of movements only. By contrast, the Action Hypothesis says that motor representations can influence experiences associated with bodily trajectories in ways that are not exhaustively visual. Note that the Action Hypothesis does not flow directly from Experience (which is, strictly speaking, consistent with the Visual Hypothesis). However, the Action Hypothesis could be regarded as one way of developing Experience. If the Action Hypothesis is right, there should be situations in which subjects can rationally distinguish bodily trajectories in ways not fully explained by their visual experiences of movements. To illustrate, consider placing a solid barrier somewhere along a possible hand trajectory. Suppose (as might in principle happen) that subjects were to judge, on the basis of observation, that the hand follows this trajectory and that they also report not seeing the hand pass through the solid barrier. If the same combination were not obtained concerning the movements of mere shapes (rather than hands), we might conclude that the judgement about the hand trajectory is not, or is not entirely, a consequence of visual experiences of movement. This would be evidence for the Action Hypothesis. Of course, it is unlikely that things would turn out so neatly. We mention the possibility merely to illustrate one virtue of the fourth candidate answer, Experience:  although perhaps implausible and complicated, it is a source of hypotheses that generate readily testable predictions.

23.6  Conclusion

We started with the discovery that those more skilled in performing certain very smallscale bodily actions are sometimes better able to acquire knowledge when observing those actions (Section 23.2). But why? We conjecture that it is because performing and observing actions involves a common element, namely motor representations of outcomes to which the actions are directed. This conjecture is supported by a range of evidence (23.3). It is also theoretically coherent (23.4). However, its correctness would leave us with a deeper and more puzzling question than the one it aims to answer.

If our conjecture is right, whether we know something about the goals of an action sometimes depends on how we represent that action motorically. That is, motor representations can have content-respecting infuences on knowledge states. How is this possible? As we have seen in Section 23.5, there are at least four distinct candidate answers to this question. Further, at least three of these are consistent with each other in the sense that, in principle at least, any combination of them could be correct. There is a gap in our understanding of how expertise matters for gaining knowledge of observed actions.

In our view, progress could be made in two ways. The first is to explore links between different interface problems. An interface problem arises wherever it is challenging to explain how representations of one kind can have content-respecting infuences on representations of another kind. This is challenging not only for how motor representations influence knowledge states but also, moving in the opposite direction, for how intentions influence motor representations (Butterfill and Sinigaglia 2014). And, more broadly, there appear to be related challenges for understanding how perceptual representations can influence knowledge states (e.g., Jackendoff 1996). Of course, the solutions to different interface problems may turn out to have little in common. But our guess is that good solutions will be reused. Linking different interface problems may therefore constrain the range of candidate solutions that need be considered. This is likely to leave several candidates in play, of course. After all, the interface problem is a question about how minds work and so not one that could be answered on the basis of narrowly theoretical considerations. We therefore need a second way of making progress:  we need to turn rough ideas into hypotheses and to test their predictions.

Corrado Sinigaglia & Stephen A. Butterfill

Notes

1 Aglioti et al. (2008: experiment 1) compared three groups: one group of players, one of coaches and sports journalists (‘expert watchers’) and one of novices. Each was shown clips of the early stages of a player taking a shot and tasked with predicting whether the shot would land in or out. The players made correct predictions significantly more often than the expert watchers did, and the expert watchers were no more likely to be correct than the novices (p. 1111; see figure 1). Modelling the impact of shorter and longer clips on correct responses suggests that only the players were able to make full use of bodily information, while expert watchers and novices may have relied more on information about ball trajectory (p. 1111).

2 Urgesi et al. 2012: experiment 2) studied three groups who all took part in 12 two-hour sessions over three weeks. The three groups’ sessions differed in how subjects were taught about floating serves: one group was shown videos and given verbal instructions (‘observation training’); another group was given training that required them to perform floating serves (‘execution training’); and the control group was given training that did not involve floating serves at all. Before the training, and then again after the training, each subject was tested on how accurately they were able to predict whether a shot would be in or out. Subjects made predictions after watching two kinds of clips. One kind of clip showed bodily movement only (stopping at just the point where the hand contacts the ball); the other kind of clip showed the ball’s trajectory (starting at just the point where the hand contacts the ball). Only execution training resulted in significantly more accurate predictions on the basis of bodily movements; and only observation training resulted in significantly more accurate predictions on the basis of ball trajectories (Urgesi et al. $2012^{\mathrm{BIB}-}062$ : see figure 3 on p. 533).
3 For an infant to track the goal of an action is for there to be a process in the infant such that how this process unfolds nonaccidentally depends, perhaps within limits, on which goal the action is directed to. We take no view on whether or not the infants have knowledge about the goals of actions. Research on their abilities is relevant given that, as we suppose, the goal-tracking processes infants manifest somehow matter for knowledge.
4 Note that our suggestion is not that infants cannot track the goals of observed actions they cannot perform. The key claim for us is that infants are like adults insofar as those more skilled in performing certain actions are also better at extracting information when observing them.
5 Infants' goal-tracking abities were measured using the much-replicated habituation paradigm introduced by Woodward (1998).
6 This is an instance of what is usually called the ^end-state comfort effect' (Rosenbaum et al. 1992, 1993). While it is important not to confate end-states with goals, in this instance there is a connection. Sensitivity to the end-state of the action implies sensitivity to the goal of the last part of the action (the placing of the book) since it is this goal that determines the end-state.
7 On what motor representations are and why they are necessary, key sources include Prinz (1990); Wolpert et al. (1995); Jeannerod (2006); Rizzolatti and Sinigaglia (2008); Rosenbaum (2010).
8 In Knoblich and Flach (2001), the subjects were informed about the identity of the agent observed; in Knoblich et al. (2002) they were not.
9 For example, Artiga (2014: 546) argues that the main motivation for embracing the Pushmi-Pullyu account is flawed’.
10 Our reply to the objection also indicates a way of defanging some of Millikan's (1995) original arguments for the existence of representations with both directions of fit.
11 For one state's infuence on another to be content-especting is for whether or how the first infuences the second to nonaccidentally reflect some relation between the two state’s contents.
12 Two outcomes match in a particular context just if,in that context, either the occurrence of the first outcome would normally constitute or cause, at least partially, the occurrence of the second outcome or vice versa.
13 Mylopoulos and Pacherie (2017) focus on how motor representations relate to intentions rather than to knowledge states; however, their view can be generalised in a natural way. The notion of an executable action concept may be related to Pavese’s (2015: 19) suggestion that ‘operational semantic values are kinds of practical senses’ (Pavese, personal communication): in both cases, a key idea is that entertaining propositions or possessing concepts concerning ways of acting enable agents to act in those ways.
14 Motor schema are internal models or stored representations that represent generic knowledge about a certain pattern of action … that is the organization and structure common to a set of motor acts’ (Mylopoulos and Pacherie 2017: 330).

References

Aglioti, S. M., Cesari, P., Romani, M., and Urgesi, C. (2008) “Action Anticipation and Motor Resonance in Elite Basketbal Players, Nature Neuroscience 11: 1109-16. https://doi.org/10.1038/nn.2182.
Ambrosini, E., Reddy, V., de Looper, A., Costantini, M., Lopez, B., and Sinigaglia, C. (2013) “Looking Ahead: Anticipatory Gaze and Motor Ability in Infancy," PLoS One 8: e67916. https://doi.orgy 10.1371/journal.pone.0067916.
Ambrosini, E., Sinigaglia, C., and Costantini, M. (2012) “Tie My Hands, Tie My Eyes,” Journal of Experimental Psychology: Human Perception and Performance 38: 263-6. https://doi.org/10.1037/a0026570.
Artiga, M. (2014) “Teleosemantics and Pushmi-Pullyu Representations,” Erkenntnis 79: 545-66. https: // doi.org/10.1007/s10670-013-9517-5.
Bakker, M., Sommerville,J.A., and Gredeback, G. (2015) Enhanced Neural Processing of Goal-Directd Actions Afer Active Training in 4Month-Old Infants, Journal of Cognitive Neuroscience 28: 47282. https://doi.org/10.1162/jocn_a_00909.
Blake, R., and Shifrar, M. (2007) Perception of Human Motion,” Anual Review of Psychology 58: 4773. https://doi.org/10.1146/annurev.psych.57.102904.190152.
Bortoletto, M., Mattingley, J. B., and Cunnington, R. (2011) “Action Intentions Modulate Visual Processing During Action Perception, Neuropsychologia 49: 2097-104. https: //doi.org/10.1016/j.neuropsycholo gia.2011.04.004.
Burnston, D. C. (2017) “Interface Problems in the Explanation of Action,” Philosophical Explorations 20: 242-58. https://doi.org/10.1080/13869795.2017.1312504.
Butterfill,S.A.,and Sinigaglia,C. (2014)“Intention and MotorRepresentation in PurposiveAction, Philosophy and Phenomenological Research 88: 119-45. https://doi.org/10.111/j.1933-1592.2012.00604.x.
Calvo-​Merino, B., Glaser, D. E., Grèzes, J., Passingham, R. E., and Haggard, P. (2005) “Action Observation and Acquired Motor Skills: An FMRI Study with Expert Dancers," Cerebral Cortex 15: 1243-9. htps: / doi.org/10.1093/cercor/bhi007.
Cannon, E. N., Woodward, A. L., Gredebäck, G., von Hofsten, C., and Turek, C. (2012) “Action Production Infuences 12-Month-Old Infants’ Attention to Others′ Actions,”’ Developmental Science 15: 35-42. https://doi.org/10.1111/j.1467-7687.2011.01095.x.
Casile, A., and Giese, M. A. (2006) “Nonvisual Motor Training Influences Biological Motion Perception,” Current Biology 16: 69-74. https://doi.org/10.1016/j.cub.2005.10.071.
Cattaneo, L., Caruana, F., Jezzini, A., and Rizzolatti, G. (2009) “Representation of Goal and Movements Without Overt Motor Behavior in the Human Motor Cortex: A Transcranial Magnetic Stimulation Study The Journal of Neuroscience 29: 11134-8. https://doi.org/10.1523/JNEUROSCI.2605-09.2009.
Cavallo, A., Becchio, C., Sartori, L., Bucchioni, G., and Castiello, U. (2011) “Grasping with Tools: Corticospinal Excitability Refects Observed Hand Movements”’ Cerebral Cortex 22: 710-16. http://doi.org/10.1093/cercor/bhr157.
Chapman, G., Cleese, J., Idle, E., Jones, T., Palin, M., and Gilliam, T. (1970) Monty Python’s Flying Circus, Series 2, London: British Broadcasting Corporation.
Cohen, R. G., and Rosenbaum, D. A. (2004) “Where Grasps Are Made Reveals How Grasps Are Planned: Generation and Recall of Motor Plans," Experimental Brain Research 157: 486-95.https://doi. org/10.1007/s00221-004-1862-9.
Costantini, M., Ambrosini, E., Cardellicchio, P., and Sinigaglia, C. (2014) “How Your Hand Drives My Eyes, Social Cognitive and Affetive Neuroscience 9: 705-11. https://doi.org/10.1093/scan/nst037.
D’Ausilio, A., Pulvermüller, F., Salmas, P., Bufalari, I., Begliomini, C., and Fadiga, L. (2009) “The j.cub.2009.01.017.
Della Sala, S., Marcheti, C., and Spinnler, H. (1991) “Right-Sided Anarchic (Alien) Hand: A Longitudinal. Study Neuropsychologia 29: 1113-27. https://doi.org/10.1016/0028-3932(91)90081-I1
Della Sala, S., Marchetti, C., and Spinnler, H. (1994) “The Anarchic Hand:  A Fronto-​Mesial Sign,” in F Boller and J. Grafman (eds.) Handbook of Neuropsychology Vol. 9, 233-255,Amsterdam: Elsevier.
Fadiga, L., Craighero, L., Buccino, G., and Rizzolatti, G. (2002) “Speech Listening Specifically Modulates the Excitability of Tongue Muscles: A TMS Study,”" European Journal of Neuroscience 15: 399-402.
Fadiga, L., Fogassi, L., Pavesi, G., and Rizzolatti, G. (1995) “Motor Facilitation During Action Observation: A Magnetic Stimulation Study” Journal of Neurophysiology 73: 260811.
Falck-Ytter, T., Gredeback, G., and von Hofsten, C. (2006)“Infants Predict Other People's Action Goals," Nature Neuroscience 9: 878-9.
Fogassi, L., Ferrari, P. F., Gesierich, B., Rozzi, S., Chersi, F., and Rizzolatti, G. (2005) “Parietal Lobe: From Action Organization to Intention Understanding"” Science 308: 662-7.
Fridland, E. (2016) “Skill and Motor Control:  Intelligence All the Way Down,” Philosophical Studies 174: 1539-60. https://doi.org/10.1007/s11098-016-0771-7.
Funk, M., Shiffrar, M., and Brugger, P. (2005) “Hand Movement Observation by Individuals Born Without Hands:  Phantom Limb Experience Constrains Visual Limb Perception,” Experimental Brain Research 164: 341-6. https://doi.org/10.1007/s00221-005-2255-4.
Gerson, S. A., and Woodward, A. L. (2014) “Learning from Their Own Actions: The Unique Effect of Producing Actions on Infants' Action Understanding," Child Development 85: 264-77. https://doi.org/ 10.1111/cdev.12115.
Haslinger, B., Erhard, P., Altenmiller, E., Schroeder, U., Boecker, H., and Ceballos-Baumann, A. O. (2005) “Transmodal Sensorimotor Networks During Action Observation in Professional Pianists,” Journal of Cognitive Neuroscience 17: 282-93. https://doi.org/10.1162/0898929053124893.
Jackendoff, R. (1996) “The Architecture of the Linguistic-Spatial Interface,” in P. Bloom et al. (eds.) Language and Space, 130,Cambridge, MA: The MIT Press.
Jeannerod, M. (2006) Motor Cognition: What Actions Tell the Self, Oxford: Oxford University Press.
Kanakogi, Y., and Itakura, S. (2011) “Developmental Correspondence Between Action Prediction and Motor Ability in Early Infancy, Nature Communications 2: 341. https://doi.org/10.1038/ncomms1342.
Kawato, M. (1999) “Internal Models for Motor Control and Trajectory Planning,” Current Opinion in Neurobiology 9: 718-27. https: //doi.org/10.1016/S0959-4388(99)00028-8.
Knoblich, G., and Flach, R. (2001) “Predicting the Effects of Actions:  Interactions of Perception and Action”" Psychological Science 12: 467-72. https://doi.org/10.1111/1467-9280.00387.
Knoblich, G., Seigerschmidt, E., Flach, R., and Prinz, W. (2002) “Authorship Effects in the Prediction of Handwriting Strokes: Evidence for Action Simulation During Action Perception,” The Quarterly Journal of Experimental Psychology Section A 55: 1027-46. https://doi.org/10.1080/02724980143000631.
Michael, J., Sandberg, K., Skewes, J., Wolf, T., Blicher, J., Overgaard, M., and Frith, C. D. (2014) “Continuous Theta-Burst Stimulation Demonstrates a Causal Role of Premotor Homunculus in Action Understanding Psychological Science 25: 963-72. https://doi.org/10.1177/0956797613520608.
Millikan, RG(1995)Pushmi-Pullyu Representations Philophical Perspetives 9: 185-200. https://i org/10.2307/2214217.
Mylopoulos, M., and Pacherie, E. (2017) “Intentions and Motor Representations: The Interface Challenge,” Review of Philosophy and Psychology 8: 317-36. https://doi.org/10.1007/s13164-016-0311-6.
Pavese, C. (2015) “Practical Senses” Philosopher's Imprint 15: 1-25.
Prinz, W. (1990) “A Common Coding Approach to Perception and Action,” in O. Neumann and W. Prinz (eds.) Relatinships Between Peretion and Action, 17201, Beln: Springr.
Rizzolatti, G., Camarda, R., Fogassi, L., Gentilucci, M., Luppino, G., and Matelli, M. (1988) “Functional Organization of Inferior Area 6 in the Macaque Monkey,” Experimental Brain Research 71: 491-507. https:/doi.org/10.1007/BF00248742.
Rizzolatti, G., Fogassi, L., and Gallese, V. (2001) “Neurophysiological Mechanisms Underlying the Understanding and Imitation of Action,”" Nature Reviews: Neuroscience 2: 661-70.
Rizzolatti, G., and Sinigaglia, C. (2008) Mirrors in the Brain: How Our Minds Share Actions, Emotions, and Experience, Oxford: Oxford University Press. - (2010) “The Functional Role of the Parieto-Frontal Mirror Circuit: Interpretations and Misinterpretations,' Nature Reviews: Neurosience 11: 264-74. htps://doi.org/10.1038/nrn2805. (2016)“The Mirror Mechanism: A Basic Principle of Brain Function, Nature Reviews Neuroscience 17: 757-65. https://doi.org/10.1038/nrn.2016.135.
Rosenbaum, D. A. (2010) Human Motor Control, 2nd ed., San Diego, CA: Academic Press.
Rosenbaum, D. A., Vaughan, J., Barnes, H. J., and Jorgensen, M. J. (1992) “Time Course of Movement Planning: Selection of Handgrips for Object Manipulation,” Journal of Experimental Psychology: Learning, Memory, and Cognition 18: 1058-73.https://doi.org/10.1037/0278-7393.18.5.1058..
Rosenbaum, D. A., Vaughan, J., Jorgensen, M. J., Barnes, H. J., and Stewart, E. (1993) “Plans for Object Manipulation,” in D. E. Meyer and S. Kornblum (eds.) Attention and Performance XIV:  Synergies in Experimental Psychology, Artifcial Inteligence, and Cognitive Neuroscience, 803-20, Cambridge, MA: The MIT Press.
Shea, N. (2018) Representation in Cognitive Science, Oxford: Oxford University Press.
Shiffrar, M. (2010) “People Watching: Visual, Motor, and Social Processes in the Perception of Human Movement, Wiley Interdisciplinary Reviews: Cognitive Science 2: 68-78. https://doi.org/10.1002/wcs.88.
Shiffrar, M., and Freyd,J.J. (1990) “Apparent Motion of the Human Body, Psychological Science 1: 257-64. https://doi.org/10.1111/j.1467-9280.1990.tb00210.x.
Shiffrar, M., and Heinen, T. (2011) “Athletic Ability Changes Action Perception: Embodiment in the Visual Perception of Human Movement," Zeitschrift fur Sportpsychologie 17: 1-13.
Sinigaglia, C., and Butterfill, S. A. (2015) “On a Puzzle About Relations Between Thought, Experience and the Motoric,’ Synthese 192: 1923-36. https://doi.org/10.1007/s11229-015-0672-x. - (2016) “Motor Representation in Goal Ascription,' in M. H. Fischer and Y. Coello (eds.) Conceptual and Interactive Embodiment: Foundations of Embodied Cognition, Volume 2, 149-64, London and New York: Routledge.
Sommerville, J. A., Hildebrand, E. A., and Crane, C. C. (2008) “Experience Matters: The Impact of Doing Versus Watching on Infants Subsequent Perception of Tool-Use Events” Developmental Psychology 44: 1249-56. https://doi.org/10.1037/a0012296.
Sommerville, J. A., Woodward, A. L., and Needham, A. (2005) “Action Experience Alters 3- Month-Old Infants Perception of Others Actions,” Cognition 96: B1-B11. https:/doi.org/16/ j.cognition.2004.07.004.
Tessitore, G., Sinigaglia, C., and Prevete, R. (2013) “Hierarchical and Multiple Hand Action Representation Using Temporal Postural Synergies,” Experimental Brain Research 225: 11-36. https://doi.org/10.1007/ s00221-012-3344-9.
Umiltà, M. A., Escola, L., Intskirveli, I., Grammont, F., Rochat, M., Caruana, F., et  al. (2008) “When Pliers Become Fingers in the Monkey Motor System,” Proceedings of the National Academy of Sciences 105: 2209-13. https://doi.org/10.1073/pnas.0705985105.
Urgesi, C., Candidi, M., Ionta, S., and Aglioti, S. M. (2007) “Representation of Body Identity and Body Actions in Extrastriate Body Area and Ventral Premotor Cortex,” Nature Neuroscience 10: 30-1. https: // doi.org/10.1038/nn1815
Urgesi, C., Savonitto, M. M., Fabbro, E, and Aglioti, S. M. (2012) “Long- and Short-Term Plastic Modeling of Action Prediction Abilities in Volleyball,' Psychological Research 76: 542-60. https:/ /doi.org/10.1007/ s00426-011-0383-y.
van Kemenade, B. M., Muggleton, N., Walsh, V., and Saygin, A. P. (2012) “Effects of TMS over Premotor and Superior Temporal Cortices on Biological Motion Perception,” Journal of Cognitive Neuroscience 24: 896-904. https://doi.org/10.1162/jocn_a_00194.
von Hofsten, C. (1991) “Structuring of Early Reaching Movements:  A Longitudinal Study,” Journal of Motor Behavior 23: 280-92. https://doi.org/10.1080/00222895.1991.9942039.
Witherington, D. C., von Hofsten, C., Rosander, K., Robinette, A., Woollacott, M. H., and Bertenthal, B. 1. (2002) “The Development of Anticipatory Postural Adjustments in Infancy” Infancy 3: 495-517. https://doi.0rg/10.1207/S15327078IN0304_05.
Wolpert, D. M., Ghahramani, Z., and Jordan, M. (1995) “An Internal Model for Sensorimotor Integration,” Science 269: 1880-2. https://doi.org/10.1126/science.7569931.
Woodward, A. L. (1998) “Infants Selectively Encode the Goal Object of an Actor’s Reach,” Cognition 69: 1-34.