…All They Can Do Is Watch

The idea for this presentation came from a video clip that resurfaced earlier this year of Prince, on the music news site Musicradar, being interviewed by the Belgian band Soulwax in 2000. They had asked Prince how does he "keep things interesting for himself" with how rapidly recording technology is advancing. They give the example that some innovative recording techniques he pioneered in the 80s could now easily be recreated by "any kid with a PC in a bedroom"; but Prince interjects…

📺 Prince interview with Soulwax, 2000.

“There's just one catch — the kid with the PC in the bedroom can't play his instrument. He can make some sounds, and then you can get a vast amount of repetition, but if they didn't take the time to learn their instrument and their scales and to actually speak the language of music then they can't play with someone like Sheryl Crow — all they can do is watch.”

To be clear, Prince is not some sort of techno-musical luddite. He is one of the most prominent examples of an artist who treated the recording studio not as just a utility to record live analog performances, but as an instrument in itself. He made extensive use of the then revolutionary LM-1 drum machine, multi-track recording, varispeed vocals, and was even one of the first pioneers of digital distribution in the 90s. Which I think, situated Prince to also be keenly aware of digital music’s shortcomings for developing musicians.

To me, in his response, he is alluding that:

  1. Technology can substitute musical skill for technological skill.
  2. Technology can turn "doing" into "perceiving".
  3. Approaching music from a purely "cognitivist" perspective doesn't capture the depth of the thinking music demands.

Part 1: Content and Technological Knowledge

Pedagogical Content Knowledge

The first thing Prince seems to be hinting at when he talks about “learning your instrument and scales” is the difference between content knowledge and technological knowledge. One way we can look to understand this is through the TPACK framework developed by Punya Mishra and Matthew J. Koehler in 2006.

The TPACK Framework is built from Lee Shulman’s idea from 1986 of “Pedagogical Content Knowledge”.

Content Knowledge
Knowledge of the subject matter.
Pedagogical Knowledge
Knowledge of the practices of teaching the subject matter.

Shulman theorized that teachers need to know more than the content and general-purpose pedagogical skills, but that they need to know how content and pedagogical knowledge intersect

  • I.E. how to teach a specific subject or what kind of pedagogical skills/knowledge are particularly effective for teaching a certain discipline.
  • I.E. Being a good English teacher requires a different skill set than being a good Math teacher.
© 2012 by tpack.org

Technological Pedagogical Knowledge

Mishra and Koehler add Technological Knowledge as a third domain, in turn creating three more subdomains:

Technological Knowledge
Knowledge of the technology itself.
Technological Content Knowledge (TCK)
Knowledge of the technology used to teach
Technological Pedagogical Knowledge (TPK)
Knowledge of how to use the technology to support teaching.
Technological Pedagogical Conent Knowledge (TPACK)
Knowledge of how technology, pedagogy, and content intersect.

While this tool is meant as a planning tool for teachers, it is also worthwhile to examine the balance of content vs technological knowledge we are asking of our students in a given task.

© 2012 by tpack.org

Content-Technological Knowledge Continnuum

While digital tools can be exceptionally helpful, there is a danger that we can be teaching too much "technological" knowledge instead of "content" knowledge. When we design learning experiences for our students, we need to be careful that the technology chosen to help teach the content knowledge, doesn't replace the content knowledge.

One key question to consider: What percentage of the students alloted time are they going to need to spend learning the technology, and what percentage will the spend learning the content? (This will also be different depending on the student. Each student brings a different level of prior knowledge and students can generalise and skill transfer at different levels).

How do you know you might be teaching more technological knowledge than content knowledge?

  1. Technology adds a barrier rather than removes a barrier.

    • Eg. Using a tablet and stylus to mark up a score with colour but it would have just been simpler on paper (literally).
  2. The technology skills aren’t transferable.

    • Eg. I know how to use Google Docs because I know Microsoft Word i.e. I know how to use a word processor.
    • Eg. I know how to use Logic Pro because I know GarageBand i.e. I can use a DAW.
  3. Explanation and instruction is mostly procedural instead of conceptual.

    • Eg. “Click this, then drag that…”
  4. Student questions are about the technology rather than content.

    • Eg. “How do you x and y?” rather than “Why would I x rather than y?”
  5. Students can't do a similar task or meet the same learning goal without tech or without a specific piece of tech.

    • Eg. I can’t compose something without notation software that will prevent me from making “grammatical” mistakes

Part 2: Action-Sound Separation & The Musicking Quadrant

Another major point Prince makes about the utility of digital music is that it can turn formerly active participants into passive observers. One way we can understand how digital technology changes our students relationship to the way the make music is Alexander Refsum Jensenius’ Musicking Quadrant from his 2022 book Sound Actions.

(Jensenius, 2022)

These roles should also not be thought of as mutually exclusive. Musical activity often blurs the boundaries between these distinctions, and someone can take on more than one role simultaneously. Because of this, musicking is sometimes easier to understand and reason with if we thinking about the type of musical engagement associated with each role.

(Jensenius, 2022)

The problem with digital music instruments or interfaces is that it increases the action-sound separation and moves musical engagement that would formerly be real-time, out of time.

(Jensenius, 2022)

Jensenius defines a sequence of Action-sound separation for Acoustic Instruments:

Embodied
Sound is produced within the body (voice, clapping).
Tactile
Direct contact with the resonating object (guitar, flute, hand drum).
Tool-based
A simple tool is used on the resonating object (sticks, a bow).
Mechanical
A mechanism sits between the action and the sound, up to instruments that play themselves (Piano).
Automatic
No physical contact between the body and object. (Music box, player piano)
Conceptual
Imaginary instruments that exist only in the mind.
(Jensenius, 2022)

The Action-Sound Separation sequence looks slightly different for electro acoustic instruments:

Embodied
The body is directly involved in the sound production or modification, such as in the theremin or cracklebox.
Analog
There are continuous (electric) relationships between action and sound, like in most analog synthesizers.
Digital
There are discrete (sampling-based) relationships between a physical input device and controller.
Virtual
There are multilayer relationships between action and sound, such as in apps and web-based instruments.
Automatic
The instrument can play on its own.
Conceptual
There is no physical instrument to interact with; the instrument and its sound production are only happening within your (embodied) mind.

How do we incorporate technology into the music education that doesn’t turn everyone into perceivers? Use low action-sound separation technology tools and controllers.

Choosing the right controller:

  • Keyboard-like controllers (typical MIDI piano keyboards)
  • Mixer-like controllers (knobs-and-sliders, like a mixer)
  • Pad-like controllers (grids of touch-sensitive buttons)
  • Acoustic instrument–like controllers (drum kits and wind controllers)
(Jensenius, 2022)

Low action-sound separation instruments and technology allow students to more easily move between all four means of music making, both in and out of real-time. As educators we need to carefully choose the interfaces our students use for musicking in order to allow them to engage in all four segments of the musicking quadrant.

Part 3: 4E Cognition

Speaking the Language of Music

Cognitvism
Thinking happens in the mind. Your brain is a computer. Your thoughts can be reduced to symbols. The body is just a means of executing the mental models and understanding you’ve built of the world in your head. The world is just input for your brain to process. Musical understanding is built through purely mental representations and understanding of metre, harmony, form, etc.
4E Cognition
A loosely organized set of approaches to cognitive science that challenge the traditional notions of cognitivism and the separation of the mind from the body and environment.
Embodied Cognition
Your body is part of your thinking.
Embedded Cognition
Your environment is part of your thinking.
Enactive Cognition
Your actions and reactions are part of your thinking.
Extended Cognition
Your tools are part of your thinking.

Embodied Cognition

In Susan Goldin-Meadow’s book Hearing Gesture (2003), she presents evidence about how body movement improves thinking. In one experiment, it was found that children who gesture while explaining math problems were much more likely to learn. Furthermore, when students who weren’t gesturing to explain math problems were then asked to start gesturing, their performance improved.

Embedded Cognition

One way to understand the concept of embedded cognition is from David Kirsch and Paul Maglio’s 1994 paper On Distinguishing Epistemic from Pragmatic Action. They demonstrated that advanced Tetris players will often rotate a piece more than is needed to place it even though each rotation doesn’t bring the piece closer to its final position. On the surface this looks inefficient, but rotating it on screen and seeing the result is faster and more accurate than mentally rotating the shape and comparing it to the gaps below because they are using the environment as part of the problem-solving process.

(Kirsh, D. & Maglio, P. 1994)

Enactive Cognition

One demonstration for how Enactive Cognition works is an experiment that was done on kittens in 1963. Two groups of kittens were raised in the dark and exposed to light only on a carousel apparatus. One group walked actively, the other was carried passively in a carousel. Only the actively moving kitten developed normal visually guided behavior. This study strongly suggests that passive exposure to visual stimulus is not enough to “learn” but we need to be receiving input from not just our environment but the reactions the environment has to our actions.

(Held, R., & Hein, A. 1963)

Extended Cognition

One way to understand how extended cognition works the thought experiment about “Otto’s Notebook” from Andy Clark and David Chalmers’ paper The Extended Mind. Igna and Otto both want to visit the Museum of Modern Art in New York. Inga recalls that the museum is on 53rd Street and walks there. Otto has Alzheimer’s disease and relies on a notebook that he carries everywhere and writes useful information in. He consults it, reads that the museum is on 53rd Street, and walks there. The notebook is part of Otto’s cognitive system, and his mind extends into it. The notebook isn’t just any external object.

It qualifies because it is:

  • automatically invoked i.e. used without thinking about using it;
  • automatically endorsed i.e the information is trusted without scrutiny, easily accessible; and,
  • information that was consciously endorsed in the past, which is why it’s in the notebook

Otto’s saxophone – would an instrument or other musical interface not also apply?

4E Cognition and Music

Embodied Cognition
Eg. You can’t create a mental representation of what is like to play the trumpet; it is understood through the breath, embouchure, and posture.
Embedded Cognition
Eg. Singing in a choir requires you to adjust to your physical location in a group, the singers around you, the room’s acoustics, an audience, etc.
Enactive Cognition
Eg. A saxophonist improvising a jazz solo is creating musical meaning from moment to moment based on the actions and reactions to other musicians.
Extended Cognition
Eg. The pianist using the layout of a keyboard to reason about harmony.

Technology Integration and Cognitive Offloading

ProcessAnalog TaskDigital TaskCognitive Offloading
EmbodiedComposing a piece of music with an instrument.Composing a piece of music with notation software.Thinking is no longer shaped by how the body interprets or feels music (breathing, technical demands, etc).
EmbeddedListening to music at a concert hall.Listening to music through headphones.Thinking is no longer shaped by the environment (how sound reacts to the space).
EnactivePracticing with an ensemble.Practicing with a backing track.Thinking is no longer shaped by real time reaction to musical stimuli.
ExtendedRealizing a lead sheet at the piano.Realizing a lead sheet with playback software.Thinking is no longer shaped by the instrument.

Managing the tradeoffs

Embodied
If the learning goal is to develop a theoretical understanding of melody and harmony, using notation software doesn't interfere with this goal; however, an embodied understanding of harmony will not develop.
Embedded
If the learning goal is to practice fingerings in real-time, playing to a backing track doesn't interfere with this goal; however, an embedded understanding of playing with an ensemble will not develop.
Enactive
If the goal is to play in tune, playing to an autoassesment tool doesn't interfere with this goal; however, an enactive understanding of tuning will not develop.
Extended
If the goal is the theoretically analyze a piece of music, then using playback software does not interfere with this goal; however, an extended understanding of the piece will not develop.

Taxonomies of Embodied Learning

In 2018, Alexander Skulmowski and Günter Daniel Rey introduced a taxonomy to categorize the different levels of embodied cognition in a given learning task based on the levels of bodily engagement and whether or not this was by design or merely incidental.

(Skulmowski & Rey, 2018)

Taxonomy of 4E Cognition

We could take this one step further and use this taxonomy, applying the two axes “Engagement” and “Integration” across each of the 4E domains.

Part 4: Classroom Applications

Applying the Frameworks to Common Music-Tech Integrations

  1. Notation Software
  2. DAW
  3. Recording and Playback
  4. Practice and Assessment Platforms
  5. Tuners, Metronomes, and Drones
  6. Backing Tracks
  7. Ear Training Apps
  8. MIDI Instruments
  9. Tablet Screen Instruments

Bibliography

Bogunović, B., Timmers, R., & Nikolić, S. (Eds.). (2024). Psychological Perspectives on Musical Experiences and Skills: Research in the Western Balkans and Western Europe (1st ed.). Open Book Publishers. https://doi.org/10.11647/OBP.0389

Clark, A., & Chalmers, D. (1998). The Extended Mind. Analysis, 58(1), 7–19. https://www.jstor.org/stable/3328150

Dylan van der Schyff, A. S. (n.d.). Musical Bodies, Musical Minds. In Urn:isbn:9780262370332. The MIT Press.

Goldin-Meadow, S. (1999). The role of gesture in communication and thinking. Trends in Cognitive Sciences, 3(11), 419–429. https://doi.org/10.1016/s1364-6613(99)01397-2

Goldin-Meadow, S. (2003). Hearing Gesture: How Our Hands Help Us Think. Harvard University Press. https://doi.org/10.2307/j.ctv1w9m9ds

Held, R., & Hein, A. (1963). Movement-produced stimulation in the development of visually guided behavior. Journal of Comparative and Physiological Psychology, 56(5), 872–876. https://doi.org/10.1037/h0040546

Jensenius, A. R. (2022). Sound actions: Conceptualizing musical instruments. The MIT Press.

Kirsh, D., & Maglio, P. (1994). On Distinguishing Epistemic from Pragmatic Action.

Mishra, P., & Koehler, M. J. (2006). Technological Pedagogical Content Knowledge: A Framework for Teacher Knowledge. Teachers College Record, 108(6), 1017–1054. https://doi.org/10.1111/j.1467-9620.2006.00684.x

Rogerson, B. (2026, January 1). “There’s just one catch. The kid with the PC in the bedroom can’t play his instrument”: Back in 1999, Prince offered his opinion on the new generation of DAW-based musicians and producers. MusicRadar. https://www.musicradar.com/artists/theres-just-one-catch-the-kid-with-the-pc-in-the-bedroom-cant-play-his-instrument-back-in-1999-prince-offered-his-opinion-on-the-new-generation-of-daw-based-musicians-and-producers-bestof25

Shulman, L. S. (1986). Those Who Understand: Knowledge Growth in Teaching. Educational Researcher, 15(2), 4–14. https://doi.org/10.3102/0013189X015002004

Skulmowski, A., & Rey, G. D. (2018). Embodied learning: Introducing a taxonomy based on bodily engagement and task integration. Cognitive Research: Principles and Implications, 3(1), 6. https://doi.org/10.1186/s41235-018-0092-9