The Game Controller: Mediating Between Virtual and Physical Worlds

Introduction

What do the buttons on a game console controller represent for you? Each button most likely has different functions which vary among each game’s unique game mechanics. Take for example, the “B” button. In Splatoon, pressing it results in your in-game avatar to jump, in Hades, it makes you dash forward, whereas in Animal Crossing: New Horizons, holding it down makes your avatar run. Our real-life act of pressing a button translates into another action occurring in-game, as depicted on the screen. One could even say that the buttons having diverse effects in each game’s unique mechanics, representing their own specific set of rules, relies on their own system of signs.  

Avatar in Splatoon 3 displaying “jump” mechanic – Footage by Christine Choi
Character, Zagreus, in Hades displaying “dash” mechanic – Footage by Christine Choi
Avatar in Animal Crossing: New Horizons displaying “run” mechanic – Footage by Christine Choi

This fascination surrounding such concepts is precisely why I chose a game controller as my evocative object. Although there are so many different variations of a game controller, I am going to use the Nintendo Switch Pro Controller as an example as it is the controller I was mediated by the most and is also one of the more “standard” modern designs when it comes to game controllers (the existence of a joystick, four buttons with letters/symbols, L and R buttons, etc.). By analyzing the game controller, I will highlight the ways that it mediates between us, the player, and the virtual world that the game exists in. 

The Controller and the Player

As mentioned before, the game controller has the unique ability to mediate between us and the virtual world displayed within the hardware (whether that is the console or the PC). When it comes to “playing” a game via a controller, there is a unique set of feedback that is inputted and outputted mediating between our physical corporeal bodies and the incorporeal in-game virtual bodies. 

You would first take in the world through senses, like sight and hearing. Certain in-game index, symbols, and icons may evoke feelings of fear, especially if it had informed the player of it causing harm to the avatar in the past. Others may evoke curiosity, enticing the player to explore more of the game and the “rules” of this digital world. Once you have cognitively processed that, your instincts—shaped by in-game and real-world experiences—would inform you to react. You would react by pressing buttons or rolling the joystick to the direction you want it to. We know, or at the very least expect, that the controller has received input when we receive the tactile feedback of the button being pushed down then springing back as we release it from pressing down on it. Then you would see the fruition of your act of button-pressing/joystick pushing by seeing the pixels on-screen change to indicate movement/change within this virtual world. 

Feedback of input and output between virtual and physical worlds – Diagram drawn by Christine Choi

The feelings evoked from the virtual information would translate in the grip of our controller; dodging enemies evoking another emotion of relief and safety, the achievement leading to satiate more of our curiosity, all driving our progression of the game. Thus, the controller is the mediating object for the player’s input and the software (which would be the game). Without the game, there is nothing for it to control and without the player input, there is nothing being controlled. 

Exerting Control over in-game “bodies”

Much of what is being said about the “control” over avatars/characters is correlated to what is said about the “body” as a medium in Critical Terms for Media Studies. After all, the controller could be seen as an extension of our own body, which extends into what is being “embodied” in-game. Wegenstein, too, utilizes psychoanalytic theories of how video games allow us to play the role of the “other”, a virtual embodiment that differs from the embodiment of ourselves. She quotes Slavoj Zizek on “a figure capable of taking on, or projecting itself into, many simultaneous roles” (28). The concept of roles that we project onto is correspondent with how the controller that mediates and perpetuates this “ego” that we project onto, making the body of the playable character another medium. 

Thus far, I have only discussed characters and avatars that have an anthropomorphic body, which is easy to visualize as we easily project our human bodies onto these characters. But what about games with no “avatar” or humanistic representation of our own bodies? I would argue that there is still a “body” or “vessel” in which we, in our physical and corporeal forms, exert control over digitally. Take Tetris for example, the falling blocks would be the body that we project ourselves onto. As the blocks fall, we move with it left or right via the joystick or D-pad. 

Game Controller as a “Black Box”

Even the most avid gamers most likely do not know the internal computational and mechanical workings of what occurs in between the space and time in which we press the button and watch the game do its magic. The game controller generally works under the “black box” heuristics (a cybernetics theory coined by Norbert Wiener), in which the processes of the input from our button to the output in the game is shrouded in mystery for the average user (Wiener, page xi). However, I would argue that this knowledge we lack of our game controller’s internal workings is precisely the tool we use to immerse ourselves in the virtual space of a video game. What we do know as gamers is that eventually, its mechanics is burned into the memory of our bodies through “muscle memory” as the controls become second nature to us, thus “mediating” our physical bodies in the real world and the incorporeal bodies that exist in the virtual space of a video game.

Citations

Wegenstein, Bernadette. “Body.” Critical Terms for Media Studies, University of Chicago Press, Chicago, Illinois, 2010, pp. 19–34.

Wiener, Norbert, et al. Cybernetics: Or, Control and Communication in the Animal and the Machine. MIT Press, 2019.

Images and footage were all taken by Christine Choi

8 thoughts on “The Game Controller: Mediating Between Virtual and Physical Worlds”

  1. Hi Christine!
    I really appreciated this post! I greatly enjoyed your demonstrations and diagrams! I found it interesting to think about how a controller can mediate our “physical bodies” and “virtual bodies”. This made me think about embodiment and how, through our virtual character, we can be an extension of ourselves. I liked how you pointed out that “embodiment” does not depend on the character having humanistic traits. Even in games where there are no characters, we still reflect ourselves into “gameplay” as we have agency over what is happening. I think it’s interesting to deep dive into how we reflect our “personas” into our virtual selves. Are we presenting our same selves in this virtual world, or are we taking liberties? Are we creating a version of ourselves that is representative of us? Is this virtual world a chance for many to be their true selves?

    1. Hi, Ami! Thanks for the comment!

      It’s interesting that you brought this up because I would definitely say that we take more liberties when it comes to the virtual representation of ourselves. We especially see this in chatrooms or Mass Multiplayer Online games (MMOs). It’s even brought up in the “Body” chapter or our Critical Terms for Media Studies book where the author brings up World of Warcraft as an example for roleplaying in mass multiplayer spaces in page 28. She even quotes Sherry Turkle on how such spaces provide opportunities for minorities are able to express themselves since it frees them from the societal pressures that comes with being one.

      It also ties a lot into the book I’m reading for another class on game studies, titled “Of Floating Isles” where the author describes their own experiences with taking on a “role” or character that they play as when it comes to roleplaying and interacting with strangers.

      Thanks for the thought-provoking comment!

  2. I really like your object of choice, I also feel a lot of memories and connections to specific game controllers, like the old Wii nunchuks. I have a lot of memories associated with the movements and buttons those controllers use. As you noted in your blog, the B button on a controller can represent different uses for any different game. Personally, my first thought was B is ‘jump’, which is interesting considering Splatoon may be one of my most played games, which as you mentioned uses B for jumping. Perhaps overtime, I associated Splatoon’s jump button with the B button itself. In fact, in other games I tend to change my controls to set B to ‘jump’, otherwise it feels wrong. Love this blog 10/10 !

    1. Hi Maxine, thanks for your insightful comment!

      It’s pretty interesting that you change the settings of the game, I would say this could even be a way of “standardizing” and adapting the existing system of signs into your own system when it comes to games at your own convenience.

      (Also same with Splatoon 3, I fear I have 600+ hours on it…

      P.S. come to UBC e-sports Splatoon events twin…….)

  3. Hi Christine! I really enjoyed reading this, your discussion of the controller as a medium between our physical and virtual selves was so clear and engaging. I especially liked how you connected embodiment theory to non-human avatars like in Tetris, that perspective really broadened how I think about “bodies” in games. The “black box” idea was also fascinating — it made me reflect on how we rarely think about the technology itself once we’ve internalized the controls through muscle memory.

    1. Thanks, Christina! I appreciate your comment. It is indeed interesting to think about how once technology becomes second-nature to us, it’s no longer foreign and perhaps that makes it lose its uniqueness. I believe that this makes us take a lot of technology that we use for granted. Like our phone, for example, is a great example of technology that we take for granted. We never realize how much of an important object of mediation it is for us and the rest of society until we lose it. We end up losing a lot of our connections to friends, family, and people across the globe without it.

  4. Hi Christine! This was a wonderful read and a very interesting way to expand on the game controller as your evocative object! The beginning portion of your piece reminded me a lot of Vilem Flusser’s understanding of technical images in contemporary life, and how we are programmed to press buttons in certain ways to interact with systems and objects. I also really liked how you made a diagram that mirrors Ingold’s triangle between person, instrument/object, and medium. Your application of other readings to your own example goes to show your strong understanding of the concepts we’ve discussed throughout the year so far!

    1. Hi, Ela! Thanks for your comment!

      I’ve never read Vilem Flusser’s work before, I would definitely have to check that out! It’s really interesting that you made that connection because I do think video game consoles are designed in a special way that dictates this sort of interaction with systems (both in system of signs and in game mechanics that the games themselves implement). I also didn’t realize it was so similar to Ingold’s diagram, but you make a great point and it is indeed connected to the sort of arguments that I’m making.

Comments are closed.