Video Analysis of VJ (Video Jockey) in Live Electronic Music Performance using Cadoz and Wanderley's Gesture-Music Taxonomy
DOI:
https://doi.org/10.24191/idealogy.v11i2.1016Abstract
This study examines how VJ (Video Jockey) hand gestures function as an expressive control layer over audio-reactive visuals in live electronic music performance. As live electronic music performance continues to face criticism for its lack of visible performer-audience connection, VJ gesture practice offers a meaningful but academically undocumented form of physical expressivity. This research aims to (1) analyse common hand gestures used by VJs in live electronic music performances through video analysis, and (2) map identified gestures to corresponding audio-reactive visual design parameters. A qualitative video analysis approach was applied to eight (8) VJ performance videos selected from publicly available online sources. The analytical framework combines Cadoz and Wanderley's Gesture-Music taxonomy (2000) classifying gestures as Ergotic, Semiotic, or Epistemic with an inductively developed Gesture-to-Visual Mapping Taxonomy that documents the physical actions and visual parameters involved. Results indicate that Ergotic gestures dominate VJ performance practice, with fader sweeps, knob turns, button presses, and touchpad drags serving as the primary gesture vocabulary for controlling visual parameters including intensity, colour, scene transitions, and effects. This study contributes the first systematic gesture-to-visual mapping framework for VJ practice, providing a foundation for the design of gesture-driven audiovisual performance systems such as HOLOCHORD.
Keywords: VJ Performance, Hand Gesture Analysis, Audio-Reactive Visuals, Gesture-to-Visual Mapping, Live Electronic Music Performance
References
Cadoz, C., & Wanderley, M. M. (2000). Gesture-music. In M. M. Wanderley & M. Battier (Eds.),
Trends in gestural control of music (pp. 71–94). IRCAM – Centre Pompidou.
Carvalho, R. (2025). Sound-to-visual mapping strategies in audiovisual performance and installations.
In L. Correia Castilho, R. Sampaio Dias, L. Rocha, & A. de Sousa Dias (Eds.), Perspectives on
music, sound and musicology II. Springer. https://doi.org/10.1007/978-3-031-67503-4
21
Davidson, J. W. (1993). Visual perception of performance manner in the movements of solo
musicians. Psychology of Music, 21(2), 103–113. https://doi.org/10.1177/030573569302100201
Dourish, P. (2001). Where the action is: The foundations of embodied interaction. MIT Press.
https://dl.acm.org/doi/10.5555/513034
Erdmann, M., von Berg, M., & Steffens, J. (2025). Development and evaluation of a mixed reality
music visualization for a live performance based on music information retrieval. Frontiers in
Virtual Reality, 6. https://doi.org/10.3389/frvir.2025.1552321
Gurevich, M., & Fyans, A. C. (2011). Digital musical interactions: Performer-system relationships
and their perception by spectators. Organised Sound, 16(2), 166–175.
https://doi.org/10.1017/S1355771811000112
Jordà, S., Geiger, G., Alonso, M., & Kaltenbrunner, M. (2007). The reacTable: Exploring the synergy
between live music performance and tabletop tangible interfaces. Proceedings of the 1st
International Conference on Tangible and Embedded Interaction (TEI '07). ACM.
https://doi.org/10.1145/1226969.1226998
Khazaei, M., Bahrani, A., & Tzanetakis, G. (2025). A real-time gesture-based control framework.
arXiv. https://arxiv.org/abs/2504.19460
Krippendorff, K. (1980). Content analysis: An introduction to its methodology. Sage Publications.
Miranda, E. R., & Wanderley, M. M. (2006). New digital musical instruments: Control and interaction
beyond the keyboard. A-R Editions.
Nasruddin, A. N., Ghazali, M., & Din, S. C. (2024). Exploring the usage of motion graphic elements
in projection mapping video to promote agrotourism in Malaysia. Idealogy Journal, 9(2).
https://doi.org/10.24191/idealogy.v9i2.564
Noor Hishamuddin, N. K. B., Anuar, I. M. H., Che Din, S., & Lazim, N. A. M. (2024). An integration
of 3D design in Malaysia digital tourism. Idealogy Journal, 9(1).
https://doi.org/10.24191/idealogy.v9i1.492
Poupyrev, I., Lyons, M., Fels, S., & Blaine, T. (2001). New interfaces for musical expression.
Proceedings of the CHI Conference Extended Abstracts. ACM.
https://doi.org/10.1145/634067.634168
Wanderley, M. M., & Battier, M. (2000). Instrumental gestural mapping strategies as expressivity
determinants in computer music performance. In Trends in gestural control of music. IRCAM –
Centre Pompidou.
Wanderley, M. M., & Depalle, P. (2004). Gestural control of sound synthesis. Proceedings of the
IEEE, 92(4), 632–644. https://doi.org/10.1109/JPROC.2004.825882
Zhang, F., Bazarevsky, V ., Vakunov, A., Tkachenka, A., Sung, G., Chang, C., & Grundmann, M.
(2020). MediaPipe hands: On-device real-time hand tracking. arXiv.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 UiTM Press
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
UiTM Press (the Publisher) has agreed to publish the undersigned author’s paper in Idealogy Journal. The agreement is contingent upon the fulfilment of a number of requirements listed below.
1. The undersigned author warrants that the paper entitled below is original, that it is not in any way libellous or unlawful in Malaysia, that it does not infringe any copyright or other proprietary right. The undersigned hereby represents and warrants that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. The undersigned represents that he/she has the power and authority to sign and execute this agreement.
2. The undersigned author warrants that the paper entitled below has not been published elsewhere, and also it will not be submitted anywhere else for publication prior to acceptance/rejection by this Journal.
3. By submitting the paper entitled below, the undersigned author agrees to transfer the rights to publish and distribute the paper in an international e-journal (entitled above) to Publisher.
4. The undersigned author agrees to make a reasonable effort to conform to Publisher's submission guidelines and to liaise with the editor to ensure that the requirements of these guidelines are met to a reasonable degree.
5. The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all coauthors. This agreement is to be signed by at least one of the authors who has obtained the assent of the co-author(s) where applicable. After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted.
