https://the-smooth.space/blog/app

---

---

WHAT DIALOGUE IS MADE OF: AN ARTISTIC EXPLORATION OF LIGETI’S ‘DIALOGO’ USING A CUSTOM SEGMENTATION ALGORITHM

‘In flowing speech, the point is the symbol of the interruption of
non-being (negative element) and at the same time it is a bridge from
one being to another (positive element). 

Kandinsky

Authors:
ADRIÁN ARTACHO
HANNE PILGRIM
MARIE-LOUISE WUNDLING
LEONHARD HORSTMEYER

Abstract

Communication in interdisciplinary artistic practices involving music, movement, and multimedia often presents challenges due to the interplay of diverse modalities. In our exploration of György Ligeti’s ‘Dialogo’, we investigated the structure of the piece, focusing on the perception and temporal alignment of its segments. Starting with a plastique animé interpretation of Ligeti’s work, we conducted a series of artistic experiments that altered the piece’s structure in different ways, focusing particularly on how these changes influenced the perception of the individual segments. In order to address the complex interplay between score, sound and movement, we developed SALTA, an intuitive application designed to generate automatic segmentations from performance data. Conceived specifically to assist artistic practice, SALTA’s analysis and graphical representations of the piece served as an external reference, fostering intersubjective reflection on the relationship between sound and movement. These visualizations illuminated the ways in which these modalities interact and influence one another, ultimately shaping the performance as a cohesive whole. Our findings demonstrated how the interplay between music and movement can be systematically mapped, analysed, and critically examined using technological tools like SALTA, offering fresh perspectives on their dynamic relationship. These insights not only enriched our understanding of Ligeti’s ‘Dialogo’ but also underscored the potential of such tools to enhance communication and drive innovation in interdisciplinary artistic collaborations.

1. Initial question(s)

Ligeti’s ‘Dialogo’, the first movement of his Cello Sonata, is often described as a dialogue between two contrasting voices. Using Plastique Animée as a starting point for their artistic investigation, cellist Marie Louise Wundling and eurhythmics practitioner Hanne Pilgrim set themselves the goal of bringing this ‘Dialogo’ to life. But is this interaction genuinely dialogic, or does it merely give the illusion of a conversation? These fundamental questioning of the piece’s premise set an artistic quest in motion which culminated in a series of structured experiments and the development of technological tools to address the questions that arose along the process.

An interesting initial insight was that, during the rehearsal period, both the cellist and the eurhythmics practitioner reported feeling as though their respective contributions formed parallel monologues rather than a cohesive exchange. This led us to consider what constitutes an effective artistic dialogue. The musical surface of the piece, as expressed in Ligeti’s score, became a field of debate for our artistic discussions. Its internal structure—composed of segments and the transitions between them—served as the key reference points we returned to repeatedly in our conversations. It became clear to us that, in the process of discussing the piece, these sometimes loosely defined handles (E.g. ‘starting sequence’ or ‘climax’), originally tied to instrumental actions (e.g., ‘switch to arco’, ‘diminuendo’) in Ligeti’s score, took on new meanings when applied to a movement progression or to certain actions the performers needed to perform in sync. To begin with, the segments of the piece became contested entities. Reaching a necessary agreement on the intersubjective structure of the performance—along with the timing of transitions and synchronization points—required reconciling the initial individual perspectives of the performers involved.

2. Artistic Experimentation

Beyond the mere exchange of information —and appropriate degrees of entropy—, we sought to identify other crucial qualities and how they might be embedded in both the fixed musical text that serves as reference (Ligeti’s score) and the composed choreography (based upon Marie Louise’s interpretation of Ligeti’s score) that arose from it. 

Henceforth, in order to address the initially formulated question about the dialogic nature of our performance, we took an approach involving a series of structured experiments designed to better understand and refine the interaction between the eurhythmics practitioner and the cellist. We began by segmenting the choreography to align with the musical phrase arcs in Ligeti’s ‘Dialogo’.

2.1 Segment-Based Experiments

We conducted four primary experiments focusing on different ways of manipulating the segments in the piece:

– Experiment A: We reversed the order of the choreography segments while leaving the musical sequence unchanged, creating a deliberate disruption that emphasized the intricate interdependence between the musical and choreographic structures. This approach introduced a layer of tension and unpredictability, challenging both performers to adapt dynamically to the altered framework. It also highlighted the extent to which the two elements relied on each other for cohesion, revealing new perspectives on their interaction. 

– Experiment B: We performed the choreography with a deliberate time delay with respect to the cellist, creating a temporal dissonance (while preserving the original sequence of segments). This intentional offset introduced a sense of asynchrony between the movement and the music, challenging initial notions of alignment and synchronicity. By maintaining the segment sequence, the experiment retained a structural familiarity, allowing us to focus on how shifts in timing altered the perceived relationship between the two elements. This approach aimed to examine the nuances of dialogue in performance, exploring how temporal deviations influence the sense of connection, tension, and interplay between choreography and music.

– Experiment C: We introduced improvisation into the performance, allowing the movement to respond spontaneously to the evolving musical phrases. This approach brought an element of unpredictability and real-time interaction, fostering a dynamic exchange between the dancer and the musician. By removing predetermined constraints, the improvisational framework emphasized the dancer’s ability to interpret and react to the subtleties of the music in the moment. This not only heightened the sense of immediacy and responsiveness but also revealed new layers of communication and mutual influence. The interplay between the cellist’s sound and the improvised choreography created a blurring of boundaries between segments, with some being abruptly cut short while others were significantly expanded based on the dynamics of the movement layer. This interaction offered valuable insights into how the different modalities—sound and movement—actively shaped and conditioned the temporal structure of the performance. 

– Experiment D: Using a single musical segment as a constant framework, we systematically varied Laban’s ‘movement factors’ to explore their individual and collective impact on the performance. By manipulating these parameters independently, we were able to isolate and analyze how each quality influenced the interaction and dialogue between the music and dance. For instance, changes in spatial focus or range altered the visual and emotional resonance of the movement, while shifts in temporal pacing or rhythm affected the perceived synchronization and tension with the music. Adjustments in weight or dynamics highlighted contrasts in energy and effort, and variations in flow—whether bound or free—transformed the continuity and fluidity of the performance. This methodical approach provided deeper insights into the nuanced ways in which movement parameters shape and are shaped by the accompanying musical structure, offering a richer understanding of their role in the creation of a cohesive and expressive performance dialogue. 

We systematically varied the parameters of transitions—space, time, weight/dynamics, and flow—just as we had done with segments. This allowed us to explore how changes in transition intention affect the continuity and quality of the performance. Ligeti’s ‘Dialogo’ features transitions such as silences or “arpeggi glissandi,” which we compared to discourse particles in spoken language, like pauses or filler sounds. 

Our findings suggested that these transitions are not merely structural markers but deeply influence the perception of dialogue. For example, a silence or pause can either create a space for reflection and openness or disrupt the flow, depending on the performer’s intention. Similarly, the “arpeggi glissandi” may symbolise the pauses that often occur in speech to allow for thought in conversation.

One key observation was that individual decisions made during transitions determined the overall shape of the interaction in profound ways. It suggested that transitions play a crucial role in shaping the overall dialogic quality of the performance. These experiments revealed a full spectrum of interactions, from cohesive dialogue to fragmented monologue-like exchanges, and underscored the complex relationship between music and movement. 

3. ‘Segments’ and ‘transitions’

At the outset of this investigation, we regarded the musical sections in the score as fixed, perhaps influenced by the strong musical backgrounds of both Marie-Louise and Hanne. However, as the investigation progressed, we became increasingly aware of the ephemeral and fluid nature of these “segments” within our performance. They existed in a constant tension between the rigid structure of the written score and the organic, playful nature of live performance. This dynamic tension was particularly shaped by the interplay of sound and movement modalities, which often caused certain segments of the piece to transcend their notated origins. At this point, we shifted our focus to the transitions that were underpinning the segment structure. These liminal moments between segments seemed especially rich in creative potential, offering a fertile ground for exploration and improvisation. This realization carried along the next question: how aligned were our intersubjective perceptions of these transitions, and what role did these moments of shift play in shaping the overall quality of communication between the performers? 

Exploring this question became central to our investigation, as transitions appeared to serve as critical junctures where the interplay between sound and movement could either strengthen or disrupt the dialogue. Understanding how each performer experienced and interpreted these transitions offered deeper insights into the nuances of their collaborative process and the impact of these moments on the cohesion and expressiveness of the performance.

4. A segment-identifying algorithm

The automatic identification of segments within a choreomusical performance presents a significant challenge due to the inherent complexity and subjectivity of the task. Unlike domains where a clear and universally agreed-upon segmentation exists, choreomusical performances lack a singular intersubjective ‘correct’ segmentation. Instead, the perception of segments often varies significantly between individuals, influenced by factors such as their artistic training, cultural background, and the specific context in which the performance is experienced.

Moreover, the data available for algorithmic segmentation is inherently constrained by the capture methods used, such as video or audio recordings. These modalities, while rich, often fail to encapsulate the full spectrum of nuances present in a live choreomusical performance, including subtle interactions between movement and sound, spatial dynamics, and the performative intent of the artists. While certain segmentation approaches have demonstrated success in narrowly defined contexts, the choreomusical domain introduces a much higher degree of variability. The interplay between movement and music is not only highly context-dependent but also subject to creative and interpretative choices that defy rigid categorization. This variability poses a unique challenge for the development of robust segmentation algorithms capable of generalizing across diverse choreomusical expressions.

The approach we ultimately adopted builds on an existing segmentation method proposed by Krüger et al., adapting it to suit the unique requirements of choreomusical analysis. This method generates “windows of possibility” for transitions, applying these to both audio features and body landmark positions. By leveraging this dual-modal data, the system creates a probability density plot, which serves as the foundation for estimating segmentation points. This adaptation not only enhances the interpretability of transitions in the performance but also allows for a more nuanced integration of audio and movement data, aligning with the inherent complexity of choreomusical interactions.

Fig. 6: Comparison of probability density plots and automatic segmentations generated using different modality combinations (audio features and body landmarks):

A | All modalities processed together

B | Only Audio features

C | Body landmarks from the cellist

D | Body landmarks from the cellist & audio features combined

E | Body landmarks from the eurhythmics practitioner

F | Body landmarks from the eurhythmics practitioner & audio features combined

G | Body landmarks from the cellist & from the eurhythmics practitioner combined

5. Development of ‘SALTA’

Insights gained from artistic experimentation inspired the creation of a tool designed to perform real-time automatic segmentation, even when working with limited or incomplete data. This tool was specifically developed to support research by processing multimodal inputs, including video, audio, and simple sensor data. Unlike sophisticated motion capture systems that rely on complex and expensive equipment, this tool prioritizes accessibility and ease of use. Its intuitive design ensures that it can be utilized by a broad spectrum of practitioners, from researchers and artists to educators and performers, making it a versatile resource for exploring choreomusical interactions.

To accommodate the varying relevance of different modalities depending on the type of performance, and to enable artistic researchers to experiment freely with the tool, we incorporated a user interface that allows users to adjust the relative weight of each modality. This flexible approach emphasizes that the automatically generated segmentation is merely an external perspective on the performance, rather than an objective or intersubjective truth. By interacting with these controls, performers and researchers can explore various modality combinations, gaining valuable insights into the interplay between audio, movement, and other data streams.

6. Conclusion
Our research into Ligeti’s Dialogo highlights the intricate dialogue between music and movement, as well as the numerous factors that shape and influence this interaction. By experimenting with the piece’s segments and transitions, we gained valuable insights into what constitutes a meaningful exchange between these elements. The development of a practical segmentation tool has further enabled this exploration, supporting both precise analysis and creative experimentation.

A broader takeaway from our study is that communication in interdisciplinary artistic practices—particularly those involving music, dance, and multimedia elements—faces significant challenges due to the distinct technical vocabularies and methodologies of each discipline. Despite this inherent subjectivity, our experience suggests that temporal segments of a performance provide a crucial shared reference point. These segments function as “handles” or a form of lingua franca, facilitating clearer communication and collaboration across disciplines.

Finally, delving into our intersubjective perceptions of a piece’s components can uncover deeper dynamics, shedding light on the fundamental structures underlying interdisciplinary works. Future research could expand on these findings by exploring the broader implications of segmentation and transition analysis in a wider range of interdisciplinary practices, offering new perspectives on how such works are understood and created.

References

Agostini, V., Balestra, G., & Knaflitz, M. (2013). Segmentation and classification of gait cycles. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 22(5), 946-952.

Charnavel, I. (2019). Steps toward a universal grammar of dance: local grouping structure in basic human movement perception. Frontiers in Psychology, 10, 1364.

Giannakopoulos, T. (2015). pyaudioanalysis: An open-source python library for audio signal analysis. PloS one, 10(12), e0144610.

Jiang, L., Zhang, R., Tao, L., Zhang, Y., Zhou, Y., & Cai, Q. (2023). Neural mechanisms of musical structure and tonality, and the effect of musicianship. Frontiers in Psychology, 14, 1092051.

Kandinsky, W. (1926) Punkt und Linie zu Fläche: Beitrag zur Analyse der malerischen Elemente. Vol. 9. Albert Langen.

Kennedy, A. (Ed.). (2010). Bewegtes Wissen: Laban/Bartenieff-Bewegungsstudien verstehen und erleben. Teil 1: Bewegtes Wissen-eine praktische Theorie. Logos-Verlag.

Krüger, B., Vögele, A., Willig, T., Yao, A., Klein, R., & Weber, A. (2016). Efficient unsupervised temporal segmentation of motion data. IEEE Transactions on Multimedia, 19(4), 797-812.

Kuehne, H., Gall, J., & Serre, T. (2015). Cooking in the kitchen: Recognizing and Segmenting Human Activities in Videos. arXiv preprint arXiv:1508.06073.

Ligeti, G. (1948/1953). Cello Sonata. Edition Schott.

Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., … & Grundmann, M. (2019). MediaPipe: A framework for building perception pipelines. 

McMains, J., & Thomas, B. (2013). Translating from pitch to plié: Music theory for dance scholars and close movement analysis for music scholars. Dance Chronicle, 36(2), 196-217.

Moulieras, S., & Pachet, F. (2016). Maximum entropy models for generation of expressive music.

Sukhova, N. V. (2012). Hesitation: Thoughts, Pauses, Gestures and Speech.

---