the ʃep of wərdz to kəm

the shape of words to come

 

I believe there is a missing transitive verb to describe our relationship to the audiovisual, which impacts on our engagement with it. We will always say we watch film, rarely will we say we listen to film, but actually we are doing something more than both: We _______ film?

Recent texts[1] tend to describe this engagement as bi-modal or multimodal, however I find these definitions immediately divisive, suggesting a partisan equality to the senses when faced with stimuli. I believe however, there is a holistic attentive process that is more than the sum of these modalities, taking into account cognitive, contextual and conditioned responses, which, in the light of our advances in understanding in this field, necessitate a new expression to define our investigative connection to the audiovisual.

To thoroughly proffer a term for this action is beyond the scope of this essay, however I will set out some of the philosophical, musicological, linguistic and artistic positions I have thus far examined this problem from and explain how these positions fed into my compositions in the Shape of words to Come and for Lichtspiel: Opus I. These works themselves were compositional investigations into the thesis of this essay, and are the beginning of my research in to this subject, which I hope will be a platform for further study into this audiovisual/linguistic field.

 

From the Ontological to the Cognitive

Our relationship to an object is in some way negotiated by the language we use to mediate it. This has proven to be an exciting field of linguistic philosophical discourse, with David Donaldson’s work being particularly relevant to my own studies[2], but it has been the Philosophical Investigations of Ludwig Wittgenstein that have had the greatest impact on the work I have made in the Shape of Words to Come.

The more narrowly we examine actual language, the sharper becomes the conflict between it and our requirement. (For the crystalline purity of logic was, of course, not a result of investigation: it was a requirement.) The conflict becomes intolerable; the requirement is now in danger of becoming empty.[3]

His reductive language games, his skepticism of the meaning in words, and his awareness of language’s reliance on context and convention led me to reevaluate my endeavor to name a thing, to create a word as it were. What is a word? Can I appropriate a preexisting collection of phonetics and attach new meaning to it? Does naming a thing bring it into existence?

Another question that arises is one of Linguistic Relativity. That my understanding of the audiovisual action is not yet formed as an English word does not mean that the same intention it is not commonly referred to in another language. This conversation is often played out in discussions on colour theory[4], but has strong implications in relation to my argument towards the necessity of a new way of discussing audiovisual analysis. The Sapir-Whorf hypothesis[5], the “strong” version, outlines the author’s belief that a person’s language determines how they understand the world. I would attach my argument to the “weak” version of this theory, whereby our language merely influences how we perceive the world. Whilst the “strong” Sapir-Whorf hypothesis is often contested, the “weak” version has been investigated thoroughly by Lera Boroditsky and her team at Stanford University, whom have gathered empirical evidence to prove the affect of language on our cognitive process[6].

I think these findings are of importance to the audiovisual world, especially concerning collaborative composition at the point of conception. One of the difficulties faced as a film composer working under a director is translating their musical ideas into reality with only a potted lexicon through which to negotiate. This often leads to abstracted meanings being used to communicate ideas, e.g. “can you make it more yellow?” Here the word yellow takes on a hypostatic function[7], what are the musical properties of yellow, what is yellow music? I traced this thought through similar audiovisual descriptors and settled on a form common to both visual and musical aesthetics: Shape.

 

The Shape of Words to Come

I would then propose ‘shape’ to be a linguistically hypostatic word. Interested in examining the collaborative possibilities of these types of words, I invited three visual artists, Lauren Printy Currie, Tom Varley and Tom Walker, in whose work I had seen parallels to my own, to respond to a brief I set them. I asked each artist to create abstract filmed images that respond to a geometric shape; a circle, a parallelogram and a triangle. The three shapes were chosen as I felt drawn towards their intrinsic musicality, but had any other three shapes been chosen the investigative process would have been the same. Having discussed each other’s interests in responding to the brief, Printy Currie, Varley and Walker then returned a few months later with a selection of video files and written statements that explained their own artistic investigations into each shape. I then processed these filmed images in a similar way to how I would improvised passages from an instrumental player: Cutting and editing them at the same time as the musical structures I composed. This soundfilm technique is part of my compositional practice, one I have been developing whereby I try to mould the narrative of my music through composing the visual material at the same time as the sonic. As such it is neither music video[8], nor music for film[9], but an attempt at symbiotic audiovisual experience, in which the visual is not just musical but an intrinsic timbre within the music itself[10]. This allows me an auteurial control over the work, giving me the opportunity to fully investigate points of conceptual synchrony even within a collaborative piece.

 

Circle

Lauren’s visual response to the circle was one of, symbolism and gender politics, of “shaping of perception, visibility, and space”[11]. My own approach was one of visualisation also, albeit from two perspectives. The first being no matter where you join the circle you are presented with two completely symmetrical paths that lead back to your point of departure, with the gradient of this movement changing at a constant rate all the way around the circumference. The main harmony of this movement is built around this notion, using the circle of fifths, with one path leading up in fifths and the other down in fourths; leaving from G, passing each other at C sharp (the opposite side of the circle, the tritone) and meeting again at the G.

The second deals more closely with the idea of the splitting of the path. The rhythms that grow at the beginning of the movement, and return throughout, come from pushing and pulling the harmonic wave of a G and an F sharp together through the quartertone and beyond. That flexing dissonance results in pulses, altering the perceived metre of the music. These pulses return at other points in the rhythm of the melody, as each line is time- stretched separately from the next, throwing up alternating congruent beats.

The visual tracks deal with these notions as well as introducing the author into the path decision, voiced both visually in her reflection on the screen and in the vocal harmony through the physical circling of lips. The text and corresponding imagery has been cut from Lauren’s previous work and reinterpreted as a narration of this voyage into and around the circle. The dance of the harmonic structure, the physicality of movement between two notes becomes marked in the visual. Ultimately this passage approaches the tangential decision of movement around the circle at both a macro and micro level.

 

Parallelogram

Tom Walker responded to the parallelogram. In his text he outlines his interpretation; “Firstly as the shape that railing bars contort to when filmed at high speed and secondly as something close to but slightly removed from conventional notions of ‘the frame’. The parallelogram is a shape that implies an active skewing of things.” [12] Conversations on “the frame” were ones that featured heavily during our conceptual discussions. With our central investigation being into audiovisual shape, it was hard to ignore that it would invariably be presented through a rectangular frame. As Brandon LeBelle notes however, sound cannot be framed; it “disrespects borders, thereby making explicit the intensity of territory”[13], an interesting question then of our understanding of the “shape” of sound. Tom filmed his visual response on the iPhone, “to utilise the natural functions and limitations of the iphone as transformative agents. Everything from the iphone’s poor resolution and low frame rate (30 fps) to its automatic focusing tool provide the means of abstraction.”[14]

I then, also constructed the music for this passage almost entirely on the iPhone, using Yamaha’s Tenori-on and Brian Eno’s Bloom visual music apps to build a score that echoed Tom’s investigations into commonplace technological mediation of our environment. The Tenori-on track is doubled but time stretched in its repetition to approach the ‘active skewing’ of the frame. This track, along with others, is gradually fed into a degenerating bit-crusher, which degrades the resolution of the sonic imagery in much the same way as the iPhone’s camera does through the visual track. Repetition and chance are part of the nature of the Tenori-on and seem a fitting aural counterpart to Tom’s visual shapes. A fugato passage is also introduced that approaches the intrusive sonic properties of phone interference, an almost ever-present danger in our current audiovisual engagements but also addressing the apparent parallelogram shape that has been present in Western music for centuries, particularly evident in 16^thC fugal tradition.

 

Triangle

Tom Varley’s reply to my initial question was to give me, wholesale, the footage from a landmark psychological experiment[15]. An Experimental Study of Apparent Behaviour (1944) by Fritz Heider and Marianne Simmel was a study in the field of interpersonal perception. Both Varley and I share a fascination with the power of psychological projection, semiotics and the relationship between the logographic (and the phonetics they represent) and the pictographic.

The International Phonetic Alphabet has influenced a lot of the work I have made over the last year or so, as a language within a language, of latent symbols within symbols. The rhythm of this movement is made of reconstituting the fragmented phonics of the shapes of investigation back into their English arrangement. Music for film relies heavily on a similar psychological projection and attachment as pictography does. As far back as 1947 Adorno and Eisler were lambasting composers’ reliance on convention when scoring for film[16]. Here, I’ve tried to draw attention to our expectation of form. It has become so commonplace to stress on screen violence with contrapuntal, “beautiful” music that it is no longer shocking. The original force that was generated between the audiovisual discord dissipates over time much like the meaning of a word can, driving us to reevaluate its function. I also brought to the fore my interest in the way vowel sounds interact when sung in harmony, an interest that first appeared when I was signing in harmony with an Australian friend. I found I was forced to change my vowel sounds to match his to create a true harmony. Here I have tried to create clashes through the vowels alone, singing a five-part harmony that moves through the vowels whilst always presenting the five together at the same time; a small violence within a larger harmony, echoing the results from the initial experiment.

 

St. Cecilia’s Hall.

In presenting this triptych as one work I aimed to underline the parallels between the each shape, and as such each audiovisual discussion that was taking place between them. However, I felt it necessary to physically coalesce these works on to a larger screen in a space that highlighted the musicality, and not the visual force. In her book Sounding the Gallery, Holly Rogers states,

“To situate the genre

etymologically within the visual arts is responsible for cordoning it off from its dual past and audiovisual present. What is needed is a reappraisal of the connotations of the term “art” in order to establish the audio as equal in importance as the visual.”[17]

In my work I seek to respond to this call for this reappraisal, to address the musical potential of the visual in video. Showing this work in a cinema space would focus the audience too much on the visual material and detract from the visual as a musical function in much the same way as a gallery space would. As such, a concert space was needed that underlined the musicality of the works and engaged the performative aspect of the visual. After a few conversations I came across St. Cecilia’s Hall in Edinburgh, the oldest music hall in Scotland and one that, as an oval space, demands that the screen stand apart from the wall on the stage, presenting it as one uncomfortable shape within another.

[18]

St. Cecilia’s has also added a historical weight to the soundfilm process’ place within visual-music history. In the film world, this history has a fairly firm date of conception within the absolute film movement, in particular the first performance of Lichtspiel: Opus I by Walther Ruttman in Munich, 1921. This early abstract cinema work was created as visual music, as painted shapes interacted through musical form; counterpoint, harmony, rhythms. Originally performed with a commissioned score by Max Butting, Ruttman himself played cello at its premier.

 

Lichtspiel: Opus I

Sound film is still waiting to emerge – because the mere reproduction of the spoken word, with the conventions of silent film still in force, would have no artistic merit and preclude any further development. We don’t want that any more than I would have wanted mere ‘mechanic photography’ in the cityscape of Berlin… My efforts were geared towards developing a complete art form. Now that we regard the basic issues of silent film as solved, it would be wrong to assume that embedding acoustic events would suffice to make sound film. Such a synchronism of sound and image would be untenable and might even achieve the opposite of the task at hand: instead of heightening the illusion, weaken it. Sound film (…) sets its own technical and artistic terms, still largely unknown in their implications and potentialities.

Walther Ruttman, Compilation of Excerpts from Interviews and Articles 1927 -1937[19]

Walther Ruttman’s Lichtspiel: Opus I seemed like the most natural counterpart to the Shape of Words to Come. His comments on our understanding of film as music read as relevant now as they did when he first wrote them, with the complex potential of sound film a long way from being fully explored. As an ardent modernist I am certain he would have welcomed my new score for his film, though likely not the showing of his work almost 100 years on from its creation.

With this in mind I was keen to create a contemporary score that was far removed from Max Butting’s original serialist one. However, understanding how and where Ruttman intended the music to synchronise with the visual was vital to composing a true sound film. The Goethe Institute provided me with a copy of the string quintet score onto which Ruttman had hand drawn markings of synchrony. I used this to inform the music I wrote, however I resisted imitating the form of Butting’s, instead approaching the soundtrack with an alternative vigor and expressiveness that was afforded to me through modern recording techniques.

I decided to write for a contemporary quintet, with each instrument representing a different period in visual music’s disparate history. The iPhone, in particular the Tenori-on, Bloom and modAxis apps that were used, are indicative of the post-millennial understanding of visual music, whereby form is created through direct visual engagement with the instrument. The digitally modeled ARP Odyssey Mk II synth is analogous with the mid 1970s-80s explorations into digital audiovisual synchrony. The guitar makes reference to the Visual Music movement a la the Joshua Light Show, whilst the cello quotes the original butting score. The percussion tracks look to our much older audiovisual experiences, in live performance and dance.

Unlike my soundfilm works however, my music for Lichtspiel: Opus I is reactive to the visual, rather than fundamentally intrinsic to it. Still, I feel it important to exhibit the parallels between historic and contemporary visual music, to highlight how little progress we have made in 100 years without an appropriate language with which to investigate it.

 

Hypostatic Shaping

To conclude then, the Shape of Words to Come and my score for Lichtspiel: Opus I are musical investigations into both the role and potential of video as musical composition, as well as being practical discussions on the language of audiovisual collaboration. A lot more research and artistic inquiry is needed before I can put forth a suggestion for this missing transitive verb. However, I hope that these works might add to the discussion on the necessity for such a word to exist at all.

When critically interacting with the audiovisual our cognitive process is being wrought by the language between us and the object, our previous cultural experience with the form affects our interpretation of what is unfolding, and we are being influenced by our conditioned responses to the space it is shown in.[20]

Not just watching then, not just listening: Shaping?

 

Bibliography

Adorno, T and Eisler, H (1947) Composing for the Films (New York: Oxford University Press).

Bigu, Dragos. (2012) Whorf’s conception of the relation between language and thought: A critical examination. Euromentor Journal. Vol 3.

Boroditsky, Lera. (2011) How Language Shapes Thought: The languages we speak affect our perceptions of the world. Scientific American.

Crutchfield, Stuart A. (2011) The phenomenal unity of perceptual experience. PhD thesis. University of Glasgow: UK.

Davidson, Donald. (1974) On the Very Idea of a Conceptual Scheme. Proceedings and Addresses of the American Philosophical Association, Vol. 47.

Heifetz, Jeanne. (1994) When Blue Meant Yellow. (New York: Henry Holt and Co).

LaBelle, Brandon (2006) Background Noise: Perspectives on Sound Art (New York: Continuum).

Peirce, C.S. (1931–1935), Collected Papers of Charles Sanders Peirce, vols. 1–6, Charles Hartshorne and Paul Weiss, eds., vols. 7–8 (1958), Arthur W. Burks, ed., (Cambridge, MA: Harvard University Press).

Rogers, Holly (2013) Sounding the Gallery (New York: Oxford University Press).

Vernallis, Carol (2013) Music Video’s Second Aesthetic. Taken from The Oxford Handbook of New Audiovisual Aesthetics (New York: Oxford University Press).

Wittgenstein, Ludwig. (1953) Philosophical Investigations. 3^rd Ed. (Oxford: Blackwell Publishing Ltd).

 

DVD

Berlin, die Sinfonie der Großstadt & Melodie der Welt DVD (Liner notes) Ralph Schermbach and Stefan Drössler. Munich, Germany: Filmmuseum München, Arte, Bundesarchiv-Filmarchiv and Goethe-Institu. (2008)

 

Private Publications

(2014) The Shape of Words to Come Programme Note, August 11^th 2014, Edinburgh. (A physical copy of the programme note should be included alongside this essay as well as in PDF form with the video files.)

---