sound - digital filemaking

sound

silence is not nothing. it comprises thousands of tiny, inconsequential sounds fused into a barely audible medley. this aural environment forms the physical underpinning of our experience on earth. it gives us the sense of being in the midst of a living world. yet, it is always space specific and it evolves through time (the sounds at 6 am are not those we hear at 6 pm). this is not the case for silence in film though. “room tone”, as it is called, is not just the absence of dialogue, ticking clock, footsteps, the rustling of clothes, etc. it also strives to extirpate from the environment anything that could possibly reveal it, say, as a set occupied by a crew of twenty as opposed to an authentic locale inhabited by the two protagonists. this silence is thus a counterfeit. because of this, room tones are largely interchangeable. so what we take to be indicative of specific surroundings can in fact easily have been borrowed from another habitat or film.

although we don’t generally pay attention to the matter, the audio track actually comes first when developing the future film. look at any script: what we will eventually hear monopolizes the writing. yet this dominance is undermined from the start by a format that rules out thinking about the rest of the audio: ambience, off-screen sounds, music, etc. quickly too, directors focus their attention elsewhere: the actors, the locations, the light, the mood, etc. in their view, there will be plenty of time to deal with the sound track later on. so, whereas production designers and directors of photography are consulted early on, sound designers, to their dismay, are called in only after the fact. instead of being able to contribute to the thinking of the film prior to it being shot, they are reduced to fasten the proper sounds onto gunshots, speeding cars, explosions, etc.

but what if we take sound seriously? instead of dishing out images from the start, cheapening their magic, explore the possibility of more complex relationships between the audio and the picture tracks. how many images do we really need in a film (letter to jane)?

the fact that audio tracks are made up of the same ingredients and positioned in the same pecking order today as they were in 1930 should alarm us. at the top of the pyramid we find the dialogue which must be crystal clear and in sync. conventionally that track alone is responsible for telling us the story. it is followed by the foleying of all the sounds connected to the action on the screen. ambient and background sounds are next. these particular tracks bring a sense of realism to the world we watch. lastly some source or score music is added as needed, generally to specify the mood or ante up the action.

today we take for granted that a film needs no accompaniment of any kind. but that was not always so. the benshi in some countries and live musicians practically everywhere were once responding to the action on the screen from within the theater itself, playing up the events, contributing to the viewers’ experience of the images. when the talkies took over however, the system mandated that all the audio had to originate from the screen. the theater was thus refashioned as a mere container, a vault within which one could observe the show but not to contribute to it. yet, a disparity immediately emerged between what was seen and what was heard. the problem, i believe, arose from the need for the spoken words to reach the last row in the auditorium. not only that, it was also decided that the protagonists had to be heard consistently well whether shown up close or far away (rick altman). two distinct problems emerged as a result. first, there is a clash between the visual seesaw on the screen (from long shot to close-up etc.) and the constant level of the dialogue track. the cadence is not the same. second, even though the principal audio speakers are located behind the screen, the dialogue doesn’t seem to originate from the protagonists. the volume that reaches us clashes with their physical location far away from us. it is too clear. it is too big. in fact the protagonists sound like they dwell in our own world, that they are close to us, somewhere in the theater. the spectators’ visual and auditory operations are therefore in conflict. on the one hand, our gaze transports us psychologically all the way out there where the story is taking place. on the other hand, the dialogue cocoons us in our own immediate environment.

why aren’t we bothered by this inconsistency? children are mesmerized by punch and judy shows even though the puppets obviously don’t speak. what happens is that in their mind they reconcile the voices they hear with the action they see. as long as the voices relate to it, the grafting will occur naturally (there is a delightful example of this phenomenon in the 400 blows). during the silent era, spectators could easily link the intertitles onto the following scene. even today, millions of movie spectators have no problem reconciling the dubbing of the original performers by actors speaking the local language. in all these cases, viewers effortlessly fuse in their head what they see and what they hear. this trompe l’oreille phenomemon is far from universal though: footsteps out of sync are simply unbearable.

so the sync we expect when someone speaks may not be crucial after all. haven’t italian films, martial arts movies, and problems with skype technology in news reports accustomed us to accept a gap between images and the accompanying dialogue? hasn’t steven soderbergh shown us in the limey that one can take great liberties with sync and get away with it? how about a film where the dialogue between people is slightly out of sync except for the moment when the protagonists fall in love and words wondrously come out of their mouths in perfect sync?

let’s go further: why not take advantage of the mismatch between pictures and sound to revive the theater as an active participant in the storytelling? why not launch for instance a second narrative located squarely in the spectator’s space? a few words at first could be exchanged by characters yet unrelated to those on the screen. these fragments could progressively develop into a more conventional dialogue. needless to say the link between the two stories would have to be orchestrated judiciously with the personages on the screen taking a break when something of importance related to those near us needs to be paid full attention to, and vice versa (the kind of balance so successfully achieved between different screens in time code). at one point, as i imagine it, the audio characters would take over the screen while those who used to be there would now carry on as mere voices near us. one could refer to this kind of film as a split track movie.

the production sound mixer and the boom operator are skilled workers, technicians who are counted on (1) to capture the dialogue as clearly as possible whether on the set or on location and (2) to make sure no hint of life beyond the actors’ exchange of lines mars the recording. hence the track that emerges makes it sound like the protagonists were the last human beings left on earth after a cataclysm of some kind. although these jobs are tremendously important, they are not expected to contribute anything creative to the film. in return, these specialists couldn’t care less whether the lines they are paid to record are actually any good.

alternative models do exist. in his films, frederick wiseman has often chosen to do the audio himself. being able to see things from a wider perspective, he could guide the camera operator toward something more interesting suddenly happening elsewhere. with only a third person to take care of the cables and the rest of the equipment, wiseman ended up with a perfect crew. all films should be made this way! other great documentary teams where the sound operator is as important as the cinematographer include the maysles brothers and the joan churchill/alan barker team. what does it all mean? recruit your sound mixer from the documentary field and give him/her a voice in the shooting process. stop thinking of the person as a mere technician: he or she should be your creative partner.

typically it is in post that one slowly rebuilds, one sound at a time, a plausible full aural environment. yet we tend to hear the same sounds time and time again: the pipes in a wall, the drip from the faucet, the clock ticking on the mantel, the wind chimes on the porch, the sprinklers in the garden, the cicadas in the fields, the crows flying over, and of course the police sirens, the screeching tires, etc.

one way to avoid this predicament is to start paying attention to the surrounding sounds in everyday life. sit in your living room, close your eyes, and listen. do you hear the footsteps in the hallway? is the person wearing flip flops? is it a man or a woman? ah, a wheezing… it must be the guy in 302. a new sound emerges from the kitchen: is it the hum from the refrigerator? is some lumber being dropped in the dumpster outside? then do the same outdoors: sit on a park bench, close your eyes and start identifying what you hear, where the sound comes from, and how far is it from you. vary the location each day. soon enough you’ll feel like a child, excited at discovering the immensity and the richness of life all around you.

hasn’t our aural experience of daily life changed radically since the thirties? aren’t helicopters constantly hovering above us (at least in los angeles)? don’t commercial vehicles beep when they back-up? don’t buses and trucks produce air escaping sounds that weren’t there years ago? what about the loud blowers used by gardeners? or the beeping that regulates pedestrian crossing at traffic lights? or the infomercials that arise from pumps at gas stations?

sound designers are not unlike historians of the annales school. their interest is not so much the protagonists and their actions (the “great men” of fiction) than the living world beyond them. this is critical for a period drama. what were the sounds heard in london during the victorian age? more commonly, in contemporary films, they like to investigate the sounds connected to a location as well as those happening off-screen. for instance a meow will make us think there is a cat hiding somewhere in the room. approaching footsteps cue us that someone is about to knock on the door. furthermore, murch, burtt and thom—sound designers par excellence—like to delay our identification of off-screen sounds. in the godfather, to take a single example, murch makes a magnificent use of a shrieking sound just before michael shoots sollozzo. nothing in the restaurant explains where that sound comes from. murch however plays it safe by introducing the sound of a rumbling train a number of times before the final scraping of the wheels against the rails. even if we didn’t pay full attention to the earlier cues, we probably have registered subconsciously the presence of elevated trains in the vicinity of the restaurant. all in all, murch not only brought in an imaginary train but he also used it dramatically to heighten our experience of the killing that changes michael’s life.

but what if we hear sounds we cannot identify? i’m not talking here of what happens when several sounds are mixed into one in order to amplify the impact of the original generator (e.g. mixing the revving of a car engine into a lion’s roar). although these composite sounds cannot actually be heard in real life, insofar as viewers are concerned they pass for the real thing. what i’m thinking of is more use of unnatural, fabricated, synthetic sounds. presently, when used in horror films for instance, these sounds tend to be grating or dissonant. this does not always have to be the case. in the same way cubism made us think differently about painting, a comprehensive exploration of this category of sounds could possibly open a dimension of cinema we are not presently attuned to.

the sound accompanying images can be deceptive at times. in point blank, lee marvin walks briskly in a long cavernous hallway. the sound of his footsteps resonates throughout the shot. it continues however over shots of his wife dressing herself and having her hair done in a salon. and it persists even though marvin is now driving a car and again as he sits still in the car, waiting for his wife to come back home. it may have taken a bit of time but eventually we realize that what we took as the sound of his footsteps was actually an expression of inner rage directed at the woman he believed had been unfaithful to him.

there are times when picture and audio are meant to collide in eisensteinian fashion, producing an epiphany of sort in our mind. in 21-87, none of the sounds we hear hails from the pictures we see. put another way, the audio decontextualizes the original newsreel situation: models showing clothes on a catwalk are accompanied by church music, people dancing in a club by someone having difficulty breathing, etc. the disparity between the two tracks allows us to see the subject matter in a new light.

score music is odd. as mentioned earlier, it migrated from the theater to the film when sound was added to the pictures. the journey however got cut short. score music managed to get on the film but not in the film. it thus occupies a position that has remained indefinite to this day. instead of overlooking this elusory location, let’s acknowledge its reality and expand its realm beyond score music and classical voice-over narration. for instance, why reserve directorial comments to special features on dvds? reflections about the fiction and other more general remarks by the filmmaker would amplify in the movies the writerly approach that barthes observed in more demanding novels. with a little foresight too, one could integrate images of the conductor or the musicians responding to specific visual cues on the screen.

oftentimes we remember scenes or, more globally, an entire film because we were moved by the score. imagine for a moment the original star wars without john williams’s music: how flat and lacking oomph the images and the action would be. there is nothing subtle about the way such music works. steiner, rózsa, tiomkin pioneered the method long ago with rich nineteenth century romantic scores that wrapped themselves incestuously onto the images, compounding their impact. michel chion has rightly compared this approach to the music that accompanies clowns and trapeze artists in the circus. it magnifies, dramatizes the action. it often gives the protagonists a poise they may otherwise lack. it emotionalizes the events. it makes us feel for the protagonists and what they are going through. if the tune starts at the same time the camera makes a move, the impact on the viewer is doubly compelling.

in action movies, drums, brass, and percussion instruments rally the audience around the action just as military tambours once gave heart to soldiers marching to battle. lately, more and more of these films favor a massive, throbbing rumbling sound. this pounding stuns us into submission. no level of resistance is possible. one must surrender to the aural avalanche. this use of the audio track is at heart profoundly fascistic. had the technology been available at the time, hitler, speer and riefenstahl would have found it irresistible.

does score music have an ideological component? who in fact is murmuring (or hollering) in our ears? is music the means by which the hollywood system corrals all films, eliminating whatever difference they might otherwise display?

a few directors decline to use any score music in their films. they look at it as a mendacious intrusion from outside the world of the characters. if they want music at some point, they simply originate it from a specific source within the scene: someone playing an instrument or some canned music heard on a device.

the beating heart of the soundtrack: an actor, a voice, the timbre of the voice. when we look at someone, all we see is the outside of the person, his or her appearance. the voice however comes from deep inside the body. it is a lot more personal and may be more reflective of who that individual is really like. why not cast a film based on people’s voices as opposed to their looks?