This was a multi-disciplinary media arts initiative as it encompassed research, on-location shooting, audio and video editing, programming, and even some basic constructing work.
The show went very well and I received excellent feedback on the installation. I was also lucky enough to be interviewed at the opening for ArtSync. The first part has Dan Bazuin talking about the exhibition and the collaboration with One Hour Empire. I come in at about the 2:20 mark.
I first started thinking about this concept on one of my many coffee visits to Balzac's Cafe at the Distillery District in Toronto after the call for Here and There submissions was posted to the Propeller site. As I walked amongst the Victorian buildings, certain angles seemed familiar to me and I knew I recognized very specific landmarks. So when I got home I started exploring The Internet Movie Database and I realized that their staggering database of information also included shooting locations, and not just cities but specific areas and even addresses.
Now, I've always known that many, many productions shoot in and around Toronto, what I never really thought about was how easy or difficult it would be to find the exact locations or buildings that were used. So I punched in "Distillery" into IMDB's search engine and got back a lot of entries. The one that stood out was X-Men because I seemed to remember a specific scene. I dug up my DVD and right there close to the start of the movie was the scene I remembered - a young Magneto being led into a concentration camp with his parents. As they are separated, the traumatic experience awakens his mutant powers and chaos ensues before the butt end of a rifle makes contact with his head.
Right there I knew I had to grab my little Sony HD camcorder, head down to the Distillery, and capture some real-world video of the actual location. But how could I match the shots and edits in the movie? That's where my iPhone came in. Using some extraction tools on my Macintosh, I grabbed just that chapter and transferred the short scene onto my iPhone to use as reference at the location.
The next step in the process had four parts:
- Extract the video again and convert it to 720p ProRes for inputting into Final Cut Studio. This required a slight "up-sample" from the source but overall the quality was good
- Extract 2 audio files from the sources. I made sure to stick to movies that had 5.1 sound so that I could separate out the dialogue (mono center channel) and the score (stereo left-right channels).
- Research and decide upon a piece of music from my collection of local artists that would compliment the scene.
- Convert the recorded video from AVCHD format to H.264 Quicktime. Then down-sample the video from 1080i to 720p ProRes and conform the 30 fps video to 23.98 fps to match the movie clips. All this was required in order to be able to edit in real time using my current Quad G5 setup and Final Cut Studio 2. Many of these steps will not be necessary when I upgrade to an Intel PowerMac and the new version of FCS.
Once all the data was ready came the task of editing the location content exactly to the movie clip's edits. Some of the scenes were fairly simple while others took a long time due to the director's extremely quick editing style. This is when the whole "Parallel" aspect to the piece started to take shape. Up until then, it was all mostly a concept in my head. But as soon as I was able to start playing the two video streams at the same time, I could see that I had something interesting starting to materialize.
Currently I have 9 scenes in the pool, 6 of which are randomly chosen at the beginning of any user's interaction, but more on the programming later. Here is a synopsis of the 9, I do plan to keep adding scenes over time.
Amelia - 2009
Directed by Mira Nair
Chicago - 2002
Directed by Rob Marshall
Queen Elizabeth Building - CNE
Flash of Genius - 2008
Directed by Marc Abraham
Kings College Circle
Fringe Pilot - 2008
Directed by Alex Graves
Kingston & Main
Kick Ass - 2010
Directed by Matthew Vaughn
Max Payne - 2008
Directed by John Moore
Repo Men - 2010
Directed by Miguel Sapochnik
Don River Bridge
Shoot 'Em Up - 2007
Directed by Michael Davis
X-Men - 2000
Directed by Bryan Singer
I also decided to use Apple's Magic Trackpad as the input device as this would allow participants to focus in on the parallel monitors instead of something like a touchscreen with its own graphics. The application's "stage" ended up being the total screen area of the two monitors (2720 x 768) as it is very simple to run multiple monitors in OS X.
When a participant begins their interaction, the application randomly chooses 6 scenes from the larger pool and distributes them randomly on the Here and There sides. They first choose a tile on the Here side which flips over a still from the video. Then they choose a tile on the There side which may or may not be a match. If it is not, the tiles flip back and they try again. This is basically a version of the classic children's memory matching game. I wanted to use this device instead of just participants choosing various scenes in order to provide a greater level of engagement and ultimately a sense of reward if all six are completed.
When the participant gets a match, they are taken to the parallel playback of the scene. The trackpad then becomes a controller for the creation of their own personal soundtrack in real time. By sliding their fingers up and down, the volume of the dialogue is controlled. By sliding their fingers right and left, they fade between the film's score and the local musician's song. Feedback to these various levels of sound is provided on the screens.
This is a short demo I put together of the installation during the exhibition at Propeller. Please excuse the sound as I could not easily patch into the computer's sound output at the time. The installation ran on a Mac Mini (2.4GHz Core 2 Duo, 4GB RAM) with an external 7200 RPM Firewire 800 drive. The faster external drive was required due to the simultaneous playback of the parallel videos. Even though all the source material was edited in Apple ProRes, I decided to encode all the final 720p videos with the Apple Intermediate Codec and Apple Lossless for sound. This conserved some hard drive space and processing power without any visible or audible reduction in quality.
Here's my artist's statement.
- Parallel City Interactive Media Installation 100 x 60 cm Being a young immigrant to Canada, my Here and There explores the duality of the city I have come to know and love, Toronto. I am Here, familiar and intimate with every aspect of my city, from the evident to the covert, exploring every corner until the landscape becomes ingrained. I am There, through the magic of cinema an unwitting sleuth, observing and detecting the almost imperceptible, when my city impersonates another. Through the interactive exhibit, viewers and participants will explore my Here and There by attempting to visually match the real world - Here, with the cinematic world - There. Once matched, they will experience parallel video playback presented through two side-by-side monitors, audio will be provided by stereo speakers. The left display is Here, and it will present real world video shot in the Toronto landscape corresponding frame-by-frame to the right display, a compelling cinematic scene where Toronto stands in for There. Supporting this visual parallel will be three sound streams, the cinematic score and dialogue for There, and befitting music from a local artist for Here. During parallel playback, participants will be able to create their own soundtrack to Here and There by using the touch control surface as a fader between all three. The challenge for the participant is to match up all six Here and There scenes. The reward is a lasting impression of the diversity, adaptability, and beauty of my Here and There, Toronto.