Creating a Multimedia Interface

I have been working on this for weeks now — and finally it is opera­tional. And that’s good news, really good news and very important to me.

But the bad news (well, not really bad, but annoying): the project I used for creating this is sent to a client for evaluation and I cannot show it at this stage and walk you thru it. So unfor­tu­nately no WOW-factor this time, only my words. But you get a pretty good sense of what is possible by taking an intel­ligent and a critical peek at the Sölden Multi­media or Helsinki Inter­na­tional Horseshow 2009 Multi­media I have mentioned earlier in this blog. Combine what you see to what I say (both were used as pilot projects).

My goal has been very simple: for the viewer, I want to offer non-linear, visually driven multi­media, incor­po­rating images, video, panoramic images, sound, graphics and text all into a single piece of multi­me­diawork. Intui­tively acces­sible for the viewer and visually interesting and challenging, something they would like to come back to. Simple and yet a profes­sional feel to it.

MULTIMEDIA: Linear versus Non-Linear approach

The majority of the so called “multi­media” which is published is simply linear A‑to‑B story­telling (if you see eg. what giants like The New York Times or MediaStorm produce). I strongly feel that if you are to call it MULTImedia, you should have a non-linear approach, something along the lines the BFC guys are doing.

Besides, I am thinking about the readers of tradi­tional newspapers (my clients), even trying to profile their needs, habits and desires: newspapers are inherently non-linear (whereas TV is linear). Thus, if you want to appeal those audiences who in the past preferred the printed page, the approach has to be more than a simple video “A‑to‑B” . The approach has to be non-linear — and it has to be a good quality one.

For the designer (ie. this time myself), I needed to create a designing workflow which was GUI-based, with minimal hand-coding necessary. To make it have flash and html-combined, with the emphasis more towards flash where possible, no AS 2/3 ‑skills needed. The workflow had to be so smooth that you could accomplish a (simple) piece within more or less decent timeframe (hours, not days) in order to meet a deadline. And you had to maintain a profes­sional look, even if you were pressed with deadlines.

That is lot to ask — as I am a photo­grapher, not a designer. But, I realized that I had knowledge of and access to some interesting programs, which when used — or should I say mis-used — appropriately, would produce interesting results. So combining some 10–12 programs into a workflow, I managed to come up with pretty fluid solution. Several of these programs are meant for panoramic work, the other programs include FCP, PS, PM, Soundtrack, Flash, etc.

2D VS. 3D

The key concept of my thinking is: normally visual content (“newscontent” as I am a pressp­ho­to­grapher) is displayed on a 2D-space, ie. with x‑y — coordi­nates. Typically an image or collection of several images — or maybe a slideshow on a webpage, along with text. A graphic — maybe animated — maybe even a small video.

When working with panoramas, you have the z‑axis as well. And that makes it a totally new ballgame.

But — and I can hear the counter-argument immediately — what if you don’t want to include panoramas? What if you don’t know how to shoot them? What if you really could not care less about them?

I realized that panoramas are actually not necessary, even though I perso­nally like to work with them. What one really needs, is the 3D- projection plane panoramas are usually projected into. Unders­tanding that and mastering the necessary geometry/software gives you a full control of x‑y-z ‑coordi­nates on a cylindrical or spherical projection plane — or both — along with the standard 2d projection plane used. In laymans terms, think of it as a projection space consisting of a screen, interior of an oildrum and the interior of a ball, all blended together, forming a perfect 3D designing platform.

This gives you limitless possi­bi­lities for displaying your media. First of all, all the visual media content (images, slideshows, graphics, video) can be either static or dynamic — ie. they can be aligned to the under­lying 3D-plane/space or to the 2D-window you have open. They can be presented straight or distorted in the projection space and they can be scaleable or non-scaleable.

Audio can be either on autoplay or have manual controls. It can be static or panned in space. Live-mixing with several interacting sounds is totally possible (ie. eg. a situation where ambient audio changes depending on the viewing angle of the plane and there is voice-over with controls super-imposed — a VO which you can listen if you choose to).

An event (ie. loading an action, image, sound, video, graphic… or loading another projection plane, etc.) — can be triggered by e.g. entering, leaving, clicking, etc. on a hotspot — or by entering a preset area in the presen­tation. And as presen­tation can be multi­layered (ie. contain several projection planes), there is practically no limit to what one can or cannot do.

One can also use a preset timer — which can be reset dynamically, multiple times, if one wants to. Events can include visual changes, audio effects, narra­tives, links to websites, basically anything. The media/action which is called up by an event can be aligned either statically to your window or aligned to your projection plane.

And as all this is accomplished within flash-framework: there is a possi­bility for fullScreen-viewing and scaling up and down of your presen­tation. Supported and preferred format for media in this workflow are jpg’s, png’s, flv’s, swf’s, mp3’s, among others.

HTML-code is used minimally, basically only to trigger outside media resources/assets into the publishing frame and for providing that frame. Most of that — if not all — could be done within the flash-framework, but that would make the presen­tation heavy and sluggish, especially if one were to introduce loads of heavy video content (maybe even distorted such). And it would increase the designing load more than time would allow in certain cases. The whole workflow is about 90–95% GUI driven and the remaining hand-coding is relatively simple.

That’s about it.… I will show examples later on, especially the one which prompted me to cook this all up — and others, as they come along.

This week, my teaching and another major project I am working on is demanding my attention. But I am slowly shifting my focus towards Vancouver and starting to plan the stuff (including multi­media) I am supposed to deliver from there — given the 10-hour time diffe­rence against us. It’ll be a true challenge — and I welcome it as such.

**************

(UPDATE 3.11.2009: Just saw Julian Treasure’s presen­tation in TED on soundscapes — and I warmly recommend it . Pause and contemplate for a moment: there is enormous relevance in his words for this line of multi­media work.)