Suppose you could take a camera - lens, film, and all - and stretch it like a blob of Silly Putty. You could wrap it around people, simultaneously capturing them from all directions. You could put one end of the blob on the ground floor of a building and stretch it up the stairs to the roof, getting one exposure of all the foot traffic in the whole building at once. You could change the shape of the camera for every image in a movie, capturing the point of view of multiple characters at once as they move around.

The storytelling possibilities for such cameras are exciting. They could let us express novel visual relationships among characters and elements of the scene and how they affect one another.

Computer graphics gives us for the first time the chance to create and use such cameras. So far most computer graphics cameras have been based on a simple pinhole camera, though sometimes we also include lens or shutter information. When we free ourselves of this physical model, we find that there's a whole new exciting grammar of visual storytelling to be explored.

Let's start out gently and look at a typical background for a traditional cartoon. Animated films often use distorted background images so that the camera can simply pan, or move, over the picture, resulting in what appears to be a moving camera in three dimensions.

In this scene, the young woman on the left is about to be sent away. She is gazing across the street at her friend, who is looking back at her, as they await the inevitable car that will take her away. This is a single continuous image - think of it as a single frame from a movie. If the characters got up and walked around, we'd see them just as usual, though the images would be distorted.

Notice that we are simutaneously looking over the girl's shoulder at the boy, and over the boy's shoulder at the girl, and that each sees the other through their windows. We're also looking at the street. The composition of the scene shows their shared isolation and yet their connection, literaly and figuratively separated by the street which looms large in both their minds. The story point here is to emphasize their connection and separation simultaneously, while also giving us the chance to see each character closely and as the other sees him or her.

Let's see how these ideas can be applied to an action scene.

Here's a hungry bear going after a salmon. In the left we are looking over the bear's shoulder, and on the right we see the situation, underwater, from the point of view of one of the fish. In the middle part of the frame we can see the hindquarters of the bear and his side as the camera plunges into the water. Again, this is a single image.

Imagine this as a frame of film, with the water swirling, the fish leaping into the air, and the bear pawing at the water hoping to spear himself some dinner. We get to see it from the points of view of the bear and fish simultaneously.

Once we've taken to moving the camera in space, we can also think about moving it in time

Here's a scene from my hometown of Seattle. On the right we see some passengers on the monorail as they pass by the Space Needle. On the left we have the same train from a very different point of view (notice the Space Needle in the background), but also some time later. If interesting things are happening in the monorail as it rides, we get to see cause and effect simultaneously.

This approach raises some interesting practical questions, such as the use of sound. We will need to develop a vocabulary and set of conventions for using sound to help keep viewers oriented and aware of what's important in a scene.

As a final example, consider this scene from a thriller movie.

This is a nice application of the free-form camera technique because it lets us catch a lot of information at once. We get to see the point of view of three different characters at once (the woman appears four times, once in a reflection). We get to see all the relative motion, and as characters move we can easily judge how far they are from each other. To heighten the tension, the dark-shirted man is approaching us in one part of the frame, while we clearly see how far he is from the woman in another part. As the action progresses, we can feel the rising tension as the man with the poison approaches the woman.

All of the drawings above were originally rendered in pencil on vellum by Tom McClure.

Rendering images with wacky cameras is trivial for almost any computer graphics program. Ray-tracing, in particular, renders each sample of each pixel independently, so it's a snap to make images such as these with ray-tracing programs.

The trick is to find a way to create and control the cameras. I've experimented with a number of approaches. The one that I've found the most satisfying so far uses a collaging style.

Here's a shot from a film inside a diner (this is early pre-production work for the short film Sheila's Diner. These are early and crude test images to debug the technique).

To create an image like this, I use a collaging technique. In the next image, we see three separate renderings, each made with its own camera, combined into one picture. When the system renders each of these component images, it saves a bunch of information in related files that the filmmaker never needs to be aware of. All you need to do is open the pictures in Photoshop and create a collage, placing each piece of each rendered scene where you want it.

In this scene, two men are sitting next to each other at a diner. Their waitress, Sheila, is facing them. One of the men has just noticed a red pill in his coffee. He is, naturally, concerned, and so are we. On the left part of the image we see the men, in the bottom-center the redhead's coffee cup, and on the right we see Sheila, who is facing them. Of course in the actual movie these stand-in characters are replaced by high-quality, animated characters.

On the left are the three images collaged together in Photoshop against a gray background. The system uses the information saved from when they were originally rendered to fill in the spaces in between, as shown on the right. This is just a visualization tool for the director and cinematographer. When the results look about right, the system re-renders the whole scene. Notice that there are no sharp creases, even though the imaging model is swinging around pretty wildly.

The stretching of the camera isn't quite right in this image - it's too blocky and I don't like the way the coffee urns appear to be tilting. But if you think about where the camera is located on the left and the right, something like that simply has to happen. I think that there are ways to specify the collaging to allow some more creative control over the interpolation without requiring the director or cinematographer to become aware of the underlying mechanisms, much less programmers. I hope to investigate those approaches soon.

The system described above was developed as a plugin for the computer graphics system 3d max. I also wrote some additional standalone code to handle the heavier mathematical steps.

You can read more information on this work and see some more examples in my technical report, which you can download here. Here is the complete citation:

Glassner, Andrew S., "Cubism and Cameras: Free-form Optics for Computer Graphics", Microsoft Research MSR-TR-2000-05, January 2000.