Panorama Stitching

Basics

In the age of the digital image, panoramic photography is a technique of shooting multiple images of a single scene and combining them to form a single image with a wide field of view, from a relatively narrow capture of two to three frames along a horizontal to a deep zoom image composed of 30 frames, encompassing far more then the human eye’s natural field of view.

Panoramic images are distinguished from wide angle images by their aspect ratio. An image photographed with a wide angle lens may show a wide field of view (even 180º or more), but contain it within a standard 1:1.33 aspect ratio. Panoramic images typically use wide format ratios – at least 1:2 – to show a scene. Some panoramic images, such as 360º “little planet” projections, may not use an unusually elongated aspect ratio, but will instead show a field of view significantly wider than that of the human eye.

Figure 1 Panoramic images can be as simple as two handheld exposures stitched by hand in Photoshop.

Figure 2 Or they can be constructed from many exposures using complex hardware and software combinations.

History

Panoramic photography as a technique began to appear nearly as soon as the creation of the photographic image itself. Its origins are found in the panoramic painting traditions of the early 19th century. Louis Daguerre, inventor of the daguerreotype, was well known before his photographic revolution for his 1822 Diorama, a Parisian attraction that was a combination of scenic painting and theater. One of the first known dedicated panoramic cameras, patented in 1843 in Austria, was based on the daguerreotype, producing plates up to 24 inches (70 cm) wide with a 150º field of view.

Panoramic cameras became commonplace after the development of flexible film, and a number of dedicated panorama camera models were manufactured, from fixed lens cameras that had flat film planes and stationary lenses, to rotating lens systems with curved film planes, to rotating or “scanning” cameras that captured 360º of a scene.

Due to the cost and difficulty of producing wide-format digital sensors, very few dedicated digital panoramic systems have been developed. Those that are available are very expensive and technically complex compared to similar film-based systems. Some digital point-and-shoot cameras include a “panorama” shooting mode, but this is simply an in-camera crop. Digital photographers are therefore left with the option of stitched panoramas, where the camera captures multiple views of a scene, that is then reconstructed with software.

Types of panoramas

The most basic type of panoramic image can be obtained by cropping a wide angle image into a wide aspect ratio. Though this is a bit of a cheat, it will serve the purpose of producing a wide format image with a minimum of stress.

After basic image cropping, we get into multi-shot “stitched” panoramas. These can be shot handheld or on a tripod, but for maximum accuracy and minimal distortion, specialized hardware must be used to rotate the image around the entrance pupil (often mistakenly referred to as the nodal point) of the camera lens. Rotating the camera around the entrance pupil eliminates parallax errors from frame to frame and allows a panorama to be properly stitched. When the camera is not rotated around the entrance pupil, foreground and background objects’ apparent distances from one another will appear to change as the camera is rotated, thus making proper stitching very difficult or impossible

Single row panoramas

The simplest multi-shot panoramas are comprised of two or more images shot along a single horizontal or vertical axis.

One of the simplest and most fool-proof ways to shoot a simple two-frame panorama is by using a shift lens. The lens can be shifted fully from one side to another, then easily stitched together. Because the lens shifts around its optical center, there are no parallax errors (as long as the camera remains stationary for both exposures), and there is no need for lens distortion or vignetting correction from image to image.

Figure 4 This HDR panorama was shot with a shift lens and is constructed from two exposure-bracketed frames.

Using a shift lens to create a panorama also makes shooting HDR panoramas simpler. Since there are only two frames to deal with and no parallax error, stitching is greatly simplified and file management is much simpler than other multi-shot HDR panorama techniques.

Single row panoramas can also be shot with a camera mounting system that rotates the camera and lens around the entrance pupil.

Multi-row panoramas

A multi-row panorama is one where more than a single row of images along an axis is captured.

In order to shoot multi-row panoramas without significant parallax errors, a dedicated camera mounting system capable of rotating the camera around the lens’ entrance pupil should be used.

Figure 5 A rig such as this one from Really Right Stuff must be used to shoot multi-row panoramas without significant parallax errors.

Because more images are being shot in a multi-row panorama, there is a greater need for carefully considered file management. A file naming or metadata convention that identifies images as part of a panorama is advisable, as is separating each set of panorama images into its own folder. Mixing multi-row panoramas and single-frame shots in a single folder of raw files can become very confusing down the road.

360º panoramas

360º panoramas have gained in popularity as the equipment and software for production has become cheaper and easier to use. Very few single-shot 360º camera solutions are available for digital photographers, and those that are available are prohibitively expensive and technically complex. Digital photographers are therefore limited to using compacts or DSLRs and to capture multiple images which are then stitched together.

Depending on the goal and the gear, 360º shots can be captured in relatively few frames...

Figure 6 This 360º pano was constructed from the five images above, shot with a 180º fisheye lens.

...or shot with many, many frames to create complicated deep-zoom images, such as this panorama of Paris

Joiners

While not a seamlessly-stitched panorama image, a joiner has an appeal all its own. The concept has been around for a long time, but its most well known champion is photographer and painter David Hockney, who uses the technique to create images that address motion and time as well as space and dimension.

This is an open-ended technique, but bears mentioning nonetheless.

Limitations and considerations

The restrictions to bear in mind when shooting are the same as many multi-shot techniques, such as HDR capture. In-scene movement, particularly movement in the areas where frames will be stitched, can cause problems.

Figure 7 The boat in this crop from a larger image was moving in the same direction as the camera was panning, and happened to be positioned across a blending plane. It is very important to be aware of moving people and objects in your scenes, otherwise you may end up with someone or something in many places at once.

Some pano stitching software, such as Kolor’s Autopano, can select a particular area of a stitched image and allow the user to determine which frame to include. This is not a guaranteed cure-all, however, and retouching work in Photoshop is almost always inevitable when dealing with movement.

When shooting outdoors, changing light is frequently a problem, especially with multi-row panos. Nothing ruins a pano shoot like getting to the last frame of a scene and realizing that the sun has gone behind a cloud bank. Speed and careful setup become the panorama photographer’s most important allies when shooting outside.

Figure 8 Because the sun wasn't in the center of a frame in this Little Planet image, the flare lines don't match up and must be retouched.

Because of the technical, multi-shot nature of digital panorama photography, a significant amount of post-production time is required for almost all panoramic images. Of the images on this page, every one required at least twice the amount of time to produce in the computer as it did to set up and shoot. Post-production is greatly aided by considered planning before the shoot, notes taken during the shoot, and careful file organization when arriving back at the studio.

(Of particular use are notes indicating how many rows and images across comprise a panorama – it can also be extremely helpful to note which frame # begins and ends a pano sequence. This will speed the process of organizing panorama sequences into “shot” folders during editing.)

Workflow

Panoramic photography and high dynamic range (HDR) photography share a lot of workflow similarities. Both have a tendency to create a large number of files, and both often require specialized software, much of which is both more technically complex and less refined (in terms of a GUI) than your usual workflow PIEware.

Considerations

You might be tired of hearing it, but file management is at a premium when shooting panoramas. A “raw files” folder of a shoot that includes three panoramas, two HDR sequences and 50 single-frame shots might not seem like a problem in the days after the shoot, but will inevitably become a frustrating jigsaw puzzle months and years down the road.

Organize your files.

Most panorama stitching software allows the user to save “project files,” which contain the parametric instructions that the stitcher used to blend and create the panoramic projection. Like HDR files, panorama project files are frequently the product of a large amount of work, particularly when a file has been “hand stitched” with custom control points. Project files, like XMP files, are usually small text files, so they won’t eat up hard drive space, and can be a boon down the road.

Shooting HDR panoramas adds another level of complexity. If you are already accustomed to HDR capture, your workflow will be greatly aided by this experience.

Stitching and post-production

Most modern panorama stitching programs are based more or less on Panorama Tools by Helmut Dersch, and most share a number of similarities. 

There is a wide pantheon of panorama stitching software, including dedicated panorama heavyweights PTgui Pro, Kolor’s Autopano Pro, and Autodesk Stitcher.

PTGui Pro is the most commonly used and most powerful, but each offers different benefits. Autopano Pro has a simple, streamlined workflow and is able to control motorized panorama heads, such as those made by GigaPan. Autopano is also able to recognize different types of pano sequences in a folder of images automatically, a very handy tool. Autodesk Stitcher has very useful ghost removal tools. All of these programs are able to manage and blend HDR panorama files, and handle 360º images.

The amount of fine tuning and retouching needed after stitching a panorama depends on the quality of the shooting and the stitching itself. Minor stitching errors are par for the course, as are “ghosts” and lens artifacts such as flares and chromatic aberration.

Projections

After shooting any panorama and stitching it, what you are left with is either part or all of a spherical image. Unfortunately your inkjet printer probably won’t print on a sphere, so we must “unwrap” the image. All software tools that handle panoramas are able to redraw a spherical image using several different types of projections. These are closely related to cartography, where a map of a spherical planet must be recreated on a flat surface. Each type of projection creates different types of distortion, so it is up to the photographer to decide what projection is appropriate for a given image.

There are three main types of projection: flat (or rectilinear), cylindrical, and spherical (or equirectangular).

Flat projections have minimal distortion of straight lines, and are frequently useful for architecture. This is the type of projection that looks most natural to us, as it is the type of image produced by our normal camera lenses. However, its practical field of view is limited to a 90ºx90º maximum field of view before the image begins to distort wildly, so it cannot reproduce a full 360º image.

Cylindrical projections are similar to flat projections along the north-south axis, but do not curve horizontal lines on the “equator” of the spherical image. The upshot of this is that 360º can be projected horizontally, and up to 120º vertically with distortion of horizontal lines. The practical field of view of a cylindrical projection is limited to around 360ºx90º. It is not possible to project the poles (nadir and zenith) of the image unless you happen to have an infinitely tall roll of printer paper.

Spherical projections are able to reproduce the entire 360ºx180º field of a sphere using nonlinear math. The poles of these images become wildly distorted, but they are there, nonetheless. These images appear similar to cylindrical projections around the middle latitudes, but become increasingly distorted towards the poles. Spherical projections for print (or anywhere they are displayed as a single image) are frequently cropped at the poles as the distortions become fairly extreme. This is the most common projection used for interactive panoramas on the web; because we are looking only at a small portion at a time, the sections that are distorted can be constantly changed as the area being viewed changes.

Other types or projection are available, depending on the stitching software. Mercator projections are based on cylindrical projections, but don’t stretch the view as much along the outside vertical edge. Vedutismo projections preserve all straight diagonal lines through the center of the image (think about landscape paintings with a distinct vanishing point). This is most useful for images with a distinct vanishing point, such as a river leading off into the horizon. Stereographic projection is a way of projecting the image sphere onto a flat plane as it would be seen from the pole. This creates “little planet” images, where the scene seems to float in the sky.

HDR panorama photography

If you’re already going so far as shooting panoramas with a dedicated camera mount and stitching software, you might as well be shooting HDR, right?

All the workflow and file management considerations associated with HDR image capture apply to panorama photography. Please do take some time and read the HDR section of our site.

The major issue with HDR pano production is when to create the HDR file. The “traditional way” would be to create a stitched panorama for each exposure level in the stitching software, then use HDR tonemapping software to create the HDRI file. But we’re living in the modern age. All of the panorama stitchers mentioned above are able to handle both the stitching and creation of HDR files in one fell swoop. It’s that easy.

It is important when exporting the stitched HDR panorama to consider how large the final image will need to be. Panorama files can be very big – after all, they can have a 360ºx180º field of view. Even a six-frame pano from an eight megapixel camera could approach a 36MP image after stitching, if not sized down. While this might not seem excessive, think about the file that would be created from a 30-frame panorama taken with a 22MP camera. Most tonemapping software simply can’t handle files that size. It is necessary to test the file size limits of your tonemapping software and computer system to determine what its file size limit is. However, there is hope; as 64-bit operating systems become the standard and tonemapping software is rewritten to use a 64-bit core, this limitation will be minimized.