Catalog Strategies

Managing your catalogs

No matter which software you’re using, there are some important strategies to employ as you set up your catalogs. The most important approach is to be comprehensive. The more unified your workflow, the easier things will be. If all your photos can fit in a single catalog, workflow is much more straightforward. However, if you have more images than can fit in one catalog, or multiple versions of catalogs on different machines, things get more complicated. Let’s take a look at some strategies for dealing with these issues.

Create a stable directory structure

Catalog software works "by reference", meaning that the catalog links to images wherever they happen to be. It's important that you are careful about how images get moved around once they have been cataloged. In general, you can move folders of images, and update the catalog with the new folder location. But if you move images around within folders, it may become very time-consuming to reconnect the catalog to all images.

• We strongly recommend that your archive directory structure be extremely stable. In general, once images are archived, they should stay in that folder arrangement until some kind of migration requires that they be reworked.
• Working file directory structure will be more fluid, since the files have not yet been archived. We suggest a "workflow pipeline" where folders of images can make their way to the archive on an orderly basis.

Designate primary catalog(s)

Once again the concept of the primary becomes important, at least for people who use multiple computers to work with catalog files. If you have more than one copy of a catalog, it’s pretty easy to do some work in Copy A and do different work on other images in Copy B. Neither will represent all the work you want to save with the images, so you’ll have to sort it out or end up losing something. To ensure you don’t lose work, you should have a designated place to keep the primary copies of your catalog files, preferably in a working files folder that gets an automated backup.

It can get tricky if you copy a catalog to another computer and do work on a second copy of that catalog. Take a look farther down the page to see how you can approach this problem.

Catalog versions and types

Once you get beyond a single user with only one catalog, you'll need to think about how to manage those catalogs so you can keep the work straight. We suggest using a variety of specialized catalogs for particular purposes. The various catalog types are outlined in Figure 1 below, and are covered in the following section. Make use of these when you have a particular reason to add the capability that it offers.

Figure 1 This chart shows the relationships between different kinds of catalogs you may want to make use of. Images can come into the Master catalog from Project catalogs. A Satellite catalog can be taken on location on a laptop. Versioned backups can help protect this valuable information, and Distribution catalogs can be used to make a subset of images available to other users.

Master catalog(s)

The master catalog, as the name implies, is the definitive source for information about the images it references. Using a master catalog gives you excellent access to your collection, since you can gather information into one place. It makes it easier to make valuable connections between images, to search comprehensively, and to manage the files themselves. Once your collection outgrows the capacity of your catalog software, you'll be faced with a decision about how to split the collection into multiple catalogs.

Split along bright lines

If — and only if — you have too many images for one catalog, we strongly suggest that you split your collection along some kind of bright lines (a bright line is a definitive distinction that is easy to find). You need to know with certainty which images are represented by which catalogs. This may be a division by date (all images from this year), or it might be a division by image type (all original images vs. all derivative images), or it might be some other division that works for your collection (such as all personal images vs. commissioned work). A division like this lets you more effectively find images you’re looking for, because you will know where to look.

When deciding on a catalog structure, think about the kind of work you do, how much of it you have, and how you want to store it. Some people, for instance, can make an easy bright line distinction between images that are commissioned work and images that are personal. You might want to split these into two catalogs and keep the derivative files cataloged with each group. If you would find yourself having to pick through assignments looking for individual personal images you shot along the way, a division between client work and personal work is probably more trouble than it’s worth.

In addition to helping you find your images more easily, tidy divisions of the collection can help you manage images more easily, as long as you make sure that there is a matching division in your directory structure. If you’re using size-limited buckets and you split catalogs by year, you should make sure you never have images from two different years in any one bucket. This makes for a much more orderly restoration and revalidation of the archive, should you lose a primary drive.

There are two kinds of problems we’re trying to prevent: omission and duplication. The bright line lets us know that all images of a certain type (say, photos from 2008) are included in this catalog, and that none has been omitted. Likewise, the bright line can help to ensure no images appear in master catalogs twice, creating unwanted branching of the work (branching occurs when multiple versions are created in different places and can’t easily be reconciled).

Figure 2 One way to split catalogs along bright lines is by date. In this screenshot, you can see there is a catalog for each year, as well as a single catalog for all derivative files. While it would be better to use a single catalog, the size of the collection may prohibit this. (In this case, the collection is more that 300,000 images). Splitting by subject matter is also a reasonable practice.

Project catalogs

One technique for catalog management that helps to streamline the handling of large collections is to use project catalogs for the early part of an image’s lifecycle. We find this particularly useful for large projects, where image preparation requires the catalog to be small, responsive, and easy to transfer (this is a particularly good technique for Lightroom users). At some point in the project’s life, you can merge a project catalog into the master catalog and manage it from there. For long-term projects, it’s possible you would manage images with the project catalog, even long after archiving the raw files. Most catalog programs offer the capability to import data from one catalog into another.

Satellite catalogs

Satellite catalogs are copies of the master catalogs that are saved to other computers for use in the field. One of the great advantages of a well-cataloged image collection is that it can be easy to bring the catalogs with you wherever you go, offering access to the entire collection. While this is a real benefit, it brings some real challenges. If you do work to a satellite catalog, and want to save it back to the master catalog, you'll need to make sure you take care of it as soon as you are back in the studio. If no work has been done to the master catalog while you've been away, it can be as simple as coping the satellite catalog back into the home of the master catalog. Reconciling this work can become difficult, however, if work is done to the catalog simultaneously.

Distribution catalogs

Although desktop-class catalog software like Media Pro and Lightroom don’t offer real multi-user access (where more than one user could work on the same catalog simultaneously), many people find that they can create an acceptable workaround by using distribution catalogs, even for large multi-user groups.

In the distribution catalogs scenario, one person, computer, or department is the “keeper of the catalogs” with access to the master versions of the catalogs. On some regular timeframe (often this is an overnight operation), copies of the catalogs are distributed to people who need to have access to the image files. The people getting the distributed catalogs can search the collection by any metadata in the catalog.

Versioned catalog backups

You will have a lot of work contained in the catalog documents, and you’ll want to take pains to protect it. While we strongly suggest that you create a double backup for any working files, including all your catalogs, that may not give you all the protection you need. It’s possible that a database could be corrupted or inadvertently altered in some way, and that you won’t catch the error until both of the backups have been updated with the unwanted changes.
Read more about backup of working files in Backup Types

Merging date between versions

There will be times when someone using a master/satellite catalog arrangement will need to swap information between the two versions. This may be made necessary because both catalogs have been updated and neither one has the freshest versions of all images. You might also want to do this because it's faster to update only a few records, rather than copying the entire satellite catalog from a laptop back to a desktop. In general, there are two approaches to reconciling partial catalogs. You can either use the image as a "carrier" of the information, or you can swap record data, directly. Let's take a look.

Swapping through images

In this method of updating a master catalog, information from the satellite catalog is "pushed" back into the image files themselves. It can then be reimported into the master catalog. Note that this method only works for information that can be synced to image files. Information that lives only in the catalog, such as Lightroom's flags, collections, virtual copies, and history, would not transfer using this method.

Record swapping

It's also possible to swap information by transferirng the record information from one catalog to another. (The record is everything that a catalog knows about one particular image.) This process is really nicely implemented in Lightroom, and is the preferred method. In Media Pro, it is a little less favorable, since you have to manage the process manually.