File Management Overview

What is file management?

The data that we work with on computers is kept in a hierarchical file system in which directories have files and subdirectories beneath them. Although we use the computer operating system to keep our image data organized, how we name files and folders, how we arrange these nested folders, and how we handle the files in these folders are the fundamental aspects of file management. The operating system's organization of our data can be enhanced by the use of cataloging programs, which make organizing and finding image files easier than simply relying on the computer's directory structure. Another feature of catalog programs is that they can streamline backup procedures for better file protection.

Know your primary

As we outline in the Backup section of the website, one of the most fundamental concepts in your image collection is the distinction between Primary and Backup. The Primary is the main copy of the data, and it is protected by the creation of backup copies. This File Management section deals with the Primary copy of the data only. The creation of a well-structured file management system can make backup and restoration relatively simple, but separate, processes.

Storage vs. Organization

It's helpful to approach file management issues by separating the concept of Storage from the concept of Organization. Storage describes how you handle the files: What's in a folder? What kind of folder names do you use? How do you design a folder hierarchy?

Organization, by contrast, describes how images are grouped according to content, usage or value. Organization lets you find all pictures of Josie, or all pictures done for the Acme Company, or all images used in my portfolio. We recommend that your organization be based fundamentally on metadata, rather than on folders.

There are two main problems with using folders for organization. The first is that the work you can do to organize with folders is very limited. There must be one top-level organizational method, which can only be subdivided in a limited way before the system becomes too cumbersome and breaks down. Is it most important to divide by date, client, project, subject matter, rating, or usage? Furthermore, information that is dependent on folder structure is very fragile. If you remove an image from a folder that designates what that image is, that content information can be lost.

While folder structure can be helpful in organizing your images, we suggest that the main job of folders should be as a storage structure.

Catalog software

If folders are not the main method for organizing files, what is? We suggest that you need to use metadata and catalog software to most efficiently organize, manage, preserve, and get the most value from your images. Catalog software keeps a record of all images, and lets you use metadata to group them in any number of ways. You can bring images together that have common subject matter, were shot for the same client, were sent out for similar uses, or any other commonality. 

 Figure 1 This video shows how you can use catalog software to organize your images in lots of different ways.

Be consistent

The work you do to manage your files will be much more valuable if you do it consistently. While this can take some work to develop a system and train yourself to stick to it, you'll be paid back in the long term. We suggest you make some effort to standardize file naming, folder structure, metadata use and more. As you do this, keep in mind that your collection of image files will be growing, and you will want your systems to be scalable so they can grow with you.

Originals and derivatives

One thing you'll want to consider as you create a file management system is the treatment of camera original files and the derivative files made from them. While it seems natural to keep these together in one folder structure, there are a number of advantages to separating them into two different directory structures.

Your camera originals (which we suggest storing as DNG files for most people) are different from derivatives in a number of ways.

• There should be one and only one primary copy of an original image. This is not the case with regard to derivatives.
• An archived original image may be reedited with PIEware like Lightroom without creating the need to re-archive the file. In general, that's not true of derivatives.
• Derivative files are frequently made or reworked after the camera originals have been archived. It can simplify the backup process when new work is not mixed in with older work that has a 3-2-1 backup.
• Catalog software, along with a good naming convention, can make it easy to link derivatives to their originals, even if they are far apart in the directory structure.

Separating originals and derivatives is not the only way to structure an archive, of course. Some photographers will always make derivatives right away, and then can safely archive the entire shoot together. In these cases, it is still advisable to group the derivatives in folders separately from the originals.

Folder strategies

Folders and backup strategies

If folders are most useful as a storage tool (rather than an organizational tool), then one of the most important aspects of your directory structure is the ability to back up the files easily and safely. You can simplify this process by keeping files together if they need the same backup treatment.

Working files folder

We suggest that each computer have a place to store works in progress. If all your "works in progress" are inside a parent folder called "Working", you can set up a single backup task to protect the entire group. Use of a single working folder also helps you keep track of your work in progress, which is particularly helpful for people who use multiple computers. Set the working folder up the same way on all computers.

Archive folder structures

While the Working folder is a temporary home for your images on the way from card to archive, there should be a more permanent structure to the image archive. This needs to be a scalable structure that can grow with the collection, and it needs to lend itself to easy and safe backup. Let's look at some options.

Date-based folders

One popular way to create a structured image storage directory is to create a folder hierarchy based on date. It's a clean and easily understood structure that scales by simply adding dates as time goes by.

Client-based folders

You can also structure your archive directory around the client or project names. In many ways, this corresponds most closely to the filing system many photographers used to store film.

Size-limited folders

Another way to structure your archive is to make folders the size of your optical media backups. These would be approximately 4.7GB for single layer DVD or 25GB for single layer Blu-ray discs. This system is also commonly known as a "bucket" system. The great advantage of this system is that it makes restoring your data from an optical media backup much easier than any other folder structure.