File Format Migration
As computer technology moves forward, there will likely be times when you will want to consider migrating the storage format of image files. This may be due to obsolescence of the current format, or because you want to take advantages of capabilities of new formats.
Formats are for saving
Know what you've got
Taking advantage of new capabilities
Formats in danger
Migration example - Conversion to DNG
Is everything coming along?
How do I find what's hiding in a directory?
When an image file is saved, all the bits in the file need to be stacked up in some way so they can be decoded. A file format is a formula for stacking up those bits so various programs can understand them. There quite a few different images formats in use, and many of the variations are slight, such as the differences between CR2 files produced by different Canon cameras. Some of the differences are large and significant, such as that between a TIFF file and a JPEG file. If an image can be opened, it can be saved to a new format. There will be times when you want to change the format, or the variation of the format, that you use to store your images. That's when it's time for a migration.
Unfortunately, there's no one who can monitor what file types you have in your collection except you. It's up to you to know what you've got, and to deal with obsolescence before it's too late. If you've been making and collecting digital files for a decade or more, you should strongly consider acquiring catalog software that can index lots of file types. You can then look through your files periodically to make sure that nothing is going obsolete.
We strongly suggest that you keep a primary copy of your archive on available storage of some sort, rather than in a drawer that is out of sight. This will help to ensure that you know when a problem arises with one of your storage formats, and that you will be able to accomplish the migration more easily.
The most critical reason to do a format migration is because the current format is in danger of becoming unreadable. This is a particular risk for proprietary files from digital cameras because there are so many of them, and because there is a natural cycle of creation. As better cameras come along, software companies have less incentive to support the formats of the older models that have fallen out of use. Images from some of the very expensive digital camera from the late 1990's are now only accessible with very old computers.
While it's very unlikely that your camera files will be totally inaccessible in 10 years, it's not out of the question. Right now there are more than 200 distinct flavors of camera raw files. When software is updated, it needs to be tested with each of these. At some point, companies may decide that it doesn't make sense to spend engineering resources on a camera not been sold in more than a decade. If we get to a world of 2000 proprietary raw files, this could become a real problem.
Rather than avoiding obsolescence, it's more likely that your motivation to migrate will be the need for increased functionality. For instance, there are a number of features that are available for DNG files that are not possible with proprietary raw. This includes validation capabilities, embedded metadata capabilities, and embedded rendering capabilities. Added security and functionality are likely to be a real draw for many photographers.
The following list of formats are in at least some danger of obsolescence in the foreseeable future.
- Kodak PhotoCD
- Kodak RAW
For many readers of this website, the most important format migration will be from proprietary raw to DNG. Between the forward compatibility planning, embedded data capabilities, and the data validation tools, DNG will look more attractive as time goes on. If your archive is made up of proprietary files only, format migration using the DNG converter is pretty simple. Send the whole directory structure to the DNG Converter and let it run. At the end of the process, you'll get a set of DNGs in a folder structure identical to the proprietary raws' folder.
|Figure 1 The DNG Converter can create a duplicate directory structure populated with DNG files instead of proprietary raw files, but it will leave all non raw files behind (such as the JPEG and PSD files in this figure).|
During the DNG conversions, unfortunately, any non-raw files will be left behind. TIFF files will generate a warning in the DNG converter conversion log, but other non-raws, such as JPEG and PSD files (not to mention non-image files) will all be left behind silently. If you suspect you may have some of these interspersed with your raw files, you'll want to look at another method.
The best way to get a comprehensive view of what's in a directory is to use a cataloging program that can index lots of file types. Expression Media is one that works on Mac and PC, and can show many file types. Here's how you can use it to see what's in a directory.
- Make sure the folder is up to date in the Expression Media catalog (right-click the folder and choose Update Folder).
- Click the folder’s dot to show all items.
- Select all items in the thumbnail area.
- See which file types light up in blue.
You can use this information to know what's in the collection and to help the migration process.