Backup Types

Mirrors

As the name implies, a mirror backup makes a copy of a file, folder, or drive. In its most simple form, this is an exact copy. Over the years, many different kinds of mirroring methods have been developed - each addressing some specific issue. Let's take a look at the various types of mirrors, and examine what each is useful for.

Real-time

In a real-time mirror, any change made to one version of a file is immediately made to the mirrored copy. This is often accomplished with a RAID 1 setup where both disks are inside the same enclosure. It can also be accomplished with RAID 5 or a Drobo unit, which uses a modified form of mirroring called a parity file.

A real-time mirror protects against the loss of a drive mechanism and allows work to continue uninterrupted. Note that a real-time mirror does not protect against most of the hazards that your data is subject to, such as theft, fire, virus, malicious damage, or human error. It’s fault-tolerant storage - protection against the failure of one drive mechanism. As such, it’s mostly appropriate for working files or other files that need 24/7 uptime. It needs to be used in conjunction with some kind of backup that adds additional protection, such as off-line swapper drives.

Figure 1 shows a real-time mirror arrangement. Data is written to two drives so that the loss of either one won't cause a data loss.

Periodic file synchronization

You can mirror your data using periodic file synchronization rather than a real-time mirror. With periodic file synchronization, the files are written to a primary drive then copied on a regular basis to a different drive, often contained in a different enclosure. If this is going to be your main backup of the data, it should be scheduled with backup software to ensure that it runs regularly. Often these are scheduled for the end of the workday or middle of the night.

A periodic file sync leaves you exposed to data loss for the time between runs, which is a risk. Sometimes, however, this can be a plus. If you accidentally erase a file, you can access the backup of the file, since the version on the backup will not be erased until the sync is run at the end of the day.

Figure 2

Swappers

When you set up a mirror, you’ll end up with two copies of the data. But best practices demand three copies, including an offline copy. How do we accomplish this for working files that are constantly changing? We suggest you add swapper drives to your system. One backup drive can be continually connected to the system for automatic backups. On a periodic basis, this drive should be rotated with one that is kept offline or, better yet, offsite.

A swapper drive is particularly appropriate for working files since those are much harder to protect with offsite storage. While the offline swapper drive adds protection, of course, it does not offer any protection for files that you have added to the archive between swaps. If you can arrange for daily swaps, you’ll be most protected. If you are adding particularly valuable files to your working files folder, make sure to arrange for the swap to happen very soon. And if you are going to leave the studio for an extended period, that may be another important time to do a fresh swap.

Virgin backups

A virgin backup refers to images that are downloaded and are not worked on thereafter. Because the files are not manipulated in any way, the possibility of introducing some kind of corruption is decreased significantly. These backups are often created as disk images or verified clones of the original camera media As long as the files have the same name (or a traceable way to get from the current name to the permanent name used in the archive copy of the images), this technique can provide excellent protection for true disaster recovery.

To create a virgin download backup, burn write-once media as soon as the images are downloaded. If you use 4 GB cards, this fits neatly on a DVD. For larger sized cards a dual layer DVD can be used for 8GB cards. The newer Blu-ray Disc format offers discs with 25 GB and 50 GB capacities. Of course, this method creates a backup that does not contain any of the subsequent editing work. Be sure to separately backup your project files and additional project elements like graphics and music.

Compressed mirror

In a compressed mirror backup, the files are copied to a new location, then compressed into a single gigantic file. This arrangement is used to keep a "snapshot" of a data set that can be restored to a particular point. IT departments that have a standard installation for multiple computers will often create a compressed backup of the entire boot drive. This is also called a compressed disk image.

While a compressed mirror backup can reduce the size requirements of the data, generally doesn't recommend these, unless that's the only way your backup software can create a backup of a boot drive. For image files themselves, we prefer standard mirrors for the ease of access.

Compressed mirrors are not often used for video workflows as there is little room for compression of a video file.

Mirror plus incremental

In a mirror plus incremental (M+I) backup, a current copy of the drive or folder is stored - usually as a compressed mirror - and is updated on some kind of periodic basis. When files are deleted or changed, they are moved to an “archive” part of the backup media so they can be recovered if desired. This kind of backup is the gold standard in office environments, where it might be desirable to save multiple versions of a text document.

M+I backups present real challenges for a photographic archive, however. At the moment, most M+I backups will make a backup copy of an entire file any time you change the file, no matter how small the change. This will soon overwhelm the backup media size for most image collections. (Use of digital tape backup may solve these problems for some users.)

An M+I setup is mostly useful for relatively small files that are always changing, like word processor documents and email, because it allows you to keep access to prior versions without having to keep a folder full of many copies of the files. We suggest using M+1 backups for business and text-based documents. This workflow can work to archive video editing project files which are typically small.

Mirror plus Incremental system can be useful for backup of a video project file. The source video files should remain unchanged in a folder structure throughout the editing of the project. The project file, on the other hand, will be constantly changing. If you use an M+I setup here, your editing work can be well-protected without overwhelming the system with duplicate copies of the video source files.

Figure 3

Additive backups

As described earlier, mirror backups can propagate unwanted changes to files. Viruses, file corruption, media failure, and human error are all hazards you need the backups to protect against, but mirrored backups are all exposed to them. And while an M+I system is supposed to let you roll back to earlier versions of the files, the large size of image files often precludes saving older versions of files.

If you use write-once media, you have some protection against these hazards, but many people find write-once media difficult to implement in a system. One solution is to use additive backups for one copy of your hard drive-based backups. With an additive backup, data is added to the backup as it is added to the primary storage, but—unlike with a mirror—the backup is generally not updated once written. Right now, we're unaware of any software that can manage backups in this way, so you have to do it yourself.

Because the additive backup is not regularly updated, it offers more disaster protection than a regular mirror since it’s much less susceptible to corruption or loss. In an additive backup system, you keep the “touches” you make to the backup files very limited. And because you don’t need to store multiple backup versions, additive backup does not require an excessively large storage solution.

Additive Backups and Video

Additive backup workflow is a very good option for the backup of Video Project files once they are sent to the archive. The screenshot in Figure 4 shows how a series of archived video projects can be backed up in this manner.

Figure 4 shows an additive backup arrangement. Each stripe represents a new group of images coming in. As they get added to the primary, they get added to the backup. If the files are not updated as the primary version changes, you can add good disaster recovery protection without using write-once media like optical disk.

 

Additive backups that are never updated may also be validated with near total certainty. It's possible to make a checksum of the files, and then use that checksum to check if the files have changed in any way at a later date.

READ MORE ABOUT VALIDATING WITH CHECKSUMS

Additive backups are most appropriate for the ingestion and archive phase of life, rather than for working files.

Media managed backups for video

Video software often has the ability to create a backup of the project and source files. This will often include the ability to trim the footage down and perform other intelligent management tasks. Due to the large storage requirements and long time-frame for editing, it’s often advantageous to let the video editing software help you manage this process.

Parity

A parity file is not exactly a mirror, but can function in much the same way. A parity file is a copy of each file written in a compressed manner so that the original data can be rebuilt in the event of a failure. It is frequently used in RAID 5, Drobo, and other storage systems that use multiple disks. The parity file can store the data in a space that is about one half the size of the original data. This creates a space-efficient copy of the data, but it takes work to both compress and restore the data.

Clones and bootable clones

A clone generally refers to a mirror that is a copy of an entire drive (although it's possible to clone a folder or file). A clone is a particular type of mirror that includes both visible and invisible files, and can be used to make a bootable copy of your operating system drive. It is important that you back up the OS, programs, and preferences on your computer, even if you have all your installation disks on hand. Most of the software that you use will have had periodic updates, and these can take a long time to download and install. In addition, there are lots of preferences, passwords, and other settings that are stored on your system that would also take a long time to recreate. Save your software configurations by cloning the boot drive once you have it configured the way you like it.