Work Flow & Digital Asset Management

A work flow is something that most photographers don’t start considering until it is too late! If you’ve only ever taken a couple of 100 images, it’s pretty easy to keep track of them. But when that 100, becomes a 1000, 2000, or 10,000 plus photos, the management of your images can get out of hand quite easily. Unfortunately, by that stage it’s simply too late to implement a system. Although it’s rarely done, a good work flow should be put into place from the very start. If you’re reading this and you don’t have a system, then I would strongly encourage you to put one in place today!

So what is a work flow and what is all this asset management about? For me, my work flow; my management system; has a few key attributes. Firstly and probably most importantly, I can quickly work my images right through from camera to final output, whether that is screen or print. If your work flow is not easy, you won’t follow it. So it should have simple steps and where possible, steps that can be performed automatically, which take your images through your flow. Secondly, it has to be structured in such a way that any image, no matter what it is of or when it was taken can be found with little or no effort. I can find any image I’ve ever taken within a few mouse clicks – I’ve never lost or been unable to find an image I’ve taken. The system has to be scalable, so regardless of how many images I have or take, it can cope with it. Finally, my images have to be safe from loss (hard disc failing, computer being lost, house being burned down). Every image you take is a unique moment in time – it can never be recreated exactly. So if you lose an image, it is lost forever. By putting an asset management system in place, there is never any need to lose an image again.

The hardware

The easiest place to begin describing my work flow, is to take you through the hardware involved. Obviously at the heart of any work flow is the camera, in my case a Nikon D3. I shoot both RAW and JPG simultaneously; the reasons for which I’ll explain in a while. My images are transferred to my PC via a USB card reader, which is probably the best value equipment I’ve ever bought. If you have Gigs of data to copy, the faster you can do it, the less frustrating the wait becomes.

Leaving the camera aside, the core of my system is a laptop, which I use for all my editing, processing etc. I believe laptops offer many advantages over desktops and any of their disadvantages can be overcome – in fact I personally don’t understand people using desktops for their photo editing and storage at all. The obvious advantage of a laptop is portability. Anywhere I go, I can bring my entire library of images with me. I’m able to bring the laptop into a studio and shoot tethered if I like. I can be in any room of my house, or indeed in my office and still have my photography environment to hand. Of course, laptops do have two pitfalls. The first is generally performance. It is expensive to have a laptop with top of the range performance (screen size, cpu, RAM, hard disc etc.) Currently I have the HP 8710w “mobile workstation”. 4 Gigs of RAM running XP. Intel Core 2 Duo processors @ 2.6 GHz, 200 GB 7200 rpm SATA HDD and most importantly for Photoshop a NVIDIA Quadro FX 3600M graphics card, with 512MB dedicated video memory. The second pitfall is the screen, which again I overcome by using an external calibrated monitor connected via a docking station.

In terms of storage (which is covered in much more detail later on) I have a set of external hard discs at home. These are USB drives which are also networked and connected to a wireless router. In the cases where I’m reading or writing large amounts of data, I can connect to them via USB, but the rest of the time I can access the drives from any room in my house via WiFi on the laptop. Another advantage of having them connected to the wireless router, is that I am able to access the images from anywhere on the web via a VPN connection. In addition to the discs at home, I also keep external discs off site (in my office) which contain another backup of my images. This covers me in the case of some major disaster at home (fire, flood, theft etc.) which may damage the discs stored there.

The final piece of hardware is a printer, in my case a large format Epson 3800 ink-jet, allowing me print anything up to A2 in size. I don’t really discuss printing beyond this point, but thought it was worth mentioning and noting where it fits into the overall scheme of things.

The software

Complimenting the hardware, in addition to the operating system (Windows XP), my system runs on 4 main pieces of software. I use Adobe Bridge for batch processing my RAW files, Adobe CS3 for editing images, Microsoft Expression Media for cataloguing my images and SyncBackSE for managing backups and synchronising discs. Many people use Adobe Lightroom, which more or less combines the functionality of Bridge and Expression Media into one application, but I prefer my current setup. The concepts I discuss however, are the very same, regardless of what applications you use.

File structure & naming

Leaving all the fancy hardware aside, I think everyone should at the very least have a simple system which handles the naming and storing of images, in such a way that they can find images easily and that every image has a unique name. My file structure is created more or less manually, using the operating systems own tools. All my images are stored in folders under a main folder called “My Pictures”. Beneath this again, I create sub folders for each year. Images taken in 2007 will be stored in the 2007 folder, 2008 images in the 2008 folder and so on. Underneath the year folder, each shoot is then given a name, generally one which means something to the photographer and associated to the shoot. For instance if you were at a friends wedding, the folder could be called “wedding_eoin”. In the diagram above, I have simply indicated these names as shoot1, shoot2 etc. to demonstrate that each folder should have it’s own unique name.

Within that folder, each file is then assigned a unique name: cwhyte_yymmdd_xxxx The first part of the filename simply identifies the image as belonging to myself. The second part assigns the file name the date which it was taken on, starting with year first, then month and finally day. The last four digits (xxxx) are a count, ensuring each file is unique. I’ve chosen 4 digits, allowing up to 9999 images to be taken on any one day/shoot. The naming of files is performed automatically within Adobe Bridge.

For every image, there will always be at least 3 files. These are the NEF file (Nikon RAW format) and a high resolution JPEG, both of which are copied across from the camera. The third file is the XMP (side car) file which contains the RAW modifications, created by Adobe Bridge. For any images which are edited within Photoshop, there will be an additional two files: a master PSD file (Photoshop format file) and a low resolution JPEG which is used for posting on the web.

The Work Flow – A Brief Overview

The diagram above pretty much summarises my whole flow from start through to finish. The pink boxes are manual tasks performed within the OS. The red boxes, are actions performed within Adobe Bridge – most of which are completely automatic. The single dark blue box is the manual task of editing within Adobe Photoshop (should any images require it) followed by two light blue boxes, which are automatic steps performed using the SyncBackSE tools. Leaving the editing aside, I can process Gigs of data from camera through to archiving in a very short period of time.

Copying Images

Within the operating system, I create a directory/folder within which I’m going to copy the images from my camera to, obeying the rules above. From the camera, I copy the RAW and high resolution JPEG via a USB card reader. With my existing Sandisc Extreme IV 8 Gig card, a full card can be copied within minutes.

Adobe Bridge – Initial steps

Once the files have been copied, I open Adobe Bridge and browse to the newly created directory and newly copied files. The first steps performed in Bridge are all performed automatically. Files are renamed via the “Batch Rename” command, assigned a one star rating and metadata/copyright information written to each image.

• Metadata & Copyright: Every image gets tagged with my name, address, website, contact details and copyright information. This information is stored within the EXIF information in the image file.
• Ratings: Ratings are a special kind of tag, which allow you grade an image from 0 through to 5 stars. They are extremely useful in helping you find images which you really like right through to finding images which should never be shown to the world. As of yet, I have never rated an image to be 5 stars, so 4 is as good as it gets. All images are automatically designated 1 star, which for me is a shot that is exposed correctly, sharp etc. but not necessarily pleasing to the eye. 2 star images, would be images that have some aesthetic appeal. 3 and 4 star images are ones I will edit and most likely go to print with or certainly post on the web. 0 star images, or unrated images, are ones which have some serious technical flaws or have caught a subject blinking, with their eyes half open etc. The most important thing about rating is consistency; it’s crucially important that a 1 star image today wouldn’t be rated as a 3 star image tomorrow. In general for every 4 star image you have, you should have ten 3 star, one hundred 2 star and one thousand 1 star images.

Adobe Bridge – Keywords

Keywords are tags which describe an image and are used later in the flow when finding images. I have a rule, where at least five keywords are associated to each image. For instance, in the image below the keywords are: Colour, Portrait, Street, Dublin, Old, Beard & Smoking

There is some effort involved here. For studio shoots, keywording is a relatively simple task, because the same keywords can be applied to large blocks of images. However, for shoots which involve mutliple topics it can take some time. But what ever time is invested in keywording accurately and comprehensively, is more than returned later on. So stick with it!

Adobe Bridge – Slide Show & Accurate Ratings

At this point, every image has had keywords, copyright information and metadata attached. They have also all been rated one star. Now is the time to review each image in turn and rate it accurately. A shot has to deserve to be moved off one star, so be tough I personally only ever rate images two stars or zeros at this stage; nothing gets rated three or more. I then select all the two star images and repeat the process, promoting the better images to three stars and repeating once again promoting 3 star images to four star, assuming of course images deserve that rating.

Adobe Bridge – Global RAW Adjustments

If required, at this point I can make global RAW adjustments to every image. Typical adjustments would be changes to white balance, exposure etc. There is no replacement for getting this sort of thing right in camera during capture, but if changes need to be applied it’s an autmoatic task. The required adjustments are made to one image and then applied to every other image, or indeed selected images.

Adobe Photoshop – Image Editing

Generally, 3 and 4 star images are edited within Photoshop. Each edit is unique and can range from applying just a few quick tweaks to detailed editing which could take hours. I’m a huge fan of non-destructive editing and each edited image is saved in Photoshops PSD format. This allows me to return to the image at any point and make further modifications or indeed remove some modifications. From this I also save a high resolution JPEG, over writing the one previously there which was copied from camera and a low resolution JPEG which I can use for posting on the internet.

Catalog – Import images and apply catalog sets/collections

At this point, we now import all the high resolution JPEGS into our image catalog – in my case Expression Media. The JPEG’s are keyworded as well as being rated. The images we chose to edit are now also saved as high resolution JPEG’s. Even if our image catalog only contained the above information, we would have a valuable tool for sorting and finding images. For instance, I could sort/find all 3 star monochrome images taken in a certain month with a few mouse clicks. But catalogs also make use of sets or collections. Unlike keywords, which get written to files, sets/collections are like virtual folders where I can store images. For instance, any image which I have had published is contained within a set. The images stay in the same place on disc, but a link to them is included in this collection. It’s a powerful tool, but one that requires manual effort in the creation of these sets. Once again, what ever investment is required to do this, is more than returned in the ability to find and sort images.

Archiving & Storage

After all the hard work in taking the photo in the first place, followed by keywording, rating and editing, it’s now critically important to ensure we backup our images. There are many, many different backup solutions out there, including external hard discs, backup to DVD, tape drives, online backup etc. My choice is one that is scalable (i.e. grows with my data) and where all my images are “live” in that I can access them at all times if required.

At home, I have two identical sized external hard discs which can be accessed both via USB and via a network connection. The network connection is plugged into my wireless router which is also my ADSL/internet modem. This means I can access them via WiFi from anywhere in the house, via USB if I am beside them or over the internet via a VPN connection. Each disc is an exact mirror of the other. I use software called SyncBackSE to synchronise between my laptop and one of the discs and then mirror that disc to the other. The software allows you to create rules which determine how the synchronisation takes place, i.e. what it does if images/files change on one disc versus another, or if they’re deleted etc. Quite simply, any new or modified images on my laptop get copied to the backup disc. If I delete a file on my laptop, it does not get deleted from the archive (this is really important later on). Once I have backed up to one external disc, I then use the software to do a direct mirror.

At this point I have 3 copies of each file. One (the master) is on the laptop. A second (the backup) is on my archive disc and a third (contingency) is on the mirror disc. This may seem a little over kill, but I personally don’t think the price of a hard disc is too much in comparison to the price of losing 1000′s of images. At any point, I can access all 3 files – of course assuming that the hard discs are turned on.

Off-site storage

My paranoia does not end there. As it is now, whilst we have 3 copies of each image, they all reside in more or less one location. If a disaster happens in this location, I lose everything. So I have a fourth disc, stored off-site in my office, which I also sync my laptop to. The same rule applies, where any new or modified images on the laptop get copied to the archive disc. Now we have 4 copies of each image, where the 3 external discs (2 on-site and 1 off-site) are direct mirrors of each other.

Running Out Of Space On The Laptop

When I begin to run out of space on the laptop, the value of a digital asset management system really kicks in. I could search for all the 0 star images and delete them. Note that they are only deleted from the laptop (live/master) disc, but are still on the archive and mirror. Next time I sync, as mentioned above, it does not delete the images from the archive. I could choose to delete all the files from last year, all the PSD files (which are generally quite large). The point is, Adobe Bridge finds what ever files we want, using the criteria (ratings, date taken, key words, size etc.) which we applied to the images earlier. We can do what ever we like to these images, as there is already a safe copy.

Running Out Of Space On The Archive Discs

The system has to be scaleable, so what happens when the archive discs begin to run out of space? In the diagram above, the top diagram shows the external off-site disc on the left and two mirrored external discs (on-site) on the right. In this case they are blue (hopefully so you can see how the stack grows). When the blue discs finally run out of space, I buy two more new mirrored discs – in this case green. I bring one of the original blue mirrored discs into the office, for safe storage. The original offsite disc on the left (grey) is now identical to the blue disc from home. I format the grey disc and this is now ready for new images.

At this point, at home I have a blue disc, an archive of images (let’s say from 2006 through to 2008). I also have two blank green discs, ready for any new archives I make. These will be exact mirrors of each other like the blue ones before and in work, I have the original disc for off site storage.

Once the green ones run out of capacity, the process is repeated. Two new (red) discs are purchased. One of the green discs is moved to the office and the original office disc is once again formatted ready for use. At all times, no matter where I am (office, home, internet) I can access every image I have ever taken and most importantly there is always at least 2 copies (in most case 3) of every image I own.

The End!

That’s about it! If you’ve never implemented any sort of asset management system, or workflow, this probably all seems massively daunting. It really isn’t! What is daunting, is where you leave it very late in the day to implement a system and then try and add historical images into that system. I can do a shoot of 200~300 images and have them all the way through the flow to archiving in less than a few hours. However, it took me months to go back over all the images I had ever taken before implementing this system. Of course, I could have just decided to start at a certain point and just implement the system on any new images I take, but I opted to put the very painful work in and rate, keyword, rename, archive all my previous work. Now, it’s certainly worth it.