RAW File Storage (1 Viewer)

I’m looking for advice on how to store and archive digital image RAW files.

Over the last three or so months I have been using my Mac Book for processing and file storage of circa two thousand or so RAW files which I note are occupying some 50 MB of the hard drive. DPP is indicating each image file is circa 24MB, at this rate the 500GB hard drive will be full in a very short time

Currently I’m shooting in RAW, download image files into DPP, edit where required and convert the image to Jpeg for web publication

Should I be be considering off loading the file storage to stand alone hard drive and deleting from the Mac Book, maybe reducing/converting the RAW file size. Am I going about this in the correct way?

Any suggestions would be appreciated..

Thanks Paul S

The first question is why are you hoarding all these images? If these genuinely are all your "keepers" then if you are happy with the post processing then you could keep them as a full size jpg.

If you genuinely have thousands of files that are keepers and that you may wish the revisit their processing then I would definitely store them on an external hard drive you can get 2TB for £70. You could zip them, most RAW files will reduce by 50% but if you are going to carry on accumulating at your present rate then you would only delay the switch to external.

Thanks Mono

Maybe I need to discarding more simular RAW images and probably more ruthless in classifying reject images and discarding.

Do you have a structured approach you could recommend I should adopted for file storage, would you advocate deleting all but a few selected RAW files once I am happy with the post production?

As a hobbyist bird photographer I'm a little concerned at my present skill standard I'm going OTT for what I need, and I'm highly unlikely to modify or edit the RAW files in the future

It's a big leaning curve !!

Thanks

Bafty said:

I’m looking for advice on how to store and archive digital image RAW files.

Over the last three or so months I have been using my Mac Book for processing and file storage of circa two thousand or so RAW files which I note are occupying some 50 MB of the hard drive. DPP is indicating each image file is circa 24MB, at this rate the 500GB hard drive will be full in a very short time

Currently I’m shooting in RAW, download image files into DPP, edit and where required and convert the image to Jpeg for web publication

Should I be be considering off loading the file storage to stand alone hard drive and deleting from the Mac Book, maybe reducing/converting the RAW file size. Am I going about this in the correct way?

Any suggestions would be appreciated..

Thanks Paul S

Click to expand...

Think that technology is your friend here.
There is no need to ever delete anything. A 4TB drive costs $US 100. That should hold all your images for a little while. If it runs short, add another.
Life is too short to waste time on pruning, just select your best shots to enjoy. The rest can stay dormant until you decide to look further.

You're not going to like my answer:

We don't hoard per se, but we often take more photos than we can process right-away. To the tune of 1,000 to 3,000 photos every time we go for a couple hours walk with the intent of "birding." This quickly started to overwhelm even my 2 TB of external firewire-based storage space (I'm in design, so you get used to have lots of storage at-hand dealing with Photoshop).

Every year, I go through the hoard and get rid of all the un-processed images, but again this takes time and we seem to break even at best, slip slowly behind at worst.

And this is mostly shooting in high-res JPEG and dealing with PSD files as final files. But a layered PSD file with multiple Topaz filters applied can get pretty large in it's own right.

So I decided to upgrade our infrastructure to support our hobby: I now have two 16 TB Synology NAS setups (4 x 4 TB drives with one parity drive each), which yields about 11 TB usable storage each. One is at the house, the other is off -site and brought home quarterly to backup the primary.

They are both Synology DS1515+ if you're curious, using Western Digital "red class" drives.

This should future-proof us for many years. At least until I get my GF's Mac set-up so she can finally wade through her backlog. |8)| It also has the side-effect of allowing us to share music with each other and with the Playstation 3, and eventually we'll be able to share our photos with the outside world via the NAS instead of flickr and other "someone else controls your content" systems. So a good investment.

Right now, the NAS is about 25% full between our photos and mundane things like computer backups (TimeMachine) and network-based music sharing.

Such is our digital world. Used to be we put photo albums in fireproof safes to be sure, now it's "protect the hard drives like your life depends on it"...because in some cases it does...your whole "life" is on that drive.

I would also strongly recommend using an external hard drive. But you should also think about back-ups - what happens if that one hard drive dies? There are several options, either back up everything to two hard drives, or consider a RAID.

Also you need to start thinking about how you are going to administer the images you are going to accrue. Do you just want to store them by date, and will you be able to find what you want later, or do you need to look at software that makes this easier?

I'm with Kevin on the `don't be hasty in deleting' issue. A lot of my pictures are taken on holiday to places I may not return to, so they have sentimental value that goes beyond their quality as a photo of some subject. I should go back though and delete those that clearly are not even good for reviving memories.

Andrea

There is a difference between keeping/hoarding a jpg to enable you to relive the memories and keeping the RAW file to enable future post processing.

Once I am happy with the image I will save a full size jpg and delete the RAW. It can be the curse of the amateur to keep fettling away to try to get the perfect image, (which is fine if you enjoy it!).

Paul,

I would agree with Andrea, Kevin and others but would also add that what you choose to keep may depend on whether you are primarily a photographer of Birds & Nature and want optimum results, or whether like me you are first a birder who happens to like capturing images and memories along the way?
I am guilty of keeping far too many RAW images, many of which are not really worthy of keeping for their IQ but also I can't necessarily be bothered to convert all to JPGs which would make sense so I could then delete the RAW versions.
For me, a poor record image taken on a memorable day or a memorable holiday of something I'll probably not see again, is just as important for my own browsing as would be an image I would process for sharing online.
There are also examples where my skill and expectations have moved on and a few images I'd processed some years back have now been revisited using later software & original RAW capture to give improved results.

I tend to:

• Go through the hundreds of images for an outing (high-res JPEGs or RAW).
• Saved the "keepers" to PSD
• Covert the original image to a Smart Object
• Process the image
• Create derivatives: for desktop images, Birdforum, etc.
• Save the "master" PSD as well as the derivatives

I always keep the "untouched" JPEG/RAW from the camera so it can be re-processed in the future if my skills increase, some magical Photoshop plugin comes out, or I need a different resolution/use.

These stay on my Mac along with the ones I didn't process. About once a year, sometimes more, I go through any sets I didn't go-back to for some reason and delete all the JPG/RAW original files I didn't use. The rest get moved to the NAS.

I realized when re-reading my post that you might ask "why a NAS and not just an external hard drive?"

One is that I've done the "keep buying more and more external Firewire drives as you need them" but since I back-up everything, that meant buying two drives every time and that starts to become a clutter and cabling nightmare. At one point I had two internal drives (Mac Pro) and four external Firewire drives. And I also realized a backup right next to the originals was great if a drive fails, but not so smart for disaster recovery.

Second is my GF and I wanted to be able to share files easier and have separate space to work on, so that would mean having to buy even more drives just for her computer. At some point the expense of a NAS (Network Attached Storage) made more sense; it neatly consolidates our massive storage needs. And most consumer and small office NAS have alot of other cool household and external-facing sharing abilities beyond just being "a huge RAID on the network."

Even a NAS with a parity-drive setup, with a good network (hard-wire, not wireless), can handle even doing Photoshop work right off the drive over the network. Me being a designer and a computer geek though, I have plans for implementing Link Aggregation to further improve speeds; suspect that is overkill for most of you. ||