Leaving Flickr - How to download all your photos

CC-BY: Dazzie D

Your "Pro" membership might be expiring, you may be nearing the 200 photo limit for free account or recurring rumors of a Yahoo!-Microsoft merger might be prompting you to consider alternative services. Either way - odd are that Flickr has all your precious photos, and you want them back! This is how I did it:

First, I'll warn you that this guide may get a bit technical, as it includes a little command line work, if you're comfortable with that - read on.

The backstory
I've been using Flickr for almost four years, and shortly after joining, I decided to upgrade to a pro status so my account could double as an image backup solution. Since then, I got my first SLR, and by 2009, I was shooting only RAW. Flickr's poor RAW support plus the fact that restoring anything larger than one photo set from Flickr would not be a simple matter - both prompted me to find an alternative backup solution.
Today, I backup my whole photo library to both a local mirror drive, and crashplan+ - a remote online backup service.
Once I no longer needed a Flickr Pro account for backup purposes, 25$ a year became rather pricey for the questionable privilege of  "Access to your original files" and "Unlimited sets and collections" - seriously Flickr, it's my data! let me have it back!

All that remained to be done before I could let my Pro account expire was to download my photostream, and make sure that I indeed had local copies of all my Flickr photos.

Downloading the photostream

CC-BY-SA: rbrwr

There are quite a few programs out there that can help you download full resolution imgaes from your Flickr account, to name a few: FlickrEdit, Downloadr the for-pay Bulkr and the now banned Flickrdown.
If you only have about 1,000 photos to back up - FlickrEdit is your best choice, it has a user friendly graphic interface, and works rather well.

If, on the other hand, you have thousands of photos to download (over 33,000 in my case) - your best bet it FlickrTouchr. This python script will log on to your Flickr account, and download all your images, into folders based on your Flickr sets!
The original script was set to download "Large" size images rather than originals, and seemed a bit slow on my connection, so I modified to to download originals and... (drumroll) ... use concurrent threaded downloads!

You can download my modified version - FlickrTouchrThreaded. To use it, you'll have Python installed. If your using Linux, you probably know how to do this yourself, Mac users should already have python installed by default, and windows users can find installation instructions here.
Once you have Python installed, backing up your entire photostream is a easy as typing:

python flickrtouchrthreaded.py FlickrBackupFolder 3

 Where "FlickrBackupFolder" is the name of the folder you want to back up to, followed by the number of download threads you want to run (in this case - three). This will begin downloading you whole photostream to the your selected backup folder, with sub folders based on your Flickr sets. Images not in sets will be downloaded to a "No Set" folder.

In my case I found that using more than 3 threads didn't improve my overall download speed, you can play around with the number of threads the script uses, but I wouldn't increase it over 10 so Flickr doesn't get too mad at you :)

The first time you run it, it will send you to the Flickr website and request authorization for your Flickr account, subsequent runs will start downloading immediately. You'll also note that if the script was interrupted, you can just re-run it, and it will pick up where it left off, just be sure to point it to the same folder.

A few hours later (or in my case, one week 33,640 photos and about 80GB later) you'll have your entire Flickr photostream backed up.

Finding Corrupt Images

Photo: Boaz Arad

Once you've got your whole photostream backed up, you'll want to be sure that all your images downloaded correctly. FlickrToucher does a rather good job, and will rarely corrupt any images in transit, but even one corrupt photo out of thousands could really ruin your day - especially if it happens to be your best wedding picture, or the only group of your speedskating group (see left).

To scan for corrupt JPG images, i used cPicture.
cPicture is a commercial product ($19.99 per license), but the freely available trial will allow you so scan for corrupt images without limitation. Just pick a folder, and click "Check Pictures". After churning through your photos, it will output a text file with a list of corrupt images.
If you used FlickrTouchr to download your photostream, the filenames of the images should be their Flickr photo-id's - so you can easily find them online by browsing to

http://www.flickr.com/photos/your_flickr_username/photo_id/

If anyone can reccomend a similar utility for Max and Linux users, I'd love to hear from you in the comments.

Finding duplicate photos

Photo: Boaz Arad

Some of the downloaded photos from your photostream will no doubt be duplicates of photos you already have on your hard-drive. In order to prevent wasted space - you should scan the downloaded photos for duplicates. I recommend MindGem's Fast Duplicate File Finder (FDFF) - again, a commercial product, but the trial version likely contains all the features you'll need.

Conclusion
Once you've weeded out corrupt and duplicate images, it's time to merge the backup back into your local photo library. Both Google's (free) Picasa and Adobe's (pricy!) Lightroom do a great job of keeping you pictures in order, and I recommend using one or both regularly.
It's also a good practice to always keep local copies of all your images, and never relay on online services (Flickr, Facebook etc.) as your only photo storage solution - as these can be notoriously unreliable.