Rsync query...with a twist

JKD · 3 May 2012 at 12:55

Hi all,

On my iMac I have around 200 gigs of digital camera movies, pictures, jpegs and raw files under ~/Pictures and various subdirectories. Many of these sub directories under my Pictures folder have spaces in.

What I would like to do is find all jpg files, mov, NEF and RW2 files in these ~/Pictures/sub directory structure and copy them onto my external drive /Volumes/MyBook/Backup. I do not want to preserve the directory structure on the target directory, I want to end up with:

/Volumes/MyBook/Backup/1.jpg
/Volumes/MyBook/Backup/2.jpg
/Volumes/MyBook/Backup/1.NEF

and so on.

Can I use rsync to do this? I seem to be struggling to meet my requirement of no sub directories on the target drive/folder.

Any help would be much appreciated.

Background to this is that I have been using Lightroom and Aperture, so the folder structure is a mess and I have duplication. I want to migrate to the new drive and start a fresh catalog.

shine · 3 May 2012 at 12:59

You should be able to use the find command to do this. This tutorial should send you in the right direction:

http://www.ibm.com/developerworks/aix/library/au-unix-find.html

JKD · 3 May 2012 at 14:01

shine said:
You should be able to use the find command to do this. This tutorial should send you in the right direction:

http://www.ibm.com/developerworks/aix/library/au-unix-find.html

Thanks. I've been messing about with find this morning, and found that I could get most of the files copied across as I wished. However, a good few GB never copied across. It wasn't to do with spaces in the file/path names as I'd handled those. I think I've got a wedge of files knocking about with dodgy hidden characters in there.

Strangely, rsync copies them across fine, just with the bloody directory names in there all the time! Aargh.

shine · 3 May 2012 at 14:02

Towards the bottom of the tutorial I linked earlier it talks about handling file names with dodgy characters with find.

JKD · 3 May 2012 at 14:04

Yeah I know, I need more typing/debug time. Certainly that's the best tutorial I've come across though. Thank you

JKD · 3 May 2012 at 15:07

Think I have it cracked. This is what I'm doing:

find . -path './Aperture Library.aplibrary' -prune -o -name "*.jpg" -exec sudo cp '{}' $destDir \;
find . -path './Aperture Library.aplibrary' -prune -o -name "*.mov" -exec sudo cp '{}' $destDir \;
find . -path './Aperture Library.aplibrary' -prune -o -name "*.NEF" -exec sudo cp '{}' $destDir \;
find . -path './Aperture Library.aplibrary' -prune -o -name "*.RW2" -exec sudo cp '{}' $destDir \;
find . -path './Aperture Library.aplibrary' -prune -o -name "*.MTS" -exec sudo cp '{}' $destDir \;

There's quite a lot of data going over, but no errors as yet. I'm not going into the aperture library as there's billions of duplicate previews in there.

JKD · 3 May 2012 at 15:13

^^ Maybe not. All finished after transferring only 5GB - still no errors though. Any ideas what I've done wrong, anyone?

Edit : sorted. Moronic file naming issue.

meghatronic · 5 May 2012 at 23:24

Incidentally, you can concatenate all those commands:

find . $ -name "*.jpg" -or -name "*.mov" -or -name "*.NEF" -or -name "*.RW2" -or -name "*.MTS" $ -exec rsync {} $destDir \;