iTunes Dupe Wrangling

Using a music management program like iTunes can be very useful when keeping a large collection in order. It can rename files, keep them organised in a decent directory structure and let you browse and edit the tags with ease. Perfect, as long as the songs you import are all properly tagged and have no duplicates. If anything is wrong, though, it can be a royal pain in the posterior.

When creating an iTunes library out of an existing music collection, it is often the case that an existing set of MP3s has to be imported into an existing iTunes library. In this case, many terrible things can happen, including duplicates and MP3s with subtly different (and sometimes incorrect) tags.

Enter Dupin, from Doug's AppleScripts. Dupin is a handy iTunes duplicate finder and remover. This is not a free tool, but if you're dealing with several thousand MP3s and don't fancy the agony of the iTunes dedupe tool, it's well worth a tenner.

The key to effectively and quickly importing an old library into a newer iTunes library is to turn off the "Copy files to iTunes Media folder when adding to library" option in the iTunes advanced settings. This will leave your MP3s where they are, but add them into the iTunes interface. Don't worry, though, if you prefer them all together. We'll get to that.

Once that option is disabled, drag the folder containing all the old MP3s into the iTunes window. Leave it for a bit, and before you know it, all the MP3s will show up. Some might be duplicates, some might just have crap for tags, but they will be there. Have a quick look through to see if there are any glaring errors in the import, and fire up Dupin.

You can match several different tags to consider files as duplicate. If you're lucky and have properly tagged all the old files, checking the Title, Album, Artist and Track Number fields should give you a nice list of tracks with their duplicates. Click the Filter button to get the list of ways to filter out the duplicates.

This is where disabling the copy option comes in. Simply select the Folder Location radio button, and leave the folder as your current library location. Click the Filter button and all the duplicates that are not already in your library will be marked for removal. This means that there is less to copy in when you consolidate your library later; only files you genuinely did not have will be copied.

To actually remove the files, select the Purge option from the Tools menu. Thanks to the MP3s still being in their original folder, you can opt to not send the files to the Trash without fear of cluttering your iTunes library folder with unused rubbish. After a little while, all the dupes will be removed from iTunes and all should be well.

You might want to run Dupin again, this time without the Track Number match. I have found that many old MP3s do not have a valid Track Number ID3 tag, and so show up without one in iTunes. This causes the Track Number match option to ignore them as dupes. Again, using the Folder Location filter should work the magic.

Now that's done, click File -> Library -> Organize Library. Opt to Consolidate your library and click OK. All the new files will then be copied and properly named in your iTunes library folder. Once that's done, you can be cavalier and delete the old MP3 folder, or you can give it a once over just to be sure it's all there. Either way, your libraries are merged and the duplicates are removed. Anything left is up to you to sort out!