Jun 24 2012
Using Automator with Calibre to Automate Tasks
I saw a post the other day about using Hazel + Calibre to automatically import your Kindle files into Calibre. While the user did this to automatically strip the DRM from the books, the tutorial is smart for any one who wants to create an ongoing catalog of their purchases. It was the first time I had heard about Hazel, a low priced software that serves as a macro utility. A macro utility essentially records a number of repetitive steps to complete a task. In the future, rather than executing each step individually, you simply run the macro.
But MAC Operating Systems have a macro utility built right in. It’s called “Automator“. I’ve just started researching Automator and I’m blown away at what it can be set up to do for you. These types of programs really are harnessing the power of your computer and making the device work for you.
I’ve set up Automator to “watch” the folders where I download my ebooks – either in epub, pdf, or Kindle – and move them into a separate folder. From there, every time a book is added to that folder, Calibre knows to add it into my Calibre library, will delete the book (to avoid duplicates) and if it is DRM free, Calibre will convert it into my preferred format.  The only thing I do to set this Rube Goldberg-esque sequence into motion is to download a new book using either Adobe Digital Editions or Kindle for Mac. The rest of it happens automatically. Onto the Tutorial.
To do this, open your Kindle for Mac program. Go to Kindle on the menu bar and selectPreferences. In the second section there is an option to change the content folder.
Step 1. Create a Folder Action
Click for larger image
Open Automator and if you don’t get the above screen, type command + N for “New”. Selection “Folder Action”. This is because you want the “workflow” (or series of commands) to run when a new file or folder is created in your ADE/Reader/Kindle ebook folders.
Step 2. Choose the Folder to attach your action.
Click for larger image
This is where you will browse for the folder where your ADE books or Kindle books are downloaded. Generally, unless you have changed this, it should be in your “Documents” folder. ADE stores the books in a folder called My Digital Editions. Sony in a folder called “My Books/Reader”. Kindle books may be stored in My Kindle folder but you may also find them in /Users/yourcomputername/Library/Application Support/Kindle/My Kindle Content.
Step 3. Start your workflow by selecting from the Files/Folders Actions.
click for larger image
We are only going to be using the Files/Folders Actions. On the left side there are two choices: Actions and Variables. Click on Actions and then drill down to Files/Folders. This way you will only see the actions related to Files/Folders in the column to the right.
Step 4. Select “Get Specified Finder Items” and then drag it onto the workflow space to the left.
click for larger image
Press “Add” and then browse to the folder where your ebooks are downloaded. (Step 2 folder). This command tells the computer to get all the items in the finder folder.
Step 5. Select “Filter finder items.”
click for larger image
We don’t want to select everything in the folder, only files that are actually ebooks. In the Reader or Adobe Digital Editions folder, you want to copy files that end with epub or pdf. If you are doing Kindle, you would want to move only those files that contain azw. It’s important that your middle column is “contains” or “begins with” and not “is”. “Is” requires an exact match. Sometimes AZW files are azw3 or azw4. A command that says “file extension is azw” would ignore files that have azw3 or azw4.
Step 6. “Get Folder Contents”
Click for larger image
I did not include this step my first time I created the script and it didn’t work right. I guess this command tells the computer to gather up your “filtered” files.
Step 7. Copy Finder Items.
click for larger image
This last step copies the filtered file items to another folder. I’ve created another folder (“autoadbooks”) that I have set Calibre to watch. Now save the automator script and close the program. You are done with Automator.
Step 8. Set Calibre to Auto Add books
Under preferences – > Add, Calibre has an option to autoadd books to the program. Once Calibre auto adds a book, it will delete the book file. This is why you use the copy feature in Step 7 instead of the move feature.
click for larger image
Note that you only check the boxes under “ignore files with the following extensions when automatically adding” when you want to exclude the files. I stupidly checked epub/pdf/azw without paying attention to the instructions and wondered for hours why Calibre’s autoadd didn’t work. Thankfully I figured it out before I posted a panicked message at MobileReads.
Restart Calibre and start enjoying your automatic goodness.
Other interesting uses of Hazel and books
- This guy uses Hazel + Dropbox + Calibre to convert epubs to mobis automatically.
- This guy writes about using Hazel + Automator to get articles from Instapaper to his Kindle.
Send to Kindle
Jun 24, 2012 @ 07:19:38
This is cool…. I use Calibre’s inbuilt feature to “watch” a folder for books and automatically add when new books are added to that folder, but this one also adds the auto-convert feature which is nice.
Jun 24, 2012 @ 07:34:01
Okay, this is like magic. I’d been despairing of ever having a backup of my Kindle books because there were just too many. But this…this worked on the first try. Thank you!!
Jun 24, 2012 @ 08:13:27
This is so cool but I’ll have to wait for the PC version :-(
Jun 24, 2012 @ 08:25:19
Thanks so much for this!
I’ve done everything above, but I’m not sure how you get the automate script to work? Do you click run once you’ve entered everything?
Jun 24, 2012 @ 08:30:33
This is an excellent post, thank you. I hated having to download my kindle books as it would always accumulate duplicates. Also, did I understand you correctly that there’s a way of stripping DRM from ebooks? I know I can covert formats in Calibre, but can’t if they have DRM.
Jun 24, 2012 @ 08:31:15
The script is triggered when a new file is added to the folder. So if you add a new ePub PDF or Kindle file, the copy script runs. When you open Calibre the auto add script runs.
Jun 24, 2012 @ 08:32:38
@Carolyn @ Book Chick City: Yes there are plugins you can add to calibre but I’m not sure the legality of them so I can’t link to them.
Jun 24, 2012 @ 08:54:00
That’s great, thanks Jane. I’m off to have a little Google ;)
Jun 24, 2012 @ 09:14:06
Thanks so much! I only just installed Calibre a little while ago, and haven’t really started using it much yet. This, however will really help, as I can make sure things happen automatically, and I need to spend very little time to set it up.
Jun 24, 2012 @ 10:30:45
Damn it all, I couldn’t do it. My Mac is too old! *sob* (I didn’t have “file extension” in step 5.)
Jun 24, 2012 @ 12:37:44
Jane, you are a genius as usual! Thanks again for the great tech advice. I’ll have to think this through a bit for my set up. I download everything to Downloads, upload to Calibre, then move the original book files to a new folder where I store all my downloads (I am a crazy backup worrier). So it sounds like this will work for me if I modify my process a bit.
Thanks for the great tip!
Jun 24, 2012 @ 18:23:27
This is so awesome!
I’m going to add something for Mac users on Lion: Lion automatically hides the “Library” folder in your user directory, and so it may be hard to navigate to it in Automator. You can fix this by hitting command-shift-period.
Jun 24, 2012 @ 19:00:09
Okay, I ran into another annoying Automator problem.
Lion has also automatically removed items from “Library” from its spotlighting/indexing facility, and apparently Automator’s Filter files bit relies on filtering things through the index. (I think if you upgraded to Lion, you might not have this problem–it’s only the computers that came with it native that do this.)
So if your version of OS X is as broken as mine is, the filter suggested returns no results.
I ended up using automator to run a shell script instead:
/bin/cp /Users/[USERNAME]/Library/Application\ Support/Kindle/My\ Kindle\ Content/*.azw* /Users/katniss/Documents/autoadd
/bin/cp /Users/[USERNAME]/Library/Application\ Support/Kindle/My\ Kindle\ Content/*.mobi* /Users/katniss/Documents/autoadd
And that did it for me.
Jun 24, 2012 @ 19:50:24
@Courtney Milan: Weird because my computer came native with Lion and I didn’t have those problems this morning! (and now I’m very very thankful!)
Jun 24, 2012 @ 21:55:21
Jane, out of curiosity, is it possible to set parameters so it only takes action if a book is downloaded FROM a certain place, in addition to being downloaded TO a certain folder? I am wondering b/c about 90% of what I read comes via email from authors/publishers for review. So I download it from gmail into a downloads folder. I also download other stuff that comes in to that same folder (via mac mail for work, etc). Just wondering if there was a way to control it so it only looked for pdfs downloaded from gmail for example. Otherwise it means some weeding out of stuff that isn’t books but is still a pdf.
I still think in the long run it is faster to automate it even if I have to pull out things that aren’t actually books periodically. Just curious though if it could be fine tuned to that degree. Thanks!
Jun 24, 2012 @ 21:59:55
@Jayhay no I don’t think that there is a way to automate based on where you download a book. You may be able to automate something with your email attachments if most books arrive via email. The other option is to set your browser so that you are asked where to download the file. Then you can direct the download of books to a specific folder.
Jun 25, 2012 @ 01:22:35
@Angela James: Then I really don’t know what it was!
All I know is that it worked on my desktop (not native Lion) but not on my laptop (is native Lion), so I assumed that was it. Maybe there’s some other setting that needs to be toggled to make it all work out, and I just don’t know what that is.
Whatever it is, if I left the filtration settings blank, it would capture the contents of my kindle folder–but if I added a filter (and I checked the settings 100 times) with any kind of filter in it (for instance “size > 3 KB”) it would shoot back no results. So weird and frustrating.
Jun 25, 2012 @ 02:29:26
Thanks so much for this post. I’m working with Macs dor years and didn’t even know Automator. And in combination with calibre this will save so much time and I won’t loose any books anymore. Absolutely awesome :)
Jun 25, 2012 @ 11:20:20
@Jane: Thanks Jane! I’ll think it through a bit. Either way it works it will be a big help. Now I am trying to think of all the other things I can automate!
Jun 25, 2012 @ 17:49:04
@Jane, one more quick question. How are you setting Calibre to convert your file formats for you? I am talking about DRM free books here. Is there a step somewhere that I can have it convert my mobi files to epub rather than doing it manually (or waiting to be prompted when I load it onto my reader)?
Jun 25, 2012 @ 19:56:04
I’m going to add one thing. Using my shell script above has the added negative that it copies all files, all the time, and that confuses Calibre when it adds them, and plus, slows down my computer. So, not a good idea. I disrecommend that as an option.
The easy fix to the above is not all my rigmarole with the finder and the cp, but if for some reason spotlight isn’t indexing the folder because it’s in the hidden systems library (as mine was), just go into Kindle Preferences and change your content folder to some visible place on your hard drive.
I also had to swap steps 5 and 6 above–put “Get Folder Contents” before “Filter Finder Items” to make it work.
From doing a bit of searching, what I can tell is that the finder search in Mac OS is kind of broken, and generates weird errors–so I’m including this in case someone wants to make it work and their computer is acting like mine.
Thanks, Jane–having an autoarchive is a huge relief for me.
Jun 25, 2012 @ 20:22:19
@jayhjay: In the auto add section of Calibre you can check to have the book converted when Calibre auto adds it. To set your preferred conversion format go to Behavior -> Preferred Output format.
@Courtney Milan: Thanks for clarifying this for us. I put a shortcut for my Kindle Content folder in the sidebar and then used that to find the folder. I don’t know if that helps or not.
Oct 25, 2012 @ 01:14:59
Thanks for this! I’ve always wondered about Automator but was too intimidated to use it.
I know you went through this forever ago, but I was wondering if you know if there’s a way to set Calibre to auto-add to different libraries? I set up my Kindle books in an entirely separate library to be able to keep track of my purchases better so when it auto-adds from the Kindle I want it to go to that library but I can’t find a way to specify. Seems they’ll all go to default.
Oct 25, 2012 @ 01:22:20
@Alicia: Not that I know of. I think it auto adds to whatever library is open, but if you select the book you can right click and copy to your chosen library without switching. (and delete after copying)
Oct 25, 2012 @ 05:53:08
@Jane: Darn. I did find the transfer option, so that’s good, thanks! I was just hoping for an automatic method because I know at some point I’m going to forget what books I need to move over.