Blog Post

Workflow!

Hello all,

I received many great comments about workflow on my post First Forays so I decided to start up a new conversation on efficiency and research workflow. I'll start with a little profile on the type of researcher I am and the kinds of research I do and then I'll go into my current workflow and finally what works and what doesn't. I would love to hear from you on tools you've used or are currently using that have made a difference (great or small) in your research process.

Profile:

I'm a third year graduate student in French Literature so I work with A LOT of pdfs (who doesn't): articles, scanned novels etc.

My work is interdisciplinary, somewhere at the intersection of French Studies, African American and Diaspora Studies, Literature, HIstory and Second Language Acquisition and Pedagogy. This makes organizing sources a challenge especially with overlapping sources. I don't want duplicate files on my computer.

I am currently compiling my reading list and preparing for comps as well as putting my dissertation proposal together. It feels like a good time to think carefully about how I want to organize the tons of material I will be using in the next couple of years to make the writing process less er, nightmarish.

Tools I Use:

My iPad: for handwritting notes, reading books (mostly Kindle) and annotating pdfs. All my handwritten notes are backed up as pdfs, and when I find a quote I think I'll need I'll type it up instead so that it's searchable. I use Notability for note-taking and recording, and Goodreader for pdfs.

Dropbox: I keep all my pdfs here in carefully organized folders, but this is a little challenging because of overlapping themes and the fact that this has been a manual process. I wish I could just dump all my pdfs into one giant folder and have some program that sorts it all out. I sync dropbox with goodreader because of Zotero.

Zotero: to manage my bibliography. Each reference has a link that takes me to the stored pdf in dropbox, and I always have the most recently modified version because dropbox syncs to goodreader where I do annotations.

How to make this better:

Working with Zotero feels like I have to do more things manually than I would like to. For example assigning tags or deciding which folder/library a given document should go into. I then have to make sure the given pdf is also in an approrpiately named folder in dropbox. On the ipad I have to tap "sync" everytime I want goodreader to sync with dropbox (for example after annotating a pdf) and it's such a clumsy process because it goes ahead and syncs ALL 1 billion pdfs everytime.

So it sounds like DevonThink would be great for managing my huge pdf library. For the DT users out there, please tell me this:

1. How would I use DT with my ipad? This is make or break for me. I've read nothing but horrible reviews of DevonThinkToGo (the ipad app). I want to be able to annotate on the ipad and see my annotations on the file in DT on my computer.

2. Does DT do citations (like Zotero or Mendeley) or is it only a database? I would hate to have to use DT + Zotero or some other citation manager...that would be too much for my little brain. Or perhaps is Sente that magic tool that will do both database and citations?

3. Does DT do OCR? I have a lot of scanned out of print texts that I would LOVE to make searchable. In fact if I could make everything in the entire world searchable (google?) that would be divine and it's a little annoying having to go to yet another tool for OCR (I use PdfPenPro but it's hit or miss).

4. Why on earth is DT so expensive!?!

 

If you use any other research tools please feel free to join the conversation. I've decided on Scrivener for writing and TimeOut to remind me to take regular breaks...my next one is.........now.

Next post: ipads and other fun tech tools for teaching. Stay tuned.

51

11 comments

I found CloudOn to be a good (free) way to edit docs with Dropbox integration on the iPad.  I have been also recommended the program Docear.  I hadn't heard of it but have a feeling it will be super useful for organizing pdfs.  Not sure if it fits your requirement for managing pdfs but maybe someone else will find it useful.

56

Thanks for the recommendations Andrew! I do use CloudOn for making typed changes to word documents and it's great! It's good for giving students feedback on drafts and tracking the way they integrate corrections (mostly grammar) into their revised paper. I'll add this to my list of teaching apps for the next blog post.

For research purposes I use the ipad to handwrite notes so CloudOn isn't quite useful in that regard.

Andrew how do you organize your pdfs?

47

I have only started looking at ways to organize PDFs so I'm not sure what the best workflow is.  My wife has been excited about Docear for her work and so I'm looking into it.  I tried using I, Librarian but the setup was quite intensive, and while it seemed promising for automatically looking up titles and annotations, it returned maybe 3 out of 100 titles.  So that was a bit disappointing.  I'll gladly report back when I've worked out a system.

64

Thanks Andrew. Looking forward to hearing what you come up with. I've found many of these programs to have a steep learning curve (some steeper than others) but given the nature of academic research I think it'll pay off in the end.

56

Hello all, 

I've been using iAnnotate for about a year now and highly recommend it. It is a little pricey, at $9.99, but I think well worth it because of the range of features (highlighting, typed comments, the ability to aggregate and send all of my annotations via e-mail). 

But my big problem has always been syncing my PDFs across different devices (home computer, iPad, computers on campus, online access in general). I've lately discovered that iAnnotate can sync with my Dropbox account. That way, I can read and annotate a PDF and then upload the annotated version to replace the older version held in my Dropbox account, where I've already used the Windows interface to organize all my PDFs. I think this may work differently than Goodreader, Annette, because iAnnotate can sync a single PDF file at a time. 

As far as I know, there still isn't any single way to integrate Dropbox, Zotero, and a PDF markup program with a single click or two, is there?

A side note about Zotero, their forums and Twitter handle are all extremely helpful for sorting some of these issues out.

72

Jim your workflow sounds closest to mine at the moment but I use Goodreader where you use iAnnotate. I'll look it up especially if the syncing process is less clumsy then that would be excellent! I think the step I'm trying to get rid of is having to organize my pdfs in folders on my computer. I never know which folder to put which article because my work is so interdisciplinary. I suppose maybe that's a small detail, but then I have to make the same calculation for Zotero i.e where do I classify a given reference so that building a works cited list at the end of a chapter is a breeze. I've been looking at Papers which seems to do everything: organization + citations but I'm undecided.

I sound slightly OCD but I would quite like to lock something down now because I anticipate using these articles for years to come and it would be great to have an efficient system early on so that I can get to the business of writing quicker and easier.

Jim could you describe your Windows interface organization process? Do you use tagging, folders labelled by author or some other system?

67

I like to use Windows (though, OS would be same) because I find it easier to move things around than on the iPad. What I do is: 

1) Download a file to my computer.

2) Place the file into the necessary folder in my Dropbox.

3) Go into iAnnotate on my iPad and download the folder to my iPad from my Dropbox account. 

4) Use the file, reading or annotating or whatever I need. 

5) Hit the sync button in iAnnotate to upload the revised file back to my Dropbox account. 

As for creating my own personal metadata, I've really done my best to under-think it and toss stuff into folders for either a research project, (if I'm already using the files for something specific) or based on the file's source (i.e. an article from  Book History goes into a folder titled Book History.) 

I've been tempted to keep an analog record in the past, just so I can see more things in less mediated ways. Let's face it, we've had more time to figure out how to use paper than we have iPads. Maybe a binder of Zotero printouts might not the worst thing in the world. Any ideas for better / easier ways to create personal metadata?

49

This is pretty much what I do too although I think I'm definitely overthinking things at this point. Tagging in Zotero is a pretty nice way of being able to find things easily. I make sure that all my material is OCR'ed and therefore searchable so that spotlight returns as many relevant items as possible.

47

 

Hi Annette and all,
 
Thanks for starting this conversation. I'm a workflow nut, and have used pretty much all of the tools that we're discussing here. Let me start by saying DevonThink Pro Office is a super powerful tool, which is probably why it is so expensive: it does OCR to make PDFs searchable, and can index and cross-refernece files, but I've actually stopped using it for some of the reasons you mention: cost, lack of citation management, weird sync options, etc. There are ways to work around these, but I always find myself creating more headaches.  Let me breakdown how I do things, using mostly free or low-cost software.
 
1) Install Zotfile for Zotero. I'm suprised no one has mentioned this yet, but it allows you to store your PDF attachments in any old folder and create links to those files in Zotero, without moving the unneccesary files in Zotero's main database and creating bizarre folder hierarchies. For a library like yours, getting this process up and running will be very time consuming, but it is all automated: let Zotfile run overnight and you should be good.
 
2) Point Zotfile to a folder in your Dropbox (a new one, not the one they are currently in), and choose how you want your files to be named. Then select your whole database, option+click, and choose "Manage Attachments -> Rename Attachments" This is the part that  will take a VERY long time. By choosing a single folder, you don't have to worry about your current subfolder issue. Zotero has pretty powerful search and sort cababilities.
 
3) Point your iPad app of choice that has two-way sync capabilities to this new folder. iAnnotate is fantastic (superior to Goodreader IMO), but I think the app ReaddleDocs is kind of underappreciated. It has less features for PDF markup, but has a way cleaner interface, I think.
 
4) Because your files are now symbolically linked, you can just push changes to your Dropbox folder when done working with them on your iPad (iAnnotate and ReaddleDocs can both do this without syncing your whole folder). As long as you don't change the file names, Zotero will always automatically call up the new files when you click on them.
 
Once the initial set-up of this workflow is done, you're basically keeping everything in sync through Dropbox and your iPad app...Zotero will be in sync with Dropbox. You can try it with a few PDFs at first before deciding to commit, obviously! I'd be happy to provide more details if needed. Sorry this "comment" is already so long! And tagging is actually an entirely separate issue...
45

Craig thanks for bringing up Zotfile! I tried it a couple of weeks ago and it seemed like the perfect combination for me in theory but in practice I had some issues with it that I will attribute entirely to impatience. I'll give it another try and report back. I love that it's free! I've acquire DT and this discussion has cleared up for me the differences between Zotero (for citation management) and DT (for organization and making connections among different texts). I think I'll probably settle on a Zotero + Zotfile + ipad workflow for annotating and bibliographies and use DT in the drafting stage to pull useful material out of my sources.

Have you (or anyone out there) used Papers? It seems like the magic all-in-one tool?

50

Hello all,

so after much playing around with tools (at the expense of reading and writing of course...sigh), I have settled on the ideal workflow for me:

Sente for bibliographic citation and management. Personally I can live with Zotero because for a grad student Sente might as well cost a million dollars! BUT I think I'll settle on Sente beacuse I am absolutely in love with:

Sente for ipad: for reading and annotating, with quck access to marginalia and notes. Both versions of Sente work very well with DTPO which I caved in and got and which is amazing.

DevonThinkProOffice: to consult my resources and pull things together in the pre-writing and writing stages. It's great for seeing connections across texts. And the fact that I can have my kindle notes in there is a huge plus! I hate the thought of having my notes and highlights scattered across kindle, sente, dropbox, goodreader etc. DTPO is where everything comes together. And for such time-sensiive "exercises" as my qualifying exam it'll be great to be able to see everything at once and quickly. Woot!

Scrivener: to write. Works well with the other tools.

Dropbox: everything is stored here because I am nervous about keeping things in proprietary programs like Sente. But if I know that I have another version somewhere (even if its not the annotated one) then I'm good.

Thank you all for the wonderful input. Feel free to drop by and tell us about more tools that may have fallen out of the conversation. I'm still waiting for a plug for Papers. It really seemed like the idea all-in-one tool.

 

~A

53