How To Choose A Reference Manager

In the last few months several friends asked me for software advice. Funny enough, all of them wanted to know about the same type of application: reference management software, sometimes called citation management software or personal bibliographic management software, says Wikipedia

That is software which helps you to keep track of reading materials, quotes and citations and supports their use when writing articles, academic papers, etc.

In this article I will present a few tips on how to be smart when choosing your reference management software. This is all about asking the right questions at the right time and thinking ahead. Also, it is about knowing your personal workflow and use cases.

If you don't know your workflow yet and are not sure whether a reference manager is the right tool for you: head over to the beginner's guide to choosing a reference manager. You'll find out why a reference manager may be a good idea and how to seek out one that works for you.

No one software to rule them all

First, the bad news: as you can probably conclude from the second paragraph, there is no one right or wrong software to do this kind of information management for everybody. You can use or force to employ all sorts of different applications to keep your sources organised.

I once worked at a university department where the staff let their teaching assistants sort all the bibliographic information in Word documents and printed them off as bullet point lists. Being the teaching assistant, I found that neither particularly useful nor an appropriate workflow for bibliographic data, which should be at least available in a non-linear fashion.

As you can imagine, one can get very creative with this sort of thing. It's important to keep this in mind: the system is not an end in itself.

If you are a writer or academic, the bibliographic manager will probably be one of your most used tools upon which your career hinges in no small measure. Your main software tools should work and support you in your endeavours and yes, they should be fun to use, otherwise you’ll not gonna use them.

Like with every other «relationship»: if you want it to work out in the long run, it does not hurt to be picky and specific.

The good news is: there was never a better time to choose a reference manager (or text editor, for that matter)! The explosion in independent software development as well as Web technology presents you with an abundance of options:

You need a Web app that gives you access to a community? Take your pick among BibSonomy, CiteULike, Connotea and co.

You need a free tool that runs on every major platform as well as on your USB flash-drive? JabRef is your friend.

You need it sweet and simple, basically an iTunes for your papers? Sente is here to help.

You do most of your discovery in a browser? Zotero might be your tool of choice.

Other candidates may be Bookends, Citavi, EndNote,… all with their respective merits and weak spots. And if you want to be part of one of the fastest growing online science & research communities, Mendeley is there to stay.

Wikipedia has a list at the end of its reference management software article as well as a comprehensive comparison of applications based on their features.

Too much choice can be hard to manage and overwhelming which may even prevent you from making a choice at all. So here are some ideas and questions to think about, when shopping for a bibliographic manager.

The questions to consider

  • What do you need?
  • What do you want?
  • How do you work?

The question of what you want is related to what you need and how you work. But it is also a matter of preference and the relative importance of those preferences. Ask yourself:

  • What are nice-to-have features?
  • What are must-have features?
  • What are the one, two, three… most important features for me?

For instance, if you are working with a group of colleagues around the world and you need to share resources, there is no point in having a Mac OS X-only software without sharing capabilities. The number of great platform-agnostic Web tools would be a better starting point.

If you must use Microsoft Word to write your project, it will only grow your pains to use an application that does not integrate well with Word.

Features

Contrary to popular believe more features are not always better. The question is: Exactly what features do you need? While preferences for Web apps or apps native to your platform limit your options significantly, you’ll still need to prioritise other available features.

I cannot possibly list all the functions that may be relevant to you, but this could be a starting point for your investigation:

  • Which platforms do you need to cover?
  • Do you need the option to share your references (outside the application) with colleagues and friends?
  • Do you need to be able to collaborate with co-authors, your supervisors, mentors…?
  • Do you ever need to access the data from computer that is not your own workstation?
  • Do you have a large collection of files already?
  • Which import and export formats do you need support for? (Do you have legacy content?)
  • Do you need in-app search capabilities?
  • Do you want to manage only bibliographic data or do you want to manage the content files (usually PDFs) as well?
  • How sophisticated are your grouping, naming, tagging, organising… needs?
  • Do you need the manager to work with a specific writing environment? Which one?
  • Do you need the software to build your bibliographies for you? What formats / styles do you require for that?
  • What is your backup strategy for this sort of data?
  • How often will you work with it? How much time can you devote to «learning the system»?
  • What is the budget or price range you can afford? (Be sure to find out what upgrade prices, subscription costs and so forth you may encounter in the future.)
  • Do you need a flexible system? Is this a one time only use case or will you build a career around sifting through this information and curating the database over time?
  • Do you need a support person or are you comfortable seeking support in an online community, such as a forum or in IRC?
  • Can your work with an English user interface or do you prefer menus, documentation and support in another language?
  • Will you re-programme / script / extend or adapt the functionality of the application in your fancy super smart ways?
  • Do you want to be able to use the data in 5 to 10 years or more?
  • Do you have colleagues or friends you can ask about recommendations and what they are using? Are these friends willing and able to help you out and get you started with a system they are familiar with?

These are only a few relevant questions off the top of my head.

I want to emphasise that you should have a tested and trusted alternative software in place which lets you pick up right where you left off, e.g. in case you don’t have internet access. Or, in case your university discontinues the campus-wide licensing programme for the software you relied upon. Or whatever can happen. According to Murphy, it will happpen.

Campus software licenses

If portability is a major issue for you, you should be cautious with the campus-wide software solutions libraries and unis buy for their staff and students. This is a biggie, as almost every student and staff member has access to technology which enables sharing, is integrated with eLearning platforms, allows direct annotations in ebooks and other nifty things.

Once I surveyed the landscape of campus-wide software solutions, I was surprised what you can actually do with it these days. The challenge is to know what to do when the day comes and the licensing is discontinued, or you leave campus, either for a new job or after uni. None of the systems I looked at allowed a clean and re-usable export of this user-generated data.

This is exactly the kind of scenario few people think about until it comes back to bite them: After processing all your research material within these campus-only tools you are locked in and dependent on getting your information out in order to migrate to a system for which you actually can afford a licence.

Data export is sometimes a neglected feature, because the software developers want to keep you using their product. If you a) want to use the convenient systems your university library provides and b) make sure your data is portable, you’d better test the export functions with a small database, before you commit years of your work to it. Find an alternative software which allows you to import the type of export your campus software provides. Go all the way to test the import:

  • Can you open the exported file without problems?
  • How clean is it? Do the special characters in the Turkish author names look funny? Are URLs, notes and file paths correct?
  • Are there crucial input fields that will not be imported properly?
  • Is information assigned to the wrong input field during the import?
  • How much manual cleaning up is necessary?
  • Does the cite function work as expected with the imported data set? Can you work with the data?
  • Are you able to save the import in a file format native to an alternative software?

You get the picture—be safe!

Know how you work

If you have been writing your nationally syndicated advice column with your favourite text processor for eight years, chances are, you’d want to write other stuff with it too. Or, you already did. Find out whether the reference manager of your choice is comfortable to use in combination with your preferred writing environment.

But which ones to choose from?

Throughout my time at uni I tried every single bibliographic software that would run on Linux or my Mac. I'd love to say that I’ve seen it all. But with advances in online technology, research blogging, popular science blogging, Open Access etc. a lot has happened since then. Mendeley happened and Zotero happened and I worked with several academic institutions all with their own solution to the problem. (Read: expensive software only the librarians ever used.)

When I started studying, the choices were quite easily comprehensible: there were the expensive software packages and the free ones. There were a few independently developed alternatives which did what they did very well, but fell short on the integration with my writing tools.

I am writing pretty much everything that is not for the Web in LaTeX, using TeXShop as an editor and BibDesk as my bibliographic manager. It worked like a charm for the past eight years or so: both are actively maintained and developed OpenSource projects and MacTeX makes it super easy to put it on any Mac, so I don’t even have to worry about installing new things after the fact. I tried and tested every alternative I could find, but always came back to BibDesk.

After studying three degree programmes in different fields with different requirements in terms of text production and course work, it is safe to say that the Pareto-principle applies to my BibDesk use: I don’t use more than 20% of its capabilities. However, the 20% of the features I do use are mission critical. I would not use BibDesk if it were not for the consistent support of those priorities, e. g. in terms of data portability, sustainability and reliability.

I realise LaTeX is not everybody’s cup of tea. But sooner or later most people settle for a preferred writing tool and workflow. It’s best to take the reference manager for a comprehensive test drive before your have to rely upon it in a production environment with real things like deadlines breathing down your neck. However, it's not necessary to learn to «do everything» with the reference manager of your choosing. Focus on the most important tasks that make up 80-90% of your work.

Cautionary tale: first try, then trust!

I cannot stress this enough: first you try, then you trust! Find the most unusual pieces of information your system will have to hold. Maybe a paper of authors with really long names, or a title with all sorts of symbols and special characters in it. Test all the types of sources you’d want to manage.

Here comes another cautionary tale from Amy which in hindsight sounds so stupid: I took Papers, a fantastic tool developed by the fine folks of Mekentosj, for a very long test drive during the research of my master’s thesis. The integrated search functionality was just too darn good; I could not resist building up quite a collection of materials in preparation of writing the beast.

After four months or so, I realised that the BibTeX export (the bibliography tool for LaTeX) was… suboptimal. Besides the dodgy export of special characters and other minor details, it would not export the correct item types for the sources. (In BibTeX you specify whether your item is an article, a book, conference paper, Web page etc.) Much to my surprise, I had to learn that an application named Papers would not deal with books as I expected. Yes, I am embarrassed about this blunder on my part, but there you go. I had to go through four months of research and clean up the export files manually. I would have been better off re-creating a database from scratch.

This is not intended to reflect poorly on Papers! It is a superb piece of software, now available in version 2 and it has matured. Papers has some mind-boggling functions under a very friendly user interface. You should check it out, if you are looking for an article management system! I have not tried it again, but I think it is a fantastic product if you only need to manage papers and similar source types.

The lesson to take away from this is: know the limitations of your final candidates before they become the sole keeper of all the important information and treasure trove of years worth of work. Test the most important functions, ease of use and potential live savers in worst case scenarios (such as backup functionality and export filters…) to assess the candidates.

Summary

This article described a few questions to ask when looking for a reliable reference manager. The main questions that help you to sort the wheat from the chaff are:

  • What is it that you need?
  • What do you want and like in an application?
  • How will you use it?
  • How much do you want to spend in terms of time and budget (incl. upgrades, subscriptions prices etc.)?

Once you found a range of applications, test all the functions you want to use with standard as well as less likely sample data. Look at the most important functions, ease of use and worst case scenarios to assess the suitability of the candidates. Try them on small scale real world projects to find out which one you are most comfortable with.

Enjoy the ride!

Details & meta

Sun, 19/06/2011 - 21:52