Mendeley is a popular citation manager but it doesn’t properly export bibtex files. Bibtex files (with a .bib extension) are used to add your references to papers compiled in LaTeX. You can find many reports of these problems, including the Mendeley support page. Some of these problems haven’t been officially fixed for over 6 years! In this post I will share some code that addresses these problems, including:
- Constraints on what you can write in the “year” field
- Extra curly braces around titles
- Incorrect format of “month”
- Ability to select what types of entries include the URL (instead of only webpages)
Why Am I Using Mendeley?
First, here’s some background on how I came to using Mendeley. I was using RefWorks for citation management during grad school. Eventually, some minor annoyances piled up:
- The website wasn’t responsive, and it was actually kind of clunky to look up papers and export Bibtex files.
- The website was behind a subscription wall associated with my institution. This wasn’t a huge problem, but it was inconvenient to need to proxy through a campus server just to access my references.
This year, I started looking for an alternative. I tried out Zotero for a while but found it lacking (perhaps more on that in a future post). I eventually settled on Mendeley for the following reasons:
- Aggregate editing. I can select multiple entries to change a single field (very handy for conferences).
- Generally effective importing of references directly from the browser window.
- A useful app that lets me save a PDF annotated in Adobe reader as a separate file. I find this helpful because I do a lot of paper reading on my tablet.
- In-text searching. Which papers mention game theory? It’s easy to look up
- I can restrict what folders are backed up online so I don’t use up all of my space (2GB currently).
- Custom journal abbreviations.
- Automatic generation of bibtex files.
Problems with Bibtex Files
Overall, I think my workflow of citation management improved. However, there was one major issue, and that was how Mendeley creates bibtex files. There are a number of small bugs in how the bibtex files are formatted, and these create problems when building your bibliography in a LaTeX document. Here is a summary of the problems:
- Mendeley adds an extra set of curly braces to titles. This means that capitalization is always based on what you enter in Mendeley. It cannot be modified to accommodate the bibliography style. This has been a known issue since 2010.
- The year can only be written as 4 digits, which prevents you from writing “submitted”, “in press”, “to appear”, or similar text where it is needed. This has been a known Mendeley issue since 2009.
- Special characters (like “{}” or “#” inside of titles, etc) don’t work right for me when I have them escaped. Perhaps this isn’t really a problem because I can just disable the option to escape them, but selecting this option gives me weird output if I try to use curly braces to force capitalization of title characters.
- The only options to control what entries display URLs are “All” or “Only for Webpages”. Many of my entries have URLs that I want to keep in my database but I don’t want in the bib entry. BUT, there are non-web page entries where I DO want the bib entry, such as for journal papers that haven’t been published yet. So the options aren’t flexible enough.
- Months don’t appear correctly if they written in the database as words (e.g., June). The problem is that curly braces are added around the month (e.g., {jun}), so it gets converted to display as “jun” instead of “Jun.”. One fix is to store months as numbers. but this would mean I have to fix every paper when I add it to my database (since writing months with words is the default).
Correcting Bibtex Files
My initial solution was to wait until I was finished with writing a paper. Then, I would copy the bib entries from my master bibtex file into a separate file. I would fix the entries manually. This worked OK for a couple of short conference papers. However, this wasn’t practical to keep repeating. So I started looking for an automated fix.
Long story short, I wasn’t happy with what I could find. I found some solutions that used the online Mendeley reference format editor (such as for adding “in press”), but I’ve had problems getting the results to show up in bibtex files. So, I decided to work on my own solution.
I wrote some code in C that fixes my database in ~0 seconds (OK, more like 0.05 seconds). This is great because I just double-click on the executable whenever I make a change to my database and then I get a fixed copy of the bibtex file. My database has over 500 references so this function should take a database of thousands of entries and still finish before you switch back to your LaTeX editor.
What does this code do? It reads the file generated by Mendeley, fixes the entries, and writes everything to a new bib-file. It imports the default bib-file named “library.bib” and creates file “library_fixed.bib”, but these can be changed using input arguments. Here’s a list of the exact fixes:
- Double curly braces around the titles are changed to single curly braces.
- If “issn” is defined for an entry but “year” is not, then the “issn” is renamed to “year”. This let’s you use custom text for “year”, such as “in press”, by omitting the year and writing it under “issn” (my reasoning is that the ISSN is usually not included in a bibliography and it would be highly unusual for a paper to have an ISSN without a year).
- Escaped curly braces are changed so that you can use them around specific words in titles and they will display properly (needed for acronyms, proper names, etc.). This is an “exception” to a problem created by the removal of double curly braces for titles.
- Curly braces are removed around the month (this will only happen if the month appears in the format {xxx}).
- URLs will be removed from all entries except a list of specified types. I currently have “misc” and “unpublished” as default exceptions, and more can be added in the source. Web pages are exported as “misc”.
Your mileage may vary with your own bibtex files, but I’ve posted the code on Github. The windows executable is also posted there if you don’t want to compile it yourself.
I will update the code if I find any bugs or add other changes. If you try it out and find it helpful then I would be happy to know!
Rambling Academic 
This is the most useful post I’ve recently read about LaTeX & Mendeley. I’ve tried lots of reference managers and I’ve ended up using Mendeley becouse of the features you described. The main advantage for me is renaming fields in scope and the metadata identification when you drag a PDF. Since I’m not a programmer, I have a main doubt about your script: Is it easy to run this C code from mac compiling it myself? and if yes.. Could you give some basic steps please?
Congratulations for this good job, I’m sure lots of people will take advantage of it.
LikeLike
Thanks Rafa, I’m, glad you found this useful. Unfortunately, I don’t have much experience with macs, or I would have provided a mac version myself. The code should compile readily with any mac compiler that can work with the C99 standard of C (which should be any modern compiler).
From a quick web search, you can get the mac version of the gcc compiler by installing XCode.
You don’t need to open XCode, since you can compile my program directly from a terminal. A command like “gcc -o mendeleyBibFix mendeleyBibFix.c -std=c99” should work.
I hope this helps!
LikeLike
A further reply in case anyone is still wondering about max or Linux access. A build script for these OSs was added a few years ago, so it is easy to compile the code if you’re running either of those.
LikeLike
Hey,
thank you for this nice post.
I have one question though, did it also happen to you that the BibTex file did not include all the documents that you have in your bibiliography?
I am currently writing in Latex and somehow, some documents are never transferred to the BibTex file, never mind if I create it for the whole library, per group or per document.
Do you maybe have an idea what the problem is?
Thank you so much for your help in advance…I am getting a bit depressed here.
LikeLike
I don’t think that I’ve had that problem. I usually have Mendeley set up to sync Bibtex for my whole library. However, I think that it withholds entries that aren’t “approved”. When you import references, they aren’t assigned a citation key (which Bibtex needs) until you manually confirm that the entry details are correct. I usually see a prompt at the top of the “Document Details” pane (View -> Show Document Details) which asks for confirmation. Perhaps this is your problem?
LikeLike
Thank you very much for your code! I actually stumbled upon this page while looking for an answer for the exact same problem that some citation just wouldn’t show up in the BibTex at all. Confirming the “Document Details” did indeed the trick 🙂
LikeLike
Just wanted to say that this script is really helpful!
I now use it regularly 🙂
LikeLike
Thanks Alvaro, I’m glad to hear it!
LikeLike
Perfect, just what I needed! Thank you very much.
LikeLike
Thank you from Montreal.
LikeLike
Thank you so much for this code. It worked really well and has saved me a lot of time!
LikeLike
Hi there
How do I permanently delete the .bib file altogether? I want to do in-text referencing(citation) with some of the files in that .bib file, but it keeps showing one article even though there are 30 linked files in there. When I trie to permanently detele so that I can re-add the files individually, I am struggling. It removes it from the library tab, but not from Mendeley. Please help.
LikeLike
Hi, it sounds like you have 2 separate problems:
1) Menedeley is auto-creating a bib-file that you want to manage yourself
2) You’re trying to cite papers in the bib-file but they are not appearing in your paper.
I will address 1) for now. To disable the auto-create/update of the library bibtex file, go to Tools -> Options -> Bibtex and disable “Enable Bibtex syncing”.
2) could depend on what software you’re using as an editor/IDE for your latex, or the cite commands may not be entered properly. There are lots of sample templates available online.
LikeLike
Thank you so much, this tool has saved a huge amount of headache trying to deal with my references for my thesis.
Thank you!
LikeLike
Thanks a lot! This has exactly what I was looking for (it’s a shame Mendeley doesn’t update these things fast enough).
But is it possible to change the set of corrections I would like to apply? I just want to get rid of curly braces around months, annote and file fields, while keeping everything else as it is.
Thank you in advance! This is really great work 🙂
LikeLike
I’m glad you found it useful. My short answer is that you could modify the source code and delete/comment the changes you don’t want. That’s not particularly user friendly, but I have no plans at this time to extend this tool to make those kinds of changes through the UI. You could always post a feature request, though 😉
LikeLike
Hi, Really glad i stumbled upon this -but im having trouble with using the windows exe to add the fix. i’m not sure what to do..
or do i add the script into the preamble of the .bib file? – is that the same as copy and pasting it at the beginning of the .bib file?
thanks!
LikeLike
Thanks for trying out the program. The readme describes how to call the executable, but I didn’t say that you first need to open a command window and navigate to the folder containing both the bib-file and the executable, and then enter “mendeleyBibFix.exe OUTPUT_FILENAME INPUT_FILENAME”, where you specify the input and output filenames. You can omit those filename arguments if your bib-file is called library.bib and you want it to create library_fixed.bib. You don’t need to modify the bib-file directly – that’s what this program is for!
LikeLike
Hi,
This was extremely helpful, already broke my brains on the question for a while.
Thanks so much! Ran like a charm on Windows!
LikeLike
Thank you, Sir! Exactly what I needed!
LikeLike
Hi,
Just tried this (exe) with a bib file generated with Mendeley 1.9.3 and it didn’t work. Generated the library_fixed bib file but none of the fields were fixed. I know you said its not future proof. The future might have arrived 🙂
LikeLike
Hi SSJ, thanks for your message and I’m sorry to hear it isn’t working for you. I’m also running 1.9.3 and a (very quick) check shows that the fields are still fixed as expected (from a freshly-generated library bib file). Is your library_fixed file coming out identical to the original?
LikeLiked by 1 person
Thank you for your prompt response. I tried it again by regenerating by .bib file through Mendeley and it worked !
I had used Jabref on the bib file i tried earlier to remove some fields like abstracts etc – which might have broken the format your code was looking for.
This has just saved me a ton of time so thanks a lot for this !
LikeLike
Great, I’m glad it worked! Yes the fixes are hard-coded so making manual changes before running the code will likely trip it up.
LikeLike
Hi Adam – thank you for this excellent piece of code. Just a heads-up: I’m using now Mendeley Desktop 1.19.5. Turns out it introduces some header lines that aren’t BibTeX code but that I reckon bibtex, appropriately, ignores. However – from what I can tell – mendeleyBibFix reads those lines, and then perhaps figures it’s reached the end of what it’s supposed to, as it just stops, emitting therefore 0 entries in library_fixed.bib. (Happy to send you the file if you wish.)
Hope this helps …
—
Danny
LikeLike
Hi Danny, thanks for the heads up. I have been using 1.19.4 and everything was still working fine. I just installed 1.19.5 (which appears to be “stable” but is not yet the version featured on the main download page), and mendeleyBibFix still works for me. There is a header in both versions and the version number is the only apparent difference.
If you try running the mendeleyBibFix script while Mendeley is actively updating library.bib then this could be the problem, or maybe you have characters in your “annote” fields that are killing the script prematurely (full details of the issue are here: https://github.com/adamjgnoel/mendeleyBibFix/issues/10). If neither of these seem to be the problem, I can take a look at your library file (preferably a minimum working example, otherwise whatever you have).
LikeLike
Hi again Adam – many thanks for the reply, and apologies for my delay getting back to you. But the time elapsed means I’ve also had opportunity to work on this issue a bit more and remarkably… the problem’s gone away. It might indeed have been a race condition of some kind, but I’ll keep a watch on this. Thanks again for the reply and for a really valuable and useful piece of code!
LikeLike
Hey Adam,
as everyone here, I am extremely happy to have found your tool. I use it for all my papers and it works like a charm. I don’t know if it is only a small fix that I think is usefull but I wanted to share it:
For unpublished work I want to be able to still apply some kind of sorting. Therefore I found this page: https://tex.stackexchange.com/questions/31384/how-to-enter-publications-in-press-or-submitted-to-in-bibtex , where Joseph Wright’s answer suggests to add to the preamble of the bibfile:
@preamble{ ” \newcommand{\noopsort}[1]{} ” }
This can be used then to have entries of the form:
year = “\noopsort{3001}in press” (when using mendeley and your tool this comes into the field “ISSN”)
I never used c before, so actually I am not sure, if this a good way to implement it, but I added another line to your code, which reads:
fprintf(outputFile, “%s”, “@preamble{ \” \\newcommand{\\noopsort}[1]{} \” }\n”);
and comes directly before your line:
fprintf(outputFile, “%s”, outputContent);
Thank you very much for sharing your code
Linos
LikeLike
Wow, it is really an excellent tool! It works well in my mac and really cool. But now I want to delete the abstract in my .bib file. Could you please teach me or update your code about how to do it in your code?
Thank you very much for sharing your code.
keep safe in these strange days.
Haiqin
LikeLike
Hi Haiqin, sorry for my delayed reponse. You can remove the abstract by copying the code that removes the “annote” field as notes have a similar structure to abstracts. The corresponding code is the if-case around lines 311-317 in current source. In the copy, change “annote =”,8 to “abstract =”,10. Re-compile and you should be good to go.
Just watch for text matching “},\n” appearing within your abstracts (which is unlikely but possible), as this is what the code looks for to detect the end of the abstract.
LikeLike
Thank you so much for your tool!
I find that Mendeley hasn’t fixed these small bugs yet since 2016. Shaking my head.
LikeLike