Is there a service/software where I can feed a bibliography, say in BibTex format, and I get all “Cited By” references as a response? In other words, is there an automated “Cited By” crawler?


I'm currently hunting down case reports about a rare disease, because I need more cases for a dataset (yes, about medicine).

At the moment, I am filling up my Zotero Library by manually walking through Google Scholar's "Cited By" feature. The first day of doing so was expectedly productive. Within a few hours, the article count went up very fast. But right now, each "Cited By" page yields one or two new articles and I have about 114 articles to go. That, in itself is great, because it tells me that I might have visited most articles within the citation network. However, towards this end, I need double-check for duplicates all the time and it's very time consuming. I find myself scrolling through entries, or searching, checking whether I might already have added a paper, or not.

This is so tedious.

The very reason to continue simply is the occasional discovery of a new cluster of case reports that were not referenced by the previous one—and I really need a larger number of cases for the analysis.

I wonder whether I should stop here and look at — Mendeley — PubMed etc.

but I'm not sure whether this would cut time or assure the coverage of most of the citation networks.

Suggestions, very welcomed.

closed as off-topic by Buzz, scaaahu, Fomite, user3209815, Enthusiastic Engineer Oct 27 '17 at 19:19


  • ""Shopping" questions, which seek recommendations or lists of individual universities, academic programs, publishers, journals, research topics or similar as an answer or seek an assessment or comparison of such, are off-topic here. (See this discussion for more information.)" – Buzz, scaaahu, Fomite, user3209815, Enthusiastic Engineer

  • I really don't understand what you're asking. The best I can make out is that your current challenge is having to manually resolve duplicates. If that's the issue, then just import everything regardless and then use Zotero's duplicate resolution folder to clean up the dataset afterwards (you can search online how to use it; I use it quite a bit and it works great). If your challenge is more involved then that, then please revise your question for clarity, and then you might get more helpful answers. – Tripartio Aug 4 '17 at 22:40
  • Changed, as suggested. Thanks for getting back. – Tom Aug 4 '17 at 23:21
  • Your revised first paragraph is nothing like what your original question seemed like, so the revision was very necessary. Unfortunately, I have no idea about a ready-made tool for this. I've written custom R scripts that could do something similar, though not iterative crawling. The problem, though, with Google Scholar is that there's no API and web crawlers get blocked after 50 or fewer results are returned. Web of Science, though, has an API. – Tripartio Aug 5 '17 at 12:15

This can certainly get tedious, and searching in both directions (i.e., using the "cited by" entries) is certainly the way to go. That said, if you start at your university library, especially a medical library, you will have access to tools that are tailor-made to do medical searches, like Medline.

If you're having issues, by all means, go talk to a librarian. Their job is to help people like you get their jobs done. They absolutely love to do it, and they're usually very good at it.

To put things in perspective, though, 15 years ago the only way to do what you're trying to do is to wade through hard copies of the Scientific Citation Index and photocopy the articles after finding them in the Medical library stacks. (i.e., -- we used to walk thirty miles to and from work, uphill both ways!)

  • Kudos on recommending going to an academic librarian. They love to help with this kind of thing. – Tripartio Aug 4 '17 at 22:37

Certainly there are. Data Cite offers such services https://mds.datacite.org/.

Yet, you might not need to pay for such service, must probably your academic institution's library has a platform and an API that can be queried for the data you are looking for.

Many universities in the US rely on exlibris.com for their libraries' platform. Here is their API's website. https://developers.exlibrisgroup.com/alma/apis. I employed it recently to retrieve metadata from 2500+ publications. It took a few days to write the code and 15 minutes to get the dataset.

So do not just ask the librarian, ask also the library's data scientists (they might be the same persons).


Have you tried this tools?


It can help you to build an author and citation network directly from google scholar.
