TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Ask HN: Is there any decent API to download a paper given its name?

18 pointsby hexomanceralmost 3 years ago
I am developing a PDF viewer designed for reading research papers[1]. One very useful feature that I would like to add is the ability to directly download and open a paper just by clicking on its name in the PDF file. I have implemented a version of this using PyPaperBot [2] which is not bad, but it is not as fast as I would like it to be (it uses scihub which doesn&#x27;t have the best servers).<p>By parsing the html from google scholar, I wrote a python script that does this which was very fast and was working perfectly, however, after using it for a couple of minutes (maybe about 10-15 requests), I can no longer query google scholar using python requests (the returned html is a request for captcha). It appears that google disallows any programmatic use of google scholar (even though this was not spammy at all, the user has to manually click on a paper to send a request to google scholar).<p>Anyway I was wondering if there is any decent and free API to get the url of a paper given its name, I have found a couple of paid ones but they are way too expensive.<p>[1] https:&#x2F;&#x2F;sioyek.info<p>[2] https:&#x2F;&#x2F;github.com&#x2F;ahrm&#x2F;sioyek-python-extensions#-paper_downloader

4 comments

wilsonnb3almost 3 years ago
&gt; it uses scihub which doesn&#x27;t have the best servers<p>you could always download the scihub backup torrents from libgen and host them yourself somewhere. its probably like 100TB of data by now though so this isn&#x27;t really a cheap approach.
评论 #32733148 未加载
boredemployeealmost 3 years ago
Nice! I would easily use your tool for the next 2 years (my research deadline). I have a question tho. How would you bypass the fact that you need to be under some university vpn or special login to download some articles?
评论 #32725801 未加载
ALittleLightalmost 3 years ago
One thing you could try is downloading the papers before the user requests them so that they will be ready instantly for a user&#x27;s request.<p>Example: User opens paper on page 1. Page 1 has citations to 3 other papers. Your tool instantly begins downloading the other 3 papers. User goes on to Page 2 which cites 1 other paper. You begin downloading the new paper. User clicks on the citation and your tool now has the linked paper already downloaded and ready to open.<p>Might be a bit of an imposition on sci-hub though.
评论 #32728710 未加载
ttpphdalmost 3 years ago
Not sure off the top of my head but it&#x27;s possible that crossref DOI lookup has an API you can use for this.<p><a href="https:&#x2F;&#x2F;openapc.github.io&#x2F;general&#x2F;openapc&#x2F;2018&#x2F;01&#x2F;29&#x2F;doi-reverse-lookup&#x2F;" rel="nofollow">https:&#x2F;&#x2F;openapc.github.io&#x2F;general&#x2F;openapc&#x2F;2018&#x2F;01&#x2F;29&#x2F;doi-rev...</a>
评论 #32725855 未加载