Is there an API which I can use to pass the link, and it scrapes the text and provide me summary?
else, I can parse text on my server and then pass that text to an AI model to summarise it.<p>What are some ways to approach this? I'm looking for a good balance between the cost and the quality of summary generated.
I think GPT-4 (or 3.5 if you are trying to save a few dollars) would be best. OpenAI's models are strong for summarizing text, you would just need to scrape it yourself and pass it to the API.<p>The API is easy to use and there are a lot of supported libraries already for whatever language you prefer: <a href="https://platform.openai.com/docs/libraries/node-js-library" rel="nofollow noreferrer">https://platform.openai.com/docs/libraries/node-js-library</a>.<p>If you are lucky enough to get off the plugin waitlist, you can even use the web browsing plugin for GPT-4 and that should let you just pass in a link.
Not perfect, but I built this to generate summaries for my HN AI related newsletter <a href="https://github.com/vlameiras/hn-ai-news">https://github.com/vlameiras/hn-ai-news</a>. I just fetch the content locally with requests and then feed it to GPT-4 API