I have been working on this project for quite a while.
Here is what is happening in the background:<p>- Scraping the entire page content, run it with ChatGPT to test if it can be converted into a listicle.<p>- Send the job into the queue.<p>- Scrape the entire page again, and pass it to LangChain with ChatGPT to create a formatted JSON of name, description, features, and URL.<p>- Iterate over all the URL, take a screenshot with Puppeteer, and send it to Remotion to make a video.<p>- Craft the main post of the thread using all the other posts<p>- Save everything to a serverless Postgres and present it.<p>This works over serverless so that it can scale (I didn't want to work with scaling servers)<p>Here is an example
Original blog:
<a href="https://links.github20k.com/blog" rel="nofollow noreferrer">https://links.github20k.com/blog</a><p>Thread:
<a href="https://listicle.club/preview/U2FsdGVkX189pQBKMKMvQwOQSrvVRNYE8kngv00pNeU%3D" rel="nofollow noreferrer">https://listicle.club/preview/U2FsdGVkX189pQBKMKMvQwOQSrvVRN...</a><p>I still need to work on the post size. Sometimes it passes 280 chars.
Thinking about waiting for gpt-4-32k; hopefully, the results will be much better.<p>Let me know what you think!