TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Ask HN: Help! Tech stack for web scraping and simple frontend graphs

2 pointsby Spinosaurusalmost 3 years ago
Hey all, starting to work on some of my own projects after a 5 year hiatus.<p>My personal projects in the past have often utilized a Node.js backend (Express) that served a React frontend, with a postgres db with Knex. I have a bit of familiarity with Rails.<p>I&#x27;m wondering how much things have changed in the past few years. I&#x27;m reading that React should probably be avoided, Vue seems popular, Svelte seems like the shiny new thing. I&#x27;m considering looking into Django as learning Python might be worthwhile.<p>My day job uses Kotlin, so Ktor or Spring seems like an option.<p>The project I&#x27;m building will basically scrape some data every day, store it in a database, and serve a front end primarily consisting of graphs. Is there a particular tech stack that is more suitable to that these days? Are there any stacks that should be <i>avoided</i> for such requirements? &quot;Overengineering&quot; is something I want to avoid.

1 comment

rustdeveloperalmost 3 years ago
For web scraping, I recommend using a web scraping API, e.g. <a href="https:&#x2F;&#x2F;scrapingfish.com" rel="nofollow">https:&#x2F;&#x2F;scrapingfish.com</a>. This solves all potential problems with getting blocked and can make data extraction easier as well.<p>For the app, I&#x27;ve recently started using Remix (<a href="https:&#x2F;&#x2F;remix.run" rel="nofollow">https:&#x2F;&#x2F;remix.run</a>) and so far it seems to have been a good choice for me. There is a good integration with Remix in Mantine for front end: <a href="https:&#x2F;&#x2F;mantine.dev&#x2F;guides&#x2F;remix&#x2F;" rel="nofollow">https:&#x2F;&#x2F;mantine.dev&#x2F;guides&#x2F;remix&#x2F;</a>. I think it&#x27;s a good full stack choice if you just want to quickly build an app for your project&#x2F;product.