Puppeteer / headless chromium if you need to deal with complicated front end applications. Or node and linkedom if you can get away with parsing server rendered markup.<p>But also, if you're lucky, you may just find if you look through network requests that there's a sensible XHR/JSON api waiting for you.