very informative, thanks! i coincidentally just posted a blogpost about the other complementary approach using pure HTTP requests without headless browsers: <a href="https://news.ycombinator.com/item?id=21770576" rel="nofollow">https://news.ycombinator.com/item?id=21770576</a>