Hey all,<p>This is Jan, the founder of Apify (<a href="https://apify.com/" rel="nofollow">https://apify.com/</a>) — a full-stack web scraping platform. After the success of Crawlee for JavaScript (<a href="https://github.com/apify/crawlee/">https://github.com/apify/crawlee/</a>) and the demand from the Python community, we're launching Crawlee for Python today!<p>The main features are:<p>- A unified programming interface for both HTTP (HTTPX with BeautifulSoup) & headless browser crawling (Playwright)<p>- Automatic parallel crawling based on available system resources<p>- Written in Python with type hints for enhanced developer experience<p>- Automatic retries on errors or when you’re getting blocked<p>- Integrated proxy rotation and session management<p>- Configurable request routing - direct URLs to the appropriate handlers<p>- Persistent queue for URLs to crawl<p>- Pluggable storage for both tabular data and files<p>For details, you can read the announcement blog post: <a href="https://crawlee.dev/blog/launching-crawlee-python" rel="nofollow">https://crawlee.dev/blog/launching-crawlee-python</a><p>Our team and I will be happy to answer here any questions you might have.