I created this tool a few months ago to automate some of my tasks.
I use it to collect prices and files and to get notified when something changes on certain websites. The task runner uses Playwright.<p>I'm not sure if it can be useful to anybody else besides me :) Any feedback is welcome.
There's a chrome extension that records all your steps and then generates a puppeteer/playwright script for you. Then you can just edit bits of that script and run it. This is my go-to method presently.<p>I feel like you could incorporate some of those features into this and eliminate some of the manual entry of css selectors.
Really interesting tool. I've always found the task of finding the right CSS selectors tiresome, so I created tool which automatically extracts the right selectors for you (and builds a scraper as well). Open Source on GitHub: <a href="https://github.com/lorey/mlscraper" rel="nofollow">https://github.com/lorey/mlscraper</a>
> Periodically check for changes<p>I remember a Firefox extension that notified me with the diffs of changed pages. I don't remember exactly what I was using it for. I think something related to CVEs for some keywords. Then the service I was using shut down and I lost tracks of that extension.
I'd love to use this as a webpage interaction/scraping component in chains of webhooks (building on e.g. low-code step pipelines built with Pipedream, could imagine fitting with something like Zapier, too).<p>Would require the ability to:<p>(1) make a web request to start a run<p>(2) collect the scraped info and return it as part of (a) a webhook call or (b) in the response of a synchronous "start run" blocked long http request<p>It sounds like #1 is already possible. 2a might be possible if there was a call webhook step or notification option? But 2b would be the simplest (if I could make a single GET or POST request, and the response gives me some outputs from an automation session).<p>Zapier/Pipedream integrations could be nice platform integration marketing for the tool as well.
This is similar tools: <a href="https://github.com/huginn/huginn" rel="nofollow">https://github.com/huginn/huginn</a>
<a href="https://n8n.io/" rel="nofollow">https://n8n.io/</a>
This is cool! Is it free? If not would be good to have pricing be visible without creating an account.<p>Also the logo in the header looks low definition. Upping its resolution will make the website way better.
Best of luck.<p>Two things that have tripped me up
1. "and save the value of this attribute", it wasn't clear that this could only be certain values, text, id.
2. API I can list tasks, get the status of a task, start tasks via API, but I don't se how I can read the results / stored data
Curious what your pricing strategy is here? The free offering is cool for sure. Guess you'll learn how serious users may want to actually use the service, what that means for your unit costs, and how it all scales in your favor? or sort of already figured that out?
Only tangentially related, but...<p>I use Firefox. Anyone know of a way I can have it close all tabs once a day? Either via an external script or an extension?