TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Show HN: Sleeping Spider – Web page change monitor

31 pointsby linkletover 10 years ago

16 comments

TheDongover 10 years ago
Does this site respect robots.txt?<p>Does it render javascript content (so e.g. &lt;div ng-view&gt; angular sites work)?<p>Does it exclude or include comments... if I watch a blogpost with comments will every new comment cause an alert or only blogpost changes?<p>Will I get a notification if the page briefly 500 errors when you hit it, or does it have to get a 200 before it reports a change?<p>Will I get a notification if a page disappears entirely?<p>Does it scrape only once per day or more often and alert only once a day?<p>What user-agent does it scrape from on the off-chance someone wants to block it?<p>Does it diff binary data.. for example, if I ask it to watch an image url will it hash the image and notify me when it gets updated?<p>Does it auto-detect pages with rss feeds or pubsub feeds and subscribe to them?<p>Does it understand twitter so if I watch a user, it only alerts me of tweets, not e.g. a change in the number of followers?<p>Does it have special code for any website whatsoever?<p>Is the webpage parsing&#x2F;diffing library &#x2F; algorithm open source somewhere?<p>Is there a monitization plan, and if not how can I trust it won&#x27;t vanish?<p>Is there a limit on the number of sites I can watch?<p>I like the simplistic design, but it would be nice if there were more information.
评论 #8513656 未加载
评论 #8510969 未加载
aw3c2over 10 years ago
If you prefer privacy and control over your own tools, there are the FLOSS <a href="http://specto.sourceforge.net/" rel="nofollow">http:&#x2F;&#x2F;specto.sourceforge.net&#x2F;</a> or the free(as in no cost)ware <a href="http://www.cmcode.co.uk/webmon/" rel="nofollow">http:&#x2F;&#x2F;www.cmcode.co.uk&#x2F;webmon&#x2F;</a> for Windows
评论 #8524700 未加载
评论 #8511632 未加载
netman21over 10 years ago
In the most amazing coincidence the first two pages I submitted had the same digest: 90c0568! One was the front page of a news site, the other was the management page of a company I follow. The mind boggles. Or, they have a bug.<p>edit: as did the 3rd. And 0 bytes returned so probably being blocked. On a black list already?
评论 #8511124 未加载
lucasjansover 10 years ago
Adding watch results in a white screen of nothing, no new watches added. I&#x27;m trying to monitor the Nexus 6 page :D<p><a href="https://play.google.com/store/devices/details/Nexus_6_32GB_Cloud_White?id=nexus_6_white_32gb" rel="nofollow">https:&#x2F;&#x2F;play.google.com&#x2F;store&#x2F;devices&#x2F;details&#x2F;Nexus_6_32GB_C...</a><p>For my use case, anything more than an hour delay wouldn&#x27;t be practical. Ideally it would be up to the minute, but I&#x27;d imagine that would cause other issues for a service like this. In the past I just used a Chrome Extension. But living in Vietnam I have to make sure my VPN is up in order to see the North American content.
评论 #8511089 未加载
sleepingspiderover 10 years ago
&gt; Does it understand twitter so if I watch a user, it only alerts me of tweets, not e.g. a change in the number of followers?<p>What do you think if I provide a numeric filter option to watches, which filters out numbers on the page before comparing if turned on.
sleepingspiderover 10 years ago
&gt; Does it exclude or include comments... if I watch a blogpost with comments will every new comment cause an alert or only blogpost changes?<p>I don&#x27;t have a good solution for this problem yet. Let the user specify a css selector or xpath?
watsonover 10 years ago
Is it only for HTML, or will it alert on changes to JavaScript files for instance?
评论 #8531059 未加载
girishsoover 10 years ago
I just built &quot;Page Poller&quot; for the Rails Rumble 2014, same concept! <a href="http://pagepoller.r14.railsrumble.com/" rel="nofollow">http:&#x2F;&#x2F;pagepoller.r14.railsrumble.com&#x2F;</a>
ideaoverloadover 10 years ago
Looks very similiar to <a href="https://followthatpage.com" rel="nofollow">https:&#x2F;&#x2F;followthatpage.com</a> . Worked for me pretty well for non JS pages for last 3 years.
sleepingspiderover 10 years ago
&gt; Does it diff binary data.. for example, if I ask it to watch an image url will it hash the image and notify me when it gets updated?<p>Can you elaborate in what case you might want to do this?
akxover 10 years ago
Typo in the footer: &quot;Sleeping Sider&quot;
评论 #8513082 未加载
评论 #8510810 未加载
评论 #8510796 未加载
callesggover 10 years ago
What is regarded as a change?
评论 #8510856 未加载
seivuiover 10 years ago
Which timezone does &quot;8 o&#x27;clock in the morning&quot; consider?
评论 #8510809 未加载
评论 #8510792 未加载
sleepingspiderover 10 years ago
Just added HTTP authentication support.
zapt02over 10 years ago
Nice branding!
_asciiker_over 10 years ago
simple, clean and useful. I&#x27;m trying it out. Good job!