In PG's Frighteningly Ambitious Startup Ideas essay he talks at length about building a new kind of search engine. I was curious, who is working on this angle and makes your search engine innovative?
I have an idea( prototype? ) for a search engine that could be disruptive, with a bit of luck. It has nothing to do with bangs and slashtags.<p>I've hired a freelancing coder to do some "wrapping up" so I could show the prototype but it is over a month now and still I haven't heard from him.<p>I do not want to engage too much time and resources into this (hiring stationary coders) but this really is the search engine that is closest to disrupting Google and Bing.<p>I believe that slashtags and bangs and non-spam results are not the way to go, my search engine tries a different route altogether which those two big engines ( Google and Bing ) couldn't probably adopt because they are simply too big.<p>I can produce results semi-manually and all I need is automation for that method but the coder I've hired, stopped responding over a month ago.<p>I really see big future in it but I do not want to invest too much effort and money into something so risky.<p>I wish I could show you what it is all about but it would reveal too much. And without the automated prototype, some good coder could simply copy it in like 2 weeks.<p>I believe that for $10000 I could present a really basic prototype of something that is disruptive to Google and Bing and if it isn't I think nothing will ever be.
Do you think Twitter has the potential to grow into a new kind of search engine (<a href="http://idealab.talkingpointsmemo.com/2012/07/what-twitters-new-search-features-mean-for-google-third-party-developers.php" rel="nofollow">http://idealab.talkingpointsmemo.com/2012/07/what-twitters-n...</a>)?<p>Unlikely I know, but their access to true real-time data presents interesting possibilities I think.
I didn't write a search engine of any kind, I wrote a search front-end of sorts which redirects you to actual search engines (or Wikipedia if your query is an exact article title match). It doesn't handle not-ASCII text very well and there are some other bugs I should iron out and sometimes Heroku is down (hosted for free so I shouldn't complain) but it works well enough for me 99% of the time so I'm not actively working on it.<p><a href="http://s.xqz.ca/" rel="nofollow">http://s.xqz.ca/</a>
I would love to see if Facebook could create a better search engine by using an individual's social network to weight links instead of backlinks.<p>It might be too much work just to see if it is useful but I'm thinking they have a ton of data about where everyone goes on the web because the Like button is so pervasive. Why not use that data to make search extremely personalized?<p>Reasons why I like this:<p>1) It's not like anyone currently uses facebook for search, they literally have nothing to lose.<p>2) It would be interesting to see if the aphorism, "Show me your friends and I'll show you your future" holds up.<p>3) It's data that google doesn't have access to, unless G+ takes off.<p>4) It would be cheaper to crawl the web, essentially your users are your web crawlers.<p>Take the same graph theory algorithms used for backlinks and apply it to views and instead of page rank use time that their friends spend on the site.<p>Then you can do some interesting stuff like weight the page rank more heavily for individuals you interact with more.<p>Or super creepy (but statistically interesting) stuff could be done like if you want someone to reconnect with a friend they haven't spoken to in a while, weight the "long lost friend's" pages more heavily. Essentially, influence people to talk to each other by converging their data about the world.<p>EDIT: Another thought - if websites know that they're only admission into the search engine is putting a Like button on their page it gives them an incentive to do so.
I am working on a very specialized search engine for software libraries. You can find it here: <a href="http://www.versioneye.com" rel="nofollow">http://www.versioneye.com</a>.<p>I don't think that we need better search algorithms for search engines. I think we just need more filters in the internet. With filters I mean more specialized search engines who are not crawling everything.
If I am typing in "Spring" into the search field, I am not interested in "Spring", I am interested in the spring framework. But because most of the people on this planet are not software developers and Google don't know that I am a developer, the spring framework will not be the first result on google.<p>Of course we need general search engines like google. But I would love to use more specialized search engines who are just showing results to a special topic.<p>I would like to hear your thoughts about this.
I consider writing Digest for a 3rd time. The first time was a proof of concept in Perl, the 2nd time was product in Java for [name of customer deleted]. The 3rd version could be in PHP for the masses.<p>Digest is not a normal search engine, but an XML description language to tell a spider where to find what on a web layout. Its aimed on the dark side of the internet, able to login on website, use cookies, and even delegate captcha solving.<p>The main problem of Digest is that its maintenance intensive because websites are changing layouts regularly. The idea of a Digest for the masses would be an open source distributed search engine, that runs on normal peoples web servers. So every (power) user has access to the search engine itself.<p>My problem, and question to this audience is: How to make money with such an project? The only idea I have currently is kickstarter.
Shameless plug: we're working on a product focused solely on opinions called Cheerboo (<a href="http://www.cheerboo.com/" rel="nofollow">http://www.cheerboo.com/</a>).<p>It's part search/discovery part social, and is trying to solve the problem that all the opinions shared in places like Twitter/Facebook aren't preserved, aggregated or made useful outside of the social graph.<p>We organize opinions around topics, make people weigh in on topics with a binary rating system (cheer or boo) and use the cheers/boos to give each topic a "RottenTomatoes-style" score. Let us know your thoughts - we love feedback.
Shameless plug: My startup, Semantics3 (<a href="http://www.semantics3.com/" rel="nofollow">http://www.semantics3.com/</a>), is building an vertical based search engine for products data - track product pricing changes, get detailed metadata etc. Our current strategy is to monetize on selling the indexed data as part of a pay-as-you-go api.<p>Later on we plan to build a consumer facing website/search engine, essentially a front-end for our api, once we have moved into building large indexes around people and places (the two other verticals we are targeting)
Does there exist any non-commercial search engine? When I say "non-commercial", I don't just mean a search engine that doesn't make money, I mean a search engine that excludes <i>all commercial websites</i>. You accept credit cards? Sorry, you're out. You have an ad on your page? Out. Simple as that. I suspect this view of the web could bring back some of the old magic feeling. It would include Terry Tao's blog, and all of Wikipedia, but exclude the SEO'd crap that Google has incentivized.
Privacy, access to different engines (when there is an API), reactive search (refine your search within the results).
<a href="http://kligl.com" rel="nofollow">http://kligl.com</a>
Speaking of which, there are 'open' indexes of the web that anyone can download and use to build their own search product. But I can't remember the name of them. Can anyone help? Googling it results in dismal failure.<p>I think there is still huge room for innovation in search. Google has taken a different track, and they're focusing on Machine Intelligence. (in my opinion) search is just a nice side effect, but I really don't think it's their focus any longer.
Maybe it's just me, but search engines suffer from a vicious circle. Noone uses your search engine unless you have enough indexed pages, and you can't index the entire web (or even a small portion of it) unless you have enough generated $$$ for high-performance servers.<p>That probably explains why search engine space is dominated by a few power players, with newer players like DuckDuckGo being used by only a dedicated bunch of fans.
My side project isn't exactly a search engine, more of a front end to get recent information from blekko, twitter, Facebook and Google plus.<p>www.unscatter.com<p>I use it to keep up to date with topics on a regular basis and also when searching for recent info on topics<p>I haven't heard about common crawl mentioned in another comment, going to go check that out now.
Not exactly a new kind of search engine, but <a href="http://www.millionshort.com" rel="nofollow">http://www.millionshort.com</a><p>I think it strips out the top however many results you get from google. It can possibly (depending on the search), get rid of SEO heavy results that don't have much useful content.
for finding stuff again (~one third of all your searches)<p><a href="http://archify.com" rel="nofollow">http://archify.com</a> - What You See Is What You Search
is your private search engine for web/FB/TW/email/..<p>Disclaimer: I'm archify team member