TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Googlebot's recent improvements might revolutionize web development

119 pointsby workhere-ioalmost 11 years ago

23 comments

jacquesmalmost 11 years ago
One potential problem here is that google will use this to widen the gap between it and the &#x27;one page apps&#x27; web and other search engines (such as duckduckgo) that can&#x27;t match it in resources.<p>How strong of an advantage that will be in the long run is uncertain, I would rather see a web that ships pages with actual content in them than empty containers for a variety of reasons (most of which have to do with accessibility and the fact that not all clients are browsers or even capable of running javascript).<p>This &#x27;new web&#x27; is going off in a direction that is harmful, coupled with the mobile app walled gardens it is turning back the clock in a hurry.<p>I&#x27;m fairly sure this is <i>not</i> the web that Tim Berners-Lee envisioned.
评论 #7805561 未加载
评论 #7806171 未加载
评论 #7806193 未加载
评论 #7805990 未加载
评论 #7805864 未加载
评论 #7807779 未加载
评论 #7808581 未加载
评论 #7806629 未加载
评论 #7805583 未加载
评论 #7806353 未加载
andrenotgiantalmost 11 years ago
Taking a step back: The &quot;Page&quot; paradigm is still very much alive, despite these recent javascript parsing advances.<p>1. Google still needs a URL-addressable &quot;PAGE&quot; to which it can send Users.<p>2. This &quot;PAGE&quot; needs to be find-able via LINKS (javascript or HTML) and it needs to exist within a sensible hierarchy of a SITE.<p>3. This &quot;PAGE&quot; needs to have unique and significant content visible immediately to the user, and on a single topic, and it needs to be sufficiently different from other pages on the site so as not to be discarded as duplicate content.
评论 #7805836 未加载
评论 #7806169 未加载
blauwbilgorgelalmost 11 years ago
Create web applications, rank as a web application.<p>Create web pages, rank as a web page.<p>This is a band-aid by Google. Developers created inaccessible websites (JS-only, no HTML fallback) and Google still wanted to give those sites a chance to be in the index. Like when Google made it possible to index text inside .swf movies. This did not mean that flash sites suddenly ranked alongside accessible websites. No, it only meant that you could now find content with a very targeted search query.<p>Don&#x27;t think you are gaining any SEO-benefit from one-page JS-only applications, just because Google made it possible for you to start ranking.<p>And don&#x27;t forget your responsibility as a web developer to create accessible content. Forgetting progressive enhancement, fallbacks, a noscript explanation for why you need JS, ARIA is devolution. If Google can index your site, but a blind user has a problem with your bouncy Ajax widget, then you failed catering to all your users. If you lazily let Google repair your mistakes, then soon you will be a Google-only website.
评论 #7809397 未加载
评论 #7809334 未加载
rcsorensenalmost 11 years ago
Sending the framework of a page to your users and expecting them to do all the heavy lifting and slow loading of constructing the page and fetching the data is still rather unfriendly if you can afford a server to construct it.<p>If you love your users, give them HTML and let the Javascript enhance it.<p>Projects like Facebook&#x27;s React ( <a href="http://facebook.github.io/react/docs/top-level-api.html#react.rendercomponenttostring" rel="nofollow">http:&#x2F;&#x2F;facebook.github.io&#x2F;react&#x2F;docs&#x2F;top-level-api.html#reac...</a> ) and Rendr (<a href="https://github.com/rendrjs/rendr/" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;rendrjs&#x2F;rendr&#x2F;</a>) let you use server rendering as well as the single page technologies on the client side.
评论 #7806191 未加载
评论 #7805996 未加载
tragicalmost 11 years ago
Like many, I am suspicious of the rather overbearing claims made on behalf of the SPA architecture.<p>I just launched a website. It&#x27;s a weekly periodical with political analysis, word-count on articles 1500-6000. It needs to carve up the content in a few different ways (categories, issue numbers etc), decorate an article with links to other relevant content, and provide a nice CMS for non-tech people to use. So it&#x27;s on Django, with the regulation sprinkling of JQuery. (If it were only techies updating it, you could probably do it with a static site generator...)<p>To me, the idea that you&#x27;d try and force something that is plainly a big collection of pages into a &#x27;single page&#x27; is just philosophically bizarre, like printing Moby Dick on a square mile of paper, using some amazing origami skills to present it to the reader, all in order to save a bit of effort at the paper mill.<p>The googlebot business is one aspect of a bigger issue, which is that a website needs to be consumable by a host of different clients. I don&#x27;t see how you can do the SPA thing without making major assumptions about those clients.<p>Sometimes, of course, those assumptions can be justified - it depends on the job. And Angular etc are enormously fun to play with, and handled well can enable a great UX for certain jobs. But I don&#x27;t think it&#x27;s &#x27;the future&#x27;. It&#x27;s another tool in the box.<p>Relevant here, a nice talk by John Alsopp from Full Frontal 2012:<p><a href="https://www.youtube.com/watch?v=KTqIGKmCqd0" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=KTqIGKmCqd0</a><p>EDIT: clarification
xpose2000almost 11 years ago
I&#x27;m not sure if this announcement changes anything. The bottom line is to make apps for the end user. Google is simply saying that those best practices are now crawlable in a way that is very mature. The same rules still apply.<p>A simple guide can be found here: <a href="https://developers.google.com/webmasters/ajax-crawling/" rel="nofollow">https:&#x2F;&#x2F;developers.google.com&#x2F;webmasters&#x2F;ajax-crawling&#x2F;</a>. Although I suspect it needs to be updated since its from 2012.<p>If you create an application, make sure it alters the URL when applicable. For simple apps, the following repos will be useful:<p>The old way, that still works: <a href="https://github.com/asual/jquery-address" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;asual&#x2F;jquery-address</a><p>The better way, preferred: <a href="https://github.com/browserstate/history.js" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;browserstate&#x2F;history.js</a> or <a href="https://github.com/defunkt/jquery-pjax..." rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;defunkt&#x2F;jquery-pjax...</a>. not sure which is better to be honest. Feel free to chime in.
评论 #7805400 未加载
ssorallenalmost 11 years ago
&gt; Single page apps are not a new concept, but up until now they were typically a bad solution for public websites that depend on hits from search engines<p>If your users (I&#x27;m talking humans, not bots) have to download a mountain of JavaScript and execute it before seeing any content, your site is slower than it could be for everyone. We should stop saying that &quot;single page apps&quot;, i.e. sites rendered in JavaScript in a browser, are bad because they can&#x27;t be scraped by a bot. They are bad for EVERYONE who wants to view the site because of the network and CPU time it takes to download the assets and render the site in the browser.
评论 #7806140 未加载
acqqalmost 11 years ago
So you want to have the URL for every content but you don&#x27;t want to provide the content as HTML, instead expecting that once the page is by client, the client only then separately loads the content? Only because it&#x27;s easier to you to program, you want to deliver me the content much slower than you can?<p>There is some strange logic there.
评论 #7806239 未加载
评论 #7806222 未加载
评论 #7806425 未加载
grey-areaalmost 11 years ago
<i>With the new improvements to Googlebot, single page apps will likely advance from being niche solutions for non-public websites to being the default way to build websites. A website will contain a single HTML page (typically heavily cached and served via a CDN). The JS on that page will then fetch content (as JSON) from the server and change the path as necessary using pushState.</i><p>I find the cheerleading for single page websites disconcerting and the proposed benefits unconvincing. Why should this be the default way to build websites? A few desultory upsides are presented without a full consideration of the multiple downsides to client-side development.<p>The biggest advantage of thick-client architecture is sending less data to the client and, if you like using javascript, writing everything in js, but there are multiple downsides compared to more traditional thin-client websites - load times which depend more on client capabilities (hugely variable and out of your control) than servers, dependence on js on the client, loading pages while your content is placed in the dom by js, forcing everyone to write in js instead of switching language on the server whenever they like, ignoring the simple document model of html served at predictable URIs, which has served the web so well and means you can use dynamic or static documents, full documents can be cached for very quick serving and by intermediaries, etc, etc. Of course some of these can be overcome, but there are serious obstacles, and the advantages are meagre to non-existent unless you enjoy javascript and feel its the only language you&#x27;ll ever need.<p>For someone who doesn&#x27;t like working in js, and&#x2F;or doesn&#x27;t have a huge amount of logic already in js (many websites work just fine with some limited ajax), trying to force every website into the procrustean bed of client-side development is not an appealing prospect. I can see why it appeals to those who have already invested in js frameworks, but predictions of its future dominance on the web, like predictions that eworld, activex or mobile would replace the web, are overblown.<p>I suspect the birth and death of Javascript will be a footnote in the history of the web, rather than taking it over as this article suggests. If anything we should be looking to replace our dependence on js, not making it mandatory.
needsalmost 11 years ago
I can&#x27;t believe that single page webapp are easier to write than true old website. Maybe you can gain some performance improvement but if you use a framework you will loose this very little gain. Those who claims that developpement is easier with framework on a single page have too learn programming, because for most case, the &quot;old&quot; way works very well and is incredibly faster than a bloated javascript page.<p>I really, really dislike this approache of doing website, it make the code and the design hard to understand and cost a lot of problem when comes the time to debug or made major changes.<p>There are no magic when using javascript, it will slow down the client, and manipulating DOM is very slow. Doing things server side cost nothing compared to javascript. Remember that loading a webpage is very fast when you have only a CSS and html code, because it very easy to put it in cache and doing some pretty nice optimizations on it.<p>With frameworks, making &quot;webapp&quot; become a huge nigthmare, things becomes overly complex and bloated, the request as to pass trough a lot of layers before end up somewhere, and the developpement is not faster than have a custom code. Good framework don&#x27;t make good programmer.<p>When the article say &quot;put the CSS inside a &lt;style&gt; tag on the page - and the JS inside a &lt;script&gt; tag&quot;, it&#x27;s just horrible, fuck it.
评论 #7806256 未加载
CMCDragonkaialmost 11 years ago
Not every single search engine will be able to scale a JS virtual machine for all the pages they need to index. There are also social network bots that you might like supporting such as Facebook and Twitter which will not be able to crawl javascript either.<p>At any case, if you want to have a solution to this SEO problem now, I created SnapSearch (<a href="https://snapsearch.io" rel="nofollow">https:&#x2F;&#x2F;snapsearch.io</a>)
bhartzeralmost 11 years ago
I wouldn&#x27;t actually call this &quot;recent&quot; improvements. I mean, Google has been handling JavaScript for years now. And they&#x27;re just now coming out and publicly saying it. Which is typical Google.
评论 #7806490 未加载
drakaalalmost 11 years ago
Optimism rather than fact.<p>There is a lot more to it. I am pretty well known as an SEO, and while I would love this to be true it isn&#x27;t.<p>Google&#x27;s improvements to GoogleBot are mostly targeting spam, and obfuscation of content. The ideas is not to discover content as much as it is to avoid having content hidden.<p>Previously you could have a webpage that appeared to be about Puppies, but then used any number o dynamic methods to instead show naked cam girls. Google worked to fix this.<p>Google is now doing some indexing of named anchors, and this allows for linking to a page with in a page as it were. But that is a Long ways from building indexable single page applications.<p>-Brandon Wirtz SEO (formerly Greatest Living American) <a href="http://www.blackwaterops.com" rel="nofollow">http:&#x2F;&#x2F;www.blackwaterops.com</a>
basseqalmost 11 years ago
This is a great improvement, but I&#x27;m struck by two things:<p>1. The state of JavaScript-only application development is still nascent. The number of JS-only sites I see that are buggy, don&#x27;t use PushState correctly, or have other shortcomings is growing faster than the overall trend. Not that it can&#x27;t be done well, but if your JavaScript-only &quot;app&quot; is really just a standard website, you might want to re-think your approach.<p>2. There has to be a better way. There are distinct benefits to approaches in caching content and providing feedback, but JavaScript seems to be a kluge-y approach. It reminds me of frames back in the day. Some of this is browser support; some of this is lack of standardization; some is perhaps a missing piece of the HTML spec; etc.
workhere-ioalmost 11 years ago
Author here. Some of you are saying that this will lead to bloated, JS-heavy websites. I disagree. The JS necessary for making a single page app can be done with something like 10 lines of JS (plus jQuery or something similar, but that is already included in most normal pages anyway).<p>A single page app isn&#x27;t JS-heavy by definition, and a &quot;normal&quot; page (with HTML generated on the server) can easily be JS-heavy. It all depends on how you program it. Just keep in mind that single page apps don&#x27;t necessarily need to use heavy frontend frameworks such as Knockout, Ember or AngularJS.
CHY872almost 11 years ago
This seems like an odd thing to be shouting about.<p>As far as I can see, throughout this thread the performance benefits of single page apps are touted as being fantastic, making it worthwhile to use the new technology etc.<p>When has performing operations efficiently ever been the domain of the web? Websites in my experience have the worst performance of almost any software I use! I&#x27;ve seen developers cite 200ms or longer to load a page as being a good benchmark - that seems pretty awful to me.<p>If getting this tiny performance improvement (which often results in poorer performance on the first load (not ideal for many)) is so critical, why do the same developers not invest in writing more performant server apps? Yes, often the database is a bottleneck, but these problems can in general be worked around (either by use of faster queries or caching etc).<p>Why attempt to get a small performance benefit by saving 30-odd kB of HTML on each page load (static and so essentially free for the server), when one could get a much larger performance benefit by optimising the backend?<p>Almost all serious sites will still see their page load being limited by the time it takes to produce the page. It&#x27;s possible to write really fast websites (try <a href="http://forum.dlang.org/" rel="nofollow">http:&#x2F;&#x2F;forum.dlang.org&#x2F;</a>) but no one seems to do it :(<p>If anything but almost all of your website is static, you won&#x27;t be saving all that much time.
评论 #7807109 未加载
PinguTSalmost 11 years ago
The single page app has another _huge_ drawback.<p>The reload via JS fails silently, when you are on a bad Internet connection, like I am currently here in Nepal. You can not simply do a reload, like with a simple HTML page.<p>For example, Facebook is here unusable, because of the same issue. It works only, when you request the mobile site of Facebook in the browser.<p>So please, get rid of that damn JS, if you care about your user base and usability.<p>BTW: that problem also happens on bad hotel Wifi in the US.
adamconroyalmost 11 years ago
Somewhat off topic, but how do single page apps deal with people hacking the JS? For example, if as a particular user I am only allowed to perform certain functions within the app, and that functionality is contained in the JS, then it doesn&#x27;t seem like it would be very hard to modify the JS to enable the functionality I shoudn&#x27;t be allowed to use.
评论 #7808763 未加载
allendoerferalmost 11 years ago
This news article seems to come up every few years. Nevertheless, the niche of apps, which can profit of this, is quite small.<p>Either you have a highly interaction-heavy web-app, where it makes sense to execute most of the code on the client and deliver the content as JSON or you have a content-heavy website, where it makes sense to deliver cached content to the client.<p>There are some apps in between, which are highly interactive and content heavy like web versions of social apps. For them additionally the question arises, if they <i>want</i> to be crawled by Google or Google wants to index their content.<p>To profit from this, you need an app, which content the users search for and interact with several times after they have found it. So i guess, &quot;revolutionize&quot; seems a bit much to me.
slashdotaccountalmost 11 years ago
Please use this instead: <a href="https://en.wikipedia.org/wiki/Progressive_enhancement" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Progressive_enhancement</a>
评论 #7808519 未加载
kayoonealmost 11 years ago
Loading off most of the work to the client has its downsides too. If that was a reality i would assume that mobile devices would require quite a bit more battery power to render basic webpages.
h1karualmost 11 years ago
google is not the only search engine.
评论 #7808854 未加载
justinphalmost 11 years ago
We&#x27;re still using hypertext transfer protocol, right? You need to send hypertext down the wire.<p>We shouldn&#x27;t let one company, google, dictate how the web works, simply because of their proprietary technological innovation.
评论 #7805459 未加载
评论 #7805485 未加载
评论 #7805961 未加载
评论 #7805556 未加载