If you use this for something, please be sure to keep the data fresh! Especially now that new TLDs are being added at a steady clip.<p>I once had an issue where a mobile user's browser would do a Google search every time they typed one of our domains into the URL bar. They weren't doing anything wrong - other domains worked fine. They were using some Android phone from 2013 which had been abandoned by the manufacturer, and it turned out the stock browser was using the public suffix list to decide if the input should be treated as a URL or as a search query. The list was as old as the browser, and since the domain ended in ".link" (added in 2014) it didn't think our domain was a domain. (The workaround was to type out <a href="http://" rel="nofollow">http://</a> or <a href="https://" rel="nofollow">https://</a> before the domain, but that's crappy UX.)<p>If you use this in a server-side app, please use cron or something to keep it updated. If you embed it in a client app, make sure to update it as part of your build process. And if you know your app won't be updated very often, consider having it update separately from the app itself. (Does anyone know if this is hosted on some public CDN somewhere? I assume having a client app fetch directly from publicsuffix.org isn't kosher.)<p>Edit: My mistake, on <a href="https://publicsuffix.org/list/" rel="nofollow">https://publicsuffix.org/list/</a> they do recommend having client apps pull directly from their site, and limiting updates to once per day.
This list is used to determine what domains and sub-domains "belong" together, in the sense that they are controlled and/or owned by the same entity. For instance *.google.com is all google, so x.google.com and y.google.com can be trusted to share the same SSL key, safely share javascript (XSS), etc. However x.blogger.com and y.blogger.com are probably two completely separate blogs, people, domains, SSL keys, javascript domains, etc. And you wouldn't want to see x.blogger.com's web pages showing up in search results for y.blogger.com.<p>Secondly, maintaining this list is a pain. You have hundreds of two letter top level domains, one for each country. Each country with its own NIC in charge of sub-domains. Each NIC with the power to add or delete subdomains (check out <a href="https://en.wikipedia.org/wiki/.uk" rel="nofollow">https://en.wikipedia.org/wiki/.uk</a> for just one of hundreds of examples). Some "countries" even sell off their top level domain like .nu . Then you have .us (<a href="https://en.wikipedia.org/wiki/.us" rel="nofollow">https://en.wikipedia.org/wiki/.us</a>) that has wild card domains like <a href="http://vil.stockbridge.mi.us/" rel="nofollow">http://vil.stockbridge.mi.us/</a> where the vil. is fixed and part of the domain and stockbridge.mi is the important domain information. Of course they're always adding more top level domains: .ninja, .wtf, etc (maybe there is a .etc now?). Then you have all the blogging and hosting platforms that use personalized domains for hosting content. Many are listed in the publicsuffix list, but I'm guessing not all!<p>I ended up writing my own publicsuffix parser in PERL a few years back for the blekko search engine. The main purpose being to be able to group web pages together by site/owner. There is nothing quite like feeding every URL on the internet through your parser to find bugs and corner cases.
A great and important list.<p>It also feels a bit like a hack. Would there a be a better way to do this, maybe in the DNS system to denote ownership/isolation?
I recently found this list and use it in a Drupal module to invoke hooks based on the domain of a URL. I didn't realize how many edge cases there are for these and originally tried to do it with regex.
I do hope they use the same care and due process as they do with root certificate inclusion in the public suffix list (as well as the ability to add/remove entries manually). I think this is a great move for better security, but with great power comes great responsibility.
Quick plug for my related project: <a href="https://github.com/QA2/public-suffix-metalist" rel="nofollow">https://github.com/QA2/public-suffix-metalist</a> - pull requests welcome.