How to build a IP geolocation database from scratch?

First, I am big fan of your articles even before I joined IPinfo, where we provide IP geolocation data service.<p>Our geolocation methodology expands on the methodology you described. We utilize some of the publicly available datasets that you are using. However, the core geolocation data comes from our ping-based operation.<p>We ping an IP address from multiple servers across the world and identify the location of the IP address through a process called multilateration. Pinging an IP address from one server gives us one dimension of location information meaning that based on certain parameters the IP address could be in any place within a certain radius on the globe. Then as we ping that IP from our other servers, the location information becomes more precise. After enough pings, we have a very precise IP location information that almost reaches zip code level precision with a high degree of accuracy. Currently, we have more than 600 probe servers across the world and it is expanding.<p>The publicly available information that you are referring to is sometimes not very reliable in providing IP location data as:<p>- They are often stale and not frequently updated.<p>- They are not precise enough to be generally useful.<p>- They provide location context at an large IP range level or even at organization level scale.<p>And last but not least, there is no verification process with these public datasets. With IPv4 trade and VPN services being more and more popular we have seen evidence that in some instances inaccurate information is being injected in these datasets. We are happy and grateful to anyone who submits IP location corrections to us but we do verify these correction submissions for that reason.<p>From my experience with our probe network, I can definitely say that it is far easier and cheaper to buy a server in New York than in any country in the middle of Africa. Location of an IP address greatly influences the value it can provide.<p>We have a free IP to Country ASN database that you can use in your project if you like.<p><a href="https:&#x2F;&#x2F;ipinfo.io&#x2F;developers&#x2F;ip-to-country-asn-database" rel="nofollow noreferrer">https:&#x2F;&#x2F;ipinfo.io&#x2F;developers&#x2F;ip-to-country-asn-database</a>

If you don&#x27;t want to do this yourself, you can actually just get Cloudflare to do it for you for free using a simple Worker since all Cloudflare requests contain approximate IP location information.<p>You can also just send a request to my URL (Cloudflare Worker operated - so it should have global low latency): <a href="https:&#x2F;&#x2F;www.edenmaps.net&#x2F;iplocation" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.edenmaps.net&#x2F;iplocation</a><p>Use it for small applications, I don&#x27;t mind. Just don&#x27;t start sending me 10M requests per day ;-)

As someone that lives in a country where the national language is not my first language, I hate websites that use IP location to make assumptions about my choice of language and it being forced on me based on a lazy assumption, when my browser is sending language headers quite clearly, and they are ignored.

It all depends on what you want to use it for and how accurate it needs to be.<p>The best way to build a geolocation service is to have a billion devices that report their location to you at the same time they report their IP to you. That&#x27;s basically Apple and Google. They have by far the best geolocation databases in the world, because they get constant updates of IP and location.<p>The trick is basically to make an app where people willingly give you their location, and then get a lot of people to use it. That&#x27;s the best way to build an accurate geo-location database, and why every app in the world now asks for your location.<p>4-square had the right idea, they were just ahead of their time.

Somewhat relevant: Google Maps can learn the location of your IP based on which locations you browse in the map. If you browse a specific location enough times, it will use that as the default location when you open Google Maps, even if you clear all cookies. (I discovered this just from using Google Maps, and I&#x27;m a little concerned by the privacy implications, considering that multiple people may share an IP address.)

Interesting but this isn&#x27;t actually how geolocation is done, right? The ARIN&#x2F;RIPE data isn&#x27;t sufficiently accurate to be useful beyond country. Commercial geolocation involves correlating client IP vs known physical location e.g. from WiFi AP or mailing a package to the user. At least that&#x27;s what I have been told over the decades.

Comments seem fairly dismissive but I actually found this really interesting. It reminds me of a task I had in my first position to add PostGIS to our database and a location based search. That was based off addresses and zipcodes.

So, at the risk of outing myself, I wrote <a href="http:&#x2F;&#x2F;www.hostip.info" rel="nofollow noreferrer">http:&#x2F;&#x2F;www.hostip.info</a> a long time ago* which used a community approach to get ip address location (&quot;is this guess wrong ? Fix it please&quot;).<p>The last time I checked (maybe a decade ago [grin]) it worked pretty much perfectly for a country, imperfectly for a region, and better-than-a-coin-toss for city resolution. All the data is free.<p>I don&#x27;t think they have it on the site any more, but I used to have a rotating 3D-cube thing (x,y,z were the first 3 octets of the address) for things like known-addresses, recent lookups, etc. I used different colours for different groups (country, continent,...) It was so old it was written as a Java applet. Yeah. I guess if I were to do it again, it&#x27;d be WebGL.<p>--<p>*: I sold it a long time ago, with the proviso that the data must always remain free. I actually didn&#x27;t believe the offer at first (it came as an email, and looked like a scam) but it went through escrow.com just fine, and I think we both walked away happy. That was almost 2 decades ago now though.

A modern version of the ping-based geoip mentioned<p><a href="https:&#x2F;&#x2F;github.com&#x2F;Ne00n&#x2F;yammdb">https:&#x2F;&#x2F;github.com&#x2F;Ne00n&#x2F;yammdb</a>

Any suggestions for geolocating datacenter IPs, even very roughly? I&#x27;m analysing traceroute data, and while I have known start and end locations, it&#x27;s the bit in the middle I&#x27;m interested in.<p>I can infer certain details from airport codes in node hostnames, for example.<p>It would also be possible - I guess - to infer locations based on average RTT times, presuming a given node&#x27;s not having a bad day.<p>Anyone have any other ideas?<p>Edit: A couple of troublesome example IPs are 193.142.125.129, 129.250.6.113, and 129.250.3.250. They come up in a UK traceroute - and I believe they&#x27;re in London - but geolocate all over the world.

These IP geolocation lookups never seen to work for me.<p>They are always multiple states off, and checking multiple different services pretty much never even seem to agree.

&quot;how to scrape an ip geolocation database&quot;<p>You know you can just run a whois query per ip you want to analyze, no point in scraping the whole ipvN space.

Step 1: Download Geolocation Database

&gt;&gt;&gt; Consider Open Source Geolocation Projects<p>Not the definition of &quot;from scratch&quot; in my book

This is a very useful .csv, what is the license? Is it free for personal and commercial use?

I think it&#x27;s interesting that the one IP range I decided to check has correct information on the ipapi.is web site, but unambiguously incorrect information in the downloadable geolocationDatabaseIPv4.csv. Somehow Bedford, New Hampshire (which came straight from WHOIS) became Bedford, Texas.<p>How&#x27;d that happen?

Question: What’s the motivation to put coordinates in one’s own WHOIS record? (geoloc&#x2F;geofeed)

I feel like a more useful and accurate way would be to buy client ip and GPS location data in bulk from one of the mobile data brokers who have their spyware embedded in zillions of popular apps&#x2F;games and then group it by &#x2F;24 or something.

Shameless self-promotion:<p>I built a page to compare IP geolocation providers: <a href="https:&#x2F;&#x2F;resolve.rs&#x2F;ip&#x2F;geolocation.html" rel="nofollow noreferrer">https:&#x2F;&#x2F;resolve.rs&#x2F;ip&#x2F;geolocation.html</a><p>I&#x27;ll work on adding ipapi.is shortly!

I expected traceroute to play a bigger part in this. If you know the route to an IP address and the location of routers, perhaps even from a few different servers, then you should be able to locate it fairly well.

Is anybody maintaining a historical archive of “IP address metadata” (which would include geolocation)?<p>If I have logs from 10 years ago, can I look up information about that IP as it was at the time?

Here is a solution for those that care about speed:<p><a href="https:&#x2F;&#x2F;www.miyuru.lk&#x2F;geoiplegacy" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.miyuru.lk&#x2F;geoiplegacy</a>

rfc8805 <a href="https:&#x2F;&#x2F;datatracker.ietf.org&#x2F;doc&#x2F;rfc8805&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;datatracker.ietf.org&#x2F;doc&#x2F;rfc8805&#x2F;</a><p>rfc9092 <a href="https:&#x2F;&#x2F;datatracker.ietf.org&#x2F;doc&#x2F;rfc9092&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;datatracker.ietf.org&#x2F;doc&#x2F;rfc9092&#x2F;</a>

a long time ago i build a project like that but instead of relying on whois. i did a traceroute to every ipv4 address avaliable. several router hops, have a reverse dns that uses some names that include city codes, (like airport codes ). most providers have a single hop for a city. so its easy to correlate the latest router hop to a city.

Thanks for sharing.<p>I have heard there is much effort to use BGP data to build GeoIP database.

Surely someone is using online shopping shipping addresses for this?

What are common use cases for needing IP geolocation?

The easiest way to get a geolocation is to ask the user. Maybe they’ll just tell you, and if that’s good enough for your application there’s no need for such solutions.

Step 1. Download Visual Basic