Hey, Do You Guys Have a List of All the Domains in [ccTLD]?

| June 12, 2013

That’s a question we get asked a lot.  So I thought I would blog some updated stats.

In order to answer that question, you have to know what ‘all the domains’ actually is, in other words, you need the denominator for a coverage ratio.  In a world where most ccTLD Registries do not publish zone files, this information is hard to come by.  Fortunately, CENTR, the Council of European National Top Level Domain Registries, published this recent report updating rounded current domain counts in the top 20 ccTLDs worldwide.  This data helps DomainTools gauge how good, or poor, our coverage is on certain TLDs, something our enterprise customers are increasingly interested in.

Unfortunately the domain count is not the hard part.  The challenge lies in figuring out which domains actually make up those lists, that is, which domains actually exist in DNS for all ccTLDs.  If you have those lists, you can actually create tools of real value for users, such as DomainTools Brand Monitor which currently follows over 100 ccTLDs, as well as all of the common gTLDs.  To put a finer point on the challenges:  when a user registers a .DE domain, for example, nobody knows but the user, the Registrar and the German Registry DENIC.  How do you find out it just got registered?

This is an ongoing challenge, and one where success is measured as a matter of degrees.  Domain discovery is one of DomainTools’ top engineering priorities, and something we are continually getting better at.  Using a combination of significant amounts of our own internal data, some external data, and a ton of computing horsepower, we’re making noticeable progress.

So with that brief overview, and in the spirit of transparency, here is a snapshot of our recent coverage of the Top 25 TLDs by volume (ccTLDs and gTLDs).  Note that .TK, as a free TLD with extraordinarily high growth and extraordinarily high turnover, is a special case that I will try to address in a future blog post.

 Snapshot of DomainTools' Recent Coverage of the Top 25 TLDs by Volume (ccTLDs and gTLDs)

            * Recent zone file count, or data supplied by the referenced CENTR report.

Share

Tags: ,

Category: brand monitor, Domain Tools Updates

About the Author ()

Tim Chen has been the CEO of DomainTools since its 2009 spin-out into an independent company. Tim has over 15 years experience as a sales and operations executive in internet-driven businesses, including 8 in the domain name and DNS space. Prior to this Tim spent seven years in business development executive roles with Internet and software companies in the Bay Area, and four years in the Corporate Finance group at J.P. Morgan in New York. Originally from Rochester, New York, Tim is a graduate of Haverford College and Stanford business school. When not in the office you can usually find Tim on his bike, on his surfboard (or falling off it), or sequestered away reading entirely too much about the Buffalo Bills' draft prospects.

Comments (12)

Trackback URL | Comments RSS Feed

  1. gpmgroup says:

    Two quick questions…

    Are you legally allowed to aggregate data from the UK WHOIS?

    Also why won’t you abide by robots.txt directives? (i.e. Why do you still show screenshots of pages where your robot is explicitly denied?)

  2. some cctlds dont publish zone file
    how do u the no.s ?

  3. Tim Chen says:

    @gpmgroup: Sure thing, happy to respond. Question 1: This is not whois data. We are just talking about knowing the list of domains which are registered in a given TLD. Question 2: unlike other screenshot services, of which there are many such as Archive.org, we do not crawl the site and pull the HTML. We simply take a snapshot of the website homepage, each of which is publicly available for any internet user to resolve in a browser at any time. We have a longer response on this issue related to the ‘fair use’ provisions used by sites like Wikipedia. Please feel free to email me at tim “at” domaintools “dot” com if you’d like to pursue this question further.

  4. Tim Chen says:

    @big data events: The ccTLD zone counts were published in the CENTR report referenced (and linked) in the blog post.

  5. Very interesting article. The growth of .cn is just incredible according to the CENTR report! It’s also interesting to see emerging ccTLD like .in, with a growth of more than 14% these last 6 months!

  6. Ajay says:

    .IN data is now 1200000 , reduced in last few months.

  7. india says:

    congratz to .in extension,
    .tk freebie, Their all domains are on parking pages.
    Thanks for the list

  8. Shailesh Tripathi says:

    These TLDs are now updating because every few months a new tld gets started. Your list is wonderful and without errors. Thanks