Hey, Do You Guys Have a List of All the Domains in [ccTLD]?

| June 12, 2013

That’s a question we get asked a lot.  So I thought I would blog some updated stats.

In order to answer that question, you have to know what ‘all the domains’ actually is, in other words, you need the denominator for a coverage ratio.  In a world where most ccTLD Registries do not publish zone files, this information is hard to come by.  Fortunately, CENTR, the Council of European National Top Level Domain Registries, published this recent report updating rounded current domain counts in the top 20 ccTLDs worldwide.  This data helps DomainTools gauge how good, or poor, our coverage is on certain TLDs, something our enterprise customers are increasingly interested in.

Unfortunately the domain count is not the hard part.  The challenge lies in figuring out which domains actually make up those lists, that is, which domains actually exist in DNS for all ccTLDs.  If you have those lists, you can actually create tools of real value for users, such as DomainTools Brand Monitor which currently follows over 100 ccTLDs, as well as all of the common gTLDs.  To put a finer point on the challenges:  when a user registers a .DE domain, for example, nobody knows but the user, the Registrar and the German Registry DENIC.  How do you find out it just got registered?

This is an ongoing challenge, and one where success is measured as a matter of degrees.  Domain discovery is one of DomainTools’ top engineering priorities, and something we are continually getting better at.  Using a combination of significant amounts of our own internal data, some external data, and a ton of computing horsepower, we’re making noticeable progress.

So with that brief overview, and in the spirit of transparency, here is a snapshot of our recent coverage of the Top 25 TLDs by volume (ccTLDs and gTLDs).  Note that .TK, as a free TLD with extraordinarily high growth and extraordinarily high turnover, is a special case that I will try to address in a future blog post.

 Snapshot of DomainTools' Recent Coverage of the Top 25 TLDs by Volume (ccTLDs and gTLDs)

            * Recent zone file count, or data supplied by the referenced CENTR report.


Tags: ,

Category: brand monitor, Domain Tools Updates

About the Author ()

Tim Chen is a proven business development and operational executive, having worked in and around Internet and software companies for over 15 years. Tim has been the CEO of DomainTools for four years, appointed during the May 2009 spin out of DomainTools from Thought Convergence, Inc. (TCI). Before joining DomainTools, Tim spent a year as TCI's Vice President of Corporate Development and 2.5 years leading the domain acquisition efforts for Internet REIT, one of the first venture funded domain investment companies. Tim started in the domain world in 2005 by incubating new business ideas with Mark Pincus (Founder of Zynga) and a small team of engineers in San Francisco. Prior to focusing on the domain and DNS communities, Tim spent seven years in business development executive roles with Internet and software companies in the Bay Area, and four years in the Corporate Finance group at J.P. Morgan in New York. Originally from Rochester, New York, Tim is a Phi Beta Kappa graduate of Haverford College, has an MBA from Stanford University, and sits on the board of NewRetirement.com, an automated retirement planning service.

Comments (12)

Trackback URL | Comments RSS Feed

  1. gpmgroup says:

    Two quick questions…

    Are you legally allowed to aggregate data from the UK WHOIS?

    Also why won’t you abide by robots.txt directives? (i.e. Why do you still show screenshots of pages where your robot is explicitly denied?)

  2. some cctlds dont publish zone file
    how do u the no.s ?

  3. Tim Chen says:

    @gpmgroup: Sure thing, happy to respond. Question 1: This is not whois data. We are just talking about knowing the list of domains which are registered in a given TLD. Question 2: unlike other screenshot services, of which there are many such as Archive.org, we do not crawl the site and pull the HTML. We simply take a snapshot of the website homepage, each of which is publicly available for any internet user to resolve in a browser at any time. We have a longer response on this issue related to the ‘fair use’ provisions used by sites like Wikipedia. Please feel free to email me at tim “at” domaintools “dot” com if you’d like to pursue this question further.

  4. Tim Chen says:

    @big data events: The ccTLD zone counts were published in the CENTR report referenced (and linked) in the blog post.

  5. Very interesting article. The growth of .cn is just incredible according to the CENTR report! It’s also interesting to see emerging ccTLD like .in, with a growth of more than 14% these last 6 months!

  6. Ajay says:

    .IN data is now 1200000 , reduced in last few months.

  7. india says:

    congratz to .in extension,
    .tk freebie, Their all domains are on parking pages.
    Thanks for the list

  8. Shailesh Tripathi says:

    These TLDs are now updating because every few months a new tld gets started. Your list is wonderful and without errors. Thanks