Game Changers

| March 25, 2013

I’m a very casual student of history and I’ve always enjoyed thinking about “game changers”.  Throughout all of human history, we have been inventors.  From fire, to the wheel, to electricity, to the computer, to the Internet, our inventions have changed (and often disrupted) our cultures and the world.

In the last 100 years, I think the electric lights, radio and television, aviation, and the Internet have been the biggest game changers.

Although invented in the 1870s, it was the widespread use of electric lights in the early 1900s that really changed the world.  It meant safe illumination that wouldn’t explode or burn the house down like gas lamps or candles.  The English language has had to add words like “nightlife” and “night shift”.  Electric lights make us feel safe at night.  Electric lights meant that our days were no longer bounded by sunrise and sunset.

The first public radio broadcast in 1910 was several opera signers. Along with television, both changed the way we think about the world. Franklin Roosevelt’s fireside chats (both on radio and TV) helped Americans through the Great Depression and World War II.  They brought the greater world into living rooms and bedrooms.  Fashion, politics, and water cooler conversations all revolved around the messages on radio and television.  Radio and television meant that we could learn about and react to something happening anywhere the world as it happened.

The Wright brothers first flew in 1904, but it wasn’t until the 1960s that commercial aviation really took off.  In 1879, Jules Verne wrote “Around the world in 80 days”.  In 2012, showed us how to travel around the world visiting 6 continents for at least 48 hours each for less than $3000.  Aviation allows FedEx and UPS to deliver a package to almost any spot in the world in 24 hours.  If radio and television told us what was happening anywhere in the world, aviation allows us to get up and go do something about it.

In the world of Information Technology, there have also been game changers.  Today, in less than a second, our systems receive a request for a whois page, look up all of the details, compose a response, and ship it to the browser.  During peak hours, our servers will answer over a thousand requests every second.  We track over 215 million active domains and have over 5 billion curent and historical whois records.  It’s no trivial feat to reliably deliver that content quickly.

For DomainTools, the game changers have been large memory systems, solid state drives, and virtual systems.

10 years ago, it was very very expensive to buy a server with more than 4 gigabytes of memory.  With improvements in manufacturing and memory technology, it’s easy to buy systems with 256 gigabytes or more of memory.  DomainTools uses a number of servers with huge amounts of memory to hold the entire index of all of the whois records in memory. It would be very difficult to answer a thousand requests per second quickly and cost effectively.

From the beginning, DomainTools used MySQL to hold most of it’s data. At first it was 10s of gigabytes to hold everything.  Today, it’s terrabytes.  Two years ago, we were approaching the breaking point of keeping our MySQL databases on hard drives.  The spinning platters of the hard drive simply could not keep up with our needs.  We faced a tough decision to either figure out how to make MySQL faster or change our software to use something else.  We ended up testing SSDs and the effect was incredible.  Despite the high cost, the SSDs were 100x faster than the hard drives and it bought us time to explore other technologies and techniques.  Today, we use more SSDs than hard drives.

When DomainTools started, as a small company, every server was precious.  Every server had to do multiple roles in order to keep the bottom line in the black.  Back in 2007, the cost of keeping our operating systems upgraded was getting to be too high.  The problem was that if a server had a MySQL database, an Apache web server, a PHP application, and a Ruby application, it was trickly to upgrade all of them without conflict.  For those who lived through those days, the “expat” library was the worst offender.  We made a hard decision to try out virtual systems.  First with VM Ware, and later the OpenVZ, the results were awesome and we have not looked back.  Over the last three years, we have upgraded almost all of our systems to virtual systems.  We have fewer hardware nodes, so our co-location costs are lower.  We use the processors we have more efficiently.  And lastly, every virtual server does exactly one thing, so the “cost” of keeping our systems up-to-date is much lower.

I’m sure there are folks out there who disagree on the three biggest technological changes over the last 100 years or in the IT world. I’d love to hear your thoughts and stories on the technologies that were game changers for you.


Tags: ,

Category: Domain Tools Updates

About the Author ()

Gustavo leads the IT and Ops team at DomainTools. More than simply supporting the network and server architecture, IT is a strategic discipline for DomainTools as the company's value is steeped in the breadth and depth of data it is able to provision for its clients. Across all the numerous data collection and processing systems the Company runs, DomainTools supports almost 200 servers and processes almost a terabyte of data per day. Gustavo's experience in software development and systems administration spans almost 20 years. He is a graduate of the University of Illinois, where he worked for the National Center for Supercomputing Applications. Gustavo has been with DomainTools for almost 6 years. Prior to DomainTools, Gustavo worked for Motorola, first in DevOps and later as a Network Admin.

Comments are closed.