Running Millions of Queries Per Day in the Cloud

When thinking about the term cloud analytics, people tend to focus on “cloud” at the expense of “analytics.” Though misguided, it’s easy to make assumptions about analytic capabilities that exist in a cloud solution when the discussion is centered on independent scaling of compute and storage, on-demand consumption, billing granularity, auto-this, and super-duper-that. The masses can become enamored with the sizzle of the new, and forget about the importance of the day-to-day capabilities behind the cloudy veneer.
I don’t blame them because it’s natural to get jazzed about cool new capabilities that were economically inefficient in the past, yet are now totally feasible (and fun!) with a cloud consumption model: things like scaling compute to “surge” processing when really needed—then reverting to baseline to optimize spend—or completely shutting down an environment and paying only for what’s used.
 
Using a real estate analogy, the curb appeal of some cloud analytics offers might seem alluring. But unless you’re thinking about what it’s like to actually live in that house on a daily basis—commutes to work and school for each family member, maintaining the garden, enduring the heat and cold, and sharing fences with neighbors—you’re not considering both sides of the coin.
 
It’s the same with cloud analytics. There’s a big difference between what many providers call analytics and what Teradata refers to as “enterprise analytics at scale.” When we say enterprise analytics, we mean advanced analyses involving many data types potentially spread across myriad geographically-disparate repositories that are constantly being updated, added to, and accessed. One may not know the questions in advance, but you can be sure that with Teradata software it will be possible to uncover the answers—and fast.
 
Mind you, this isn’t basic reporting on how many widgets were sold at location X in month Y, but rather complex modeling for variables such as market basket analysis, fraud detection, churn prediction, next best offer, and more. Consider trying to understand a customer profile who bought Product A in conjunction with Product B when the price of B was below C dollars and the temperature was above D degrees on days when there was a home game for the local sports team with humidity above E percent but not when Product F was positioned at eye level within G feet of Product A.
 
You get my point. There’s analytics, and then there’s Teradata analytics.
 
Likewise, when we say “at scale” we mean it: dozens or hundreds of terabytes (if not petabytes), hundreds of applications, thousands of users, millions of queries, and billions of rows and columns being accessed—per day. This is the opposite of kid stuff. We’re talking about enormous workloads that drive what industry-shaping companies do to get and stay ahead of competitors.
 
Here’s a specific example: a global consumer packaged goods company and longtime on-premises Teradata customer had a mandate to vacate their data centers and move to the cloud—without changing any application code. That’s right. The mandate was to move their entire enterprise analytic environment to the public cloud without any changes to applications and do it quickly. Because their Teradata environment was at the heart of their ecosystem, they were extremely sensitive to any potential reduction in performance, capabilities, reliability, or availability.
 
We understood and embraced the challenge. Working closely with counterparts within the customer organization, as well as teammates working for the public cloud provider, we migrated the entire environment—production, test/dev, and disaster recovery systems—from on-premises to the cloud in just a few weeks.
 
The result is impressive: 16 nodes of large, high-performance compute instances; 320 terabytes of customer data space; thousands of users querying the system constantly; and more than 25 million queries per month—all with zero changes to applications. The customer is happy, end users are delighted, data is being added, and the Teradata system is doing its job in the cloud with gusto. Check out this short video that illustrates their newfound cloud elasticity capabilities.
 
When Teradata thinks about cloud analytics, we take an “analytics first” perspective—analytics in the cloud rather than the other way around. Enterprise requirements are central to our mission, and cloud is simply a Teradata consumption model with many flexible, portable deployment options that make it easy for organizations to move, span environments, and change. We’re careful to not confuse the “how” (cloud) with the “what” (analytics).
 
In the end, we recognize that cloud—indeed, analytics itself—is a means to an end for our customers. Ultimately what they want is the ability to quickly, conveniently, thoroughly, and accurately extract the most value from their data to make the best decisions possible. The rest is just context. As Clayton Christensen would put it, we never forget customers’ jobs to be done.
 
To learn more about analytics in the cloud, follow the conversation at #CloudExperts or #BuiltForTheCloud, or reach out to your Teradata account executive.