We do way more than just surveys | Medallia Engineering Blog

Medallia Engineering Blog

A look inside what we are building

We do way more than just surveys

Guido De Caso / 2018-01-04 / medallia surveys

So the story goes like this: I’m chatting with you at a Meetup or event explaining what we do. I introduce you to Medallia’s vision which is to create a world where companies are loved by their customers and employees alike. You empathize, we’ve all had crappy experiences when dealing with a company; we rejoice in having found someone who’s been through the same.

I then tell you about our growth and career opportunities, it’s all going oh so well… until you pop the question. “But you guys just do surveys, right?”.

An awkward silence ensues. Now I not only have to explain that we do a lot more than that, but also that, even if it were “just doing surveys”, that is so darn challenging!

What we do poses pretty unique challenges: dealing with extremely wide data points, supporting real-time data exploration of these and providing high availability while doing so, just to name a few. Since we have a few minutes, let’s go over all three in detail.

Thousands of values for every data point

We have hundreds of large customers and work with some of the most prestigious Fortune 500 companies. Some of these companies that have been with us for many years have hundreds of millions of feedback items in our system. “Isn’t that a couple orders magnitude lower than big data systems such as real time bidding or recommendation systems?”. You are right, but here’s the thing, each of our records consists of hundreds or even thousands of values. While other industries deal with tall, thin datasets, at Medallia we became experts in dealing with incredibly wide datasets.

There are various reasons for why our records are so wide. First, the companies we work with are now able to capture much more data from their customers compared to a few years ago. As they race to understand their customers and perform more comprehensive correlations and analytics, the number of data points captured for each business transaction is at an all-time high. Furthermore, there is nothing indicating that it is just a fad; the number of data points captured per business transaction is surely going to keep growing.

Another reason behind the ever growing dimension of each of our records is the calculated fields. Imagine you own a hotel and you capture for each of your guests their check-in and check-out dates. If you want to analyze patterns like which seasons are the busiest then that’s all you need. But let’s imagine you want to understand the trends around stay length. Well, with a complex and clever set of DB queries perhaps you could manage, but that is not very helpful if you are not a SQL expert. What would be ideal is if each record had an additional derived value which was automatically computed from the check-in and check-out dates and gave you the stay length in days. With such additional field you would easily configure all relevant analysis and reports. With hundreds of thousands of users with hundreds of different roles you can imagine how the number and diversity of calculated fields can skyrocket.


Drawings by Rodrigo Fernández.

“But, how do you handle users going wild with these calculated fields?” Yeah, end-user friendliness and Turing Completeness are hard to balance, yet with a few tools, such as an integrated step-by-step visual debugger we built, we can manage that complexity.

Any aggregation, and any slice on your latest data, now

Perhaps wide records and their challenges aren’t enough, and you are still looking at me like I’m just a survey junkie in recovery. Operational Customer Experience Management implies dealing with huge amounts of feedback at scale and empowering every user within an organization to quickly get the information they need in order to improve their customer’s experience. Let’s split that up and take it one part at a time:

To deal with huge amounts of feedback we need to scale every piece of our infrastructure to meet the needs of even the vastest organizations on earth: public or private, NGO or for profit. This goes from data storage all the way up to API gateways.

In order to empower every user we need to properly understand and model the business of each of the companies we work with. OCEM is not just for the C-suite; everyone should tap on to how customers are feeling. Complex organizations have hundreds of different roles and our system has to support bringing them the information that they need, at the right time and with the right context. Furthermore, some of these companies we work with have tens of thousands of business units. Each of those has unique rules in terms of how to treat or manage the inflow of feedback. Our analytics engine has to honor these rules even in the most complex of organizations.

We want everyone in the organization to quickly get the information they need. While some users may be interested in aggregating feedback per region over a long period of time, others will want to review feedback items one at a time and apply complex filters to find those nuggets of insight they are after. With this in mind, we built an analytics engine from the ground up to support arbitrarily complex aggregations, slicing and dicing of data, always returning the freshest data.

You’re probably thinking: “just do a SELECT SUM(xyz) FROM surveys WHERE something GROUP BY something else” or at most, “MapReduce et voilà.” But when you have hundreds of thousands of users requesting completely unknown queries, you can’t parameterize or plan each and every one of those unless it’s done in real time.

Always capturing feedback

So hopefully you’re looking at this whole thing with a new perspective now. But there’s really more to share. The companies we work with rely on our feedback gathering services as part of their customer-facing systems. If we were to have even the slightest hiccup in uptime it is not to our customers that we’d owe our apologies, but to our customers’ customers.

Designing for high availability requires you to do something quite unnatural which is to think of the worst possible scenario. Humans are not good at this; even the most pessimistic engineer will fail to think of all possible failure scenarios for a given design: What if a disk fails? What if a server goes down? What if an entire rack gets isolated from the network? What if an entire datacenter goes dark?

Our newest infrastructure is built to cope with these situations and many others. Failing to capture any customer feedback item is a really tricky situation to be in and that’s why we have to have enterprise-grade availability.

And all this without mentioning how feedback can come from other sources like social networks and review sites, or interfaces with our customer’s systems, or how feedback will increasingly come from usage data and be implicit more than explicit. But let’s leave those for future posts.

Hopefully at this point you are convinced that dealing with feedback implies trickier technical challenges that you thought it would. “So, yes,” I nod, “we do way more than just surveys.”

comments powered by Disqus