Why It Is So Hard to Make Data Work For You

In 1947, Thor Heyerdahl sailed from Peru to Tahiti on a balsa raft called the Kon-Tiki to give credence to the theory that Polynesia was populated by peoples of South America, and not Asia. They sailed 4300 miles across the Pacific Ocean, faced nasty storms and many deadly threats. Yet, surprisingly, the point at which they truly feared for their lives was the inevitable part of the journey that nobody had given much thought to: the landing.

Upon sight of land with no engines, they found themselves going straight for a lagoon separated from the sea by a razor sharp coral reef creating massive waves. The crew had to improvise and surf the waves to reach solid ground.

They made it, but by all accounts it was the deadliest part of the trip.

This story is sobering. These men were all sea professionals who had seen battle in WWII, understood the value of preparation and forethought, and had had 100 days at sea to think through the landing…..and somehow, after all of that, it was fortuitous improvisation at the last minute which saved them.

Now, why do I like this story? Because I am a data geek.

I have been in the business of helping customers with data for more than a decade. I have worked with many great professionals (data warriors not unsimilar to the likes of Thor and his men) whose projects failed to reach production. I’ve worked on price optimization for retailers, segmentation and churn models for telephone operators, early warning systems for car manufacturers, affinity models for banks. All of these projects suffered from the same problem: the analytics was the easy/fun part. The hard part was:

• getting fresh data
• making sure the result was actionable

The “getting fresh data” bucket includes nasty things like: Is your data the latest one? How do you know? Has anything changed in it? How would you know? Did it change to the point where your initiative still make sense?

And what is especially difficult about data is working out the seams of all the systems, as the seams are always the most vulnerable part – kind of like the landing point for our old friend Thor Heyerdahl.

What is even more frustrating is that none of what is traditionally cool about working with data happens at the seams. This is not where you find the limits of computing. This is where very silly boring things break everything. Examples that will make you cringe include:

• column titles that are not expected
• separators that are not expected
• new columns that are not expected
• empty values that are not properly handled
• files that are too large and get cut off arbitrarily
• getting the same data twice (my personal favorite….)
• getting data late
• getting a partial data update

Now, let’s tie this into marketing. Think of what marketing departments do: they segment their customers so that they can offer them relevant items in order to get them to buy more stuff. Examples of this are demographic and behavioral segmentation.

A very common segment is activation: getting someone who just signed up to make a first purchase.

Steps include:

1. Pull List
2. Update promotion system
3. Get creative
4. Merge creative with list and send to Execution System (email sender, push sender, ….)
5. Execute campaign
6. Measure results of campaign

For this to work, the data that you pull must be fresh and the delay from the pull to execution must be as short as possible. Otherwise you risk including people in that list who do not fit the bill.

Notice that many systems (and then also those dreaded seams!) are involved, and this is inevitable. Now, all of this could be automated, but in most places it is not. It is an actual human being making sure that there are no loose seams about to fray the system, the campaign, and precious relationships with potential customers. That’s a lot of pain for marketers to deal with on a daily basis.

We, ActionIQ, are revolutionizing the above. Providing a system which ingests your data – seamlessly, accurately, in minutes – not in hours, days or weeks. In doing so, we make sure the dreaded seams don’t derail your most precious resource – your data – your portal to your customers needs, dreams and desires.