Improving Time To Insight

Forward-thinking companies can now generate a great deal of real-time, customer and market insight from their day-to-day operations.

We’re now in a world where Uber can rack up 2 billion rides across 400 cities since it launched in San Francisco just 5 years ago. A world where, in just 8 years, Airbnb now has half a million stays every night.

The data these companies generate is helping them understand what to do next, and where their future demand will come from.

These companies are developing markets before our very eyes, often using data-driven strategies. They are monetizing live data. In many cases, it’s the threat from disruptors like Uber, Facebook and AirBnB that drives real change in the marketplaces that other established companies are entrenched in. And those disruptions are becoming more pervasive, complex, and frequent, especially with the advent of Internet of Things (IoT) solution applications on the horizon, which will dramatically increase the level of data generation and analytics requirements in the next 5 years.

The result of all of this disruption spurs unstructured data models, competitive complexities, and marketplace confusion; all enormous challenges for Customer Experience (CX) and Competitive Intelligence (CI) practitioners trying to understand how to manage this tidal wave of information.

Teams are under huge pressure to drive great decisions at the speed of light, to have even a hope of keeping up. CI teams are striving to demonstrate value and ROI, and to become even more integrated in the decision-making process of the business. And in a world moving so fast, decisions need to be made more quickly and more effectively. CI has little intrinsic value in and of itself; It’s the quality and timeliness of the decisions it drives that matter.

Many current CI practices do not support the new reality, and the rapidity of information-provision it requires. “Batch” processing of information yields actionable insights that are months old by the time the organization has the ability to act or implement. Corporate organizational structures and environments are struggling. Consequently, leaders should be moving towards more reactive, nimble and flat structures that enable quick decisions and faster execution.

Additionally, the sources for data collection are mushrooming, but analyzing that data and driving action is hard. Add to that an environment where customer experience and customer journeys are so key to corporate success, and it adds up to a maelstrom for CI practitioners.

Organizational decision-making is surely at its best when based on present conditions and when decisions are made by those closest to and most responsible for the customers and products. Therefore, it is those companies that employ smart Insights teams that move away from simply amassing more data, to swiftly creating value from highly focused data, and getting that into the right hands in real time. Those companies will be successful in the 21st century.

As a result, a key insight area where CI teams should focus their efforts in order to transform their services and contribution to the business is TTI (Time to Insight). The development of skillsets and solutions in the management and reduction of the total TTI will enable CI teams to apply credible, near real-time information towards organizational strategy development. And a critical element of this real-time reflection of user preferences is to acquire and manage information from large groups of people to understand their view and generate instantaneous feedback. This will become an arbiter of success for Insights teams going forward.

Time to Insight (TTI)

Historically, CI has been done in a ‘batch’ process approach. Teams go off and collect data, analyze it, come up with a solution, propose it to management, management mulls it over, and then they decide how they’re going act. That batch timeline can be weeks at best but much more typically, it takes months.

That sort of life cycle is useless – potentially even dangerous. The way environments and our markets are changing (the amount of market disruption that’s happening, the advent of IoT, Big Data etc), huge information about customer behaviours and preferences is being generated instantaneously whilst actions in the current organizational decision-making life cycle are lagging way behind.

Additionally, customer experience and customer feedback are notoriously prone to discoloration over time, and current processes only exacerbate this issue. Therefore, it is critical for Insights teams to develop tools to quickly and unobtrusively get hold of regular feedback on a fluid, ongoing basis.

In some ways, CI teams are constrained by the very things that should allow them to go faster. There are more data flows, and a proliferation of tools and collaboration techniques. But…the quantity of data itself is frequently huge and it is often badly analyzed.  In most cases data can’t be used in its “Big” form to be actionable and actually change things. As a result, companies are drowning in data, but acting on very little of it. Accordingly, 90% of all the world’s data has been collected in the last 2 years and only 0.5% of Big Data is even being analyzed today.

As Nan Bulger, SCIP’s (Strategic and Competitive Intelligence Professionals) CEO mentioned in her opening remarks at the SCIP Europe Conference 2016, the world’s most innovative, fast-moving brands know that “technology is pointless without human touch, human action.” The sheer volume of data available to us now, in the age of the Internet of Things and Big Data, means that data is easier to come by – but unfortunately, proactive and inspired analysis is not.

time to insight

As Pirelli’s famous strapline has it “Power is nothing without control.” Computer-crunched customer data is impressive. But is it as effective as it could be? CI practitioners have the illusion of influence in the vast amounts of data they collect. But the key determinant in the success of a CI team is its ability to immediately bring that data to action.

And that is the key to successful companies  – the ability to acquire, analyze and execute on marketplace data and user experiences. This is what Time to Insight (TTI) is all about. Those companies that possess a fast CI lifecycle that arrives at the quickest Time to Insight will win! No longer can organizations operate on a “Library services” function which is largely need-based. CI teams need to push as much as pull the information into the business.

The CI process lifecycle of the future will need to be done quickly; in almost real-time. It will need to be flexible enough to allow modifications and adjustments during the CI process and be able to accept and analyze thousands of data points and process them in a constant fluid motion to arrive at the shortest TTI – while the company must be able to be nimble enough to understand those changes and act accordingly. Obviously, this is going to be a huge challenge for organizations.

Insights teams should look towards building a speedy, lightweight process that allows them to first assess groups of people of interest, and then get instantaneous feedback from those people to drive decision-making. It’s worth accepting it will not always be possible to utlilize Big Data all in one go, especially in companies whose legacy systems are older than they might like. But Insight teams can use real-time tools and techniques to selectively investigate trends and ideas, test them in real time and move forward.

The idea here is therefore is not to speed up the entire research process in one go, but to use selective, targeted insights into smaller groups to aid learning and information gathering. The benefit is that insights can be gained in minutes and not months. They can then be put into the hands of the right people to drive informed decision-making rapidly.

The choices and preferences of customers are predictive of the future

CI teams have access to a great deal of market data. But the choices and preferences of customers are potentially more predictive of the future. To go back to the earlier AirBnB example, the customer data collection and action process is so embedded in AirBnB that it’s clearly helping their execs see and act on fundamental trends.

Chip Conley, AirBnB’s Head of Global Hospitality and Strategy said in a recent interview that they were looking to their data scientists to “help understand the customer better based upon the choices that customer has made.” He went on to say that in the future, AirBnB “will be seen as a travel company more than an accommodations company.” As this quote highlights, competitive advantage is increasingly driven from customer insight.

Customers are one of the richest sources of insight, and the generation of their insight is credible and virtually perpetual. Arguably, the customer experience may be one of the only true differentiators in the future. Quick determination of Customer insight will vastly improve the comprehensive nature of the overall analysis. As we move forward it is important that CI practitioners can reach out directly to customers for insight that can be assimilated into market and competitive intelligence to further drive the TTI effort. Companies that can rapidly assimilate and act quickly on that data will win big.

customer source of insight

Getting hold of customer insight is one thing; taking advantage of it is another. Transforming this torrent of customer data into actionable intelligence requires new tools and techniques. It will be critical in the near future to be able to carve out markets, sub-markets, customer groups, partners, suppliers and more, and get real-time intelligence from them, using technqiues like micro-surveys, that can be sent, assimilated and rapidly-actioned by decision-makers.

Insights from customer data need to go to the people in the organization who ask the best questions, have problems they want solved, so they can come up great ideas based on what they are seeing happen. Those who build these real-time capabilities into their CI practices will have a strong lead on their competition, and will build real growth in their markets.

Low intrusion microsurveys

This is an area where key development efforts are most effective in improving TTI. Historically, surveys have been lengthy and unwieldy. Hard to administer, difficult to gain respondent traction, and long periods of time to develop analytics and action plans make them archaic in an age of data analysis transition. Now, the turbo-charged pace of change will herald the need for unobtrusive microsurveys that improve upon nearly all of the aspects mentioned.

Uber, AirBnB and many more companies at the forefront of customer experience offer in-app or via email simple surveys that allow them to collect data in real time to improve service. As Zander Lurie, SurveyMonkey’s CEO recognizes, “Today, every time you take an Uber, you take a 2-second survey where you’re rating your driver, and obviously those data points are helping inform them about which drivers are doing a great job, as well as [informing Uber about] the customers who drivers like. It’s using what we call people-powered data in a really refreshing way to drive their product forward.”

If practitioners are able to deploy a quick and simple survey in seconds, and see those results in real-time, they are positioning yourself, by design, of acting on the information before it’s too late. The Harvard Business Review has written on the real-time microsurvey subject, saying businesses “have long sought a research method that can capture customer reactions immediately, does not intrude into those reactions, minimizes bias, and can affordably be applied to customers in relatively large numbers. We believe that real-time experience tracking (RET)…rises to this challenge.”

Companies can now replace many traditional lengthy surveys with micro surveys to garner quick, real time insights from respondents. Micro surveys offer more real time data with a scope to quickly interpret and generate trends and patterns and to customize future insights.

In summary

Legacy CI models will be essentially ineffective in the Idea age of our markets. The onslaught of data management challenges will pose serious threats to organizations as a whole; the best practitioners need to take the knowledge from their data and tools and generate real, actionable steps in the shortest TTI.

In combination with improving TTI, greater importance must be placed on obtaining real-time customer insight. Customer behaviour today is hugely predictive of market shifts of tomorrow. CI and CX teams should proactively drive competitive strategies and new markets based on inflows of actionable customer insight. This is how organizations will be able to respond to market disruptions and become truly competitive in the 21st century.

About the authors

Paul Santilli

Paul Santilli leads the WW Business Intelligence and Customer Insights Organization for Hewlett Packard Enterprise’s OEM Business, and has been with HPE for over 20 years. He is responsible for Business and Competitive Intelligence Modeling and Customer Insights analytics, where he is the Chairman of HPE Executive Customer Advisory Boards worldwide. Additionally, Paul heads up the WW OEM Marketing & Evangelism Team, focusing on Marketing & Sales Enablement, WW Communications and Social Media platforms. Paul also is on the Strategic Competitive Intelligence for Professionals (SCIP) Board of Directors and has presented worldwide on various topics related to Intelligence and Insights in both keynote and workshop forums. Paul has a Bachelor’s degree in Engineering from the University of Michigan, and a Master’s degree in Engineering and Business at Stanford University.


lindsay willotLindsay Willott is the CEO and Founder of real-time microsurvey tool, Customer Thermometer. Customer Thermometer is used by leading brands like Dollar Shave Club, Sonos and IBM to get real time, actionable customer insight through unobtrusive surveys. Lindsay was previously the Founder and Chairman of a leading technology marketing consultancy and the Global Head of Marketing Communications for a leading CRM business. She specializes in the area where customer insight and communications meet.


Footnotes, references and sources



[3] Stephanie Hughes, Founder, Super-h, Keynote, SCIP Europe Conference, Prague, 8 November 2016

[4] Nanette J Bulger, CEO, SCIP Keynote and Opening Remarks, SCIP Europe Conference, Prague, 8 November 2016