{'fraction_non_alphanumeric': 0.025952960259529603, 'fraction_numerical': 0.006758583400919167, 'mean_word_length': 5.197654941373535, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Big Data , Big Headaches : An Agile Modeling Solution Designed for the Information Age

The surge of data and data sources in marketing has created an analytical bottleneck in most organizations. Analytics departments have been pushed into a difficult decision: either purchase “blackbox” analytical tools to generate efficiencies or hire more analysts, modelers, and data scientists. Knowledge gaps stemming from restrictions in black-box tools or from backlogs in the work of analytical teams have resulted in lost business opportunities. Existing big data analytics tools respond well when dealing with large record counts and small variable counts, but they fall short in bringing efficiencies when dealing with wide data. This paper discusses the importance of an agile modeling engine designed to deliver productivity, irrespective of the size of the data or the complexity of the modeling approach. INTRODUCTION We live in an era where the exponential growth in big data is common knowledge. Words such as petabyte, exabyte, zettabyte, ... are starting to enter common business lexicon. In an article published by Analytics Magazine, IDC reported an increase in big data technology and services from $3.2 billion in 2010, to $16.9 billion dollars in 2015 (IDC). Pressure on analytics teams to evaluate this ever-growing flow of data in search of insights is intensifying. IDC further identifies a shortage of analytics resources as "an inhibitor of adoption" (IDC). In an article recently published by Forbes Magazine, McKinsey Consulting addresses the challenge of achieving better marketing analytics. Organizations have the problem of "trying to analyze ever greater data sets in an effort to uncover a killer insight. Or they look for a tool that can solve every problem." However, the core issue at hand, is a disconnect between analysts and decision makers. According to McKinsey, effective application of analytics can free up 15%-30% of total marketing budget. Forbes concludes that the key to a successful marketing analytics is to use tools that are comprehensive, easy to use and fast... (Thomas Bauer, Lars Fiedler, Jeff Jacobs, and Dennis Spillecke, 2016) While the Forbes paper carries on discussing the need for an overarching tool to evaluate marketing efforts to concentrate on higher valued projects, this paper approaches the challenge from a different angle. It focuses on the importance of rebalancing analytical resources, so they can shift energy towards more strategic and data-driven insights with their business counterparts. As a step towards this objective, this paper also presents a SAS®-based tool to safely free-up resources in more tactical analytics such as a campaign targeting. It outlines the problem with many existing tools when dealing with big data with very high "variety" of information, and proposes a solution to consider. RESOURCE ALLOCATION PARADOX Explosion in data sources and organizations' growing investment in data has lead to a continuous growth in analytics demands. Typically, request for analytics grows exponentially as data sources grow. Higher variety in data raises more challenges from business. CHALLENGE OF THE GROWING RESOURCE MODEL According to a DataMatics survey by McKinsey (international survey of large firms), organizations are increasingly more focused on customer-centric and data-driven insights for customer-centric decisioning. Surveyed firms pointed out to increased investments in all aspects of data and technology, including human resources. Unfortunately, most organizations identified that their investments in big data have fallen short of delivering noticeable improvement in performance (DataMatics, 2014).