FINRA Manages 50 Billion Daily Transactions Using The Public Cloud

Steve Randich has been a CIO many times over at organizations like the Chicago Stock Exchange, the NASDAQ, and with Citibank prior to taking on his current post as CIO of Financial Industry Regulatory Authority, Inc., better known as FINRA, which is a non-governmental organization that regulates brokerage firms and exchange markets.  When asked about the evolution of the CIO role, he indicates that he has not seen much evolution, but that may be because he was a strategic leader, driving innovation from the CIO post long before others were presumptuous enough to think to do so.

At FINRA, Randich’s innovations center around leveraging artificial intelligence and machine learning to better surveil markets and broker-dealers. He also has led one of the most dramatic implementations of the public cloud. So extensive is the implementation that Amazon Web Services considers FINRA a best case example of the use of its technology. Randich has become an evangelist of the public cloud, citing it as the single technology across his career that actually gets cheaper as you use it more.

In this interview, he offers insights into all of the above and more.

(To listen to a podcast version of this interview, please visit this link. To read future articles like this one, please follow me on Twitter @PeterAHigh.)

Peter High: Could you give a rundown of FINRA’s operation and your role as the Chief Information Officer?

FINRA CIO Steve Randich

Credit: FINRA

Steve Randich: FINRA goes back to 1937 when it was known as the National Association of Security Dealers. After going through some mergers, other regulatory functions, and stock exchanges, the company became known as FINRA ten years ago. FINRA, which employs roughly 3,500 people, focuses on protecting investors by surveilling both the markets and the broker-dealer activities. As CIO, I run a 1,100-person organization, which focuses on building surveillance systems that assist our staff in examining firms and regulating the markets.

High: How are the new weapons in the IT arsenal implemented into FINRA’s strategy?

Randich: FINRA processes enormous amounts of data as the company handles upwards of fifty billion transactions on a daily basis, including all of the quotes, orders, and trades collectively across the equity markets in the United States. Additionally, the company looks at the historical data to identify trends over time, which exposes market manipulation, insider trading, and fraud in the markets. Several years ago, FINRA decided to use open-source, big data technologies on public cloud platforms to handle the large amounts of data efficiently and with scale. Today, those efforts are largely completed, and we are now moving into machine learning and advanced analytics. This will enable machines to do more of the surveillance, which allows our surveillance analysts to avoid the work that is better suited for the machines.

High: The large amounts of data you handle gives you knowledge on how to collect the right data and synthesize it appropriately. How would you advise others who walk in your footsteps to do some of what you have done?

Randich: As early adopters of public cloud computing, open-source, and big data software platforms, we got substantial help from many of the founding vendors, such as Cloudera, Hortonworks, and Amazon Web Services (AWS). These companies assisted us in our transformation by training us on how to use these technologies. While there are far more consultants that are available today, companies must first build the skills in-house so that they are not reliant on third parties over time. In the first year of our transformation, we soaked in an immense amount of information on how to use the technologies. From there, we worked with most of the major public cloud computing vendors to determine which we wanted to go with, and we ultimately decided to partner with Amazon. While many companies take a lift-and-shift approach to get to the cloud quicker, that strategy limits their ability to leverage Moore's Law and the cybersecurity advantages. In our approach, we built an architecture that was public cloud-enabled, which allowed us to take full advantage of all the opportunities across the software stack.

Along this journey, we started by acquiring the skills, building the appropriate architecture, and then, starting the re-platforming work, which began in early 2014 and concluded two years later. As we went along, we were continuously moving infrastructure into production as this was not a big bang type of implementation. Regarding the data, the databases that we previously had were small due to their commercial limitations. Because of this, we had to stage the data to make it available for our analytics users to do their queries and their surveillance. However, we now have large-scale big data databases on the public cloud, which enable us to always have our data online. This allows all of our data to be regularly available to our users without needing IT operations intervention. Furthermore, this enables our users to complete their queries and do machine learning prototyping on the data with little IT intervention. The combination of these elements provides a self-serve approach for our users, which gives them a significant advantage.

High: With the topics we are talking about, there is a paucity of great talent. What methods do you use to overcome that challenge and find that talent, bring it in, and ensure that people have the desire to stay with the organization?

Randich: We are participating in industry events, such as hackathons at the various universities along the East Coast, and we host our own annual “Createathon.” We are extremely noisy in the press, at conferences, and with opportunities to get awards. For example, we have been ranked as one of the "Top 100 Best Places to Work" in Computerworld Magazine each of the past five years. Additionally, we have sent dozens of speakers to the AWS annual conference in Las Vegas, re:Invent. This combination creates a high-profile approach, which gets the message out that we are moving fast into new technologies such as the cloud and open-source. Showcasing our innovative mindset has allowed us to become an attractive place to work for highly-skilled IT individuals.

High: FINRA has an office in Washington, D.C, and while the DC metro area is not a financial services hub, that may be an advantage. Could you talk about your own perceptions of the labor force in Washington compared to Chicago, New York, or in other places that you have worked in the past?

Randich: I believe that the DC metro area is highly-skilled in terms of IT because of the wide variety of industries in the region, including digital publishing. The highest concentration of data centers and internet providers in the U.S. are located in Virginia, and Maryland has surprisingly great talent as well. The financial services skill is not necessarily required because most of what we do is large-scale, big data processing. While that skill is essential for surveilling markets, it is usually a secondary skill for data management and processing.

High: At this point, most companies would agree that the transition to the cloud is an important part of their strategy. The debate on whether to go to the cloud is largely settled, but there is still a debate as to the public, private, or hybrid approach. Why are you such a major proponent of the public cloud, rather than either hybrid or strictly private cloud?

Randich: The debate has become more one-sided with the three core advantages that we have seen with public cloud computing. The advantages are as follows:

  1. The public cloud provides a near infinite scale because you are paying on-demand. With the private cloud, you have captive hosted hardware, and you are paying even when you are not using it.
  2. The ability to evolve at pace with Moore’s law. In our case, AWS is continuously upgrading their hardware, which would not be done at the same rate in a private data center.
  3. The public cloud provides immense cybersecurity benefits. This is because it is essentially impossible for a commercial firm doing it themselves to invest at the level of resources that are needed to get the cybersecurity controls of a public cloud vendor.

High: You have mentioned in the past that the economics of this makes sense as well. This is an unusual situation where with the more you use, the better the economics get. Would you mind elaborating on that?

Randich: Our data volumes have been going up dramatically as a result of the market volumes and the bull market that we have been experiencing over the past decade. Likewise, our data volumes have increased significantly since we have moved additional applications to the cloud. While our usage has gone up, our cloud costs have gone down, which is something I have never seen with any technology platform or vendor in my career.

High: Can you talk about how your partnership with AWS has benefited your company?

Randich: AWS’ service philosophy is to help their clients reduce their cost. The company has implemented dozens of unsolicited price reductions since the product was created in 2006. From an engineering standpoint, AWS’ work is focused on teaching their clients to use their platform more efficiently. Their efforts have been one of the main reasons why our work in the cloud has become more economical over time, which is not something that you are going to see with other vendors.

High: You have a diverse background as you received a Bachelor in Computer Science from Northern Illinois University and an MBA from the University of Chicago’s Booth School of Business. After the conclusion of your education, you worked as a consultant with IBM, and you held CIO roles at the Chicago Stock Exchange, the NASDAQ, and with Citibank. With your broad experiences, combined set of skills, and multiple decades as a CIO, could you talk about your perspective on the evolution of the position?

Randich: From my perspective, I do not believe the role has changed all that much. I believe there has been a trend towards the centralization of IT for efficiency, which is a transition I saw at Citibank. However, my CIO roles at the Chicago Stock Exchange and at the NASDAQ were largely centralized IT functions, and I have seen a similar model at FINRA. Furthermore, while there is a great deal of talk between the difference in the CTO and CIO positions, I do not see much of a contrast between the roles. Generally, I believe that a healthy IT team serves as trusted advisors and partners to the business they support. Rather than serving in an order taking role, a successful IT team looks for innovative solutions that the business could not otherwise find. After building a strong team of IT managers and strategists, my first priority in a new CIO role is to ensure that the relationship between the IT function and the businesses is that of a trusted advisor.

High: Are there other rising technology trends on your personal or professional roadmap that particularly excite you?

Randich: I have been able to spend a good amount of time studying robotics and automation, and I have had a couple of speaking engagements on the subject. While it is a bit futuristic, I believe concepts such as “the second half of the chessboard” and Moore's Law are fascinating. Since 2006, Moore’s Law is getting to a point where we are now starting to see exponential improvements in price performance compared to what we saw in the 1960s, which was the early stages of Moore's Law. Going forward, we are going to see the opportunity to automate technologies that were not previously possible to automate.

Because of where we currently are with technology, there have been many comparisons to the industrial revolution. It took fifty years for the machinery that fed the industrial revolution to mature, and we have likewise seen technology progressing over the past fifty years. What we are going to see in the next twenty years is going to be incredible compared to what we have witnessed over the past fifty. These advances are going to drive the ability to do robotics and automation in a way that had not been achievable until now. Furthermore, I believe we are going to see a massive displacement of labor by machines, and I believe that these elements will strengthen the CIO role. What we are seeing will result in a significant increase in work in terms of programming machines to replace labor, and I would encourage everyone to have their kids get engineering and computer science degrees because of it.

