All the studies show an increasing pace of change in and around technology. It’s not just that we run the back-office with enterprise software, it is also invading the front office and the products and services offered by organizations around the world.

The Pace of Change

Would you use a bank for your checking account that does not have online banking? I don’t think so. I wouldn't bank with one that doesn't have a mobile app either.

Would you prefer an airline that offers online booking, check-in, flight status and boarding passes, or do you still use a travel agent and get printed tickets?

Are you as amazed as I am by some of the things that Amazon is doing? Consider the fact that as they build replenishment centers across America, they are staffing them with robots! Robots in the warehouse are far cheaper than employees in China and enable Amazon to set targets of "click to ship" of no more than two and a half hours, with some items shipping 20 seconds after the customer clicks the purchase icon. They are also using dynamic pricing (see here for an explanation) to balance inventory and demand.

CEOs are saying that technology is now the number one driver of change in their organization, and they use some of the latest tools to maintain contact with their customers. My bank is suggesting that I contact them on Twitter if I have a problem, and the best way to complain to United Airlines is on their Facebook page.

Add the fact that the combination of advances in analytics (of all kinds, from mobile analytics to sentiment analysis to predictive and visual analytics) provides the capability to make leap changes in the quality of decision-making. Now, instead of relying on their experience and intuition, executives can have timely, current, insightful and useful information on which to base their decisions. The insight can be on customer experiences and views of their current or future offerings, of the level of risk, or simply of their ability to fine-tune their manufacturing and other processes to drive revenue up and cost down.

But have the practitioners (and by extension those responsible for board oversight) kept pace?

The End of Isolated IT Risk

Protiviti has shared with us their 2013 IT Audit Benchmarking Survey. It contains some useful information, which I will comment on momentarily.

First, though, I want to address the use of the terms "IT risk" and "IT audit." Personally, I look to ISACA for insight on "IT risk" and they don’t disappoint. This is what they have to say:

COBIT 5 for Risk defines IT risk as business risk, specifically, the business risk associated with the use, ownership, operation, involvement, influence and adoption of IT within an enterprise.”

In other words, we should stop talking and thinking about "IT risk" as something separate. Instead, we should be talking and thinking about technology-related business risk.

This is very important: It is much more than semantics!

For most of us, IT is a department. It has processes and addresses risks that may arise from failings in those processes through the operation of IT general controls (ITGC).

But we should be expanding our view to consider all the technology that is relied upon across the enterprise.

In my years as Chief Audit Executive (CAE), I ran into several situations where a focus on IT would have been too narrow:

  • At Tosco Corporation -- at that time the largest oil refining company in the US -- every refinery was run using sophisticated process control and other computer equipment. It was relied upon for the safe and reliable operation of the various units (catalytic cracker, hydrogen plant, etc.), blending of fuels, measurement of receipt and shipping of crude oil and finished products and much more. All of this equipment was acquired, maintained and operated by individuals in the Engineering and other refinery departments. IT had very little involvement, and arguably all the technology-related business risk was outside IT’s span of control -- and awareness.
  • At Maxtor Corporation, a $4 billion manufacturer of hard drives (later acquired by Seagate), the Engineering department (responsible for product development) managed its own network and devices. The IT team only managed the wider network.
  • At Business Objects, a major software company (later acquired by SAP), the largest number on the balance sheet was the warranty reserve. This reserve is for potential repairs and replacement of units that had been sold and failed within their warranty period. The software used to calculate the potential cost of such repairs or replacement was maintained outside the IT department.

I think it is time to stop talking about IT risk and instead talk about technology-related business risk, which I would shorthand to technology risk. Similarly, it is time to stop talking about IT audit and instead talk about specialists with a deeper understanding of technology.

In addition, the word “information” in IT leads us to focus on data, rather than the business that relies on the technology and how a failure in the use of technology can affect business strategies and objectives.

At the same time, we need to avoid letting our technology specialists live in the weeds. While they may love playing around with all the cool stuff, they need to focus on how the business depends on technology to work as designed and not to fail. In other words, they need to understand business objectives and how they are both enabled and potentially affected by technology successes and failures.

An Integrated Approach

That brings us back to the Protiviti survey and report. Have they taken the technology-related business risk approach, or are they in the weeds with the IT auditors?

I will let you decide for yourself, but for me I suspect that the survey they sent out and which therefore drove the answers they received was all about IT risk. There seems to be little focus on getting the most from technology. Fear wins the day, with a major focus on what can go wrong -- such as the statement that “Data security is of paramount concern.”

When I consider an organization making massive bets on big data and analytics, on the provision of new services via mobile apps, running warehouses with robots, and changing prices every hour or more using dynamic pricing algorithms, I suspect the “paramount concern” of the executive team is not “data security.”

OK, I hope I have made my point and we can consider all the useful information in the study.

The first point of discussion is whether organizations have a separate or integrated IT audit function (leaving aside the issue of whether it should be a technology audit function). Most have an IT audit function within internal audit, but many -- especially small and medium sized companies (SME) -- do not. However, it is difficult to draw any conclusion from the survey results as many of the companies that responded “no” to this question probably answered “yes” to the question of outsourcing/co-sourcing IT audit.

However, with technology being probably the greatest source of risk at every organization, I would agree with Protiviti that those companies with no internal IT audit resources may not be paying sufficient attention to this critical area.

It is interesting, although perhaps "alarming" is a better word, to see that the individual (typically a Director) leading IT audit sometimes does not report to the CAE or a direct report to the CAE. Some report to the CIO or through the compliance function, which I would consider unacceptable.

No audit committee should tolerate an IT audit function that is not integrated within the overall internal audit function.

An interesting chart can be found on page eight of the report. It shows the percentage of internal audit headcount designated as IT audit. I find the chart a little confusing, but if I am reading it correctly the great majority of organizations (around 70 percent) have less than 20 percent of their resources (presumably including co-sourced staff) designated as IT audit. I agree with Protiviti that the level of resources is probably out of balance with the number and level of technology-related business risks.

When I was running internal audit, I certainly had more than 20 percent of my resources working on technology-related risks. In today’s environment, I might target 40 percent or more -- quite likely more than 50 percent.

The next troubling area is whether technology-related risks are identified and assessed as part of, or separate to, the overall internal audit risk assessment process. Even in the largest companies (those with over $5 billion in revenue), 26 percent do not have an integrated risk assessment process. Many do not identify and assess technology-related risks at all!

How can this be?

How can any organization say they are providing assurance on the effectiveness of management’s processes for governance, risk management and the related controls if they do not consider the whole landscape of risks?

How can any organization understand technology-related risks if they don’t do so within the context of the business as a whole, its strategies and objectives?

At the foot of page 16, Protiviti reports that 65 percent of organizations only assess IT risk annually. That is far from best practice. The risk assessment should be updated as risks change, and I find it difficult to believe that risks change only once a year. My congratulations go to the 8 percent who update the risk assessment continually, with a silver medal to those who do so monthly (2 percent).

It is very disappointing that few organizations have a major involvement in “significant technology projects.” Fourteen percent have NO involvement.

The greatest risk is where there is change.

The change driven by technology these days is often likely to make or break an organization. The investment is huge and typically the foundation for the organization’s primary strategic objectives.

When I was with Tosco, the CIO told me that he would not go live with a major technology project without first hearing from my IT audit team -- and he was right to take that position! I had as many as four auditors assigned to every significant technology project. On the project that led the CIO to say what he did, my team (led by Tim Cox and Bruce Taylor) reported to the project board that if they went ahead, the risk of failure was high.

After the board decided that for business reasons they needed to accept the risk, my team came back and told them where the technology (a combination of hardware, software and security tools) was most likely to fail. The technology failed precisely where my team said it would, but IT and users had deployed teams to monitor those locations and were able to respond and take corrective action promptly.

Less than half of the organizations surveyed had assessed IT governance. Personally, although it is mandated by IIA Standard 2110, I would only audit IT governance if I considered it a high risk. The portion of IT governance that I would be most concerned with relates to the leadership of IT in enabling business strategies (see this earlier post), and its processes for managing significant technology projects.

This is a useful report and I commend Protiviti for it. My suggestion to them is that they focus more on technology-related business risk in their next survey, including questions around whether internal audit assesses technology that is not managed by IT.

I welcome your views and commentary.

Editor's Note: Read more from Norman in Why I Worry About Uncertainty First, Then About Risk