My First Encounter with Shadow IT

In my recent post on what I learned about enterprise IT at Smith Tool, I mentioned that I needed another few posts to cover some of the more interesting lessons learned. I already covered what I learned from a failed waterfall development project in 1980. But the lessons kept coming, shortly thereafter, in my first encounter with “shadow IT.” 

Shadow IT commonly refers to information systems that are purchased or developed apart from the corporate IT department. 

An Inventory System for Tooling

In 1981, I got a new assignment: to develop a system to manage inventory of perishable tooling in the manufacturing plant [1]. Our manufacturing systems, some of which I had developed, did a fairly good job of managing inventory of direct material—raw materials and parts that went directly into the finished product. But they did not yet fully manage the inventories of tooling that were needed to make those parts, such as cutting tools and grinding wheels. Managing tool inventory was important, because a stock-out of tooling could delay a production order just as much as a stock-out of direct material could. 

We had already built systems to maintain tooling bills of material (tool lists) and to associate those tool lists to production orders. We had also built a system to track tool usage on production orders. But we had not yet closed the loop to track on-hand inventory of tooling and to plan for replenishment based on production plans. The existing manual system was nothing more than an intricate paper-based min/max system that required a physical inventory count three times a day! Expediting to cover shortages of tooling was a way of life. As a result, my analysis showed that 90% of manufacturing production order delays were the result of tooling being unavailable. The benefits of an automated system would be huge [2]. 

Some perishable tooling items were purchased. The rest were fabricated in Smith’s own tool-making shop or subcontracted to outside tool makers. The tool room was, in essence, a factory within a factory, and it would ultimately require a manufacturing planning and control system linked to the main MRP system at Smith Tool. This is a key point in the lesson learned to come. 

A Startling Discovery

My first step was to take a walk out to the tool crib to meet with the Manager of Tool Control (“Fred”), gather some data, and talk to him about his requirements. The conversation, as I recall it, went something like this. 

Me: “Hi Fred, as you may have heard, we’re starting to gather requirements for a new Tooling Inventory System to replace your manual system.” 

Fred: “Oh, no need to do that, we’re going to install the computer system that they’re using over in the San Bernardino plant.” 

Me: “Wait, what?” 

Fred: “Yeah, the guys over in San Bernardino didn’t want to duplicate our manual system, so one of the NC programmers put together a system on one of those TRS-80 computers you can get at RadioShack[3]. It only took him a few weeks to program it.”

Me: “Fred, wait a minute. I’m not out here setting up my own tool room. So, you guys shouldn’t be setting up your own computer system. That's my department.” 

After this, I managed to calm down and got a walk through to see the tool crib operations and to gather some sample documents. 

When I returned to the IT department, I went straight to Rodger’s office to tell him what happened. He told me that I’d better take the 90-minute drive out to San Bernardino to see this rogue TRS-80 system. 

Evaluating the Shadow IT System

The IBM PC would be on the market in just a few months (August, 1981), opening the floodgates to what would later be called end-user computing. But this was my first encounter. What stunned me the most was not just that my users had usurped my job of systems development but that they appeared to have done in a few weeks what we had planned to do in a six month effort. However, as I would soon learn, the scope of what they had done on the TRS-80 fell far short of what we were planning to do on the mainframe. 

The first thing I saw involved limitations of the TRS-80 hardware as a business computing platform [4].  These were easy observations. But, as we all know, personal computers would soon overcome those limitations and become a real disruption to mainframe computers.

My more strategic observation, however, was that the TRS-80 system only addressed a single need in what should be a closed-loop end-to-end process for managing tooling. There were no tooling bills-of-material (tool lists), no tracking of tool usage, no association of tool lists to production orders (at multiple revision levels), and no determination of tooling demand based on planned or released production orders—all functionality that we had already built or were about build on the mainframe system. I concluded: 

The TRS-80 system in use now by San Bernardino Tool Control basically serves their need for the maintenance of inventory data. It is a simple alternative to the card file used by Irvine Tool Control for this data. However, the long-range use of the TRS-80 has serious limitations as outlined above.

Beginning of a Major Disruption in Enterprise IT

Although I didn't realize it at the time, the world of corporate IT was changing. In reviewing an early version of this post, my manager, Rodger Beard, offered this analysis (lightly edited). 

Our Smith Tool TRS-80 experience demonstrated a trend that was already unfolding regarding the nature of business computing.  Dramatically cheaper computing hardware and operating system software had already started to come on scene with the introduction of mini-computers, from DEC and HP especially. But the TRS-80 and IBM's hurried, clumsy, poorly conceived PC initiative soon after had a far, far bigger impact.  Mini- and micro-computers enabled the rapid movement away from the IBM computing castles that were then the norm, with budgets that only kings could afford.  Because the dollars were so great and castles took so long to build (with high implementation costs and high risk of failure) there was a critical business need for better and cheaper ways to deploy automated business systems.  The TRS-80 and then PCs offered a way to fulfill that need, and to work around IT departments that were mostly seen as being in the way.

That said, low-cost hardware was just the first leg of a 3-legged stool of disruptive technological innovations that would become manifest over the coming years. The second leg was the faster development time that these platforms offered to build business software.  The TRS-80s at Smith Tool clearly demonstrated that software could be developed more cheaply, faster, and more easily, albeit with certain downsides that we, the knights guarding the IT castle, thought were important, but not as important to many in the business. 

The point is that shadow IT was conceived as the direct solution to an already very well known problem. There was too high a cost, as well as too much delay and pent-up demand for business software.  Packaged software suites, higher level and advanced programming languages, 4GLs like Focus,  emerging SDMs, software engineering and coding being taught in public schools, software training as a industry, computer engineering majors offered on every college campus, H1b visas, outsourcing, and then offshoring, all were solutions intended to solve this problem.  Net of it all, software cost is now tiny compared to when we found out about the TRS-80..

The third leg of the new stool was obviously overcoming the primitive data communications networks of the time as well as costs and delays associated with creating node-to-node communications.  The creation of the internet, and with it fast, cheap, available connectivity was the disruptive change that gave the new IT stool all three legs. (Wow did it ever!)

Lesson Learned: Bring Shadow IT out of the Shadows

Rodger is right. Although my analysis of the TRS-80 system may have been correct in identifying its shortcomings, it took me a few more years to understand the bigger picture. When the business has a need and the corporate IT organization does not have the resources to meet that need, the business will find a way to solve the problem. The days when the IT organization could just say no, or wait until next year, were coming to an end. 

Ultimately, the technology disruption brought its own new set of challenges. For example, user departments purchasing or building their own software, were soon asking for the IT organization to connect them to corporate systems. Many IT leaders were understandably distraught with these requests when they were not involved in the original development or procurement of the shadow system.   

 

Over the years, the most healthy way to deal with shadow IT is to bring it out of the shadows. It is really a matter of IT governance. Best practices in dealing with shadow IT include having a multi-year IT strategic plan that addresses major needs throughout the organization, guidelines to determine which systems are best deployed by corporate IT and which can be left to end-user development, an overall enterprise architecture, and budgetary flexibility where in some cases the business funds new system development, with the IT organization delivering or managing the services. 

Postscript: My recollection is fuzzy concerning the events that followed. My personnel file indicates I finished implementing the corporate tooling inventory system in 1982, and I moved on to an even more interesting project. But this all took place just before the multi-year decline in oil prices and collapse in the US drilling rig count, which devastated Smith’s business. The San Bernardino plant was shut down, so the use of the TRS-80 system became moot. 

End Notes

[1] In reviewing the system specification I wrote for this project, I notice that I applied the lesson learned from my previous project, where we had a failure with the traditional waterfall development approach. In my system specification for this project, I wrote: 

“Because a project addressing all the known requirements for a tooling inventory system would take over one calendar year to develop, we have adopted a two-phase approach. This allows Manufacturing Services to receive benefits from the project within six months of product initiation. It also allows us to evaluate the use and effectiveness of the system delivered in the first phase before beginning the second." [Emphasis added.] 

In other words, I was determined to test the users’ commitment to adopt initial capabilities of the new system before IT would spend the time and effort to develop the rest of it.

[2] When I started writing this post, I assumed that tooling inventory control systems would be commonplace today as modules within manufacturing ERP systems. Although SAP appears to have a solution, I am hard-pressed to find many others, outside of a few point solutions. I have a feeling that many customers today manage tooling inventory as a special item type in the production bill of material, which may be adequate for many manufacturers, although this approach has its shortcomings. If readers have insights on this, please leave a comment on this post. 

[3] RadioShack, founded in 1921, was a one-stop shop for all things electronic, from components to personal electronics to micro-computers. It essentially went out of business in 2015. The TRS-80, launched in 1977, was one of the first widely available microcomputers. 

[4] As I wrote in my trip report, 

“The TRS-80 is a microprocessor, and it is not designed for large-scale business systems. It has limitations on file sizes and key lengths…. There are limitations in real storage…. The hardware is designed to be run only 8-10 hours a day. It is designed for the occasional hobbyist or for a light back-office business, not for the day-to-day operation of a heavy manufacturer.” 

Moreover, the user-developed tooling system was only intended to satisfy needs around tooling procurement and inventory control. There was no functionality for tooling bills of material (tool lists) or ability to associate them with production orders. In Irvine, we had already built a system to automate these functions on the mainframe, but San Bernardino de-automated those functions and put them back onto a paper-based system. 

Image Credit 

TRS-80. Attribution: Blake Patterson. Source: Wikipedia Commons.