Three Decades of Software Vendor Selection

This post continues my series on lessons learned in my career. Chronologically, we left off with my work on a major system development project at Smith Tool. By the time I finished, I had been an independent consultant for about six years. But during that time, I also began to branch out to other clients. These included travelling around the US and beyond, doing mainframe conversion training, 4GL software development, and other consulting engagements.

All these clients had one thing in common. They all came directly or indirectly by referrals from former coworkers at Smith. As it turns out, I was lucky. When a sole proprietor, like me at this time, is busy with project work, it is difficult to develop new opportunities. And when a project ends, it is not easy to develop a new opportunity on short notice. Those independent consultants I know who have been successful over many years generally have reputations as experts. They also tend to have multiple revenue streams in addition to their consulting work, such as paid speaking engagements, books, or paid media contributions. Balancing project work with business development is easier for a consulting firm with some scale, something I would learn later in my career.

But in 1989, there was still one more referral for me. A former coworker from Smith Tool, Rick McGough, had just been hired as the top IT executive for Toshiba America Medical Systems (TAMS) [1]. TAMS was the U.S. distribution and field services unit for Toshiba’s medical diagnostic imaging device business [2]. Rick was looking to replace a homegrown business management system with a commercial ERP system. TAMS already had two Deloitte consultants on the project [3], but Rick wanted “his own guy” on the project team as well. So, he brought me in. This would my first real exposure to a full-blown ERP selection project, and this was the basis for my continuing this type of work over the following three decades, evaluating and selecting all manner of software, including ERP, but CRM, HCM, supply chain, and other categories of enterprise systems.

Lesson #1: Focus on Differentiators

The Deloitte consultants had recycled a requirements document of several hundred pages from a previous client and attempted to modify it for use at Toshiba. This turned out not to be a great approach, as the document included a number of requirements that didn’t apply to TAMS, and it missed several important requirements.

One key requirement was serial number tracking, which is an FDA regulatory mandate. Each unit of equipment and even critical components within a finished assembly need to be tracked with a unique identifier. So, if Toshiba receives 10 X-ray tubes from Japan, it’s not enough just to record a receipt of 10 units. It must record the serial numbers of those 10 units and track them in inventory and through distribution to the installed customer base. But guess what. The system that the team ultimately chose (BPCS from System Software Associates, or SSA) did not have serial number tracking. It did have lot number tracking, and the reseller’s proposed solution was to use the lot number as the serial number and modify the source code to force a lot quantity of one. They also had to do a modification to the inventory receipt program to allow the data entry user to enter the individual serial numbers at time of receipt. As you can imagine, there were other modifications needed for inventory management, sales, and distribution to accommodate the “lot number as serial number modification.”

To make a long story short, this was an example of a requirement that was so important that it normally should have disqualified that vendor from consideration [4]. I now call these types of requirements “differentiators.” They are more than priority A. They are show-stoppers. As a result, BPCS did not last long at Toshiba. It was replaced a few years later by Oracle Applications (today, E-Business Suite), which could satisfy the serial number tracking requirement as well as other differentiators.

Based on this lesson learned, I made a practice of identifying five to 10 differentiators for the client. I would use that to qualify vendors for the initial short list. In developing a short list, you really don’t need an RFP with hundreds or thousands of requirements. But you do need to know these key criteria. You can specify additional requirements (though I recommend avoiding hundreds) later in the selection process. Over the years, I had other experiences where the client made a vendor selection decision without following this best practice [5].

Lesson #2: If You Must Modify, Do It with Bolt-on Modifications

Another concept I developed during this project was the difference between good modifications and bad modifications. I recall standing at a whiteboard, discussing this with the Deloitte consultants. I told them we should avoid customizing vendor source code. The reason is that it may disrupt the logical integrity of the system—it may break the system in unanticipated ways. Second, it makes it difficult to upgrade to new releases of the software, because all those customizations will need to be carried forward to the new version.

Instead of customizing the vendor’s source code, I said that we should develop what I called “bolt-on modifications.” I drew a diagram on the white board, showing the modifications as a separate program or subsystem external to the vendor’s code, and calling it by means of an API. This approach is common today in what is known as a service-oriented, modular, or composable architecture [6].

Sadly, the modification to the lot number logic I mentioned in the previous lesson could not be accomplished with a bolt-on modification. It required modification of the core source code. As a result, it took the project team six months shortly after the initial implementation to do an upgrade to the next version.

Lesson #3: ERP Can Be Hard to Justify with Tangible Benefits

An ERP implementation is a major expenditure, and as such it would need management approval in the form of a capital expenditure request, which would need to budget both the costs and the anticipated benefits. The cost side of the equation is usually easy—hardware, software, and implementation costs could all be estimated based on vendor proposals.

The benefits side, however, is more difficult, especially if you want to identify tangible benefits. The project team for improvements in inventory accuracy, order fill rates, customer service improvements, and other areas. Time and again, we would identify a possible improvement but would conclude, “it doesn’t move the needle” [7]. Later in my career, our research at Computer Economics confirmed this finding [8].

When there are tangible benefits, they are mostly in two areas:

• Productivity savings. If the organization required 15 accounts payable personnel and the new system could cut that number to 10, that would be a productivity improvement. But few organizations really want to bank on that—they hate to lay off people. They would rather redeploy them into other roles. Or, they would rather say that the new system could allow future growth without adding to administrative headcount.
• Discontinuation of legacy systems. If the new system is replacing an old system—and preferably several old systems, there will be savings in the hardware, software, and personnel that supported the old system. If the system was highly modified, the new system might require fewer personnel, although this is often not the case, especially in the early years.

Therefore, in selection projects over the years, I’ve concluded that ERP benefits are mostly intangible—difficult to put a dollar value on. If done right, they give management better data to support decision-making, and they provide a single source of truth. Most importantly, they provide a better platform for future systems that rely on ERP data, such as business intelligence. There are also other more strategic drivers: the old system might no longer be supported by the vendor, or it might be on a legacy hardware platform. I’ve seen more situations where this was the driving force for a system replacement than I have with a business case based on tangible benefits [9].

Lesson #4: Software Selection Is More Than Selecting Software

But the greatest lesson learned from this first selection project and over the following three decades is that software selection projects are really mis-named. They are not just about selecting software—they are about business transformation. As such, there are some key success factors:

• Business sponsorship. Just because computers are involved does not make these projects computer projects. They should be driven by the business, with the IT organization as part of the project team. Top management should actively sponsor the project and not delegate it to middle management.
• Strategic alignment. Many ERP selections arise from organizations changing their business model, acquiring new lines of business, or expanding into new markets. To provide context, ERP selection should start with a review of the current and future business strategy and how IT is aligned to support it.
• Application portfolio rationalization. Many clients have dozens of enterprise systems, and many of them are inter-connected. When replacing a major system, should those other systems stay or go? Most organizations would do well to first address the entire portfolio of applications and lay out a strategic road map to understand which should be replaced or consolidated.
• IT organizational impact. Does the current IT organization have the needed skills to support the future applications road map, including the new system? If not, what new skills are needed and how should the IT staff be organized?
• System integrator selection. Selecting the new software is not the only decision. Most organizations will need to select an ERP implementation partner. Having the right ERP system but the wrong implementation partner often leads to failure. In fact, some of my projects over the next 30+ years involved finding a new implementation partner to replace the one that failed.
• Client-side planning. Generally, the system integrator will have a good handle on what it needs to do. But the client’s project team must also plan for the activities required on the client side, such as training, data conversion, procedure writing, and acceptance testing. The system integrator will generally expect the client to be responsible for these activities.
• Business process improvement. New systems can involve major changes in business processes--hopefully for the better. The selection team should also anticipate what business process improvement will be needed as part of the new system and include those in the project plan.
• Change management. Enterprise software implementations rarely fail because of technology problems (though, it does happen). They often fail because of organizational resistance to change. Engaging the organization during new system selection is how you gain buy in for the decision, setting the stage for cooperation during the implementation.

Further Industry Specialization

I look back at my experience with Toshiba as a major steppingstone in my career. As was my habit, once again, I immersed myself in the business—in this case medical devices. I interviewed business leaders and studied the various imaging system modalities, such as X-ray, MRI, CT, and ultrasound. Short term, this enabled me to continue consulting for Toshiba on the business side after the implementation was finished. I moved into the customer service group, defining requirements for future field service systems. Longer term, my experience at Toshiba became a foundation for my later industry focus on medical devices and life sciences in general, including FDA regulatory compliance.

Photo credit: Frank Scavo.   

End Notes

[1] Rick had had been hired at Smith Tool as a programmer analyst in 1978, about the same time I was. But whereas I took a career path to get deeper into the business, eventually leaving the IT department, Rick chose an IT management path. He rose through the ranks eventually becoming the IT director. As noted in an earlier post, the 1980s were a tumultuous time for Smith, and in 1989 Rick left to take the top IT job at Toshiba. He is now retired after holding several senior IT positions at other firms.

[2] Toshiba Medical was acquired by Canon in 2016 and is now Canon Medical Systems Corp. The US headquarters is still located just a few miles from where we lived at the time.

[3] One of those two Deloitte consultants was John Olszewski. John became a good friend, and we worked together on and off for something like six to eight years for other clients. He continued with Toshiba long after my work there was completed. He eventually played a leading role in replacing BPCS with Oracle Applications. With that experience, he then went to work for Oracle, where he is now Senior Director E-Business Suite Service Product Management.

[4] There was one key requirement that tipped the scales in favor of BPCS and that was its ability to specify sales features and options in a multi-level planning bill of material. None of the other systems on our short list, which Toshiba had restricted to IBM midrange systems, could satisfy that requirement.

[5] Several years later, I had another client where the chosen vendor did not satisfy a show-stopper. Coincidentally, John Olszewski and I worked together on this project. We came in after the vendor selection to help with the implementation. Here the client was a well-known multi-level marketing (MLM) company. In this industry, customers (i.e., independent distributors) would deposit cash into their accounts against which they would place orders for their customers. The MLM company would then fulfill those orders. In other words, the independent distributors had to pay in advance. But the chosen system was a typical B2B system. It assumed the independent distributor would place an order, and then the MLM company would fulfill it, invoice the distributor, collect the payment, and apply it to the invoice. As a result, we had to modify the system to accommodate this fundamental process misalignment. In retrospect, this system should have been dropped from consideration just based on this one showstopper.

[6] The term “bolt-on modification” is in common use today, but I am convinced that I invented the term. I have searched and have not been able to find a reference to the use of this metaphor prior to 1989. Yes, it was used in the automotive industry, but I can’t find anyone using it relative to software engineering. If anyone can find it, I will be happy to give up my claim. But until then, I am claiming authorship.

[7] As in the previous note, I’m convinced that I coined the phrase, “it doesn’t move the needle.” In trying to come up with the business case, I used it as shorthand for, “Yes, that may be a benefit, but it is too small to really make a difference.” I had in mind an automobile fuel gage where adding a small amount of fuel would not be enough to make the needle move. Again, I have not been able to find use of this term as a metaphor prior to 1989. But I’m willing to see evidence otherwise.

[8] Our research at Computer Economics (now part of Avasant) over the years compared the ROI and TCO experience of a number of IT investments. We found consistently that ERP ranked dead last in economic success. We argue that this does not mean organizations should not invest in ERP, but rather that the benefits on the one hand can be difficult to quantify and that the costs need to be carefully controlled to stay within budget.

[9] What about growth in revenue? In my experience, it is difficult to claim top line benefits from ERP systems, which are mainly focused on back office processes. If a new ERP system is needed to support a new line of business, there would be top line benefits, of course, but the contribution of ERP is indirectly tied to that outcome. CRM systems on the other hand might be justified in terms of improved customer service, improved upsell/cross-sell performance and other measures that increase revenue.