Saturday, January 22, 2005

Benchmarking in buying and implementing enterprise systems

Management fads come and go, but one that seems to have lasting appeal is benchmarking: measuring your company's performance against others. Like former New York mayor Ed Koch, executives constantly like to ask, how am I doing?

Enterprise systems, such as ERP, CRM, and supply chain management, are largely sold on the basis of their promise to improve business performance. Therefore, it is no surprise that software vendors are using benchmarking as a way to determine where there are the greatest opportunities to improve a prospect's business performance and to establish credibility with top management.

Benchmarking in the system sales cycle
To show the extent to which the major software vendors use benchmarking as a sales tool, let me relate a recent experience. Last year, I began preliminary work with a large company that was considering purchase of a new ERP system. Before long, word hit the street that this company might be shopping for software, and several software vendors tried to get a foot in the door.

SAP and PeopleSoft were two of them, and I was surprised that, without any prior contact with this client, both vendors sent benchmarking reports that compared the financial performance of this company with others in the same industry. (Public financial metrics for this company were readily available, making it possible for the vendors to do this.) Both vendors also identified possible opportunities where this company could improve its financial performance by implementing SAP or PeopleSoft software.

Based on work we had already done for this client, I didn't feel that the vendor benchmarks were particularly relevant or useful. But the point is that both vendors saw benchmarking studies as a key part of their initial sales contact. Both vendors must find this approach to be effective, otherwise they wouldn't be using it so early in the sales cycle.

One mid-market vendor's approach
Use of benchmarking in the sales cycle is not limited to larger vendors such as SAP and PeopleSoft (now Oracle). I interviewed several mid-tier vendors during the APICS International Conference last year, and I was surprised to find that at least one such vendor, MAPICS, uses the same approach, but with some interesting twists.

MAPICS's use of benchmarking is interesting for two reasons. First, it's free to anyone visiting the MAPICS web site. Several years ago, MAPICS purchased benchmarking data on 4,000 manufacturing companies, which it updates quarterly. It allows visitors to its web site to use this database to compare their performance against other companies of similar size and industry.

Second, if the prospect turns into a client, MAPICS uses the benchmarking data to set objectives for the implementation and configures MAPICS business intelligence reporting to measure how well the client actually achieved those objectives after the implementation. This holds the client, and MAPICS, accountable for delivering real business value.

In a follow up discussion with MAPICS VP Joe Marino, I asked what led MAPICS to the use of benchmarking in selling new systems. Joe said that in the late 1990s, with the rush to implement new systems for Y2K, many companies lost focus on the business value of new systems. MAPICS was already using an earlier version of its benchmarking tool at the end of the sales cycle, to plan implementation. But they realized that with the downturn in the tech sector, it would be helpful to introduce the benchmarking tool at the beginning of the sales cycle, while prospects are simply gathering information from vendors. Their experience showed that introducing benchmarking early in the sales cycle can help focus attention on the business value of the new system.

At Joe's invitation, I've been playing around with the benchmarking tool. After initial registration the tool asks you what industry you are in and asks you to fill in eight financial metrics and five operational metrics. From these 13 metrics, the tool then plots the performance of your company against the average performance for your industry and against the performance of the top 25% of the sample for each metric, which MAPICS uses to indicate world class performance. Performance is compared along seven so-called keys to success:
  1. Speed time to market
  2. Streamline outsourcing processes
  3. Exceed customer expectations
  4. Reduce lead times
  5. Cut operations costs
  6. Manage the global enterprise
  7. Improve business performance visibility
The tool also provides a separate benchmark comparison of your company on a purely financial basis, against the average and 25th top percentile in your industry.

My overall assessment of the MAPICS benchmarking tool? I like the intention: getting companies to focus on what it is they want in terms of business value from new systems. I have some question about how the tool can go from 13 metrics to the seven "keys," but, again, I like the overall intent of the exercise. It's a good first step for someone that wants to start thinking about benchmarking business performance. Joe estimates that about half of new MAPICS sales involve use of the benchmarking tool, confirming that the approach is appealing to prospects.

If you want to try out the benchmarking tool yourself, you can find it under the link for the MAPICS World Class Performance Calculator on the MAPICS web site.

Limitations of Benchmarking
Although benchmarking can be a useful exercise when planning for the implementation of an enterprise system, executives should understand the limitations of benchmarking and the danger of taking any benchmark at face value. Here are four errors that I see when companies attempt to do benchmarking:
  • Only considering financial performance. Because publicly held companies openly disclose financial statements, financial metrics are the easiest to benchmark. But financial performance is a lagging indicator. A key driver of strong financial performance is operational excellence. For example, if your company's gross margins are lower than the industry average, is it because your product quality is sub-standard, requiring you to sell at a lower price than your competitors, or because your raw material costs are higher, or some other reason? A more detailed analysis would be needed to determine the operational metrics that are key to your success and then to compare your performance on those metrics against others in your industry. Therefore, the best benchmarking study is one that combines both financial and operational metrics.


  • Not taking into account operational differences. For example, manufacturing companies that make to stock typically have much shorter customer delivery lead times than companies that make to order. If your company is "make to order," and the companies in the benchmark sample are largely "make to stock," it will appear that your delivery performance is sub-standard. But it won't be a meaningful comparison. The same problem occurs when comparing a company that outsources much of its manufacturing against a group of companies that do most of their manufacturing internally. Comparisons of inventory turnover in such situations will be meaningless.


  • Not understanding differences in strategy. For example, if your strategy is to compete on delivery lead time, allowing you to command a higher price, and the companies in the benchmark are largely competing on price, it will appear that your gross margin performance is superior to the benchmark. But the reality might be that your gross margins are still too low to make up for the sales that you lose on price. You'll never know that unless you understand more about the companies in the benchmarking sample.


  • Only comparing yourself to companies in the same industry. The problem here is that you may look good relative to other companies in your industry but be missing opportunities to achieve truly world class performance. For example, if you are an industrial manufacturer and you want to achieve world class performance in your customer call center, who do you want to compare yourself to? Other industrial manufacturers, or companies such as Lands End that are truly world class in call center management? Sometimes you need to look outside your own industry, or you'll simply become the "best house" in a bad neighborhood.
Benchmarking continues to hold great appeal to executives. Therefore, enterprise system vendors, such as SAP, Oracle, MAPICS, and others will continue to use benchmarking as a way to get the attention of prospects during the sales cycle. Companies should hold themselves and their vendors accountable for delivering business value, and benchmarking is a useful tool to do that. At the same time, companies should be aware of the limitations of benchmarking. Use benchmarking for focusing the implementation effort, but use it wisely.

Related posts
Escaping the ROI trap
Escaping the ROI Trap, Part 2
Four problems with ERP
Solving the four problems with ERP
Business changes needed to ensure enterprise system success

No comments: