Friday, October 22, 2010

Best practices not always best

This article in Computerworld by my friend Tom Wailgum, The Trouble with Supply-Chain Best Practices, got me thinking again about this whole subject. What are best practices?

The problem, as I see it, is two-fold. First, the term best practices has at least two major and totally different definitions, and second, in many areas of business, there is not generally agreement on what are the best practices.

Two definitions
As just noted, practitioners often use the term best practices in two completely different ways, and you have to be sure you understand the context. The first is in the sense of "the best way to do something." The current definition in Wikipedia is typical:
A best practice is a technique, method, process, activity, incentive, or reward which conventional wisdom regards as more effective at delivering a particular outcome than any other technique, method, process, etc. when applied to a particular condition or circumstance.
For example, conventional wisdom might define a best practice in recruiting new employees as establishing a formal employee-referral program, as this channel often results in the highest quality candidates, lowest cost of recruiting, and best retention rates.

The second definition, which I run into occasionally, has to do with best practices in the sense of metrics. Here, folks use the term not to describe the best way to do something, but the best performance that is attained among peers against some metric. For example, again using the recruiting example, the median retention rate after six months for newly-hired nurses in US hospitals might be (I'm making this up) 75%. But the "best practice" (i.e. the retention rate achieved by the best performing hospitals) might be 92%.

So, when someone asks, what are best practices for recruiting, you have to ask, do you mean what are the best policies, procedures, and practices in recruiting, or do you mean, what is the best performance against some metrics by the organizations that are the most successful in recruiting.

Best practices not always universally applicable
For now, let's go with the first definition. Are there really ways of doing business that are generally accepted as best? In some cases, yes, but in many cases no.

Let me illustrate with an experience I had several years ago. My consulting firm, Strativa, was bidding on a business process improvement project for a mid-size medical device manufacturing firm. Our first meeting with the selection committee went quite well. We outlined our approach to business process re-engineering and business improvement and described some case studies for similar projects.

Based on the committee's recommendation, we then scheduled a one-on-one meeting with the President. That didn't go so well, and it came down to this issue of best practices. After describing our proposed work plan for the project, he asked, "Where do you compare our practices against industry best-practices?" I indicated where such an evaluation could take place. He then asked, what are your sources for best practices? I indicated that there are a number of professional societies that are good sources for best practices, such as APICS, which is the generally accepted source for best practices in materials management. In addition, we would use our own knowledge and experience from other clients as to where this organization could be viewed as having a need for improvement.

We went back and forth on this subject for some time, and I had the feeling that he wasn't satisfied with my answer. In a debriefing session afterward, the VP of Information Systems, who was sitting in on the meeting, confirmed that the President didn't feel our approach to best practices was strong enough.

The VP was sympathetic and still hoping we could win the deal. So I shared with him why I felt that the President's emphasis on best practices might be misplaced.

I said, "Isn't it true that your company uses significant part numbers?"

I had learned earlier that this company had the practice of letting each each digit or character of the product item number stand for something meaningful. For example the first two characters might indicate the product family, the second two might indicate the sub-family, the third digit might indicate the size of the product, the fourth digit might indicate the material, and so forth.

The VP said, "Yes, that's right. We've always done it that way."

"That's not a best practice," I replied.

"Really, says who?" asked the VP.

"APICS," I said. "They've been preaching against the use of significant part numbers since the mid-197o's. The reason is that it creates all kinds of problems. Invariably, as companies grow and their product portfolio changes, they outgrow their numbering schemes. Either the part number becomes extraordinarily long, or people just give it up. If you want to describe the product, use other fields on the item master. You don't need to make the part number work that hard. "

I continued, "Now, here's my point. In spite of what I just said, if we do this project, we're probably NOT going to try to change your part-numbering scheme. You've got it, and it's probably too difficult to change at this time, even though it's not a best-practice. A good consultant will look at what you are doing and will weigh the pro's and con's of changing it. That's how you've got to apply so-called best practices."

You can't just go to some database of best practices and say, here's what you should be doing. You need to apply judgment, based on experience.

Of course, this approach does not scale for large consulting firms, who like to staff business improvement projects with a lot of junior associates. It's much easier to give them a database of best practices and tell them, find out if the client is doing these. If not, recommend they do them. It's much harder to go in and evaluate the situation according to the client's specific situation. But that's the way to create meaningful change.

Best performance not universally attainable
There are problems also with the second definition of best practices: the best performance attained among peers against some metric. The problem is that which is common to all benchmarking exercises: defining the peer group and identifying the reasons for superior performance.

For example, my IT research firm, Computer Economics, publishes metrics on IT spending and IT staffing ratios. In addition, we provide a IT spending benchmarking service where we calculate the client's metrics, compare them to our published ratios, and provide our analysis of the gaps in performance.

Invariably, there is almost always a significant amount of judgment that we need to apply in our analysis. For example, just recently, a benchmark client (a public utility) showed that the number of users per IT help desk staff member was near the 25th percentile in comparison with other organizations of this size. The number of PCs supported by each help desk staff member showed similar sub-standard performance.

However, when interviewing the client, we discovered that the agency had an online permitting system that builders and developers used to submit permit applications. The IT help desk, which normally would only serve internal users, was also serving the general public as users of this application. Knowing our data, we were sure that this was not the case with the majority of our survey respondents.

So this client was well under what we would consider a "best performance" (something at or above the 75th percentile for this metric). But when we factored in the percentage of help desk incidents fielded from the general public, we found that the agency was, in fact, well above the median.

The concept of best practices can be useful, if properly understood and applied with judgment. Certainly, in terms of how to do business, organizations can learn from one another. And in terms of measurements, it is quite useful to have a sense for what levels of performance are achieved by peer organizations, or even by organizations outside of one's own industry.

But in both cases, there's no magic formula.

Related posts
Computer Economics: IT Management Best Practices
ERP implementation: putting processes and people first
Solving the four problems with ERP
Four problems with ERP
Business changes needed to ensure enterprise system success
Large system implementations require organizational discipline

Wednesday, October 13, 2010

Key success factor for SaaS suites: functional parity

As we all know, software-as-a-service (SaaS) has been one of the bright spots in the enterprise systems marketplace these days. The advantages are becoming more widely recognized: lower total cost, faster time-to-benefit, little to no capital expenditure, and less pain in system upgrades.

In fact, in some segments of the enterprise market, SaaS is already where most of the action is. For example, in CRM system selection it is difficult not to consider one of the leading SaaS solutions, such as Salesforce.com, Oracle's CRM On-Demand, RightNow.com and others. For HR management systems (HRMS), likewise, we see SaaS providers such as Workday, Taleo, and Success Factors gaining significant market share for net-new deals.

Reaching for full maturity
So, why haven't SaaS solutions completely taken over the enterprise software market, especially for full-suite ERP? Because there is one area where SaaS providers still lag behind: functional parity. For full-suite ERP, there are still precious few SaaS providers. And those that do attempt "full suite replacement" still have major gaps in their functional footprint.

However, the landscape is changing quickly. For example, until recently, we have been reluctant to short-list SaaS providers as full-suite options for manufacturing firms. Those that had ambitions to be full-suite providers simply lacked basic functionality needed for manufacturing, especially vertical-specific requirements. But we are now finding that SaaS providers at least deserve a look. These include the following:
  • Plex Systems was the first SaaS provider out of the gate with a full-suite ERP offering for manufacturing firms. More on Plex in a minute.
  • NetSuite is a full-suite provider. But until recently, it has only been a viable option for service businesses, because it lacked many fundamental features for manufacturers, such as standard costing, shop scheduling, capacity planning, and MRP. However, since I last visited this issue, a NetSuite partner, Rootstock, has been building extended manufacturing functionality on top of NetSuite's platform. I've spoken at length to the developers at Rootstock, whom I know from their previous work at Relevant (since acquired by Consona). They are making very fast progress, thanks to the ability to rapidly develop on NetSuite's platform. Rootstock's extensions to NetSuite are claimed to operate seamlessly with NetSuite's core financials and CRM.
  • SAP is moving in the same direction with its Business ByDesign (ByD) offering. Although SAP has been slow to move the product into general release, you can't fault the objective, which is to become a full-suite offering. I especially like the use-case for large SAP installed-base customers, which have a hard time justifying use of SAP ERP in smaller divisions. Such organizations frequently adopt a "two-tier ERP" strategy, where SAP runs at the corporate office and in larger business units, while a smaller footprint ERP, such as Epicor, QAD, or Microsoft Dynamics AX, runs in the smaller divisions. The use of ByD in this scenario, should be an attractive alternative.

    Update: Since this post was first published, I have learned that in FP 2.0 (released Sep. 2009), ByD now has what appears to be very complete functionality for manufacturing operations, including integrated quality control, as well as supply chain planning.
  • Workday is a SaaS provider, currently serving the HRMS and financials functions, but they have ambitions to be a full ERP replacement, at least for services-based organizations. With Dave Duffield as one of the founders they appear to be following the path that Dave took at PeopleSoft: start with HR, add financials and purchasing, then fill out to become a full-suite offering. Personally, I think Workday is underestimating how long it will take them to get there, but I have little doubt they will reach the goal. And they are going after the large company segment, which is unusual for most SaaS providers.
  • Update: Infor's Syteline is now being newly launched by Infor in a SaaS deployment model. Based on a lengthy briefing I received from Infor, it appears that Syteline qualifies as a full-suite SaaS offering. The new deployment option uses a single instance of Syteline's application server, with separate databases for each customer. While some might argue that it doesn't fully meet the pure definition of multi-tenancy, it more than makes up for it in functional parity with on-premise ERP suites, which is the point of this post. I'm hoping to write more on this new offering soon, as it appears to be another good option for those looking to go with SaaS for full-suite ERP.
  • Update: Epicor Express is another full-suite SaaS offering. Running as a pure multi-tenant deployment of Epicor 9, the offering today is supporting about 15 live customers, with another 15 in implementation. Epicor is currently targeting this offering at small job shops, with customer friendly subscription terms and very low fixed-fee implementation services ($7,500, including a defined set of data migrations). But I can see this product being up-sold to larger customers with more complex requirements as well.
The number of SaaS providers with full-suite offerings still pales in comparison to traditional on-premise ERP. Note, however, that I don't consider hosted versions of on-premise software in the SaaS category, such as hosted versions of Lawson's ERP products, or Oracle's E-Business Suite. Single-tenant hosted products simply do not offer the full benefits of SaaS outlined earlier.

Full-suite SaaS gaining traction
Although the number of true multi-tenant full-suite SaaS offerings today is limited, they are rapidly becoming a viable alternative to on-premise products or single-tenant hosted offerings.

The latest example is a big win for Plex Systems, at Invensys Controls, one of three business units of UK-based Invensys plc. This business unit provides components, systems, and services used in appliance, heating, air conditioning, refrigeration, and residential thermostat products. It has locations in 15 countries which include 22 manufacturing sites, two distribution centers, and seven engineering centers--and Plex Online will be implemented in all of them.

It sounded like a pretty big deal for a SaaS provider, so I followed up with Plex to find out more. Here are the details:
  • Plex will be replacing 11 different traditional on-premise ERP systems in the various locations at Invensys Controls, which has revenue of approximately $900 million.
  • The win by Plex comes not only against traditional on-premise ERP vendors--SAP, Infor, and Oracle, but also against NetSuite. So, Invensys actually was willing to consider two full-suite SaaS options: Plex and NetSuite.
  • The decision in favor of Plex came down to two factors: (1) functionality--a single, comprehensive solution that covered all of their functional areas, and (2) the SaaS deployment model with associated benefits of cost-savings, speed of implementation, and scalability.
Interestingly, according to my correspondence with Plex, Infor was de-selected as it did not have a true SaaS offering, and both SAP and Oracle were eliminated for non-response to the RFP. One can only speculate that Invensys may have become focused on going with a true SaaS offering, and neither SAP nor Oracle couldn't come up with one. If so, we may be coming to the point where even organizations as large as Invensys Controls see true multi-tenant SaaS as their preferred deployment option.

If that's the case, why didn't NetSuite win the deal? According to Plex, NetSuite had gaps in meeting key functional requirements. It would appear then that the deal came down to Plex versus NetSuite--two true SaaS offerings. If so, this underlines my point that the only thing holding back full-suite SaaS offerings from taking further market share is functional parity.

Back in the 1990s and early part of this decade, ERP selection often came down to long checklists of functionality, becoming more and more detailed as full-suite offerings matured. Eventually, such long checklists became less useful, especially for large deals, as the Tier I providers (SAP and Oracle) could pretty much check every box. (As a result, in our own ERP selection consulting at Strativa, we prefer these days to focus more on key differentiators and industry-specific requirements.)

Furthermore, in software selection deals we've worked lately, we are seeing much more interest on behalf of buyers in true SaaS deployment than we saw even one or two years ago. As buyers hear about the success of other companies with solutions such as Salesforce.com, the benefits of SaaS are much more well-understood, and the traditional objections (security, reliability, "where is my data," etc.) become less of an issue.

So, as the focus shifts to SaaS for full-suite ERP, we may be seeing functional requirements again becoming the key selection criteria. If the deployment option of true multi-tenant SaaS is superior to traditional on-premise deployment or single-tenant hosted offerings, then the only thing standing in the way of SaaS is functional parity. But, as development platforms such as NetSuite's make addition of new functionality much easier, the functional gaps are being closed much more rapidly than many realize.

Traditional on-premise vendors beware: the full-suite SaaS providers are catching up quickly.


Update, Oct. 15. Updated the post with newly-learned information about SAP's Business ByDesign and rearranged the sequence of solutions. Updated also to clarify Workday's intent to support services-based businesses with a full-suite offering.

Update, Oct. 21. Added information about Infor's Syteline offering in a SaaS deployment model.

Update, Nov. 17. Added information about Epicor Express, which is deployed as full multi-tenant SaaS offering.

Related posts
Ensuring win-win in SaaS aggregation of customer data
Plex Online: pure SaaS for manufacturing
Workday pushing high-end SaaS for the enterprise
Lawson's cloud services: good start, but no SaaS
NetSuite a viable alternative for SAP customers?
A game-changing play in enterprise software
The inexorable dominance of cloud computing
Computer Economics: The Business Case for Software as a Service

Friday, October 08, 2010

Ensuring win-win in SaaS aggregation of customer data

Proponents of software-as-a-service (SaaS) have been looking for reasons that shared architecture is superior to on-premise solutions. The latest argument is that a multi-tenant system (one where all customers share a common system instance) allows the SaaS provider to aggregate data from many customers and report general economic trends and or other statistics that add value beyond that of the service itself. Such aggregation of customer data is difficult if not impossible for on-premise vendors, which typically do not have ready access to each customer's system. It is also difficult for single-tenant SaaS providers, who would need to build an integration layer to access many separate instances of customer systems.

I first noticed this thought in the recently published book, The New Polymath, by my friend and associate Vinnie Mirchandani.

Vinnie writes in a section about a leading SaaS provider, NetSuite:
"We have some of the best leading indicators on the economy. We can aggregate order value, cash flow, and several other metrics instantly in our base of over 6,000 customers and watch trends," gushes Zach Nelson, CEO of NetSuite.... Soon, those customers will be able to benchmark themselves against aggregated data of their peers. That would obviate the need for mailed-in surveys. That capability has been the domain of benchmarking firms like Hackett, not of the software industry, so that is another innovation NetSuite is working to deliver. Nelson explains: "Take those benchmarks and some of the creative BPO [business process outsourcing] partnerships we are exploring, ...and the industry could see SLAs [service-level agreements] that don’t just monitor technical metrics like systems availability but business process metrics that have been elusive to codify."
Dennis Howlett then picked up the thought in his post on ZDNet last month, where he wrote about the value of SaaS being beyond lower total cost-of-ownership. He wrote (emphasis mine):
I have long argued that multi-tenant architectures offer transformational benefits from the ability to aggregate data. That is not possible in a single tenancy situation. Workday isn’t ready to contemplate that notion just yet any more than any other vendor. Except perhaps NetSuite and a small handful of the very small business apps vendors that are building in these capabilities. I recall a conversation where it was said that NetSuite saw the recession coming before it became public knowledge by virtue of the transaction trends it saw in its customer portfolio. Try doing that from your single tenant application.

How about selling anonymized data to banks, other financial institutions, telcos, insurance companies, healthcare organizations…the list goes on. Why would they buy these data? Application tech vendors tend to specialize in certain business sectors. Multi-tenant SaaS vendors are collecting prime data that has genuine value third parties cannot get.

A bank may see ins and outs of bank accounts but those numbers are meaningless without context. A first step I am seeing is where some vendors are suggesting giving client bank managers shared access to aggregated financial data in real-time. Why? Because the bank can marry what they are told with what they see and so make much better informed lending decisions. Add in the ability to offer comparative trend data and you’ve got something extremely powerful....

Long term, I believe the ability to powerfully slice, dice, form and reform data out of multi-tenant systems will become the place where customers see huge value that is way beyond TCO. If I can benchmark performance in real-time or spot trends and compare, again in real-time, then my ability to take corrective or revenue enhancing action is vastly improved. Do I maintain a competitive edge? Of course because it isn’t the availability of data that matters but the ability to execute plans against what I am seeing.

Does this require careful handling to ensure that confidentiality is not breached? You betcha. But just as in the consumer world we have forgone privacy in the name of getting help from Google, there is no reason to believe the same won't hold true in the enterprise. The upside potential for benefit is simply too big to ignore.

This seemed like a bit of an overstretch to me, so I commented on Dennis's post:
I am fully in agreement that multi-tenancy is in the client's best interest, for the simple reason that there is no way a vendor can deliver the service as cost effectively under a single tenant model.

However...I am having a bit of a problem with this concept of "selling anonymized data to banks, other financial institutions, telcos, insurance companies, healthcare organizations." Whose data is it? It's the customer's. The vendor has no more right to take that data in any form, shape, or level of aggregation than a telco vendor has the right to intercept my phone call. The MT vendor is a carrier of the data, that's all.

I'm pretty sure vendors who want to do this have something in their contracts to allow it. But if I'm a customer of such a vendor, I'd either say no, or demand compensation for it.
Now stirred up about this matter, I tweeted:
Having a problem with @dahowlett's view of MT vendors aggregating and selling customer data.

Seriously, ppl go ballistic about Google showing ads based on content of Gmail. Now it's OK for NetSuite to poke around cust's AR files?"
An off-the-cuff, poorly-chosen example. NetSuite--monitoring Twitter--promptly called me about it, and assured me they are doing no such thing. So, I corrected the record on Twitter: NetSuite is not poking around customers' accounts receivable files.

Actual practices of SaaS providers
That out of the way, the folks at NetSuite then offered to brief me on their actual practices as well as how their customer contracts are written. NetSuite was even kind enough to let me review its standard contract, which I won't directly quote here out of respect for confidentiality. But I did tear into it pretty aggressively. In addition, I interviewed two other SaaS providers (Workday, plus one other) to gain a broader understanding of this issue. As you'll see, the actual practice lags far behind the vision of what's possible.

Obligatory disclaimer: I am not a lawyer, I do not practice law, and nothing in this post should be construed as legal advice.

Here's what I what I've been told, based on interviews with these SaaS providers:
  • Currently, NetSuite is only looking a customer activity in terms of usage: how many users are being added or removed and how often do they log-in, for example. NetSuite has nothing in place to look at actual customer transaction data (e.g. customer A/R files).

  • NetSuite has considered the opportunity to let customers benchmark their own key metrics (e.g. average days sales outstanding in A/R) with those of other customers in aggregate, but it has not yet moved forward to do so. If it were to do so, it would allow customers to opt-in or opt-out of this service.

  • NetSuite's standard contract for customers includes prohibitions against unauthorized disclosure of customer data and against unauthorized "use" of customer data by NetSuite. Anything that NetSuite would do in the future in the way of new services to aggregate customer data would be done in a way that is consistent with its customer contracts.

  • Workday takes a similar approach. It considers customer data at three levels: (1) performance monitoring--e.g. response time, service levels, (2) usage data--e.g. what features customers use most often in the system, and (3) best practices--e..g. what is the average time for a customer to bring a new employee on-board.

  • The only customer data that Workday accesses today is performance data, as described above. Workday is considering services that would involve aggregation of customer usage data and best practices, and under Workday's standard customer contract, this would require customers to opt-in.

  • The third SaaS provider, who will remain unnamed, says the investment community has encouraged them strongly to develop customer data aggregation services, as they see this as increasing shareholder value. This provider has no current plans to develop such services, however.

  • This third SaaS provider indicated that its standard contract explicitly gives them the right to aggregate customer data and report aggregate statistics. However, this clause often gets stricken by customers in contract negotiations.
Although my research only involved three SaaS providers, it does show that the actual practice today is lagging far behind what is possible in terms of customer data aggregation.

Data aggregation not a new idea
This does not mean that there is no precedent for customer data aggregation, however. NetSuite pointed out to me that ADP, for example, has been reporting employment trends for many years, based on actual customer data in its payroll and HR systems. The description of ADP's National Employment Report shows the value of being able to aggregate a large sample of customer's day-to-day operational data.

In addition, Coremetrics, an online marketing SaaS provider, recently acquired by IBM, aggregates its retailer customers' data to provide its Coremetrics Benchmark, which it describes as follows:
[A] peer-level benchmarking solution that measures online marketing results, including commerce data, against those of the competition. More than 500 leading U.S. retailers, contribute their analytics data to Benchmark. All data is aggregated and anonymized.
Coremetrics also uses this data to provide its widely-quoted next-day flash results of sales from so-called Black Friday. ("Black Friday" is the major shopping day that occurs each year in the U.S. on the day after the Thanksgiving holiday, when many retailers finally can report sales "in the black" for their year-to-date results.)

So, in fact, the aggregation of customer data by SaaS providers is not a new idea. It's just somewhat new to most new enterprise SaaS providers.

My take
I agree there can be great value in reporting statistics based on aggregated customer data. But that value should be shared between providers and customers.

In reviewing SaaS contracts, therefore, customers should consider the following:
  • Non-disclosure. At a minimum, be sure your SaaS contract ensures confidentiality of your data. Most if not all providers appear already to have addressed this point. But verify.

  • Fair consideration. If your SaaS provider wants the right to aggregate your data with that of other customers, be sure that there is some quid pro quo for your participation. If the provider is somehow preparing benchmarks based on your data, you should be able to received the benchmarks either at a discount or at no charge in exchange for your participation. If the provider is proposing to sell aggregated statistics to third-parties, and this wasn't contemplated at the time you originally signed up, you should receive a discount against your subscription fees in exchange for your participation.

  • Opt-out provisions. If you are not comfortable participating in data aggregation, you should have the right to opt out. You should also have the right to opt out of future aggregation after having previously opted-in. This is an important check against providers getting too creative in how they use aggregated data, for example, segmenting the data so finely that it becomes uncomfortably close to identifying specific customers.
There's a lot of innovation going on with SaaS providers these days, as they search for ways to add value. That's great. But SaaS providers are still vendors, and a SaaS agreement can be just as lopsided as on-premise software agreement. As I've said before, "you don't get a pass just because you're SaaS."

Although I count myself as a SaaS proponent, first and foremost I am a customer advocate. Know what you're getting into with a SaaS provider and be sure you agree.


Considering a SaaS solution? Let me know if you would like assistance in evaluating SaaS providers or reviewing proposed agreements. My email is in the right-hand column.


Related posts
Workday pushing high-end SaaS for the enterprise
SaaS contingency plans need more than software escrow
SaaS: plan to get out before you get in