I have been working with a SaaS provider to help them define metrics that should be measured, monitored and correlated to business metrics. Thought it would be useful for others who might be in SaaS business. There is no denying the fact that follow critical metrics

• Annual Contract Value (ACV),
• Total Contract Value (TCV),
• Monthly Recurring Revenue (MRR),
• Average Monthly Revenue per Customer (AMR)
• Churn Rate
• Customer Acquisition Cost (CAC)

should be tracked and measured from a finance/profitability point of view, but in this post, I will focus on operational metrics that help you build customer success.

Usage Metrics

• Application Logins: it is always a critical statistic to measure how many users are signing onto the application. It not only talks to the scale of the application, it also demonstrates the critical nature of the application. If you broke down the logins by the role of users in the application – for example a HR Manager versus a VP of Talent Management, and individual usage patterns should help drive focus areas in your product roadmap. Marketing can use this same information to create day-in-a-life documents or case studies on the critical nature of the application. Operations can use the metric # of logins as a way to demonstrate the scale, SLA and also determine peak/off-peak usage patterns for performance benchmarking, planning scheduled downtime etc.

• Time Spent in the application: Time spent in the application is something a company should always be proud of. It talks to the stickiness of the application. While that is something marketing can use to demonstrate the value of the product, product teams should inspect the same metric to identify potential areas for optimization. In this day and age, users seek smart business processes that are not click-hungry and easy to accomplish. So users spending longer time on a given process could imply productivity loss and in the long run could lead to unhappy users. Client Services should look at this metric to identify opportunities for training (or lack thereof). It is always critical to benchmark a typical lifecycle of a business process and see if there are reasons to be concerned if deviation from the benchmark is large.

• Access Mechanisms: As such most SaaS applications are accessed using a browser. But with the browser wars looming again it is important for a product team to measure and compare the different browser usage – Firefox, Chrome, Safari and the Goliath – Internet Explorer. Besides the different browsers, it is critical to measure the versions of each. With effective logging of sessions, this information should be easy to capture. Besides browser, it is also important to capture information around
  • Desktop Operating System (Windows, Mac, Linux)
  • Monitor Resolution
  • Platform (Desktop Vs Mobile)
  • Languages used
  • Bandwidth used (DSL, T1, Dial-up)

  Product teams can use this information to improve testing coverage,   support for specific browsers (versions). Marketing can use this same information to highlight to the broad capabilities of the product/platform. Sales will require this information for the RFP they complete as part of a sales deal.

• Source of Users: Where your users are accessing your application from is a key important metric from multiple angles. If you are in Sales, you probably know your customers in various geographies. Facts about the density of users from a particular geography might indicate better adoption rates and need for increased sales efforts. For product team, this might drive decisions around caching strategies, internationalization or localization needs, increase in latency based tests. If you are in marketing you will now be able to leverage customers from various geographies to provide you localized references in marketing efforts.  Operations can factor this information to gauge the coverage of the redundant data centers created to cater to global users.  For those with one active data center, this could provide insights to support that second data center plans you were putting in place.

• Business Transaction Density (Quotes created, Search conducted): Capturing metrics around the key activities performed in the application like creating orders, creating versions, search conducted, user roles created, projects created, surveys conducted, documents uploaded are all great ways to measure the coverage of usage of the application. The product team can use this information to crosscheck with the roadmap and identify the cause for lack of usage in certain areas. Follow that up with discussions with customers that requested those features to better understand the effectiveness of the product feature. For instance a large number of document uploads might indicate companies using documents in lieu of  structured business process and identify opportunities for expansion of product footprint. Operations can use the same metric of large number of document uploads to determine if storage configuration should be optimized or potential for using de-duplication technology.

Operational Metrics provide additional insights in the user behavior and indicate hidden opportunities to improve.

• Tickets logged: Not the most favorite metrics for any constituents in the service provider company.  But I have a bright side. It is much better than not have any tickets at all – atleast you know your product is used. While it is standard to bucket tickets into product areas, I recommend you break down tickets into those logged by new users, critical areas of business process and specifically those logged during critical dates (month end, quarter end and year end). The easier you make new users to adopt the application, reduce the instances of fall-over the more purpose with which they will use the application. The more they will talk about your application. Conversely, the more troubling it is to get accustomed to the application the sooner they will desert it. Critical areas in business process and critical dates need no highlighting as to why they are important.
• Time outs: Timeouts in my opinion are the worst kind of issues. In addition to creating a bad perception of the product, they also could point to infrastructure issues, missed test cases. They also put client services in a bad spot where they cannot explain the cause unlike a product deficiency. Considering that it is not always possible to root out all time out issues due to the varying nature of access (cable, dsl etc), it a great idea for client services to have a “Have you checked this?” list. Worst of all are those that happen during crucial demos to prospects.
• Downtime: While unplanned downtime is bad and puts your CEO in the news for the wrong reasons, planned downtime is equally painful from the customer point of view. Given the round-the-clock nature of world we live in, people extend their work lives to evenings and weekends. So having excessive downtime and more so, those that went over the announced window need to analyzed. No one likes to work on weekend and if you cancel plans to work on weekends only to find out that downtime window has been extended would not make for a happy user on Monday.
• User growth over time: This is a no-brainer of a metric. Tracking user base growth and charting the patterns gives you a view of buying habits, correlation of success of campaigns and what you do well. It also show the times when you should increase thrust on your sales/marketing campaigns.

This was a representative sample of long list of metrics we identified and started tracking. The list extended to sales, marketing and implementation to track success in converting leads, success in converting trial users, investment done in implementation cycles, training cycles etc – I am sure you catch the drift. If you have channels, then you will need another set of metrics to measure the effectiveness of channels and give you insights into what does (not) work.

All these metrics once identified and tracked, can be part of a dashboard that all employees in the company has access to. Make sure to include those as discussion items in company meetings and goal settings for each executive.

In an on-demand business, the risk is heavily tilted towards the software vendor. It is incumbent on the company to automated, measure, monitor key statistics and adapt the business based on those insights. Any/All decisions made in roadmap, client services, market can critically impact the success/failure of the company.

The challenge of coming up with a SaaS incarnation when you have a established on-premise customer base is nothing short of what the Big Three Auto manufacturers are going through in re-inventing themselves. The entire thinking software design to delivery and maintenance changes. I will take a look at some of the key challenges and potential solutions.

Lay of the Land

Let us start with a look at a the footprint of a typical large on-premise application deployment.

• Global 2000 company with global deployment of a suite or integrate set of applications covering Financial Management, Supply Chain Management, Human Capital Management and Customer Relationship Management besides some industry specific vertical applications.
• Extensive customizations, localizations, integrations to other applications
• Reporting infrastructure supported by a large data warehouse or some form of redundant data store with aggregated data from one or more sources.
• Company specific security implementation to meet the governance mandates.
• Scalability related deployments like WAN Optimization, Caching, Replication.
• 5-10yrs of historical data.
• All of these managed over private hardware infrastructure that needs large upfront investment and ongoing care-and-feeding.

Just so we get a true sense for what they are up against by establishing the key characteristics of a SaaS application. Granted, the definition of SaaS along with Cloud Computing, Web 2.0 form the troika of terms that have had a hundred interpretations, if not more. But in my mind a true SaaS application would have the following characteristics

1. Single Code base shared across all customers
2. Multi-Tenancy architecture to host all customers in a single instance.
3. Blue-prints/Templates to facilitate rapid on-ramp of new customers.
4. Self Service Administration model
5. Framework to easily integrate external applications
6. Framework to move data from existing applications in bulk

The Challenge

The incumbents see the on-coming SaaS train shaking the very foundation of guaranteed maintenance revenues. In the face of mounting pressure from customers to reduce TCO and also to combat lost sales to upstart SaaS vendors, are responding to this challenge is different ways.

• Some have gone onto create a new product, albeit scaled down version with limited success.
• Some have sprinkled some web-based services to their on-premise offering and claimed victory with some marketing around it.
• Some have just plain made claims that their products have been designed as SaaS from the ground.

To me all this is posturing in deferring the inevitable. They all know SaaS is here to stay (Sorry for ruining your wish Harry) and the on-premise gravy train has run its course and entering its last leg. If the recent customer pushback to a SAP’s one-price-fits-all maintenance contract (driven increase) is any indication, customers are clearly sending the message that they are tired of bearing all the risks, overheads and whims of software vendors.

The Journey

Different companies have embarked on this journey in different ways. There are companies like SAP and Callidus who have created a alternate line of products for SaaS and along with it came a parallel organization who will invariably end up competing against each other. Then there are companies like Oracle, Infor who are re-architecting existing products or new version of their products to support both models. While this seems like nirvana, it is rife with challenges.

Business Model: The foundation of any business is its business model. Moving from a license model based company to a subscription based model creates ripples in the business model. It creates challenging questions around the R&D budgets, revenue streams, revenue recognition and cost of sales as they are going to be dramatically different from what it is in the on-premise world. It is easy to just hope that this Cloud/SaaS stuff would go away.

Sales: Of all people, Sales will have a rude awakening. There will no longer be those front loaded large contracts that will bring in big commissions. SaaS deals are going to be much smaller in size to begin with and then ramp over time. Save for some exceptions like Flextronics deal for Workday or GE deal for Aravo, deal sizes are going to come down a notch from millions to thousands. Just so SaaS sales does not cannibalize the maintenance revenue “gravy train” from existing on-premise customers, they will be out of bounds for SaaS sales team. Hunter & Farmer model, if adopted, will create more heartache for sales guys. They will not be able to engage with customers after the initial sale as they do now.

Marketing: Marketing will no longer be the “all vapor no results” and now unwillingly be bed fellows with sales. Their activities will be scrutinized and tied to ROI more so than ever before. Budgets will be constrained unlike days of the past. As I explained in my post scope of marketing will now expand from demand gen activities to lead qualification and the primary goal would be reduce sales cycle. Webinars, online marketing campaigns, customer/partner communities, customer engagement assume critical nature.

Partners: System Integrators in the good old days would take a product that does not really fit the real needs of the customer would make it work by customizations, integrations, migrations – all for a lowly price of some million dollars. If you had just recovered from the sticker shock of the product, the after shock from SI would enough to make you dig deep into your IT budgets. Now with SaaS, the provider takes onus for many a activity that a SI would have performed. Customers expect the try-before-you-buy deal during which they expect to spend very little, if at all. As a SaaS vendor, you are expected to have integration APIs, Web Services that connect to their in-house apps or other Cloud based apps. This also puts onus on you to have a more finished product and eliminates a shield that SIs provided for product issues in the past.

Product Architecture: This to me is the most under-estimated issue. To say,  “Our architecture is designed from ground up to work for on-premise as well as SaaS” is gross underestimation of the challenge. Just stripping the database for multi-tenancy architecture while essential, makes not a product SaaS ready. Here are things you need to factor in

• You are now going to be responsible for scaling of the application, fail-over, almost zero-downtime maintenance, all this while one issue is enough to cause most, if not all, customers to be at your throat.
• You will have to continuously tweak a single deployment to adapt to the varying workloads in terms of volume, user habits, areas of the application usage, geography.
• If you said, same product will work for both sets of customers, on-premise and SaaS, brace yourself. You will have two sets of customers each expecting different rates of change. Having a product team go full throttle once and hunker down another time is easier said that done. Remember – they say in Good Driver guide – Rapid Acceleration and Sudden Slow Down might not get you where you want to go faster, but it guarantees damage to the engine.
• If you had two different teams for building SaaS and On-premise, then you are dealing with fragmentation of knowledge and skills. Domain experts will now need to stretch themselves to meet the needs of two teams.

Operations: This is a completely new area for a software company. If internal Development Operations was challenging enough, now you are dealing with Data Center challenges, Redundancy, Disaster Recovery, Intrusion Detection, SAS-70 Audits and constant demands from sales team to support them in sales cycle allaying fears of customers. The SLAs asked of you would put you on the hot seat while the budget constraints will continue to ask more of your for less.

Support: In general, the customer support of most enterprise software companies is ordinary at best. Customers are left to fend for themselves and at the mercy of System Integrators, IT Consultants and Community Q&A forums. This in addition to the small army of IT resources in-house. With SaaS, the support tiers suddenly collapse. Vendor is now becomes the helpdesk for not just product issues but also for its availability and performance SLAs. It would be in your own interest to make the product as much self-service as possible to alleviate the strain it is going to put on your support. Fostering a vibrant community to support itself via community owned product documentation, how-tos, case studies would go a long way.

Roadmap: No not the product roadmap, the company roadmap. There is no way a company can keep going with two product lines that demand such divergent needs of the company. There should be plans for the product lines to converge and so also the plan to move customers over to SaaS. While SaaS would continue to drain a lot of money upfront for infrastructure investments while on-premise gravy train continues to fund it. But this can go only for so long. Ask Callidus Software that embarked on such journey as to the amount of (financial) stress it put on them for some quarters.

A few companies like Infor, Plex seem to have made the transition or almost there. It will be interesting to see how this journey shapes up for SAP, Oracle and how they transition from old to new.   While the ever growing list of upstart brand new SaaS startup with no baggage keep creeping into their customer base.

Now let us look at the reality. Despite all the hype and promises of cost efficiencies and barrage of me-too announcements we get daily from technology vendors purporting Cloud capabilities, all the evidence around usage of  Cloud services points to it being limited to

1. Early stage startups building edge-apps or small business focused applications. (Okay, I know there are some exceptions and they are anything but)
2. Enterprises replacing file shares with cloud based storage for redundancy, backups.
3. Enterprises conducting a proof-of-concept/R&D projects on cloud based computing power before building the real-thing in-house.

While the value of Cloud is definitely the way of future, I still do not see any real evidence of large scale customers moving their mission critical technology deployments on to the cloud based infrastructure. Not just yet.

In fact, there is no evidence that Big 4 SaaS vendors Salesforce.com, Intacct, NetSuite, SuccessFactors themselves don’t claim their services are based on commercial cloud based infrastructure. Note: I will stand corrected if someone from those companies can provide some facts.

In this three part series, I will focus on the Infrastructure-as-a-Service (IaaS)  and try and establish the needs yet to be met before a wider, large company adoption can be expected. In the follow-up posts, I will delve into PaaS and SaaS related challenges and needs.

Infrastructure-as-a-Service (IaaS), for the uninitiated, is the delivery of IT Infrastructure capacity as a service. Companies can consume ubiquitous elastic Computing Power, Storage Capacity, Network Capacity, Security, Backup, Redundancy on a usage based subscription fee.

IaaS aims to relieve companies from the burden of

• Having to make large capital investments in IT Infrastructure to build capacity and consequently move to a more predictable OPEX model in tune with your business needs.
• License, Install, Upgrade and maintain all software
• Buy, Configure, Upgrade and Maintain Hardware
• Hire and Retain teams of System Administrators, Network Engineers, Database Administrators
• Negotiate and Maintain Vendor contracts.

All these capabilities delivered by achieving unprecedented economies of scale by using commodity hardware, open source software, virtualization and continuous automation.

All the virtues listed above and then some make for a undeniable value proposition. So why all the concern and inertia in large scale adoption?

Large enterprises work in cycles that are much slower than the pace of innovation in industry. With good reasons, I might add. IT has become the backbone of running any business today and changing that in-flight is akin to changing the foundation of a house that you continue to live in. So, not withstanding all the catastrophic predictions by the so-called experts, for those not embracing the Cloud, we should carefully look at the reality.

I will use GE as a prototypical large global company with heterogeneous businesses, each with its own demands, technology and resource footprint. So if I am Gary Reiner and I am considering to move the GE IT infrastructure to Cloud based services, here are somethings I would definitely seek answers for

1. Given that I will not be moving my entire IT portfolio to a single Cloud provider, how do I keep my Cloud Deployments in sync? Are they interoperable? Can I move things from one cloud to another? Can I move a VMWare vCloud app to Amazon? Do I need to maintain multiple flavors of them if I use different cloud vendors?
2. Can I move workloads from one cloud provider to another? Or can I burst workloads from one to another to meet seasonal or periodic demands of the IT infrastructure? Think Best Buy during holidays.
3. Can I get guarantees on the performance, latency that I get from my dedicated network?
4. Do I have robust tools to manage the various cloud platforms out there? Can the same set tools work for all the cloud platforms?
5. Can I meet the governance mandates, I have around SOX, HIPAA, PCI-DSS? Can I have control on defining access policies to the data all the way to the storage?
6. The most important, can I tie the Quality of Service that I need to provide to the business, with the elasticity of the cloud. Especially, given that the cloud provider’s architecture has been built to provide a particular service and not envisioned as participating in another transaction.

Let us go through each of the points in detail

Interoperability: As with any new technology, the cloud standards are yet to mature, if at all. Each infrastructure-as-a-service vendor is pushing their own standard implementation API – AMI for Amazon, vCloud for VMWare. They do not have standards for a common API that service providers or application vendors can integrate into to access those services. Standards around security, cloud security, infrastructure protocols and data artifacts are still in the early stages in DMTF, the working body that is making an attempt to draft standards. Cloud interoperability standards that result from their work it is hoped will reduce lock-in and increase agility for cloud computing adopters taking advantage of a multi-provider, mixed cloud environment.

Portable Workloads: Given that companies have different applications that meet different needs, their capacity and scalability needs might be different too. It is fair to assume that companies would have a federation of clouds to manage different needs – much the same reason companies have multiple internal networks to compartmentalize things. The departmental lease management application, core financial application and the corporate data warehouse all might be hosted on different clouds if not different providers. If a company decides to use a hybrid model with some of those applications in house and spare capacity for them on cloud, today it is not possible to burst out workloads from one cloud to another or from an internal network to the cloud. This goes back to the lack of standards.

Network Latency:  In the new soon-to-be-all-in-cloud world we will all be accessing the applications through the good old internet. This is like saying we will all take the same freeway to work in the morning. So every time there was the Victoria Secret live broadcast on the internet we will all be log jammed on our way to the critical application we need access to finish work. This today is less of an issue since most large enterprises have dedicated networks and WAN optimization implemented.  All the SLAs provided by the IT to business and Cloud provider to the IT organizations is moot if the packets don’t travel fast enough. Another kind of Net Neutrality discussion you say?

Manageability: Most of the enterprise applications that are managed on-premise today have sophisticated tools from System Management vendors like HP, BMC, CA that allow you to manage almost all the aspects of the enterprise IT footprint. Once we move the applications to the Cloud, most of those system management tools, configured to your current environment might not work. Load Testing, Monitoring, Quality, Configuration Discovery tool vendors do not openly claim to support cloud based deployment. Most of this goes back to the fact that there no standards for Cloud APIs. Adapting the tools for each cloud provider is a R&D spend tools vendors have not committed to.

Data Governance: In the hyper optimized virtualized environments in which the Cloud platforms operate (they have to so they achieve economies of scale) data is virtualized, sharded, replicated, cached. This brings the very critical needs around data retention, protection, purging, access requirements mandated by SOX, PCI, HIPAA that large companies need to comply with. The current cloud vendors save some announcements do not openly claim they are ready to sign SLAs to agree to these needs. The Cloud platforms today do not have the quality of tools that are needed for companies to dive into the logs, access and troubleshoot.

Availability and Reliability: Unplanned outages in Amazon, Google are but rare publicly discussed occurrences of Cloud Service reliability issues. Those kinds of odd occurrences can happen even in a private network. But when you talk about multiple large customers co-locating on the same infrastructure, it raises concerns of many more of those outages. The Cloud architecture that are available today have been designed with a certain set of assumptions of how applications work. Case-in-point,  Amazon.com has no understanding how SAP Payroll system would work, neither are they interested. So not having an understanding of application workload patterns, architecture any SLA provided upfront is meaningless. Without tying the Quality-of-Service(QoS) metrics that you have in place for your customers/business and the SLAs being signed with Cloud vendor there is no way availability can be guaranteed. If you consider the scenario of multiple clouds or hybrid clouds then the integration of SLAs between all of them and tying it to QoS is practically impossible. Service credits to compensate for the lack of availability, by themselves, might not guarantee his/her job for the CIO who made the decision to move to cloud.

I know an argument could be made as to how things like portable workloads are handled today in an internal network, if at all. I admit, companies are probably not geared to handle these needs today, whilst on their own network, but then if we discount future needs then all the switch-to-cloud would bring is a like-for-like swap,  a cloud network for private network. The cost advantages that cloud brings will be offset by the loss of existing investment. So what gives?

I also hear people talking about companies not being ready to absorb sunk costs into legacy IT infrastructure as the reason behind lack of cloud adoption. To which I say – I don’t see people swapping old perfectly well-functioning cars/houses for new energy efficient ones. Maybe a cash-for- clunkers program is in order here too.

If at the end of this elaborate post, if it seemed that I am a tad bit anti-cloud, I am not. I am a die-hard fan on the cloud computing. In fact, I am one of those who thinks – IT shops have no business building custom applications, regular companies have no business building their own power generators and people like me have no business buying a cow when all I need is just milk. The world works best when specialists are left to do things they are the best equipped to do. That said, I would tread carefully and evaluate my risk before making bets on next best technology innovation since slice bread.

Anyway I digress. Now that we have had a successful year of market share gains for SaaS vendors behind us, it is time for CEOs of SaaS companies to make their new year resolutions. Having spent some time meeting CEOs of SaaS companies and their clients, I thought the least I could do is to create a new year resolution template to help them out. So here goes.

Resolution #1: Improve Customer Service: My customers have been incessantly complaining of lack of adequate customer service. This coming year we will spend enough money and resources to provide A+ service, excellent documentation and foster a community that can support itself. After all, we will need customer references to gain new customers now that we have cornered the easy pickings. The last thing we want when the economy recovers is for the customers to move in droves, to a competitor.

Resolution #2: Provide better on-ramp process: We managed to get a bunch of customers online – kicking and screaming. Not to mention, our profit margins on those customers went down the toilet. Considering that we do not need to spend all that money on cross-platform porting/certifications, on supporting multiple versions concurrently, we should make it easy to get new customers online and using the product.

Resolution #3: Provide a real integration framework: Following up on my previous resolution, we should make sure the engineering team designs the product with the knowledge that we will not be an island onto ourselves. Companies require that the information loop is closed with their other cloud applications or existing on-premise (or do those fall under the category- Clunkers now?) applications. Standard APIs/Web Services should be moved from nice-to-have bucket to must-have bucket early this year. In fact, we should be working with our customers to identify the adapters that we should be providing out of the box. This will then make good on all the blabber we made about TCO during the sales cycle.

Resolution #4: Be the best customer advocate I can be: I MUST become the biggest customer advocate in the company. I don’t need to be the great visionary all the time. Customers have made big commitments by taking a chance on us and signing up to our service. Now it is my job to support them and help them succeed in their business. While I am at it, I should make it a point to ensure my entire organization makes only those commitments that they can follow through. Memo to Sales team – “SaaS is not a hit-and-run sale, we will be engaging with the customers for a long time, so let us not start on a wrong footing by promising the impossible/non-existent.”

Resolution #5: Be Transparent: Every time we had service outage this year, we have had to have a embarrassing meeting to customers/press. This year invest in being transparent. Trust builds when we are transparent. Do what Intacct and Big Dog Salesforce.com has done with their service level dashboards. We definitely do not want to have a public boo-boo day like Workday did. While am at it,  I must put in place a process to share the audit certification and governance reports as well with our customers.

As a CEO if I follow-through on all these resolutions and we execute we  should be able to have another great growth year ahead while keeping the customer churn down. Now that I have captured my resolutions, it is MBO time for VP of Products, Service and Sales !!

Starting with delivery model, architecture, sales, support companies, employees and customers need to get used to a new way of doing things. If you are one of the decision makers on either side of the transaction, the SaaS vendor or the Customer considering buying SaaS, there are a variety of legal issues you need to contend with.

Bruce Cleveland, a SaaS veteran and a pioneer of on-demand business while running the Siebel On-Demand, now a VC with Interwest Partners, must have been one of the first few to enter this uncharted territory. Defining new pricing model, subscription agreement, Service Level Agreements (SLA) is just the beginning. As a vendor you need to ensure you have backing agreements with your service providers like hosting company, license software providers for you to be able to meet all your commitments to your customers.

Bruce shared a detailed Q&A session on SaaS business model and legal issues, he had with his legal attorney during Siebel days, Cary Platkin of Platkin Law, on his blog. If you are starting off in your SaaS journey, this serves as a good starting reference.

Cary goes on to explain the basics of a Subscription Agreement and risk mitigation/sharing strategies by using similar or better back-to-back terms with your vendor. The larger your customer base, larger you share of the risks are.

SLAs are critical in providing services that customers run their business on. Most SaaS companies guarantee anywhere 99.5% to 99.9% up-time as part of their SLAs. As Cary rightly points out, most and the best SaaS providers have outages or unplanned downtime. So keeping that in mind, factor the availability, response times, performance commitments, Disaster Recovery commitments, while drafting a SLA. Service credits are becoming a critical part of SLAs. But in my experience after a service has delivered enough value to the them, (make sure you keep that as your focus), customers are more forgiving that you might think. We once had a service credit report of 250k (across a year) whittled down to a mere low thousand of dollars, when all was said and done.

Besides outage, data breach or leaks are the most concerning issue that will be raised by customers during contract negotiation. Customers are getting more educated on the Data Security concerns and the necessary process and infrastructure needs around Data Security to meet their regulatory mandates. As you saw from the Merrick v Savvis case, the service provider can be held liable for incidents of breach. Cary has the right advice for SaaS vendors is ensuring sufficient insurance, avoiding unlimited liability and avoidance of any ASP like terms.

If revenue recognition was not already complex, SaaS has some new twists considering that the agreements are signed upfront but the corresponding revenue recognized over the life of the contract.

Cary also explains the complexities surrounding multi-year agreements, international contracts, Data Privacy requirements, Data Ownership are all key areas to focus on during contract negotiation.

As with any legal issue, consult your attorney to ensure you have worked out the details around the complex legal issues involved while the SaaS model and the legalities continue to evolve.

Large Upfront investment: The first and foremost benefit to a company is that it is hosted by the vendor (or a third party) and a company does not have put in place the IT infrastructure, team to host, manage the application. This frees up the large IT spend that now could be diverted to other projects that require it, now.  Remember the infrastructure includes application deployments for QA, Production, Pilots/Acceptance, Development and Fail-over besides production.  If you are global company then add to it the other needs around replication/WAN optimization to counter the latency issues. It is the Total Cost of Ownership (TCO) remember!. Then there is the Storage. With SaaS, your vendor will address all these needs for a fixed all-rolled-into-one subscription price. The beauty of the model is you can ramp up your costs commensurate with your consumption as opposed to making all the investment upfront often resulting in under utilizing the hardware/software.

Ongoing Costs: This is one of the most contentious one. Traditional vendors will have you believe that over time (5yrs and above) the infrastructure investment will have paid for itself. After that SaaS is going to cost you while an on-premise solution will have no incremental cost if you add more users. This is a bogus argument.  On the face of it, the subscription costs might look like a redundant cost after the “infrastructure pay-off period” but that is only one side of the story. Anyone who has managed IT for a while, will know, IT infrastructure is a living thing and needs constant care and feeding. Software updates, hardware additions, performance tuning, monitoring, backup and recovery. Then there is the burden of negotiating contracts with the hardware and software vendors. This on top managing the application and the upgrades will clearly be much more than the subscription costs. Each upgrade cost could come up to tens of thousands of dollars, if not more.

One thing which was until recent not a consideration was the power consumption and needs. With all the power glut, now energy needs are becoming the first or the second line item on every on-premise software consideration.

What else? Oh! yes. Did I mention that there is the small matter of manpower turnover that you will have to deal with when managing things on-premise?

On-demand Elasticity: SaaS affords you a quick on ramp with costs commensurate with the size of the team. Once you have conducted the smell test and decided to move further, SaaS also gives you the opportunity to scale up and/or scale down depending on your demands. Say your company makes an acquisition – you scale up your usage with just the subscription cost going up – easily quantifiable. Say 6 months later you rationalize the acquisition and decide to reduce a workforce reduction, you can now scale down the usage on the SaaS application. Contrast the same with a on-premise deployment. You buy it – you keep it. You will have to continue to pretend that those phantom users who were part of your organization during the post merger/acquisition integration are still part of the organization and keep on assuming those costs for the hardware procured, higher energy costs for that extravagant hardware footprint.

Anytime/Anywhere Access: Being in the cloud, your users will have access the SaaS applications just by using a browser and internet connection. Contrast that with a on-premise application that needs provisioning of access, you will need to go through a IT root-canal for arranging for VPN, access control etc.

Short-term ROI: Depending on the scope of implementation a SaaS application can quickly allow you to define target ROI and achieve it. Implementations are shorter in matter of weeks for a new implementation and maybe 2-3 months in cases where there is data migration, back-office integration. Contrast that with a long drawn out on-premise implementation. By the time the software is installed, pilots done, it will be a good 6-9 months if not more. ROI is elusive, if ever.

Risk Mitigation: In case of a traditional on-premise implementation, besides the large upfront investment for IT infrastructure, there is substantial investment to be done to get the project off the ground. This could include staffing a team and a prolonged project wrought with risks of project failures and cost overruns.  With SaaS, you could have a defined implementation schedule and near term ROI, which if not met, can allow you to terminate the project at the lowest cost.

Frequent Product Updates: One of the key benefits of SaaS is the product updates are frequent. This nimbleness of the vendor allows them to deliver incremental functionality faster than any IT organization can deliver to business. This represents a opportunity cost that a company will have to bear in a traditional software management model. Also given that the vendor is responsible for upgrades, relieves you of that cost burden as well.

Support Cost & Quality: With the vendor themselves providing the support (included in the subscription price), the buck stops with the vendor. You can define SLAs and measures to hold the vendor accountable. In a on-premise case, the IT organization is responsible for SLAs for a product that they would not be the ultimate experts in.

Get ready to be drilled

While we went into all the virtues of SaaS, there are some landmines that you should avoid as well.

Portability: SaaS has been perpetuating this principle that – “Customers can easily move to another vendor, if they did not like the service. So they have nothing to loose by signing up to a SaaS service”. This is b.s. of the first degree. This is much easier said than done. Never sell yourself into a deal where your bluff can be called. A smart buyer can push you to a corner with this and have it in the contract for you to provide data in a normalized form when/if they decide to move to another vendor. If you are tech savvy then you know what you are getting yourself into with that argument. Talk to your engineering team before you sign-up for this.

Integration: “We can integrate to everything/anything on earth through our APIs”. This is one area were every vendor exaggerates. Integration is not an off-the-shelf offering. Remember – Even a band-aid needs peeling, sizing cleaning the wound before you apply it to the wound. It is not magic. So take time to understand the need before you oversell your integration capabilities.

Disaster Recovery: While you are new SaaS company, it is unrealistic to assume that you will be able to afford Disaster Recovery implementation. While this is one of the gating issues – you would do well to highlight the Service Levels, Uptime statistics and if you do have a roadmap for DR share it. Remember you do not have the same excuse that a on-premise sales person has – it will be ready by the time you are live (read 18 months). On SaaS you are live tomorrow.

If you are an IT Executive
One of the first things you will see that is different, is the primary application management is not your main imperative. That said, while your team might not be directly responsible for managing the actual infrastructure and its upkeep, they will still be responsible for the following

• Program Management of on-boarding business community
• Planning and Uptake of upgrades
• Managing and Certifying integration with the back-office (on-premise) applications
• Provisioning of Access to the applications
• Service Levels to the business community

Once it is established that the application and the data is going to be hosted outside your premise and managed by others, three things that pop into your mind should be data security, scalability, vendor lock-in. Besides those, a company should understand many nuances of buying a SaaS solution before they ink the contract.

Being on the receiving end of these questions myself for the last couple of years, helping key deals at a SaaS vendor, I thought it would be useful to put together a reference guide for the buyer to help them procure a SaaS solution.

Let us start with the high level vendor qualification stuff

Financial Stability

Given all the turbulence in the market, ensuring that the vendor of your choice is financially stable is paramount. Outside of Salesforce, RightNow, Omniture and handful of others, most SaaS vendors are private and hence there will not be much financial information available to you publicly. So as a proxy to that I would look for some of these characteristics -

• Check up on the VCs that are backing the company. The key thing to  check is to see if those VCs have a good track record in guiding companies in similar space. Good VCs back their investment to a hitch. If you can get information on how the funding levels are, revenue run rate etc that would give you a better idea. As much as you prohibit, nothing is out of bounds to a sales guy trying to close the deal so go ahead and ask anything else you need about the company – off the records, obviously.

Management Team

Any vendor is only as good as its management team. While the product fit is critical to ascertain, ensuring the vendor has a good management team should give you more comfort. Having a good team should give you the assurance that they will execute on the vision they are selling you on despite where it is at the moment.

While looking at the management of the company, here are some things to look for

• A team that includes executives from the industry domain.
• A industry leader as a CEO. As goes the leader, so goes the company.
• A passionate leader of customer support. Insist on talking to the executive in-charge of Support as part of your finalization process. An executive with no passion for serving customers in not someone you would want to deal with. As IT organization responsible for the support you don’t want to be caught between rock-and-a-hard-place. A angry business user and a non-compassionate support team from vendor.

Focus

SaaS model works best when the solution is purpose built, niche with a set of configurable options to tailor to specific nuances at each company. Any (small) vendor that claims that their solution can meet the needs of large and small companies alike clearly highlights the lack of strategy on their part and should throw a caution flag. There is no way a smaller, leaner SaaS company can afford or be able to meet all the needs of small and big companies alike. A General Electric and a small electronic parts manufacturer operate completely differently and so will be their needs. If you are the smaller electronic parts manufacturer then the vendor better be focusing on smaller customers like you or you will be in the queue behind the big customer who contributes to more than 30% of the revenue.

In terms of the target market, the needs of multiple industries might be similar to an extent, but the business demands will eventually diverge. So it is important to ensure that the current customer base of the software vendor has enough names that belong to your industry to ensure your industry will continue be a focus area.

Organization Structure

While understanding the vendor also try and understand how their organization is structured – # of engineers, # of support analysts, # quality engineers. Each of those point to the value a company puts in those corresponding disciplines.

Here are somethings to put things in perspective. For example:

• if the # qa engineers are less, guess what, you are going to be their qa. Vendors across the board are trying to cut costs and stay slim and if you know software vendors guess who they cut first – you are right – it will be QA.
• Less support engineers would mean you will have to compensate by having a team on your end to answer to your business.

Pricing Terms and Billing

Typical pricing in SaaS is by subscription. Vendors usually require customers to sign a one year or three year contracts. Based on the unit price, be it a user-based, transaction-based or tiered price or some combination of user+module price you should make sure the terms are elastic. The turnover in people, mergers-divestitures-layoff can create changes in usage so pay special attention to escalation clauses. Also ensure the billing/payment terms are clearly outlined in the contract. If the you plan to put uptime guarantees in the SLA terms, service credits, penalties should also be accounted for in the billing.

One of the things you will need to look at would be the switching costs. You should also make sure termination process is clearly worked out as part of identifying the vendor. More of that later in the Governance part.

Technology

Majority of the SaaS solutions are built on some flavor of Multi-Tenancy architecture. That is the only way the vendor can establish economies of scale and provide you a all-wrapped-in-one subscription fees. You will hear a lot about web services, but check to make sure it is the flavor you are looking for. Most of them would involve you buying additional paid IT services to adapt them instead of working out of the box. To effectively integrate the technology into your business process, including customizations on your end, you should also look at the technology being used by the vendor. Specifically, you should have your architects look at the architecture, redundancy infrastructure, scalability considerations, provisioning made by the vendor.

Customers

Checking who uses the solution before making your decision is absolutely critical. As the first order of business, check the news and events section of a site to check the press releases. Any company worth its salt would do press releases when they sign new customers (or a set of new customers). Check for the quotes from those customers in those press releases. The names there would serve as good reference checks. If you are lucky you might already have them in your network. Besides throwing names, a vendor should be able to provide references and case studies of successful implementations. While you are at it, also check for customers who have renewed their contract at least once. A customer who has renewed the contract (after the initial term of 1 or 3 years) is much more convincing reference than a new customer just couple of months old.

Product Roadmap

Evaluating a product fit to your business need no doubt would be the first thing you look for. Granted that you would verify and ensure the product meets most of your current needs. A product roadmap with concrete list of items for at least the next six months and candidates for future releases are a good measure of a quality vendor. That said, you will also want to ensure that the vendor is agile enough to change course and priorities as business needs demand.

Governance

If you thought the imposition of regulatory mandates in recent times in onerous already, implementing them with a bunch of SaaS vendors might create some additional challenges. The governance processes implemented by the SaaS vendor should be of particular interest to you. Who manages the application, backups, upgrades, change management, infrastructure, security model, data center resiliency, data retention policies, disaster recovery all should be high in your criterion while considering a vendor. Most small vendors, understandably, have some form of ad-hoc processes but it is important for you to vet that against your own SOX-404 process needs. Despite the fact that software is hosted outside your premise, the burden of proof of compliance rests with you. Your auditors might not have yet gotten their arms around how to audit SaaS applications in their audits, but rest assured it is coming. So stay ahead of the curve. Ensuring your vendor has SAS 70-Type II certification done – regularly – should also be something you should insist on – and incorporate in your SLA terms.

The Technology Evaluation and Governance areas around buying a SaaS solution merit their own posts and I will cover more details in their own posts.

Update:

You can read them at

Part II: SaaS Buyer’s Guide: Technology Considerations

Part III: SaaS Buyer’s Guide: Governance Controls