Salesforce.com, RightNow, Oracle's Siebel CRM On Demand and NetSuite provide CRM software-as-a-service (SaaS) solutions. Although the base user subscription fees provide some level of comparison, users must look further to get a truer picture of overall fees and commercial risk.

• CRM SaaS vendors have a variety of contractual and noncontractual approaches to pilots, uptime guarantees/service-level agreements (SLAs) and penalties, charges for sandboxing/test and development environments, storage, backup and recovery, which are included in basic support vs. premium support and the differing fee structures and termination obligations.

• Pilots are a good idea, but don't end up with a contract that ties you in for many years.
• Seek contractual uptime guarantees that meet your business needs, and beware of exclusions to those guarantees.
• Beware of overpaying for sandboxes (that is, test and development sites).
• Avoid paying for extra storage per user where possible; if forced to pay extra, then minimize fees in advance.
• Because basic support is so minimal, you will probably need to buy a premium support package and negotiate the rates upfront.

Many organizations want to compare the price per user per month for CRM SaaS for salesforce.com, RightNow, Oracle's Siebel On Demand and NetSuite (see Note 1). Although the base user fees provide some level of comparison, users need to look further to get a truer picture of overall fees and commercial risk. In particular, customers need to understand these vendors' varying approaches to pilots, uptime guarantees/SLAs and penalties, charges for sandboxing/test and development environments, storage, backup and recovery, which are included in basic support vs. premium support and the differing fee structures and termination obligations because they all affect overall costs. This research compares these approaches. Although these are not the only areas to evaluate, they are the key areas where there are differing policies/fees (for more information on general terms, see "Software as a Service: Negotiate Key Terms to Avoid Unexpected Costs").

Some CRM vendors offer prospects a free trial, during which the company can pilot the CRM application without a charge (apart from professional-service fees). The length of these free trials vary:

• Salesforce.com and Siebel On Demand offer 30-day free trials; after this period, you have to decide or fees become due unless you negotiate exceptions. With salesforce.com, Developer Edition is available for free, which allows test and development and permits up to 2MB each of storage for two users. This could be useful when the 30-day trial phase is over if work needs to continue in a limited fashion, within these restrictions.
• RightNow's pilots usually range from 30 to 60 days; however, there have been exceptions where a pilot has run longer.
• NetSuite's standard test drive (which is typically a pre-built demo environment, but custom environments are available on request) lasts 14 days. If NetSuite provisions a company with a demo account, then it has no time limit and is free.

Prior to piloting, negotiate all contract terms and minimum discounts that would be due if the pilot is successful, on the basis that there will be little leverage if it succeeds. Minimum discounts will also ensure some flexibility for renegotiation if the pilot reveals unexpected use needs (for example, more users or data than expected). If there is a 30-day time restriction on this pilot, then ensure there is some flexibility, such as a two- to three-day free extension and a reminder of the deadline by the supplier; therefore, if the deal is not canceled beforehand, the remaining 11 months of fees do not suddenly become due on the last day. If required, ensure that your trial data and configurations can be carried forward into your production environment without redoing them, as salesforce.com offers.

Service levels regarding performance and uptime are critical for most CRM applications (see "Beware of Unplanned Downtime When Using Software-as-a-Service Providers"). The uptime guarantees vary, as do the penalties awarded for failing to meet them:

• Salesforce.com is the least favorable, with no contractual SLA unless customers negotiate for it. Because many salesforce.com contracts have no SLA guarantees, when the application goes down, the customer has very little leverage. Salesforce.com states that it provides an application called trust.salesforce.com that provides "ballparks for uptime and downtime," but customers should note that it does not provide customer-specific reports.
• RightNow, NetSuite and Oracle offer an SLA with 99.5% uptime per quarter, which is offered as a contractual promise. In the case of Siebel On Demand, it is included as a standard support entitlement. In general, this appears to be standard in the CRM SaaS market.

At the top end, vendors such as Entellium offer a standard uptime of 99.7%, and Siebel On Demand customers who pay for Gold support can get 99.7% uptime.

When presenting these uptime guarantees, CRM SaaS vendors, like all SaaS vendors, clearly state that they are only including the storage, server, application server, application, Web server and the service provider's connection to the Internet in their calculations. They exclude any downtime of the WAN, Internet or LAN; any third-party failure; or "force majeur" (such as natural disasters or war) when calculating the downtime number. Beyond that, there are also other exclusions, usually for scheduled maintenance. For example:

• Oracle/Siebel states in its System Availability Addendum to the On Demand User Agreement that it will try to do scheduled maintenance Friday 9 p.m. PDT/12 a.m. EST until 5 a.m. PDT/8 a.m. EST for U.S. customers, and during the appropriate nonbusiness hours in non-U.S. time zones. It also offers private infrastructure "pods," where customers can specify their own maintenance windows; however, that maintenance could fall out of the stated times, and no notice is required to be given. To make the SLA more meaningful, document it in the contract as an SLA.
• Salesforce.com states that it will try to do scheduled maintenance during the weekend hours — from 6 p.m. PST Friday to 3 a.m. PST Monday; but it also states that this is excluded from any uptime SLA promises, if it made any, as is any downtime event where it gives the organization at least eight-hours notice.
• RightNow excludes from its 99.5% uptime SLA anything where at least 24-hours notice has been given, but it does state that no more than 20 hours of scheduled maintenance can take place per year and that scheduled system maintenance will occur on the first Friday of each month.
• NetSuite is the most concrete, stating scheduled downtime will only take place on Friday and Saturday 10 p.m. to 3 a.m., and not on the first or last weekend of the month.

In reality, the vendors most often use less than the total time stipulated, but these are the contractual promises. In several cases, the time frames for downtime in standard contracts are based on Pacific Standard Time, which, unless negotiated, can be a problem for organizations in other time zones — particularly those with a 12-hour difference (primarily the Middle East, India and Pakistan).

The uptime service level required will depend on the criticality of the application; but for many CRM applications provided internally, most enterprises fall between 99.5% and 99.9% uptime, which translates to between 3.5 hours and 45 minutes down per month. Gartner considers 45 minutes per month (or nine hours down per year) as best-in-class availability. However, because of the above exclusions, the uptime SLA would have to be greater than 99.9% to be best in class, which is higher than we typically see, even in negotiated agreements, so push for 98% to 99.5% per month (see Note 2).

Ensure you have the ability to audit uptimes or get the vendor to provide an application to track uptimes for your environment. Document the recovery process that should be followed by the vendor and your organization. Also, ask if the vendor has a Web-based discussion forum available that enables users to ask peers if they are experiencing the same availability issues.

If downtime service levels are not met, then negotiate penalties and escalation clauses. If penalties are offered, they often take the form of credits, rather than refunds. Money back is preferable in terms of negotiating leverage and putting pressure on the SaaS provider:

• Salesforce.com does not offer any penalty clauses in standard contracts. We believe it has included some penalties in negotiated agreements that have documented SLAs. In these cases, salesforce.com will sometimes agree to credit for one full day of its subscription agreement for each full or partial percentage point (for example, at least 0.1%) below the service level during such month, applied to each active subscription. Salesforce.com has also, in some cases, agreed that if it fails to meet the SLA in two consecutive calendar months, then the customer can terminate its subscription agreement and get the fees back for the remainder of the contract (even if it's a multiyear contract).
• RightNow does not offer penalties, but it does offer a right to terminate the agreement and get back all prepaid fees relating to the unused portion of the agreement if it misses SLAs twice during any 12-month period, as part of the standard agreement.
• In its System Availability Addendum, Oracle's Siebel On Demand states that if there is sustained downtime for a full 15 minutes, and the downtime is below the agreed SLA for the period and you log a service request for a credit, then it will credit one-day's fees for each active subscription, up to a maximum of 10 days in a calendar quarter (10 days in a month for Gold customers). Termination with a prorated refund is only possible for Gold customers, and only if they meet the maximum credits in two consecutive months.
• NetSuite has the most favorable penalty clauses, stating that if it misses an uptime SLA at any point, and by any margin, then the customer gets all that month's fees back as a credit to future payments.

Recognize that penalties may not cover the business damage done. For example, if a sales force automation application is down for two hours, then the business impact may not be huge. However, if a high-transaction call center is down for two hours, then the penalties will unlikely cover the loss. Try to seek contractual protection against the loss of configured templates or reports in any maintenance or upgrade activity. Ensure that the wording around downtime is specific enough to enable you to collect penalties — for example, some vendors put the application into read-only mode, which severely diminishes usability, but still claim it to be "up," or claim that one of the modules is operating, even if the ones you use are not. Define performance SLAs as well as uptime SLAs to avoid suffering from impinging performance degradation that does not qualify as downtime. Negotiate the right for a refund of any prepaid fees if the CRM SaaS provider fails to meet the SLA two months in a row.

Some, but not all, CRM SaaS vendors charge incremental fees for the test, development and deployment environments (or sandboxes). In some cases, for smaller deployments, a single sandbox will be sufficient for all three of these roles; however, for larger deployments, most organizations typically want at least three sandboxes.

• Salesforce.com offers Developer Edition for free, which allows for testing and development with user and storage restrictions, as noted above, which may suffice in certain situations. However, a sandbox offers more because it provides an exact mirror of data, configurations and code running on the same production environment, and users can migrate changes from test to production environments with a single click. For this, salesforce.com charges a 20% premium on per-user fees for all users, unless customers sign up for the premium-priced Unlimited Edition (see Note 1).
• NetSuite charges from $40,000 to $100,000 per year for its sandboxing, depending on whether it is shared or has a dedicated database and hardware allocated to it. Its technology supports automated migration of changes from test to production environments.
• RightNow customers get a free sandbox site when they schedule an upgrade. The sandbox site is available for the duration of that upgrade. Outside of upgrades, there is a fee for sandboxing/staging sites. List price for an application test site starts at $208 per month. However, if a customer uses RightNow professional services for implementation, then they have their own sites for sandboxing/staging, which are used at no additional charge. The sandbox environment is a mirror of production with single-click upgradability, but customers sometimes create integrations or customizations that require some manual work to carry over to the new version.
• Oracle has the most favorable policy. Customers receive a staging environment with their production account at no additional charge, but this is not formally in the contract. Oracle's sandbox is a clone of the production environment, but migration of the changes from test to production must be done manually.

Organizations with more than 200 users should try to reduce sandboxing fees by leveraging more-favorable policies from competing vendors, or try to only pay for those users who are doing the sandboxing work. If an automated migration of changes from test to production environments is required, then ensure that it is provided in the service or contract.

Most CRM SaaS providers limit the amount of storage they grant with the standard monthly fee. For example:

• Oracle grants the least amount of storage, with 10MB per user, but it can be shared in a pool (that is, one user may use 15MB, while another only uses 5MB). If customers require additional storage, then the cost is $10 per extra 100MB charged per user per month. This is not stated contractually.
• Salesforce.com includes up to 1GB per site or 20MB per user, which can be pooled, in the subscription fee for Enterprise Edition. After that, customers must pay additional fees. Salesforce.com states that document storage is not included in this allowance, but this is not evident in its agreements. The other option for additional storage is to purchase Unlimited Edition, which includes up to 120MB per user.
• NetSuite includes 10GB of storage with the standard subscription fees per site. Additional storage is available for $250 per gigabyte per month.
• RightNow offers 5GB of storage free with each site. Additional storage is purchased in 5GB increments. List price for 5GB storage starts at $21 per month, but can be lower depending on the length of the contract.

Try to eliminate additional fees for storage on the basis that one of the value propositions of SaaS is that customers should not have to consider infrastructure requirements. If unable to eliminate such fees, and/or if you anticipate significant increases in storage requirements (particularly with unstructured data), then lock in discounted fees in advance. For longer-term contracts, it is advisable to tie the price paid for storage to the cost of storage with some form of cost plus model, because disk prices have notoriously fast price declines (see "Managed Storage Pricing and Practices Revealed: 2Q06 Update").

The CRM SaaS providers listed in this research provide updates and upgrades as part of the monthly fee; however, ensure that this is documented contractually. The support entitlements (for example, response time, access to phone support or number of permitted contacts) that are bundled into the standard fee vary:

• In the standard support embedded in the subscription fees, neither RightNow nor NetSuite provides any telephone support, just e-mail support for issue and enhancement reporting and access to a knowledge base and support portal 24/7. RightNow includes an "annual tune up," and NetSuite includes free training clinics.
• Salesforce.com promises only 12/5 support. However, given its centers in the U.S., Ireland and Australia, it can deliver 24/5 support. It also offers an unlimited amount of incidents in the basic offering, but with only a two-day response time. Otherwise, customers can use online self-help/customer portal.
• Siebel CRM On Demand includes phone support from 6 a.m. to 6 p.m. Monday through Friday local time, with a response time of one business day. Online access is also included 24/7.

Given the basic support in the standard offering, premium support is necessary for many customers, but it can be expensive. For example:

• For an additional 15% of list subscription fees, salesforce.com will provide Premier Support services. In addition to the basic support, this includes 24/7 phone support (toll free in the U.S.), a priority phone queue, a two-hour response time, an assigned representative and an annual "health check."
• RightNow's Preferred Support package costs 18% of subscription fees and offers live 24/7 phone support for Severity-1 and Severity-2 incidents via a toll-free number, with a 30-minute initial response time on Severity-1 issues, and a two-hour resolution time SLA. It also offers a designated support contact, biannual tune ups and an annual success scorecard review. A Premier offering is priced at 25% of fees for customers who additionally want a 15-minute initial response time on Severity-1 issues, a one-hour resolution time SLA, management escalation and a biannual business review of metrics, and an assessment of success criteria.
• NetSuite provides telephone and e-mail support during normal business hours, and weekend e-mail support with a two-day response time for an unlimited number of cases via e-mail for 22.5% of net license fees. Response times on the phone are not specified, but integration support is also included. For 32.5% of the fee, NetSuite offers Gold Support, which, in addition, provides global 24/7 e-mail and telephone support (via a toll-free line). E-mail cases are aimed to be resolved within one business day. For 37.5% of fees, customers can also get a dedicated customer support representative during normal business hours.
• Siebel CRM On Demand's standard support is bundled into the $70 per user per month Gold support package. It is $15 per user per month for fewer than 50 named users, and $10 per user per month for 50 or more named user. The Gold support package provides 24/7 phone support (and may include a pager service during off-peak hours), but with a four-hour response time.

Understand what is in the basic offering, and document it in the contract with a clause that states that these services cannot be diminished for the money paid. If you have leverage and only need one component of the premium maintenance offering, then try to negotiate that into the standard price. If you take premium support offerings, then negotiate the right to cancel or switch from one program to another (for example, basic to premium and back) during the contract term.

CRM SaaS vendors have contingency plans and disaster recovery arrangements. They have their primary data centers in California, Montana and Texas. Some also have their data centers in different countries. None have contractual stipulations about how often they test these disaster recovery facilities, or how often operations will be switched. They also have backup and recovery capabilities for recovering data in nondisaster situations, but not all of them document these as contractual obligations.

• Salesforce.com states that it has multiple levels of backup and fully redundant hosting facilities on the West Coast and East Coast, but has no specific contractual obligations for backup and recovery.
• RightNow states that hosted data shall be backed up at least once in each 24-hour period.
• NetSuite is the most explicit in its contracts, stating that it will do daily backups to tape library taken to secure off-site locations 100 miles away.
• Oracle has stated that it performs automated backups daily and weekly on the full database containing all customer and related data for the service. Incremental database backups are written to disk daily and copied to tape twice a week. An integrity validation check is performed on each backup. For an additional customized fee, Oracle offers a separate disaster recovery service. This service provides a mirror of an individual customer's applications to Oracle's Rocky Mountain Data Center, and has a recovery point objective of one hour, and a recovery time objective of four hours.

Ensure that your SaaS provider has arrangements for disaster recovery, conducts regular testing and that the capabilities of its recovery site meets your requirements. Ensure you can export your data as frequently as you like, rather than a limited number of times or at specific times. Ensure that there will be no additional cost to process at other sites if a disaster occurs and the primary location is shut down. This cost should be borne by the SaaS vendor as part of the monthly fee.