You are currently browsing Ventana Research’s articles.
The market for big data continues to grow as organizations try to extract business value from their own masses of data and other sources. Earlier this year I outlined the dynamics of the business opportunity for big data and information optimization. We continue to see advances as big data and associated information technologies deliver more value, but the range of innovation also has created fragmentation among existing systems including databases that are managed onpremises or in cloud computing environments. In this changing environment organizations encounter new challenges not only in adapting to technology that is more efficient in automating data processing but also in integrating it into their enterprise architecture. I’ve already explained how big data can be ineffective without integration, and we conducted more in-depth research into the market, resulting in our benchmark research on big data integration, which reveals the state of how organizations are adopting this technology in their processes.
The research shows that use of big data techniques has become widespread: Almost half (48%) of all organizations participating in this research and two-thirds of the very large ones use it for storage, and 45 percent intend to use big data in the next year or sometime in the future. This is a significant change in that most organizations have used relational database management systems (RDBMSs) for nearly everything. We find that RDBMSs (76%) are still the most widely used big data technology, followed by flat files (61%) and data warehouse appliances (46%). But this is not the direction many companies are planning to take in the future: Hadoop (44%), in-memory database (46%), specialized databases (43%) and NoSQL (42%) are the tools most often planned to be used by 2016 or being evaluated. Clearly there is a revolution in approaches to storing and using data, and that introduces both opportunities and challenges.
Establishing a big data environment requires integrating data through proper preparation and potentially continuous updates of data, whether in real time or batch processing. A further complication is that many organizations will not have only one but several big data environments to be integrated into the overall enterprise architecture; that requires data and systems integration. Our research finds that some organizations are aware of this issue: Automating big data integration is very important to 45 percent and important to more than one-third. Automation can not only bring efficiency to big data but also remove many risks of errors or inaccurate and inconsistent data.
Data integration technologies have evolved over the past decade, but advances to support big data are more recent. Our research shows a disparity in how well organizations handle big data integration tasks. Those that are mostly or completely adequate are accessing (for 63%), loading (60%), extracting (59%), archiving (55%) and copying (52%) data while the areas most in need of improvement are virtualizing (39%), profiling (37%), blending (34%), master data management (33%) and masking for privacy (33%). At the system level, the research finds that conventional enterprise capabilities are most often needed: load balancing (cited by 51%), cross-platform support (47%), a development and testing environment (42%), systems management (40%) and scalable execution of tasks (39%). To test the range of big data integration capabilities before it is applied to production projects, the “sandbox” has become the standard approach. For their development and testing environment, the largest percentage (36%) said they will use an internal sandbox with specialized big data. This group of findings reveals that big data integration has enterprise-level requirements that go beyond just loading data to build on advances in data integration.
Big data must not be a separate store of data but part of the overall enterprise and data architecture; that is necessary to ensure full integration and use of the data. Organizations that see data integration as critical to big data are embarking on sophisticated efforts to achieve it. The data integration capabilities most critical to their big data efforts are to develop and manage metadata that can be shared across BI systems (cited by 58%), to join disparate data sources during transformation (56%) and to establish rules for processing and routing data (56%).
Other organizations are still examining how to automate integration tasks. The most common barriers to improving big data integration are cost of the software or license (for 44%), lack of resources to use on improvement (37%) and the sense that big data technologies are too complicated to integrate (35%). These findings demonstrate that many organizations need to better understand the efficiency and cost savings that can be realized by using purpose-built technology instead of manual approaches using tools not designed for big data. Along with identifying solid business benefits, establishing savings of time and money are essential pieces of a convincing rationale for investment in big data integration technology. The most time spent in big data integration today is on basic tasks: reviewing data for quality and consistency (52%), preparing data for integration (46%) and connecting to data sources for integration (39%). The first two are related to ensuring that data is ready to load into big data environments. Data preparation is a key part of big data and overall information optimization. More vendors are developing dedicated technology to help with it.
For a process as complex as big data integration, choosing the right technology tool can be difficult. More than half (55%) of organizations are planning to change the way they assess and select such technology. Evaluations of big data integration tools should include considerations of how to deploy it and what sort of vendors can provide it. Almost half (46%) of organizations prefer to integrate big data on-premises while 28 percent opt for cloud-based software as a service and 17 percent have no preference. Half of organizations plan to use cloud computing for managing big data; another one-third (32%) don’t know whether they will. The research shows that the most important technology and vendor criteria used to evaluate big data integration technology are usability (very important for 53%), reliability (52%) and functionality (49%). These top three evaluation criteria are followed by manageability, TCO/ROI, adaptability and validation of vendors. Organizations are most concerned to have technology that is easy to use and can scale to meet their needs.
Big data cannot be used effectively without integration; we observe that the big data industry has not paid as much attention to information management as it should – after all, this is what enables automating the flow of data. Organizations trying to use big data without a focus on information management will have difficulty in optimizing the use of their data assets for business needs. Our research into big data integration finds that the proper technology is critical to meet these needs. We also learned from our benchmark research into big data analytics that data preparation is the largest and most time-consuming set of tasks that needs to be streamlined for best use of the analytics that reveal actionable insights. Organizations that are initiating or expanding their big data deployments whether onpremises or within cloud computing environments should have integration at the top of their priority list to ensure they do not create silos of data that they can’t fully exploit.
CEO and Chief Research Officer
At this year’s Dreamforce more than 140,000 people gathered in San Francisco to share the excitement about the use of technology for business. Salesforce.com’s annual conference has reached megashow status, which is a mixed blessing: Dreamforce remains social in its design, but it has become impersonal due to its size. In any case Salesforce had plenty to show off. The company has continued to enhance its cloud-based business applications for sales and customer service, and in the last year it has added marketing through acquisitions. It also has advanced the attraction of its cloud computing platform; even IT departments see its approach as a simple way to use and build applications, especially mobile ones which the ubiquity of smartphones and tablets have made critical to business. Cloud computing is becoming the defacto approach for new applications and software for business and now IT, and its importance continues to grow: Our benchmark research on business technology innovation shows that it is important or very important to more than half (57%) of organizations. At Dreamforce, Salesforce announced Salesforce1 Lightning (available in 2015), a way to assemble mobile applications that can operate across platforms. Salesforce makes the technical details of the mobile platform transparent and facilitates assembly of mobile applications.
Probably the largest announcement was of Wave technology, that is part of the Salesforce Analytics Cloud, which originated in its acquisition of EdgeSpring in 2013. It fills one of the holes in Salesforce’s product portfolio. For years the company has tried to provide in its SFA product reports and dashboards but has not been able to keep pace with the technological advances in analytics. Wave uses analytics to create metrics in dashboards that are easy to use and interact with on the Web and through mobile devices. Initially Salesforce will support the Apple iPhone and iPad, with Android to come later in 2015.
To address the potential weakness of not supporting data outside of its own applications, Salesforce pitched its partnerships with Dell Boomi, Informatica, MuleSoft and Talend that we cover in our research, all of which help move data from other systems and applications in the cloud and on-premises. Other partners like SnapLogic who have taken a pure cloud computing approach were not referenced but can play a critical role in supporting Salesforce Analytics Cloud. This collaboration will make Salesforce analytics more robust and able to access data anywhere including the Internet. This focus on integration is critical because in the past Salesforce has left it to partners while taking the position that customers should make Salesforce their operating environment. But the next generation of analytics requires access to big data, which is becoming widely dispersed and should be integrated to derive full value from investments; Salesforce has not referenced this need in any substantive manner, but it ought to be more inclusive of other big data technologies. Our research in big data analytics finds that half of organizations are already engaged into this activity.
At the conference Salesforce executives emphasized that its differentiator is its platform and its own database technology powering its analytics. I am not as convinced and because none of the current analytics and BI partners are operating on its platform, this claim from my analysis is more hopeful marketing than reality. The push to position its platform as the differentiator is meant to support Salesforce’s effort to be seen as a key provider and to deflect notice of any tool limitations in the first release of Wave. I do not see the latter as an issue as it has many positives that are more important to business users than the platform. The first is the user experience of interactions with the dashboards and the elegant presentation of charts that materialize in a wavelike manner. Having differentiation in the usability of its products is a strong position; that is the top software evaluation criteria for organizations today according to all of our benchmark research.
Another potential differentiator for Wave is its interoperability across mobile devices. As noted Salesforce initially is focused on Apple but will expand to Android and hopefully to Microsoft Windows Phone and the Surface tablet, which slowly are gaining adoption as organizations update from legacy notebooks running older versions of Windows. Salesforce has designed the native mobile user experience to adapt to the design of devices so users do not have to resize windows; it also is good at displaying the context and attributes of what is being shown. It also enables synchronization across devices to enable collaboration for coaching and interacting within the organization. Its annotation capabilities help supply context on issues or opportunities that can be shared by users. Our latest Value Index on mobile BI providers found many analytics and BI providers have lagged in full support of mobile platforms, often just publishing to Web browsers or just supporting Apple, which is ineffective in business and frustrating to users. In our next-generation business intelligence research usability is an important purchase consideration for the most (63%) organizations. Mobility is essential to the future of analytics, and Salesforce is smart to emphasize its work in this area to support analytics along with the rest of its applications.
With Wave Salesforce is not disrupting the analytics market but moving into the existing market of providers in the cloud. Some have offerings across the lines of business while others specialize in areas such as sales analytics. Many of these other providers are current partners of Salesforce including Birst, Domo, GoodData, MicroStrategy, Qlik and Tableau Software; they are now direct competitors that focus on the same audience of buyers and users in the lines of business. While Salesforce and its partners tried to downplay the competition, this is just deflecting from the reality of the situation. I predict that these partners will see that Salesforce will make it harder to close deals as they have in the past with customers in the lines of business, which have been the major growth area for analytics deployments featuring dashboards and visualization. On the other hand, partners like FinancialForce.com and others that use Force.com as their application platform will find Wave can improve the robustness of their applications through embedded analytics.
The Analytics Cloud should help Salesforce in customer service and marketing, for which until now it did not offer analytics or access to metrics. It will especially help in sales force automation, whose existing dashboards and reports are more than challenging to use. Potential customers will have to decide how important it is to have state-of-the-art dashboards. For reporting Salesforce Analytics Cloud is not going to solve the needs of organizations as it is limited in tabular presentation and formatting of data that most know and use in reporting, and anyway reporting is free in its existing Salesforce Sales Cloud. Merely addressing the need of sales organizations for dashboards and reviewing information on past performance is not certain to meet other needs to manage and plan quotas, territories and compensation, forecasting, quota and other areas for which it is not designed. Salesforce has its Analytics Cloud Excel Connector to provide any flexibility and address limitations in its current product through providing access from spreadsheets. There is a need for this, as the majority (59%) of organizations in our research on sales forecasting said that spreadsheets undermine efficiency.
Salesforce1 Lightning will exploit further the potential of assembling custom applications that use components of Salesforce Analytics Cloud. Helping organizations and partners build analytic apps is likely to be significant for the company’s future. Salesforce wisely is encouraging people to work with it as doing so quickly demonstrates its differentiation in the user experience and mobile support, and I spent some time at Dreamforce reviewing Wave and trying it on my mobile device. Salesforce also made sure its roadmap in 2015 was open to everyone at Dreamforce. Its plans include support for mapping as visualization or storytelling; the ability to record and play back so others can see and learn; Android support, real-time collaboration and offline support. What was not clear was the roadmap to improve or replace dashboards that are currently part of its Sales Cloud offering that are less than intuitive for sales but are used by many organizations. Salesforce is not being secretive like many vendors; being late to enter the analytics market, it has to build trust and confidence in its customers.
As well as its pluses, Salesforce Analytics Cloud has negatives that will prevent it from being the only tool needed for analytics; again, it is designed for assembling and deploying dashboards and has elegant selection methods. Analysts and operations personnel who perform data discovery and exploration and want forward-looking forecasting and planning or predictive analytics will need a separate tool. Therefore organizations will have to budget their allocations for the needs of various roles. Salesforce has set a premium price for Wave: $125 per user per month for those that consume and interact with it and $250 per user per month for the analyst and administrator license. Salesforce is likely to gain adoption among Global 2000 customers that can afford the price point, but small and midsize organizations will find it challenging: The price is twice as expensive as Salesforce’s basic SFA offering, which is $65 per user per month. Comparatively its major applications competitor Oracle embeds the price of analytics in its Sales Cloud at $100 per user per month. Salesforce’s analytics and SFA combined will cost at least $190 per user per month. Whether it is worth this steep price will require a thorough assessment by organizations that do not have unlimited budgets for technology.
Salesforce Analytics Cloud is a good first step that will enable Salesforce to be taken seriously as an analytics provider. Our research finds that analytics is the top technology innovation priority, called important by 39 percent of organizations. Salesforce Analytics Cloud will gain attention for its sophisticated dashboards, but it is likely to complement established analytics products rather than replace them. Potential users excited by the new product and its engaging marketing should keep this context in mind. This reminds me of the advent of sales force automation, which helps in automating the recording and review of accounts, contacts and opportunities during the sales process but does not automate or manage the sales force itself. Many sales executives excited about SFA did not allocate sufficient budget and resources to applications that focus on other essential aspects of sales, such as managing coaching, compensation, quotas and territories. Organizations looking for interactive dashboards should examine Salesforce Analytics Cloud closely. While the price is high, it could eliminate the pain of using less intuitive approaches. Organizations also can examine alternatives from partners that are integrated with Salesforce today and are less expensive to deploy and use.
Organizations using Salesforce applications for sales, marketing or customer service should evaluate Salesforce Analytics Cloud. Its elegance and interactivity in dashboards for both the Web and mobile devices are ahead of many competitors. Its sophisticated dashboards will attract organizations that want to present metrics more intuitively to business professionals. They should balance the price against the capabilities and determine whether to take this step toward being an analytics-driven organization. Salesforce Wave will impact the analytics market now and in the future, changing what businesses will expect from providers.
CEO and Chief Research Officer