You are currently browsing the tag archive for the ‘CIO’ tag.
Big data has great promise for many organizations today, but they also need technology to facilitate integration of various data stores, as I recently pointed out. Our big data integration benchmark research makes it clear that organizations are aware of the need to integrate big data, but most have yet to address it: In this area our Performance Index analysis, which assesses competency and maturity of organizations, concludes that only 13 percent reach the highest of four levels, Innovative. Furthermore, while many organizations are sophisticated in dealing with the information, they are less able to handle the people-related areas, lacking the right level of training in the skills required to integrate big data. Most said that the training they provide is only somewhat adequate or inadequate.
Big data is still new to many organizations, and they face challenges in integrating big data that prevent them from gaining full value from their existing and potential investments. Our research finds that many lack confidence in processing large volumes of data. More than half (55%) of organizations characterized themselves as only somewhat confident or not confident in their ability to accomplish that task. They have even less confidence in their ability to process data that arrives at high velocity: Only 29 percent said they are somewhat confident or not confident in that. In dealing with the variety of big data, confidence is somewhat stronger, as more than half (56%) declared themselves confident or very confident. Assurance in one aspect is often found in others: 86 percent of organizations that said they are very confident in their ability to integrate the variety of big data are satisfied with how they manage the storage of big data. Similarly 91 percent of those that are confident or very confident with their data quality are satisfied with the way they manage the storage of big data.
Turning to the technology being used, we find only one-third (32%) of organizations satisfied with their current data integration technology, but twice as many (66%) are satisfied with their data integration processes for loading and creating big data. A substantial majority (86%) of those very confident in their ability to integrate the needed variety of big data are satisfied with their existing data integration processes. Those that are not satisfied said the process is too slow (61%), analytics are hard to build and maintain (50%) and data is not readily available (39%). These findings indicate that making a commitment to data integration, for big data and otherwise, can pay off in confidence and satisfaction with the processes for doing it. Additionally, organizations that use dedicated data integration technology (86%) are satisfied much more often than those that don’t use dedicated technology (52%).
New types of big data technologies are being introduced to meet expanding demand for storage and use of information across the enterprise. One of those fast-growing technologies is the open source Apache Hadoop and commercial enterprise versions of it that provide a distributed file system to manage large volumes of data. The research finds that currently 28 percent of organizations use Hadoop and about as many more (25%) plan to use it in the next two years. Nearly half (47%) have Hadoop-specific skills to support big data integration. For those that have limited resources, open source Hadoop can be affordable, and to automate and interface with it, adopters can use SQL in addition to its native interfaces; about three in five organizations now use each of these options. Hadoop can be a capable tool to implement big data but must be integrated with other information and operational systems.
Big data is not found only in conventional in-house information environments. Our research finds that data integration processes are most often applied between systems deployed on-premises (58%), but more than one-third (35%) are integrating cloud-based systems, which reflects the progress cloud computing has made. Nonetheless, cloud-to-cloud integration remains least common (18%). In the next year or two 20 to 25 percent of organizations plan additional support for all types of integration; those being considered most often are cloud-to-cloud (25%) and on-premises-to-cloud (23%), further reflecting movement into the cloud. In addition, nearly all (95%) organizations using cloud-to-cloud integration said they have improved their activities and processes. This finding confirms the value of integration of big data regardless of what types of systems hold it. With a growing number of organizations using cloud computing, data integration is a critical requirement for big data projects; more than one-quarter (28%) of organizations are deploying big data integration into cloud computing environments.
Because of the intense need of business units and process for big data, integration requires IT and business people to work together to build efficient processes. The largest percentage of organizations in the research (44%) have business analysts work with IT to design and deploy big data integration. Another one-third assign IT to build the integration, and half that many (16%) have IT use a dedicated data integration tool. The research finds some distrust in involving the business side. Almost one in four (23%) said they are resistant or very resistant to allowing business users to integrate big data that IT has not prepared first, and the majority (51%) resist somewhat. For more than half (58%) the IT group responsible for BI and data warehouse systems also is the key stakeholder for designing and deploying big data integration; no other option is used by more than 11 percent.
It is not surprising that IT is the department that most often facilitates big data and needs integration the most (55%). The most frequent issue arising between business units and IT is entrenchment of budgets and priorities (in 42% of organizations). Funding of big data initiatives most often comes from the general IT budget (50%); line-of-business IT budgets (38%) are the second-most commonly used. It is understandable that IT dominates this heavily technical function, but big data is beneficial only when it advances the organization’s goals for information that is needed by business. Management should ensure that IT works with the lines of business to enable them to get the information they need to improve business processes and decision-making and not settle for creating a more cost-effective and efficient method to store it.
Overcoming these challenges is a critical step in the planning process for big data. My analysis that big data won’t work well without integration is confirmed by the research. We urge organizations to take a comprehensive approach to big data and evaluate dedicated tools that can mitigate risks that others have already encountered.
CEO and Chief Research Officer
In recent years line-of-business applications including accounting, human resources, manufacturing, sales and customer service have appeared in the cloud. Cloud -based software as a service (SaaS) has replaced on-premises applications that were previously part of ERP and CRM environments. They have helped companies become more efficient but have also introduced interoperability challenges between business processes. Their advantage is that cloud software can be rented, configured and used within a day or week. The disadvantage is that they don’t always connect with one another seamlessly, as they used to and when managed by a third party there is limited connectivity to integrate them.
Smooth interoperation is critical for business processes that use ERP. The hybrid computing approach to ERP was assessed by my colleague Robert Kugel, who identified the challenges in these early approaches to ERP in the cloud. As on-premises ERP, these applications were fully connected and integrated with process and data integration wired together through additional technologies. This configurability of ERP has been a large challenge and has led to failed implementations, as Robert noted. Understanding the complexities of ERP today is key to coming up with a solution.
One of the most fundamental business processes in organizations is tracking an order to fulfillment. That needs to be connected without manual intervention for transactional efficiency and to ensure data, analytics and planning are available. In fact our next generation of finance analytics benchmark research finds that ERP is the second-most important source of data being analyzed, after spreadsheets, and is the first-ranked source in almost one-third (32%) of organizations. In the transition to cloud computing this has become more complex. Applications that support the order-to-fulfillment process, for example, are increasingly being used individually in cloud computing, and many are being mingled with on-premises applications that require integration and automation to avoid manual intervention.
To start, orders are created by sales. Sales opportunities are created and closed with products and services purchased by contract or electronically. These must be placed in an order management system through which the record can be used by a multitude of systems and processes. Purchasing and fulfillment details are moved into an invoice as part of a billing application, then entered into accounting and accounts receivable where it is managed to payment. Finance takes orders and consolidates them into reports, typically in a financial performance management system. The order is then provided to manufacturing or fulfillment applications which build and deliver products, then fulfilled through warehouse and distribution management. The customer name and order number is also placed into an application that handles support calls or deploys services.
This is the reality of business today. Many departments and individuals, with different applications, are needed to support orders, fulfillment and service. If these applications are not connected organizations have to perform manual intervention, re-enter data, or copy and paste, which not only wastes time and resources but can introduce errors. Half (56%) of organizations do the integration through spreadsheets or exporting data, with custom coding being second-most popular (for 39%), according to our business data in the cloud research. (I will leave HR applications and the supporting people component out of this process though they play a critical role as an enterprise resource to support the order-to-fulfillment process and are part of many ERP deployments.) In addition performing any level of business planning is not simple as data is needed to determine past performance and plan for the future.
There is no simple way to make all this efficient. It has historically been managed by ERP, usually through a single vendor’s application suite. Now businesses want it done in the cloud, either on-premises or through other cloud applications. Proper attention must be paid to the needs and competencies of the departments and business processes involved. Thus migration and transition to ERP are not simple, nor is building an application architecture that supports process and data efficiently. Assessment and planning are necessary to ensure that risks are addressed. Switching to a new ERP system in the cloud will still require an application architecture to maintain proper operations across departments and existing applications, whether they are on-premises, in a private cloud, in the public cloud or in a hybrid combination. This integration among sales force automation, customer service and other systems is outside the scope of most cloud ERP deployments who have not provided the most robust integration points for the applications and data.
Robert Kugel writes that ERP must take a giant leap in order to operate in the cloud. I agree. Our firm often gets requests for assistance in finding the right approach to ERP and business process. While midsize companies find ERP in the cloud increasingly attractive, there are significant challenges to adapting and integrating such applications as part of business processes, which many customers overlook in their desire for a cloud-based solution. The majority of cloud ERP vendors have not provided integration and workflow of information from their applications to others in an open and seamless manner, complicating deployments and adding unexpected costs for customers.
ERP suppliers moving from on-premises to cloud computing have acknowledged the complexities. Many of the legacy ERP vendors have struggled to enable application interoperability and the differing management requirements of IT and business. This struggle has resulted in a lack of confidence by organizations wishing to migrate to ERP in the cloud and makes them wonder whether to look at alternative approaches with individual applications using integration across them and data to support business processes. Automation is another major concern. The lack of it across business processes has impeded finance groups in closing the books efficiently; our research on the fast, clean close shows many organizations losing four or more days.
In the meantime technological advances in integration technologies that operate in the cloud computing environment can interconnect with on-premises systems. The ability to simultaneously distribute transactions and data across systems makes the ability to architect business processes and workflows a reality. For some organizations the use of business process management and other integration technology approaches are being adopted. The new technologies are able to blend into many applications, so that users do not know they are working on applications from different vendors, nor do they need to know. These advances enable application architecture to interoperate and automate the flow of data from on-premises and cloud computing environments, providing new opportunities to interoperate from ERP or other applications. Many organizations are doing this today, and more than one-third of companies are planning or need to move data from cloud to cloud, cloud to on-premises or on-premises to cloud, data according our business data in the cloud research.
No matter whether a company is in manufacturing or services, they must address the integration complexities to gain efficiency and support growth without adding more resources. While many new ERP providers in the cloud are taking simpler approaches from the applications interface to how it processes transactions, most have not learned the importance of integration to other systems and the need for accessing and integrating data for transactions and analytics. Having consistency of data across applications and systems is a major obstacle in more than one-fifth (21%) of organizations and a minor obstacle in two-fifths (43%), and they find similar difficulties in the complexities of accessing data, according to our business data in the cloud research. In addition they lack effective planning (which of course is the P in ERP), and reporting is less than sufficient and often must be complemented by third-party tools.
All this introduces yet more complexity for business and IT in determining how they can move forward with ERP in the cloud, adapting existing and new applications to interoperate. The outlook for ERP in the cloud is thus uncertain. If these vendors do not adapt to the reality of what their customers want (as opposed to what they want their customers to do), it will remain cloudy. Responding to pressure to take an open approach that is easy to integrate, however, ERP providers could see a sunny forecast.
CEO & Chief Research Officer