You are currently browsing Mark Smith’s articles.
Big data has become a big deal as the technology industry has invested tens of billions of dollars to create the next generation of databases and data processing. After the accompanying flood of new categories and marketing terminology from vendors, most in the IT community are now beginning to understand the potential of big data. Ventana Research thoroughly covered the evolving state of the big data and information optimization sector in 2014 and will continue this research in 2015 and beyond. As it progresses the importance of making big data systems interoperate with existing enterprise and information architecture along with digital transformation strategiesbecomes critical. Done properly companies can take advantage of big data innovations to optimize their established business processes and execute new business strategies. But just deploying big data and applying analytics to understand it is just the beginning. Innovative organizations must go beyond the usual exploratory and root-cause analyses through applied analytic discovery and other techniques. This of course requires them to develop competencies in information management for big data.
Among big data technologies, the open source Hadoop has been commercialized by now established providers including Cloudera, Hortonworks and MapR and made available in the cloud through platforms such as Qubole, which received a Ventana Research Technology Innovation Award in 2014. Other big data technologies are growing as well; for example, use of in-memory and specialized databases also is growing like Hadoop in more than 40 percent of organizations, according to our big data integration benchmark research. These technologies have been integrated into databases or what I call hybrid big data appliances like those from IBM, Oracle, SAP and Teradata that bring the power of Hadoop to the RDBMS and exploit in-memory processing to perform ever faster computing. When placed into hosted and cloud environments these appliances can virtualize big data processing. Another new provider, Splice Machine, brings the power of SQL processing in a scalable approach that uses Hadoop in a cloud-based approach; it received a Ventana Research Technology Leadership Award last year. Likewise advances in NoSQL approaches help organizations process and utilize semistructured information along with other information and blend them with analytics as Datawatch does. These examples show that disruptive technologies still have the potential to revolutionize our approaches to managing information.
Our firm also explores what we call information optimization, which assesses techniques for gaining full value from business information. Big data is one of these when used effectively in an enterprise information architecture. In this context the “data lake” analogy is not helpful in representing the full scope of big data, suggesting simply a container like a data marts or data warehouse. With big data, taking an architectural approach is critical. This viewpoint is evident in our 2014 Ventana Research Technology Innovation Award in Information Management to Teradata for its Unified Data Architecture. Another award winner, Software AG, blends big data and information optimization using its real-time and in-memory processing technologies.
Businesses need to process data in rapid cycles, many in real time and what we call operational intelligence, which utilizes events and streams and provides the ability to sense and respond immediately to issues and opportunities in organizations that adapt to a data-driven culture. Our operational intelligence research finds that monitoring, alerting and notification are the top use cases for deployment, in more than half of organizations. Also machine data can help businesses optimize not just IT processes but business processes that help govern and control the security of data in the enterprise. This imperative is evident in the dramatic growth of suppliers such as Splunk, Sumo Logic and Savi Technology, all of which won Ventana Research Technology Innovation awards for how they process machine and business data in large volumes at rapid velocity.
Another increasing trend in big data is presenting it in ways that ordinary users can understand quickly. Discovery and advanced visualization is not enough for business users who are not trained to interpret these presentations. Some vendors can present locationand geospatial data on maps that are easier to understand. At the other end of the user spectrum data scientists and analysts need more robust analytic and discovery tools, including predictive analytics, which is a priority for many organizations, according toour big data analytics research. In 2015 we will examine the next generation of predictive analytics in new benchmark research. But there is more work to do to present insights from information that are easy to understand. Some analytics vendors are telling stories by linking pages of content, but these narratives don’t as yet help individuals assess and act. Most analytics tools can’t match the simple functionality of Microsoft PowerPoint, placing descriptive titles, bullets and recommendations on a page with a graphic that represents something important to these business professional who reads it. Deeper insights may come from advances in machine learning and cognitive computing that have arrived on the market and bring more science to analytics.
So we strong potential for the outputs of big data, but they don’t arrive just by loading data into these new computing environments. Pragmatic and experienced professionals realize that information management processes do not disappear. A key one in this area is data preparation, which helps ready data sets for processing into big data environments. Preparing data is the second-most important task for 46 percent of organizations in our big data integration research. A second is data integration, which some new tools can automate. This can enable lines of business and IT to work together on big data integration, as 41 percent of organizations in our research are planning to do. To address this need a new generation of technologies came into their own in 2014 including those that received Ventana Research Technology Innovation Awards like Paxata and Tamr but also Trifacta.
Yet another area to watch is the convergence of big data and cloud computing. The proliferation of data sources in the cloud forces organizations to managed and integrate data from a variety of cloud and Internet sources, hence the rise of information as a service for business needs. Ventana Research Technology Innovation Award winner DataSift provides information as a service to blend social media data with other big data and analytics. Such techniques require more flexible environments for integration that can operate anywhere at any time. Dell Boomi, MuleSoft, SnapLogic and others now challenge established data integration providers such as Informatica and others including IBM, Oracle and SAP. Advances in master data management, data governance, data quality and integration backbones, and Informatica and Information Builders help provide better consistency of any type of big data for any business purpose. In addition our research finds that data security is critical for big data in 61 percent of organizations; only 14 percent said that is very adequate in their organization.
There is no doubt that big data is now widespread; almost 80 percent of organizations in our information optimization research, for example, will be using it some form by the end of 2015. This is partly due to increased use across the lines of business; our research on next-generation customer analytics in 2014 shows that it is important to improving understanding customers in 60 percent of organizations, is being used in one-fifth of organizations and will be in 46 percent by the end of this year. Similarly our next-generation finance analytics research in 2014 finds big data important to 37 percent of organizations, with 13 percent using it today and 42 percent planning to by the end of 2015. And we have already measured how it will impact human capital management and HR and where organizations are leveraging it in this area of importance.
I invite you to download and peruse our big data agenda for 2015. We will examine how organizations can instrument information optimization processes that use big data and pass this guidance along. We will explore big data’s role in sales and product areas and produce new research on data and analytics in the cloud. Our research will uncover best practices that innovative organizations use not only to prepare and integrate big data but also more tightly unify it with analytics and operations across enterprise and cloud computing environments. For many organizations taking on this challenge and seeking its benefits will require new information platforms and methods to access and provide information as part of their big data deployments. (Getting consistent information across the enterprise is the top benefit of big data integration according to 39 percent of organizations.) We expect 2015 to be a big year for big data and information optimization. I look forward to providing more insights and information about big data and helping everyone get the most from their time and investments in it.
CEO and Chief Research Officer
Managing investments in people and their performance is critical to every organization. It also is complicated. To support the various aspects of human capital management (HCM), organizations often use a variety of technology including systems for human resource management, talent management, workforce management and payroll management. Often these separate systems use their own information and are not well connected to each other. Today they are deployed both on-premises and in cloud computing environments, which further complicates integration. This situation disrupts processes and challenges HR departments and leaders to invest time and resources to correct it.
Unifying applications across HCM is a critical concern that organizations are addressing. One intersection point is between core HR systems and payroll management; as well, payroll management must connect to talent management for salaried employees and workforce management for hourly earners. Our benchmark research finds that time and attendance (51%) and compensation (45%) as the two areas that most organizations need to integrate with payroll management. Application and data integration can support a range of key activities such as recruiting, onboarding, performance management, learning management and compensation management, all of which are crucial to motivate, develop and engage employees. It is essential to integration to have a single, consistent source of employee data that all these systems can draw upon; an employee master can provide a reliable source of information and simplify interactions of managers and employees and facilitate collaboration among workers. Many organizations are examining ways to create a consistent platform on which human capital management applications can interoperate.
Integration of systems will be a major focus of our research agenda for human capital management in 2015. We will closely examine the current state of core HR management, talent management and workforce management systems with an eye on possibilities for them to work together smoothly. Managers in HR departments and the lines of business realize the importance of providing the best possible employee experience in attracting and retaining all types of talent; to achieve this they must effectively use technology for all the people aspects of their organization. Developing a strategy and a plan to unify HCM systems should be the first step. To succeed this will require buy-in from executives and management across finance and operations. Our research this year will strive to identify best practices in making technology a more valuable asset for managing the most important component of any business – its people.
Our 2015 agenda will leverage two recent benchmark research projects, payroll management and next-generation learning management. In addition, soon we will have new insights from benchmark research onnext-generation human resources management systems, and we will explore the expansion of talent management platforms and applications. As usual we will continue the in-depth product and vendor evaluations of ourValue Index; in 2014 we released Value Indexes on compensation management and payroll management systems, and this year we will expand to assess learning management and HR management systems.
Examining new methods to recruit and retain employees will become critical as competing for available talent becomes more difficult, and our research in 2015 will explore this area. Being efficient at recruiting has become a top priority for organizations. To achieve that goal will require a more comprehensive approach than most take. It starts with marketing positions and moves on to how candidates are assessed and interviewed; both increasingly rely on technologies, now including video. Later on organizations seeking to enhance their employees’ skills and potential should consider using learning management as a platform not just for training but also to improve their onboarding, performance and employee development processes, and we will continue to focus on this area. In addition we’ll examine new ways to address the work/life balance, a major concern for today’s workers, including using wearable computing devices for wellness and fitness tracking as part of creating a more flexible work environment. As I have pointed out wearable computing has a larger potential for organizations. This innovation along with mobile technology could help transform management of hourly and salary workers over the next couple of years.
Technologies like these are part of the new landscape in human capital management as businesses work to support the needs of all employees, from management to front-line workers. As well as mobile and wearable technologies these include analytics, big data, collaboration, cloud computing, and social media. Combined they have opportunity to simplify workforce processes and increase understanding of performance and engagement. As many organizations have both cloud-based and on-premises environments it is more challenging to get timely data that can be used for a range of analytics and reports. And they more data than ever before, which big data tools can help them manage and which analytics can enable them to gain insights from. In this data-driven world HR can’t continue using spreadsheets and reports and should transition to dedicated analytics tools, including predictive analytics to determine future hires or risks involving current employees. Analytics also is essential for helping maintain regulatory compliance, particularly the new Affordable Care Act (ACA). Our research confirms that the foremost issue driving investment into human capital analytics (in almost two-thirds of organizations) is improving efficiency and productivity. Analytics will remain a major focus of our HCM research.
The rapidly expanding use of mobile technology won’t slow down, we predict, especially as smartphones grow into mini-tablets, and tablets become like ultra-lightweight notebook computers. This pace of innovation is challenging for HR, many of whose applications do not run well on them. Apple is the preferred platform for business mobility, followed by Android and a small but growing presence from Microsoft, and organizations must figure out to accommodate the workforce in using the variety of mobile technology. HR should take a leading roleto determine the best strategy for the workforce and mobile use of human capital management applications. For example, our recent research on learning management systems finds mobile devices being used in more than one-third (35%) of organizations and almost half (43%) planning to use them.
The final item on our 2015 research agenda is the advancing use of systems that help individuals and teams collaborate. We observed embedded collaboration capabilities becoming a common component of talent management applications in 2014 and expect the trend to spread to workforce management in 2015. Organizations are rapidly turning to social collaboration methods, and while conventional discussion forums are most often used (by 39%), social recognition, wall posting or activity streams and earning badges and awards are being planned or evaluated for use in more than 40 percent of organizations, according to our next-generation learning management benchmark research. Organizations should be looking more closely at how to use collaboration to improve employee engagement, and we will provide recommendations on doing that.
Our HCM research agenda for 2015 continues to focus on the major issues in which technology plays a key role in addressing business issues. Please follow my analyst perspectives and our research and education in our community this year as we continue to evaluate the use of information technology to improve the effectiveness of HR professionals and lines-of-business managers.
CEO and Chief Research Officer