Teradata Brings In-Memory Computing and Data Discovery to Big Data

Teradata recently gave me a technology update and a peek into the future of its portfolio for big data, information management and business analytics at its annual technology influencer summit. The company continues to innovate and build upon its Teradata 14 releases and its new processing technology. Since my last analysis vr_bigdata_big_data_technologies_plannedof Teradata’s big data strategy, it has embraced technologies like Hadoop with its Teradata Aster Appliance, which won our 2012 Technology Innovation Award in Big Data. Teradata is steadily extending beyond providing just big data technology to offer a range of analytic options and appliances through advances in Teradata Aster and its overall data and analytic architectures. One example is its data warehouse appliance business, which according to our benchmark research is one of the key technological approaches to big data; as well Teradata has advanced support with its own technology offering for in-memory databases, specialized databases and Hadoop in one integrated architecture. It is taking an enterprise management approach to these technologies through Teradata Viewpoint, which helps monitor and manage systems and support a more distributed computing architecture.

By expanding its platform to include workload-based appliances that can support terabytes to petabytes of data, its Unified Data Architecture (UDA) can meet a broad class of enterprise needs. That can help support a range of big data analytic needs, as my colleague Tony Cosentino has pointed out, by providing a common approach to getting data from Hadoop into Teradata Aster and then into Teradata’s analytics. This UDA can begin to address challenges in data activities and tasks in the analytic process, which our research finds are issues for 42 percent of organizations. Teradata Aster Big Analytics Appliance is for organizations that are serious about retaining and analyzing more data, which 29 percent of organizations in our research cited as the top benefit of big data technology. This appliance can handle up to 5 petabytes and is tightly integrated with Aster and Hadoop technology from Hortonworks, a company that is rapidly expanding its footprint, as I have already assessed.

The packaged approach of an appliance can help organization address what our technology innovation research identified as the largest challenges in big data: not enough skilled resources (for 56% of organizations) and being hard to build and maintain (52%). These can be overcome if an organization designs a big data strategy that can apply a common set of skills, and the Teradata technology portfolio can help with that.

At the influencer summit, I was surprised that Teradata did not go into the role of data integration processes and the steps to profile, cleanse, master, synchronize and even migrate data (which its closest partner, Informatica, emphasizes) but focused more on access to and movement of data through its own connectors, Unity Data Mover, Smart Loader for Hadoop and support of SQL-H. For most of its deployments there is a range of complementary data integration technology from its partners as much as it is a Teradata only approach. For SQL-H Teradata takes advantage of the metadata HCatalog to improve access to data in HDFS. I like how Teradata Studio 14 helps simplify the view and use of data in Hadoop, Teradata Aster and even spreadsheets and flat files for building sandbox and test environments for big data. (To learn more, look into the Teradata Developer Exchange.) Teradata has made it easy to add connecters to get access to Hadoop on its Exchange which is a great way to get the latest advances in its utilities and add-ons to its offerings.

Teradata provided an early peak on the just announced Teradata Intelligent Memory, a significant step in adapting big data architectures to the next generation of memory management. This new advancement can cache and pool data that is in high demand (hot) across any number of Teradata workload-specific platforms by processing data to determine the importance of data (described as hot, warm or cold) for fast and efficient access and applying analytics. This technological feat can then utilize both solid-state and conventional disk storage to ensure the fastest access and computation of the data for a range of needs. This is a unique and powerful way to support an extended memory space for big data and to intelligently adapt to the data patterns of user organizations; its algorithms can interoperate across Teradata’s family of appliances.

Teradata has also invested further into its data and computing architecture through what it calls fabric-based computing. That can help connect nodes across systems through access on the company’s Fabric Switch using its BYNET, Infiniband and other methods. (Teradata participates in the OpenFabrics Alliance, which works to optimize access and interconnection of systems data across storage-area networks.) Fabric Switch provides an access point through which other aspects of Teradata’s UDA can access and use data for various purposes, including backup and restore or data movement. These advances will significantly increase the throughput and combined reliability of systems and enhance performance and scalability at both the user and data levels.

Tony Cosentino pointed out the various types of analytics that Teradata can support; one of them is analytics for discovery through its recently launched Teradata Aster Discovery Platform. This directly addresses two of the four types of discovery I have just outlined : data and visual discovery. Teradata Aster has a powerful library of analytics such as path, text, statistical, cluster and other areas as core elements of its platform. Its nPath analytic expression has significant potential in enabling Aster to process distributed sets of vr_bigdata_obstacles_to_big_data_analyticsdata from Teradata and Hadoop in one platform. Analytic architectures should apply the same computational analytics across systems, from core database technology to Teradata Aster to the analytics tools that an analyst is actually using. Aster’s approach to visual and data discovery is challenging in that it requires a high level of expertise in SQL to make customizations; the majority of analysts that could use this technology don’t have that level of knowledge. But here Teradata can turn to partners such as MicroStrategy and Tableau, which have built more integrated support for Teradata Aster and offer easier to use that are interactive and visual designed for analysts who do not want to muck with SQL. Teradata has internal challenges in improving support for analysts and the analytic processes they are responsible for; its IT-focused, data-centric approach will not help here. Our big data research finds that staffing and training are the top two barriers for using this technology, according to more than 77 percent of organizations; vendors should note this and reduce the custom and manual work that requires specific SQL and data skills in their products.

Regarding analytics specifically, Teradata has continued to deepen its analytics efforts with partner SAS. A new release of Teradata Appliance supports SAS High-Performance Analytics VR_leadershipwinnerfor up to 52 terabytes of data and also supports SAS Visual Analytics, which I have tried and assessed and tried myself.

Through its Teradata Aprimo applications Teradata continues its efforts to attract marketing executives in business-to-consumer companies that require big data technology to utilize a broad range of information. Teradata has outlined a larger role for the CMO with big data and analytics capabilities that go well beyond its marketing automation software. The company announced expansion to support predictive analytics and has outlined its direction for supporting customer engagement. It needs to take steps such as these to ensure it tunes into business needs beyond what CIOs and IT are doing with Teradata as a big data environment for the enterprise.

Along these lines I have also pointed out that we should be cautious about accepting research that predicts the CMO will outspend the vr_CRM11_Inbound_InteractionsCIO in the future. What I have seen in these assertions is flawed in many facets and often come from those who have no experience in market research and the role marketing and dealing with technology expenditure in that context. As we have done research into both the business and IT sides, we have discovered the complexities of making practical technology investments; for example, our research into customer relationship maturity found that inbound interactions from customers occur across many departments; they occur in marketing (in 46% of organizations), but more often through contact centers (77%), where Teradata should strengthen its efforts. On the plus side Teradata continues to demonstrate success in assisting customers in marketing, winning our 2013 Leadership Award for Marketing Excellence with its deployment at International Speedway Corp. and in 2012 at Nationwide Insurance with Teradata Aprimo. Our current research into next-generation customer engagement already identifies a need to support multichannel and multidepartment interactions. Teradata could further expand its efforts in these areas with existing customers; KPN won our 2013 Leadership Award in Customer Excellence after connecting Teradata with its Oracle-based applications and supporting BI systems.

Overall Teradata is doing a great job of focusing on its strengths in big data and areas where it can maximize the impact of its analytics, especially marketing and customer relations. While IBM, Oracle, SAP and other large technology providers in the database and analytic markets tend to minimize what Teradata has created, it is has a loyal customer base that is attracted to the expanded architectures of its appliances and its broader UDA and intelligent memory systems. I think with more focus on the processes of real business analysts and further simplifying usability, Teradata’s opportunity could grow significantly. In helping its customers process more of the vast volumes of data and information from the Internet, such as weather, demographic and social media,  it could make clear the broader value of big data in optimizing information  from the variety of data in content and documents. It could expand its new generation of tools and applications to exploit the use of this information as it is beginning to do with marketing applications from Teradata Aprimo. If Teradata customers find it easier to access information and share it across lines of business through social collaboration and mobile technology, that will further demand for its technology to operate on larger scales in both the number of users and the places where it can be accessed even via cloud computing. Exploiting in-memory computing along with providing more discovery potential from analytics will help its customers utilize the power of big data and trust in Teradata to supply it.


Mark Smith

CEO & Chief Research Officer

Technology Innovation in 2013 is a Business and IT Priority

The proper use of technology enables businesses to be more efficient. Our recent research into technology for business innovation found that 56 percent indicate innovative technology is very important, yet only 9 percent are very satisfied with theirs, showing plenty of room for improvement. As we enter 2013, businesses have more choices than ever for technology to improve business and IT. Our firm has identified six key technologies that give organizations significant competitive advantages: big data, business analytics, business and social collaboration, cloud computing, mobile technology and social media. Our research agenda for 2013 is designed to help organizations assess and analyze these technologies and make the best possible decisions.

Big Data

Big data helps business and IT organizations manage and use information. Our technology innovation research finds only 14 percent of businesses today are very satisfied with their existing big data technology. At the same time, organizations that utilize big data vr_infomgt_information_management_initiativeeffectively have improved their business significantly, according to 28 percent of organizations. Our research in 2013 will build on our assessment of big data in 2012. We will do a benchmark research study on big data analytics and another on information optimization. We see that organizations are investing in information assets that require big data and the analytics associated with it to refine information and optimize business activities. Big data can have significant business value, but using it requires that IT coordinate with business on the benefits they can achieve. Making a business case for an investment in big data technology can help organizations address the top issue of it being too expensive, as found in almost half (44%) of organizations. At the same time, organizations need to ensure they have the competencies to meet big data needs, which include not just the technology for storage and access but also underlying information management issues such as data governance, data integration and data quality, which our research in 2012 found are still in embryonic form in most organizations.

Business Analytics

Our recent research on technology innovation found business analytics to be the top-ranked priority, and very important to more than half (52%) of organizations. Business analytics is not just a technology to get metrics faster but a set of processes to operate smarter with information, to better visualize and apply advanced methodsvr_predanalytics_predictive_analytics_obstacles such as predictive analytics, and to identify ways to better search and present information for a broad range of business constituents. Our research finds only half (51%) of organizations are satisfied with their existing processes due to lack of skilled resources. More than half of organizations say analytics are too hard to build and maintain or data is not readily available. The most time-consuming aspects of the process are data-related ones, according to 44 percent of organizations. The technology too still has room to improve, with only 20 percent being very satisfied. The most critical capabilities are in the areas of predictive analytics (49%), visual discovery (48%) and taking action on the outcomes of the analytics (46%). We also found in our next-generation business intelligence research that the use of mobile technologies such as tablets is growing across organizations. We also found in our research a high priority to use social collaboration technology with business analytics to work together on making improvements in shorter period of time than traditional email or phone calls. Our research agenda in 2013 will investigate big data analytics and next-generation business analytics approaches building on top of our research on predictive analytics, which found that organizations still struggle to integrate predictive analytics with information architectures to support analysts and data scientists. Advancing the competencies and focusing on analytics are critical processes, but businesses also need simpler communication of results to help those responsible understand situations and consider potential recommendation actions. We hope that technology suppliers will work to better align to the human dynamics of what really happens with analytics to better support communicating observations and insights to ensure that end goals are achieved.

Business and Social Collaboration

vr_socialcollab_supports_talent_managementThe revolution in social media has expanded into business, bringing with it social collaboration and helping business processes by connecting people to achieve goals personally, departmentally or across an organization. This new technology was ranked the second most important technology in our research, but only 17 percent of organizations are very confident in their ability to use the technology well. With most organizations (86%) using shared folders and documents, it should be no surprise that part of the issue is related to technology. Organizations are evaluating new methods such as wall posting (45%), social recognition (41%), earning badges and awards (40%) and broadcast or Twitter-like capabilities (39%). With only a quarter of organizations being satisfied with their existing approaches, we see a lot of changes coming in 2013 in regards to the technologies selected and deployed. Our research in 2013 will examine where collaboration is critical in areas of human capital management, sales, customer engagement and even finance. Building on top of some groundbreaking research across business and vertical industries, we see business advancing rapidly with or without IT support, since business and social collaboration can be easily onboarded through the use of cloud computing. Unfortunately organizations are mixed in the methods they prefer to use to access collaboration – embedded in applications, through Microsoft Office, embedded within tools like business intelligence or stand-alone – making it complex to have consistency for users and their interactions. Using social collaboration with business analytics is a growing priority and organizations will need to assess their technologies to see if they meet this need. We believe that social collaboration will help bridge generational divides between workers as it becomes more easily accessible through web and mobile technologies, allowing managers and workers to engage anytime or anyplace. Focusing on the benefits of social collaboration, such as knowledge sharing, is critical as our research finds as the top need in 49 percent of organizations.

Cloud Computing

Our research finds that businesses don’t see cloud computing as innovative technology but rather as a utility and becoming a standard method that can be easily accessed and leveraged as part of their portfolio of computing options. These faster methods to onboard applications have become easy and in most cases require little IT involvement.vr_datacloud_data_in_the_cloud_concerns But beyond the simplicity for business and potential chaos for IT to eventually govern and support, the cloud computing environment is now a viable platform for IT to leverage in a multitude of methods, from IT infrastructure to developing and operating applications. The cloud computing environment can be used as a central point for integrating data and storing it for the enterprise or for customers and suppliers, but most organizations have not automated the integration of data to support business processes or business analytics and decision support. The lack of automation has increased concerns for data security, which 63 percent of organizations in our data in the cloud research find to be a major concern. In all of our research in 2012, the preference for cloud computing is growing across lines of business and especially in areas like sales, customer service and human capital management. In 2013 we plan to further assess the advancements in cloud computing, from big data and analytics to information that can be leveraged from a broad range of applications and services.

Mobile Technology

The use of smartphones and tablets has become common among consumers who are also workers in organizations. Mobile technology is a new platform on which organizations can deploy applications and tools for a wide array of business needs. Yet our next-generation vr_ngbi_br_goals_for_mobile_bi_deploymentsworkforce management research finds only 8 percent of organizations indicate they have everything they need on these technologies, and only a quarter more indicate they have most but not all they need available, which leaves a large number of organizations not able to meet their mobile business needs. This might be why only 20 percent are very confident in their use of mobile technology today. The debate on whether to use native applications and tools or operate across a web browser environment still looms, with native (39%) outpacing browser (33%) and a fifth having no preference. Bring your own device (BYOD) is another area of friction, where 39 percent of organizations allow this approach with smartphones and 45 percent with tablets. Organizations have many opportunities to determine how to use mobile technology effectively, and can derive many benefits. Our next-generation business intelligence research found increased workforce productivity was at the top of the list in 55 percent of organizations. Our research in 2013 will further investigate the use of tablets across the line of business, since this was found to have the largest growth planned (34%), while smartphones are more established.

Social Media

Social media is a new path for organizations to use to expand their corporate footprint to a broader audience and to gain brand awareness by marketing products and services. This new channel of opportunity enables organizations to rethink how they operate many of their business processes, including the ones that they use to find newvr_socialcollab_factors_driving_use_of_social_media talent and track candidates into an organization. In 2012 our research into social media and recruiting found that only 7 percent of organizations are very confident in use of this channel, but half of organizations are planning to change how they use social media over the next year; for instance, 81 percent of organizations have identified it as a method to identify new talent pools. In 2013 we will continue to examine best practices and benefits of investments in this channel. We will also assess social media as a new channel for customers to engage with organizations through a new benchmark in next-generation customer engagement. Our research in 2012 found organizations benefit from using this channel to handle a broad range of customer questions and issues.

While new technologies can help business innovate, what’s old is still new, and requires a foundation of skills and resources. For example, with big data, those organizations that have information management competencies to automate big data efforts will find themselves further ahead, as they leverage the core skills of data integration to handle more data environments. Those organizations that use business and social collaboration to connect people and processes more efficiently than conference calls and email will better leverage their human capital investments.

At the same time new techniques can make it simpler to gain value from existing technology investments, such as advancements in the use of text to present analytics in a readable form, new methods to use visualization as a discovery tool on analytics, and the ability to engage employees by using new and more social collaborative methods. Taking advantage of this technology requires smart use of best practices, leadership from the top and agreement about the desired outcome. Organizations need to have technology in place to develop a business case with balanced evaluation criteria that are not about the vision a vendor has but rather about what the vendor can provide to advance business efforts. We use this practical approach in our vendor and product assessment methodology and rating called the Value Index which we will be assessing over 100 vendors in 2013.

vr_bti_br_barriers_to_use_of_innovative_technologyOrganizations should explore resources in the company to determine if necessary skill sets exist, since their lack is the top barrier to adoption of technology according to 51 percent of organizations as found in our research. Part of this process is ability to know whether the technology can adapt to the workers’ needs and capabilities, and whether it requires weeks of training. Organizations should also look to the future and examine how to use cloud computing to rent technology, and how to use mobile technology to enhance collaboration. They should also keep pace with peers and competitors through the use of benchmarks and industry comparisons.

You can depend on Ventana Research to provide sound facts and pragmatic guidance to help you leverage technology to gain a competitive advantage in your business and innovate in your processes and with your workforce.

Read 2013 Research Agenda


Mark Smith

CEO and Chief Research Officer