A new research report by MIT Technology Review Insights, in collaboration with Databricks and dbt Labs, delves into the changing landscape of data engineering and the essential role of data practitioners in the era of artificial intelligence. The report elucidates the impact of generative AI on enterprise data strategies and underscores the increasing significance of data experts in driving business decisions and innovation.
A new research report by MIT Technology Review Insights, in partnership with Databricks and dbt Labs, examines significant shifts in data engineering and the evolving role and skill set required of data practitioners. Published during the Databricks Data + AI Summit in San Francisco this week, the report titled “The Data Practitioner for the AI Era” sheds light on how generative AI is transforming enterprise data strategies and the imperative role of data practitioners in this evolution.
The study is based on interviews with executives and data leaders from various organisations, including Apixio, Tibber, Fabuwood, Starship Technologies, StockX, Databricks, and dbt Labs. It underscores the critical importance of data in the burgeoning AI landscape and presents several key findings.
As AI continues to demonstrate its business value, data practitioners face new challenges and increased organisational significance. The report highlights an emerging trend where data practitioners are increasingly required to understand the business context, while functional teams within organisations are developing their own internal data expertise to harness their data effectively.
The integration of data and AI strategy has become central to business strategy, necessitating substantial investment from business leaders in terms of resources, data governance, and technology platforms. The report argues that data practitioners will play a pivotal role in determining how generative AI is deployed within enterprises, thereby influencing the future of these organisations.
Reynold Xin, co-founder and chief architect at Databricks, emphasised the fundamental role of data engineers in AI applications. “The reality is that what data engineers do lays the foundation for AI. They have the power to make or break all of these AI applications,” Xin stated.
Laurel Ruma, global director of custom content for MIT Technology Review, also noted the increasing demand for data practitioners’ expertise in the current AI-driven landscape. “The connection between data practitioners and business outcomes underscores critical decision points for unlocking the full potential of data and AI strategy. As generative AI evolves, data practitioners hold significant influence in transforming the enterprise,” Ruma explained.
The Databricks Data + AI Summit, where the report was debuted, focuses on how data intelligence can empower organisations to leverage generative AI using their own data. This annual summit gathers professionals from across the industry to discuss innovative solutions and the future of data and AI technologies.
For further details or to download the full report, interested readers may visit the MIT Technology Review Insights website. MIT Technology Review Insights is the custom publishing division of MIT Technology Review, a globally recognised technology publication associated with the renowned Massachusetts Institute of Technology. The division conducts extensive research on contemporary technology and business challenges, producing a variety of content aimed at senior-level executives and industry professionals.
The findings of this report are poised to shape how enterprises approach AI and data strategy, underlining the critical and evolving role of data practitioners in this dynamic field.
