{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"Smart Metals Podcast","title":"Ensuring Data Quality in Metals Manufacturing: Techniques and Challenges with SCADA and Databricks","html":"","width":"100%","height":180,"duration":1798,"description":"In this episode of the Smart Metals Podcast, hosts Luke van Enkhuizen and Denis Gontcharov explore the critical topic of data quality in metals manufacturing, with a strong focus on SCADA systems and modern cloud platforms like Databricks. Denis kicks off with a big announcement: his business is now refocused on integrating legacy SCADA architectures with scalable cloud-native environments such as Azure Databricks. Together, Luke and Denis dive into the key challenges of aligning SCADA data with business use cases, the erosion of trust caused by bad data, and the urgent need for automated monitoring. The discussion emphasizes how companies—from SMBs to enterprises—can implement robust data quality testing using open-source frameworks like Soda and Great Expectations. You’ll learn how to embed testing into ETL pipelines, use Databricks to store and analyze data reliably, and ensure high-quality inputs within a Unified Namespace (UNS).  Timestamps: 00:00 Introduction to the Smart Metals Podcast 00:44 Big Announcement: Refocusing Business Activities 01:12 Understanding SCADA and Data Quality Challenges 04:37 Importance of Data Quality in Manufacturing 07:22 Real-World Data Quality Issues and Consequences 11:04 Steps to Ensure High Data Quality 27:00 Open Source Solutions for Data Quality Testing  Notable Quotes: “SCADA is essentially the second layer of the automation pyramid—supervisory control and data acquisition. It collects data from PLCs and individual machines. The challenge is moving this high-frequency, millisecond-level time series data to the cloud. Data quality is one of the key problems in this area.” – Denis Gontcharov“My new focus is helping companies integrate legacy SCADA systems into modern platforms like Azure Databricks, where they can finally get control over their industrial data.” – Denis Gontcharov“Almost any factory using modern machinery has multiple layers—sensors, PLCs, SCADA, MES, ERP, and eventually the cloud. Much of this may be hidden...","thumbnail_url":"https://img.transistorcdn.com/Hv9X5-K-Xg_jobixWbYrDC7-MP2M1esqIHHaXkW-xJ8/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS8zZTJm/YjY3NjUzZTUyYjBm/YzA3MGI5MDg5NWIw/N2I0ZS5qcGc.webp","thumbnail_width":300,"thumbnail_height":300}