{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"Smart Metals Podcast","title":"Ensuring Data Quality in Metals Manufacturing: Techniques and Challenges with SCADA and Databricks","html":"","width":"100%","height":180,"duration":1798,"description":"In this episode of the Smart Metals Podcast, hosts Luke van Enkhuizen and Denis Gontcharov explore the critical topic of data quality in metals manufacturing, with a strong focus on SCADA systems and modern cloud platforms like Databricks. \nDenis kicks off with a big announcement: his business is now refocused on integrating legacy SCADA architectures with scalable cloud-native environments such as Azure Databricks. Together, Luke and Denis dive into the key challenges of aligning SCADA data with business use cases, the erosion of trust caused by bad data, and the urgent need for automated monitoring. \nThe discussion emphasizes how companies—from SMBs to enterprises—can implement robust data quality testing using open-source frameworks like Soda and Great Expectations. You’ll learn how to embed testing into ETL pipelines, use Databricks to store and analyze data reliably, and ensure high-quality inputs within a Unified Namespace (UNS).  \nTimestamps: \n00:00 Introduction to the Smart Metals Podcast\n 00:44 Big Announcement: Refocusing Business Activities\n 01:12 Understanding SCADA and Data Quality Challenges\n 04:37 Importance of Data Quality in Manufacturing\n 07:22 Real-World Data Quality Issues and Consequences\n 11:04 Steps to Ensure High Data Quality\n 27:00 Open Source Solutions for Data Quality Testing  \nNotable Quotes: \n“SCADA is essentially the second layer of the automation pyramid—supervisory control and data acquisition. It collects data from PLCs and individual machines. The challenge is moving this high-frequency, millisecond-level time series data to the cloud. Data quality is one of the key problems in this area.” – Denis Gontcharov\n“My new focus is helping companies integrate legacy SCADA systems into modern platforms like Azure Databricks, where they can finally get control over their industrial data.” – Denis Gontcharov\n“Almost any factory using modern machinery has multiple layers—sensors, PLCs, SCADA, MES, ERP, and eventually the cloud. Much of this...","thumbnail_url":"https://img.transistorcdn.com/Hv9X5-K-Xg_jobixWbYrDC7-MP2M1esqIHHaXkW-xJ8/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS8zZTJm/YjY3NjUzZTUyYjBm/YzA3MGI5MDg5NWIw/N2I0ZS5qcGc.webp","thumbnail_width":300,"thumbnail_height":300}