{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"Technology Explorations in Data & AI","title":"Data Ingestion using PyAirbyte","html":"<iframe width=\"100%\" height=\"180\" frameborder=\"no\" scrolling=\"no\" seamless src=\"https://share.transistor.fm/e/309a5050\"></iframe>","width":"100%","height":180,"duration":1050,"description":"Move your Google Drive documents straight into Postgres using Python and PyAirbyte. In this Technical Explorations episode, Jonny and Tarik from Dataminded show how they ingest internal meeting transcripts (Facts at Breakfast, Learning Over Lunch) from Google Drive into a relational table, ready for querying and AI use cases.\nYou'll see how to:\nConfigure PyAirbyte to read from a Google Drive folder\nAuthenticate with a Google service account (JSON key)\nConvert Airbyte output into a clean pandas DataFrame\nLoad the processed data into a Postgres table\nDiscuss performance limits, API rate limits, and batching\nReflect on when PyAirbyte is great for PoCs vs. production setups\nWe also touch on:\nHow many connectors Airbyte offers and what PyAirbyte can reuse\nTrade-offs of code-first ingestion vs. point-and-click UI\nIdeas for the next step: using MindsDB and LLMs to query this knowledge base\nResources:\nDemo code: https://github.com/datamindedbe/demo-technology-exploration/\n\nFull playlist: https://www.youtube.com/playlist?list=PLJ_da7qdfL80rA7byzC_CmyrfJWjcCTnb\nChapters:\nData & AI: Technology Explorations is a biweekly show from Dataminded. Each episode a Dataminded engineer demos a tool or technique worth knowing about -- working code, honest takes, no hype.\nMusic by Aleksandr Karabanov from Pixabay","thumbnail_url":"https://img.transistorcdn.com/z5RDJLT6bqK4m6zhRkosVXEjgssOVw4nBJpnb_UIEaQ/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS83Mzcx/ZDNiNjIzNzMxZGE3/ZmNmNmUyMjBkMWFk/MTJlYS5wbmc.webp","thumbnail_width":300,"thumbnail_height":300}