Energy Bytes

Reishin Toolsi, Senior Solution Architect and Lead Energy SME at Databricks discuss how Databricks is transforming the energy sector through innovative AI and data management solutions. Reishin shares insights into Databricks' Lakehouse architecture, which bridges data silos and accelerates AI deployment. The conversation also explores the benefits of open-source technologies like Delta Lake and how they enhance data governance and operational efficiency in oil, gas, and utilities.

Join the conversation shaping the future of energy.
Collide is the community where oil & gas professionals connect, share insights, and solve real-world problems together. No noise. No fluff. Just the discussions that move our industry forward.
Apply today at collide.io

00:00 - Intro
00:50 - Reishin's Journey to Databricks
02:45 - Importance of Energy
05:27 - Energy Addition
08:56 - Databricks History
12:50 - What is Databricks
16:56 - Databricks as a Unified Data Platform
18:20 - Databricks and the Future of Data
20:00 - Databricks on Any Cloud
21:40 - Challenges of Edge Computing
23:20 - Future of Edge Computing
29:02 - DALL-E
30:30 - ChatGPT
32:49 - Open Source vs Proprietary Models
35:23 - How to Use Dolly
37:35 - AI to Capture Tribal Knowledge
40:00 - How to Use GPT-3 Plugins
43:15 - AI's Impact on Society
48:31 - Databricks in Action: Use Cases and Case Studies
52:35 - Getting All Your Data in One Place
53:33 - Interesting Energy Projects by Databricks
57:56 - How Databricks Optimizes Your Data
1:02:40 - What Are You Excited About
1:04:05 - Speed Round
1:07:29 - Have You Tried the New PSVR 2
1:09:31 - Apple TV
1:12:05 - Favorite Visualization Tool
1:12:41 - Favorite Book

https://twitter.com/collide_ai
https://www.tiktok.com/@collide.io
https://www.facebook.com/collide.io
https://www.instagram.com/collide.io
https://www.youtube.com/@collide_io
https://bsky.app/profile/collide-ai.bsky.social
https://www.linkedin.com/company/collideai

What is Energy Bytes?

Welcome to Energy Bytes with John Kalfayan and Bobby Neelon. Your essential guide to the intersection of data and energy. This podcast dives deep into the world of energy, shedding light on how data, AI, and technology are revolutionizing this sector. Each episode equips listeners with insights into the most efficient tools and resources, paving the way for a data-driven future in energy. From technical nuances to broader industry trends, Energy Bytes offers an unparalleled perspective on the evolution of the energy industry. Join us as we decode the algorithms of energy, one byte at a time.

All right. Well, welcome to another episode of Energy Bites. I'm Bobby Nean. We got uh Rad John Calan with us here. How's it going, guys? Welcome. Welcome. And uh you before we get introduce our guests, you want to make sure please make sure you go review our podcast. Um you know, hopefully give us five stars, but if not, please just give us some really good feedback and make sure you you subscribe. But um really excited. We've got uh Raian Tulsi with here with us from from data bricks to not just I mean I know John and you've got some you some history but um I'm excited you know I'm pretty well in tune with the whole snowflakes data bricks you know that side of the world understanding like kind of the you know strengths weaknesses you know if there are weaknesses of you know data bricks and some of the different stuff in the market. So definitely interested to pick your brain on it. Coca-Cola versus Pepsi. Yeah. Right. [laughter] But uh let's let's jump into it. Tell us a little bit about yourself, where you're from, you know, kind of how you got into energy andor tech. Oh yeah. So uh Rian Raian Tulsy, I'm now a resident of Houston, Texas. And you know, I like living here. So I'm going to stay here for a while, you know, until the heat gets to me like most most folks would see. Uh maybe, you know, a few more summers and then we'll decide. Uh I actually originally I I grew up in in South Africa and uh you know I ended up living in England. So I'm a dual national as such and you know now I live in the US and I started my career in in pure tech software engineering medical diagnostics and then ended up in um ended up in in I worked for a company called Schlumbumberj back then which now changed it name to SLB SLB and kind of moved around ended up in Houston and then during co actually which is kind of an interesting experience. I um uh I was dealing with data bricks as as from a client perspective. Okay. And I was kind of giving them a hard time about saying, "Hey, you know, you could do these things better and these things better." And actually a guy in there said, "Hey, you know what? You should come work for us." And that's usually how most of these these conversations go. So we had this conversation. I ended up I like what they were doing. Co was a interesting time to make decisions. So yeah, I joined the startup company back then. And when I joined, I think there was like uh maybe 800 people or so. And it's definitely definitely definitely exploded. Yeah. Exploded now into a much larger there's a number that they're targeting by the end of the year. I think yeah, it's they'll they'll get there as well. So, it's it's doing well. It's it's they call it the rocket ship. It's my first real rocket ship, and I'm excited to be part of it. That's awesome. That's I love I love being able to see not just people from the energy space do well but also people from the energy space getting into like bridging that energy tech gap so to speak. Yeah because yeah I think a lot of people in tech think maybe our problems are easy you know or I mean like from the outside or or that they've solved them but I mean until again you get someone who actually understands both sides of it and you're like no this needs to work better. So let's let's talk about it for a second. Let's just just jump into what what is easy right? So fundamentally you are let's let's take energy as being because where the way I look at energy and the reason I I joined data bricks is because I get an opportunity to work with energy not just so I have a passion for oil and gas because I have a lot of respect for the industry but energy itself I keep saying this right so you know it's this connected ecosystem now of energy oil and gas powers the utility industry utility industry powers our homes without electricity our world and our lives will be a very different place it's not a place I want to live in right so the the challenges around those industries are immense. You got, you know, heavy assets. You're dealing with transporting lots of of of dangerous material as such. You're working in high pressure, high dangerous environment, drilling deep into the air at high pressures, drilling out the sea, middle of nowhere and and it's also a global kind of industry where you have to deal with the geopolitics and wars and and all sorts of consequence of it. Those all, you know, and that all adds to it. So, you know, I have all the respect involved for all the other industries, but I still think without energy, none of those things will actually exist. Yeah. And we'll be living in a very different world. So, uh I'm I'm very passionate about keeping the industry going and and you know being successful in it, making others also successful in that industry. Yeah. I think I don't think a lot of people realize how directly correlated economic growth, wealth, innovation, technology are all directly tied to cheap energy, cheap, reliable energy, right? Well, let's take um um dire situation. you out in the middle of nowhere uh and basically you're sick and uh right now if you have telecoms, a helicopter could get to you to rescue you. An ambulance could drive out to you and then transport you to a hospital. rubber for the tires, silicon for the tires, the oil that actually can keep those vehicles running, the communication and transportation, and then when you get to the hospital, the supply chain that provided you the the medicine that's going to keep you alive, and then the R&D over the last 50 years that basically produced something that's affordable. Mhm. And the knowledge that we got from basically communicating with people around the world to basically bring you the best care is there and you don't think about that. Yeah. I mean from everything to the actual medicine itself to the IVs to the transportation around everything just keeping the lights on at the hospital. I mean like I sound like a similar you were describing before we got on like about like situation in South Africa now I think people in the you know and that's not that still is a developed world but I mean say United States western Europe like we just say for granted that I flip a switch and the light comes on and energy comes from the wall air conditioning you know keeping us warm in June and July and Houston you know the air dehumidified so yeah that's that's a great example I grew up I grew up in South Africa and I still have a you know some some linkage to the country I don't want to get too much into the politics of it all because I find politics uh not just divisive but also very very boring right because there's much more exciting things to talk about right but basically due to uh mismanagement corruption and just lack of like um maintenance the infrastructure is basically failing around maintaining their their power grid yeah so they go through daily they've been this is for years every day there's a dramatic amount of power failure and this is easily documented and there you know there's lots of cases about where now businesses can't can succeed because you're going to have to have diesel to power the generators but just the cost of keeping your business going, groceries and transportation, it's impossible. So now it's it's failing and it's fundamentally because lack of infrastructure investing also some diversification like they they too invested in their coal and only coal they could be looking at other sources of energy to kind of manure that's something to think about too right so the reality of like you know being hyper myopic about energy yeah well I think that's a big probably one of the biggest I think problems misconceptions about the energy space as a whole is that there is a silver bullet onesizefits-all solution And it's like that's not necessarily the case in most areas, right? Like yeah, we need need all energy. Like it's it's about energy. I'm like I haven't really said it much on the on the podcast, but we have been actively pushing it more at DW. It's about energy addition. As long as that energy is cleaner than the prior form and can be more affordable, accessible, reliable, more energy is better than less energy generally and and we kind of in a unique like let's just take because we live all I live in Houston, you guys probably live in Houston or you live you know in the area and so we have a lot of sun here, right? I mean maybe there's a lot [laughter] there's no shortage of sun during summer but like even during the the winter times, you know, so solar is ramping up. So there's a a a really great solar companies in Houston that are kind of like taking off. Um and and what we finding is that now that if you have a smart home or you're very costconervative about how you manage your homes and most most new homes are basically kind of built for this, you're overproducing energy. So either the energy goes back into the grid which you can sell off at a fraction of what you're generating or and and now we're seeing we're seeing vendors out there that basically finding ways to basically create a marketplace for energy so you can sell it to to you right at peak prices local regional a local regional right and then all of that and this is now going to be my pitch all of that basically is um a challenge around data right so fundamentally you got billing data you got operational data you got uh functional data coming into unstructured, structured, real-time batch data coming into it. And really, this is where we're seeing these kind of like uh central platforms of data kind of picking up, right? And the more flexible and the more a platform allows you to kind of work with different types of data and streaming and batch and and and ingesting tools and then also in real time at the same time. And the most important part is actually the data format around it needs to be in an open format so you can communicate and allow you to plug in and play with other people. So this is where we're seeing like you know the data bricks product which is the lakehouse kind of like be very pervasive actually then gain a lot of market share because of this right so we're seeing this happen because the world is moving towards this kind of like connected ecosystem for communicating and collaborating and even competing right we're seeing competitors kind of use our product but also share data sell data and then work with data so yeah so I mean it sounds like we're kind of pivoting into data bricks one thing because I kind of know it but I want to make sure other people you know listening or watching kind of are Can you kind of walk through the kind of the history of data bricks like what it started as and what it's kind of morphed into? Um I mean was starting from Spark and because they were some of them were the original Spark people, right? Yeah. The founders were the uh founders were the original creators of Apache Spark when they basically created the the libraries around Apache Spark and they created Spark and they made it open source which is great and they basically you know created a comm a community around that and that community I think there's millions of of spark users around that and then I think they they they thought about you know the ability and there's a whole great story about um called truth tellers out there and they're the data bricks story it's much more well written and much more elegant than I can I can you describe right now but they formed the company data bricks based on this foundation and they actually created a kind of an open source format called Delta which is really what the the core the core engine for data bricks really is and it's core engine for spark is the ability for us you know kind of like process this this data in this like format uh so we you know we have this delta format which is still open source park format uh there's no uh there's no data vendor locking into it it's yours you can you know you can work with it hit it from R Python whatever you can we can work in Python Scala uh we can work in SQL it's great you know and that's it and they created data bricks as a company that basically has they built these optimization tools around it so they brought in their knowledge they love for this passion around and they kind of built a a series of optimization engines around data bricks and they kind of created this platform as they evolved uh called the lakehouse and on that it was a single platform where you can basically do business insights you can do your visualization you can work in multi cloud so we we partner with all three cloud vents uh and then we having Of course we we now bring in more governance. We have this thing called unity catalog which is really around creating good governance because once you have good data you need better governance to kind of manage that. Uh and then you know security is kind of an issue as well. So we we manage cyber security as well. We also work with the cloud vendors. You that's where your data kind of sits in it. And then we have technologies for like sharing information like data sharing. So that's the ability for you to you know share data in a cost-effective way that was never possible before. Right. And and what's interesting is this isn't me trying to push a product or not, but you it's a consumer producer type relationship with delta sharing in the sense that you as a consumer or you as a consumer don't have to be a data risk customer, but the producer does. So let's say you're running an operator and you basically want to share some real-time data or some just some near real-time data with a transportation company or vendor. You could share that Yeah. with them. They don't have to be a database customer. So it makes you more efficient in terms of like vendor management. Yeah. And that's say relative to like say snowflake which because I mean snowflake has the being able to share across accounts which is really nice couple clicks of button and shared across but that's in their own proprietary format and you have to be a snowflake customer to consume it on the other side. That's absolutely right. So there is a proprietary format around it and and actually you know as we will we'll put it out to the customers and say we believe that your data is your data and fundamentally there should be no vendor lock in right. So ultimately, you know, we'd love customers to work with data bricks, but you know, for some reason, if you choose not to work with us after, you know, being part of our our community, it's still your data. There's no lock in with us, right? But we we are so confident you'll remain with us that that people do. And I think I mean I think this past week or so has really highlighted that where Azure Microsoft has adopted the Delta format for their uh fabric. Fabric. Yes. Yes. So it's platform. So it's like a it's like a little love letter, isn't it? Yeah. Exactly. That it was built on this. They all have their own kind of like park format and they have some specialization around it but but essentially it is the core of what we've done and and that is that is that is that is interesting right there. Yes. Yes. They also still encouraging their their customers to still pursue investments back into data brick. So it's still called Azure data bricks. So there's you know there's a whole write up about that. You know I'd love to we could obviously share that with you guys and your readers if they need to. No 100% probably link to it even. Exactly. Yes. Tell me let's back up a second. So for the more generalist technologists out there, give me the the fifth grade uh summary of of what data bricks is maybe an example or two of like in the energy space how somebody would use it, things like that. Yeah. So data bricks um you know if you think about it from in terms of like the well let's let's take a step back as to as to um we we know that AI is basically a technology that will be more productive than it'll be you know I wouldn't say destructive is such not not a good word but it's definitely a much more productive technology and we see a lot of CIOS and cos and people investing in AI Right? We know that and then we also know that there's you know uh billions of dollars invested in that for success right and people have you know not just invested resources but they've invested you know in terms of technology and they they've made big bets on AI but actually what we also see is that uh the data does say that a lot of these AI products are unsuccessful and they don't go into production so we looked into why that is right and it has to do with kind of this challenge around the managing of data right so if you look at data how it is created origin We had warehousing and that's really what Snowflake focuses on is data warehousing. Um and then and in data warehousing you had really good things like good business insights but you had challenges around data warehousing uh sorry get challenges around uh artificial intelligence uh but also data warehousing is very structured and you tenure exactly and you also you laugh you also lack like the scalability and the volume of data as it changes to deal with unstructured data and change. So then these lake these data links kind of popped up over time and as the data links kind of popped up and basically we found that yeah you could scale data and it was great you could do unstructured data and you could the volumes of data was great and you could do AI on it but you kind of lacked governance right right and you kind of lacked that security and governance that you you kind of turned into a mess even like it was so unstructured it's just the data garbage so so what the technologist did was and this is data they kind of built technology to kind of build the gaps but by creating more technology and for sources you had different data formats you were communicating you actually created more bottlenecks in terms of communicating cost goes up so those AI projects were kind of delayed because the cost and delays go to market was was kind of failing uh you couldn't reproduce uh training sets couldn't do any of that stuff or you had your data say in a snowflakes or a SQL data warehouse but then you had to get it out to run your models you know rather than you know yeah yeah yeah and and so like you know time to market was kind of failing and if you're a CIO you're seeing a cost for AI projects kind of rising and you think I don't want to invest So it's challenging right so what we did was basically take that that technology which was delta which is our core engine use data bricks and kind of create this thing called a lakehouse so we kind of took the best of both worlds took the best of the data warehousing world which is the structure and governance and then we took the flexibility of a data lake and kind of created a lakehouse so all of this in one place so like the communication issues have all been kind of resolved so you create a single platform for people to work so from a fifth grade I know this is kind of long definition but from a simplified version it's perfect a single place to actually work allows you to communicate faster and share information faster. Right? A single format that's open, that's flexible, that that there's no vendor locking, there's no proprietary locking means that you can engage with lots of other open source technology. You're not vendor locking and you can work in multiple languages. So there's no v there's no language locking. SQL, R, Scala, Python, great. You can kind of work in all these things. You can share we work in notebooks, you can share notebooks easier, you can share data easier. Uh so it just makes it faster time to fail, right? So um bad ideas would fail faster and good ideas go to the front. Y like we've seen some data where where we've increased training set time and also sorry we've we've decreased training set time but we've doubled the uh the data set size. So all by kind of just just creating this kind of ETL right platform for just kind of doing this thing. So that's really it. No I mean it's pretty fascinating platform. Um I mean looking into because again it serves all the different pieces of that data team right like the data engineer can use their spark and everything create the pipelines and again to your point you can write those in a jupyter notebook type thing and you can deploy them that way right you don't have to like I don't have to pull it out put it into python script now and now schedule with airflow I mean you can do that within the platform right but then that data gets put into the lakehouse and you have your delta tables right but then you know the business analysts can hit it with the SQL photon engine right and they can pull it into spotire and do descriptive analytics, but then your data scientists can use that same data and you know run their AI models. But then the thing that fascinated me, I went to like the data bricks in a day or whatever it was a couple months ago. It was just like all right now the outputs of their models go into their own delta table and then those are actually accessible say to the business analyst if they want to pull those in. So I mean it's it is truly one kind of unified compounding and and even if you don't want to use and and I love and I love typical right so I like I like Spotify but like even if you don't want to use that if you just want to create visualization like the notebooks we have our own notebooks uh and even if you don't want to use any of these high-end visualization tools you could we have like DBSQL you can get some visualization within the notebook that can immediately give you results like so a business analyst or like a business stakeholder wants to see some results from either data engineering or data science they can see that immediately. Yeah. Right. and they can make quick decisions. They can make decisions faster all in the same place. Well, because and maybe some of this is driven by it, but I and I know was it a couple years ago you guys acquired the bamboo lib or whatever it was from uh and that was kind of the the ability to write some was it pandas or something that kind of with a guey use pandas. Yeah. Um so I mean I think that's interesting too like once you again you get again full scale you don't like you're saying you don't have to leave the data bricks platform. You don't have to and and we are growing right now. We've been making some new acquisitions and uh we are growing. We're trying to make it we we love feedback from the customer base out there. This is why we we increasing our our kind of like user groups and we want to have and the way we do user groups is we actually have a client hosted because that's much more authentic. Did you attend the one that was in Houston? Yeah, this was pretty recently. It was at uh right there I work in city center so it was right over there at the Microsoft building above Avanad I think or Yes, that's right. That was hosted by Microsoft. Yeah, exactly. So we have our clients our partners hosted and that's uh that's that that's that's great. We we're going to having more of these in Houston. Uh we have them all around the all around the world right now. So this is kind of good and it's actually community based. So the communities speak about it and you know we love feedback and we want to improve. So yeah, we want to we want to welcome people to grow especially you know in you know like Texas itself is a it's an energy capital itself and you know there's challenges around there and you know to John's point earlier he said it's kind of nice to see someone from the oil industry kind of go into tech. There's a lot of that. I mean what else you do with your time right now? take that kind of knowledge and you know help tech understand what's going on and and then when you you you speak to your fellow colleagues in the industry you add more kind of um um credibility to that conversation. Y right because you understand the problem and you you've worked in the in the same field so you understand that. Yeah. Yeah. And I think one thing I would want to highlight too and maybe you can highlight a little more is like because I know at one point especially I thought of data bricks is like you Azure data bricks was one of the first ones but like I think make sure people understand like data bricks can be run on any of the right cloud you know well I don't know about agnost but like any of the major cloud platforms it works on all three major cloud platforms. Yeah. You can you can deploy data bricks on Google or on Azure or on AWS. Yes. So Google, Azure, AWS work the the three the three major clouds and we now have and this is public. We now even have a um I don't want to call it onrem but there's a way so we have a partnership with Dell. Okay. Great. We we can add a link to that where basically you can have data basically stored you know on if you have your own private cloud on private cloud basically send that to database to be created. So it's a great example of especially in the oil industry where for data residency issues, communication issues, you may have to have some data on prem you can basically store the data and then push it into the cloud and basically manage that. Yeah, that's fascinating and because then also you can also deploy some of the stuff as say IoT devices as well, right? So models that you can deploy the models but we'll use the um model deployment tool. So the curation will be in database and then we'll use MLflow which is our orchestration tool to kind of manage that entire uh entire cycle. uh and then the deployment will will be via whatever mechanism that you use out there. But yeah, right and and that's how it should be, right? It should be everything all the high-end compute should be in the cloud uh developer should be because it's cost effective makes more sense and then you push to the edge for doing some some running the models running the models and basically you know doing errors. Yeah. Now that's how we linked up was when I was at high yeah that was a great product. I like that. RIP. [laughter] It just even it looked pretty cool. Like there's there's so many things I think they were doing right. And then of course there's so many things just holistically with edge that is still so early and new that there's a lot like CI/CD for I'll just take that category right no one has figured that out for edge yet. That's going to be a very tricky thing specifically in oil and gas where you have connectivity intermittency and things like okay I'm pushing an software update to my remote server all 100 of them across a field or across the country well two of them the internet went out in the middle of the update. Of course it would. What do you do? You know like we just bricked those servers. So now what? you know, and so there's there's so many so many uh nuances and and things around edge that need to be solved. But I'm I'm still pretty optimistic and I'm glad companies like data bricks are incorporating and looking at you know they are it's not just for today and it's not just gas this could be manufacturing be retail because you know like cloud is it's just access to the data that's the challenge right so cloud cloud technology is fantastic right now it's cheap it's effective it's everywhere the data centers are are getting much more global right now but where we where our clients are and where our background comes from you in the field a lot you could be on on a rig, you know, on a ship somewhere and satellite communication may exist, but it's usually the bandwidth is is actually usually channeled for ship operations rather than actually internet communication. So, yep, even if there's there is internet, they'll basically choke that internet. So, updates will be will be restricted. Yeah, that was actually one of the first uh I think one of the first pilots that they did when I was there was with a or that we were investigating was with a international shipping company, right? and they've got, you know, a ship that h it basically has its own SCADA system running on it, but it's now connected to a cloud-based SCA platform that doesn't really work because they've got satellite comms that are incredibly intermittent. Yeah. And but they've got all this data that they would love to be able to do something with, especially around, you know, the the thing that makes you the money, which is your boat. Yeah. And so, how do we do that? Right. Um, but that's now it's I'm I'm excited to see what the future for for edge in general holds. Like I said, I think there's a ton of things that need to be solved, but I think there's a lot of promise there. Um, and I'm glad companies like data bricks are I mean set up for it at the very least. Think about yourself as a person as a you know in your private life, right? So we have a lot of devices on Yeah. Yeah. I mean that's an edge device. It's a incredible edge device. It's an edge device. It's just that it's just normally there's a te telecom's communication too, but I don't know. Maybe [clears throat] out in Yellowstone somewhere and you know it's like you know Yeah. You know it's possible. Yeah. There's no telecom. So what do you do that does that become useless? No, you should be able to have functionality on that that device. Yeah. Now, that's uh I think one of the case studies that I found when I was first getting into it was uh this article around like one of I don't know what algorithm it was, but Facebook Meta put out about how they're doing like a hybrid kind of setup with their processes where they're essentially using the phone as the edge device. So they'll run certain processes on the edge device first if they can or if the communication is low then they'll try and localize it versus pushing it up to the cloud and just things like that and you're like oh that makes a lot of sense you know if you you think about that way and so it's it's uh yeah it's it's fascinating. I mean, then you look at like the flip side of that, which is like a Tesla, right? Where they custom built their own edge computers for all of their devices and they're trying to do federated learning and all this really interesting, fascinating stuff. And still we're going to be, I don't know, five, 10 years away from completely self-driving. Maybe if we get to that, I don't know. Well, then you need to get the adoption, people feel good about it. The adoption, the policy, the technology is almost there. My issue is with the outliers with most AI, right, is it's even with Tesla, you know, having constant data and calculations and models running, you still, you know, run into those random outlier cases where it doesn't know what, you know, a garbage truck that's painted all white looks like, and so it runs into the back of it. Yeah. The guy on a motorbike was once hit by a Tesla because it's just undetected. didn't recognize. Well, yeah. I don't want to get too far down this rabbit hole, but like then the thing comes into like insurance. Who's at fault? Yep. No, I remember vividly when they first started talking about autopilot, there was a whole paper about like, well, we're going to have to program the computer, you know, just to to make a decision of if, you know, do I hit this pedestrian or do I turn my car into a brick wall and kill who who's going like there's this ethical piece of it of like, well, if someone's going to die, how does it determine like or potentially if both outcomes are bad? Oh, I I have one. I have one for you. So, MIT runs this kind of uh this use case, this AI, a terrible use case scenario, a no- win scenario, and it's basically I'm driving a car and uh it it has automation in it and uh it's worst case scenario. It's snowy, it's icy, you're kind of driving down here and you're on this no- wind scenario and essentially you're on a bridge and basically your brakes fail mechanical, nothing. So the car has to kind of like the mechanics of the car can only steer to basically move. You can't actually slow down. So it's something it's scenario like on the left is a bridge and if you go off the bridge you will basically perish and on the right it's basically a bunch of like school kids, right? It's a school bus and then in front of you is basically a crossing area for like senior citizens and they're crossing at the time. What should the AI do? Like who should it endanger? Right? That's a no scenario, right? And so that they they ask you what you should do and people like oh well you know they make this horrible scenario it's like well you know the senior said they've lived a lot longer so you know they should take it starts getting it starts getting hit like I don't know those kids you know it's like you it's like and so ultimately what it comes down to is that what people don't want to say is that is what we all think is like no the car should save my life. Yeah, right. That's what it should chew. It should save me and my family, right? But I'll the answer to that is I think the AI just goes, we don't know. Yeah. It shouldn't make a moral decision based on those things, right? Because it's impossible. We can't make that, right? We always say I would do my best to save everyone's life, but it's a no-win scenario. Yeah. No, that's I remember in college take some very interesting my the one psychology class I took and it was fascinating because you have all those Yeah. those moral ethical that you throw out there, you know, would you kill a baby to save the world or like Yes. So these are you know heavy questions and and fundamentally they're never really about the scenario. They're always about yourself and what you do in it. And actually what we realize is we'll do everything for self sustain and actually that's what we want our AI to do is to help us to try and mitigate as much external damage as possible but also to save us as as us right the best worst case scenario the best worst case scenario is like yeah I think the perfect opportunity for a Billy Madison reference but business ethics business but um no while we're on AI um let's go or LLMs so data bricks released their own dolly LLM open source model, right? And and then I always had to correct people on this cuz I was like, wait, isn't that like Dolly the painting? No, no, not that one. Yeah, cuz there is the Dolly thing. This is this is like Dolly like a sheep. This isn't This isn't the Open AI Dolly image generator. This is the sheep. Dolly the sheep because it's a clone, the first sheep that was ever cloned. And uh I don't know if Dolly is still alive right now. Yeah. Yeah. I don't know if she's got that. Probably not. Probably not. Probably not by now. But yeah, so we have this really expensive. Great. I mean, so I I I even if I don't want to and and I I always want to, but even if I'm talking about something else, the whole world is talking about LLMs. Yeah. And we had no idea what they were like 3 months ago. Like to be perfectly honest, like well, I mean, we kind of knew what NLP was. And if you're in the business, you know a little bit about what large learning models are. You know what natural language processing is, but you never really got too invested into it because we all were deeply focused on on machine learning, on deep learning. That's really where we were. And then suddenly, I mean, I think the the techies out there were playing with open AI. I was playing with Dali back in the day because I liked it was generating gimmicky images. I was kind of excited about that. And then chat GPT came up and the whole world just just suddenly it's like that's a definitely an inflection point. That's what Yeah, definitely. Uh, so will the will will chat GPT be the technology that will change the world? I don't I don't think it I mean it's definitely got us talking about that. It might not be chat GPD itself. Um Open AI itself is not necessarily open because you have to pay for it. I liked how their uh what their CEO was on, you know, Congress basically saying that now it should be regulated now that I've got now that I've got a foothold, let's regulate the I saw yesterday they're on pays for a billion users already and it's like it's tremendous. It's tremendous like and actually in all honesty I suspect we all use chat GPT. Yeah, I use it for like most random things for like my kids like daycare stuff or like my kid he's in, you know, maybe you know for his like elementary school stuff like tell me how I can create a report on you know these things that you know for a fifth grader or something you know something like that. No, it's so flexible. Although it's interesting because I don't know if you told me about it or if Jake told me about it, but uh the the leaked uh Google memo, I think it was within the last month or so, but it was a memo from some Google exec seuite whatever basically that got leaked. It was an internal memo got leaked externally, but the summary of it was essentially uh we thought large language models were going to be better. So that's what we focused all of our time and efforts on and that was a complete failure and looking at you know open AI is clearly in the lead at the moment. However, we believe that open source is actually uh going to be the the route in the future because of how quickly it is accelerating things while also keeping things light like yes the whole thing was talking about how actually lighter models are being surprisingly just as effective as a lot of these big ones. Plus they're easier to run, they're faster, etc. But looking at it from essentially the open source community is allowing that the innovation and the the rate of innovation to change so quickly with this stuff that we don't and efficiently most importantly that we don't even have to necessarily throw a ton of R&D at it. we can just take some open source models and kind of do the Microsoft Google model, which is take somebody else's really good idea that got proven out and integrate it into everything that we have and make people rely on it, right? And so it's I'm I'm very curious to see how that plays out. I'm also curious on your kind of opinions on that just because open source has its place. So this comes up quite a bit, right? So let's go back to the the energy industry itself. So and everyone's data is fundamentally confidential and very important right. So what we are seeing is that there are challenges like that that people that have open AI and they have chat GPT as part of their you know corporate agreement is that they can't do their actual operational work in this thing because you can't share confidential data and client data and your own data in this. So you can do personal things, you have to lead the environment. But also the challenge is what we're seeing is that also is like how do you train the cost of training these models are quite expensive, right? So you know uh so Dolly our version our LLM out there you know we trained I think with like three with three computers out there and it was cost like $30 and we're able to train a set and you know it's a fairly young you it's a it's a young you know data set itself and it's you know doesn't have the maturity of obviously of of chat GPT was never designed to it's still growing and learning but the fact that we could do it with like you know a cost of less than $30 right means that anyone can do it rightc incredible and also being open source and following open standards means that the model itself can be interrogated and this is where kind of like bias comes into right so we can't eliminate bias I think from human nature that's a challenging psychological concept you know I don't think it's possible but we can certainly look into if bias has become an issue in terms of like you know giving results and decision-m like if the model is open you can interrogate it you can see where decisions have been made and you can train it right but if you can also but also if somebody's going If decisions are being going and results are being generated in one direction, you feel that that direction doesn't represent what maybe the the reality is of of of of an of an answer because remember generative gener anything generative is a probabilistic answer. Right. Yeah. It's never really 100% right. It's as close to it as could be. But if you feel that it's not really closer to what you should feel like, well, you can add to [clears throat] that model. Right. Right. And that's that's what it is. And and now we're seeing like especially in operators oil and gas operators they can they've got like wow I would say thousands of acronyms and words they use and historical information that's written on paper written on context ingest all that into an LNM model but you can train within your environment. So Dolly and and and the open source models that train within your environment and never have to leave your environment well then you also reduce the possibility of leaks right of actually you know [clears throat] errors that are coming out. And that's why I wanted to ask when I brought up Dolly like I mean because I feel like I high level understand like can I I can I take Dolly and I can train it on some geologic data or I can train it on or some interpretations that we've done and then can I feed you know new ones into that and it should be able to you know give outputs similar to what you know we trained them or I mean just trying to think fundamentally how how could I use an open source LLM like within my company or yeah I mean it's not really meant to be yeah you you you could do something like that. I mean you could train it right on different models and it is text based. I mean it's fundamentally giving you text out there. So it has to understand what's going on. So what you could do is actually ingest maybe not just the operational data around you know a thing. What about all the email communication? Yeah. All the reports around right. So let's say you want you want to basically get a report um let's say a drilling report. A drilling report. Daily drilling report. Right. A daily drilling report P on uh you know 100 wells, right? And like normally it just it'll just be a whole bunch of text out there. But actually, you know, with Dolly, you can probably ingest all that text and summarize that into something more contextual based on on based on what the expectation could be. Right. Right. This is it's like in agile dev, right? It's red, yellow, green, right? Like because in those morning meetings, they don't care if things are going well. They want to know where the bad things are. Exactly. What's behind slow down? It's not interesting. And they kind of want to detect something. And you can, you know, you can create a lot more in context demos and training and contextual information that's actually closer to kind of the language that the corporate language you use in your organization, right? Because a lot of the language that's used in the oil fields and and in service companies are not I mean I don't mean the grammar, I meant the actual terminology are not normally uh public information. They won't be in any of the training models that exist. It's jargon, right? The industry jargon is basically very specific. You could even have corporate jargon. Yep. That's specific to your your operations. Need an MBA library. [laughter] So in you can you can use NLP to ingest all that information and basically then start producing a model that can just you know give you a much more informed near humanistic response to that. Right. And what's important is what we see this is identified is that the oil industry is changing. The utility industry is changing in the sense that it is also this isn't an aegis comment but a lot of people with great domain knowledge have left the industry. They've moved on. They've taken their time on boats and they living great lives. Hopefully they moved on. And so you have a younger generation that uh are very smart but they don't have that in context domain knowledge and a lot of that is being lost. So that tribal knowledge that's missing you can use AI to basically and NLP and large learning model to maybe help you create tribal knowledge that can be trained. institutionalize it and no I mean that's so that's where we're kind of thinking where it'll be that's a I think that's a huge deal because I think that's I mean you look at any uh EY insert consulting company here report you know about energy executives and the things that they worry about and where and people and knowledge is like the top every year regardless and it's how do I retain the knowledge of these people that are retiring or that are leaving especially with in oil and gas specifically the you know crew change generational gap that there is um and I think like I don't think a lot of people realize that like this is a huge benefit of these models if you can internalize them with your company you know it's not just it's like we use notion right for our kind of uh wiki so to speak it's like the wiki now gets fed into a model and now the model knows everything about everything in your company wiki without you having to go find it or without someone who doesn't even know where it lives. They can just ask it a question. You nailed it right there, right? It's it it's reducing speed of time to find information like even with Google right now like like we didn't know we needed uh very interesting open you know we didn't know we needed chat GPT for information and because we had Google like who would have ever thought that that's how you start searching for things right like even though it's like a very to me intuitive way right like instead of me chopping up my Google search of like you know top restaurants yes Charleston South Carolina right like instead of that I can just tell it, hey, I'm going to South Carolina in this states and I want to eat at seafood restaurants. I would you could even have it more realistic like where can I find great seafood restaurant? Let's serve great calamari. Right. Yeah. Right. And then it'll probably pull up it'll probably take information from reviews and then put it up there and give you something. That's the most interesting thing with the the plugins yet or with GPT. I haven't used it with the plugins, but Colin has. And he said it's actually it's pretty interesting as far as like, you know, Yelp, for example, or Trip Advisor, like you've got all these, especially around like the travel stuff, right? Like being able to tell it, hey, I want to plan a trip to some kind of destination that's going to have good weather during this week, of this month, of this year, and I want to, you know, go to a sporting event, and I want to eat the best at the best steakhouse. tell me where and it'll do it. And it's now it shouldn't be used for like back you know in the industry like let's say you're looking at you know power turbines right or wind turbines you know in you know renewable or even you know turbines for the utility industry like it shouldn't be you shouldn't be able to say things like tell me how turbine 13 is doing right because actually it's not going to ingest the operational data it'll probably just ingest the reports around you that's where it's going to make sense and also it won't be real time because it has to be trained and models around it and it won't be you you know, minute by minute, right? But it's never designed for that. Yeah. It's designed to basically, I guess, summarize information, contextualize it, and make it easier for you to find things. But I mean, but I can see for like for legal a huge I mean you get these massive legal documents like can it you feed it that and then like give me the cliff notes you know like that's honestly like so in my masters I took whatever uh I can't remember the specific name of the class but it was legal and policy I think and uh you know one of the things of it is going through legal documents, MSAs etc. And a lot of that workflow is literally just this is from the hacks from my professors that they told us it's like control F and you have a list of keywords that you go and you search for to try and find in this 60page legal document right and it's like yeah feed this into a LLM and let it tell me the things that I need to be looking down to you know I don't speak 300 characters but you have to also be careful with LM and because what we've also found at L&M is as powerful and is fantastic. Even a chat GPT very confident even if it has no right to be. I found that this term recently and you know and I'm proud to say that I'm honest enough to say that I've never heard of this term but machines hallucinate. Mhm. This that's now apparently a term for it just like if it doesn't know something. I think it's like I think it's because it's still a kid. It's it's still growing and it's still basically we think it's this like mature professional out there but it's probably only in his third grade. Yeah. or like fourth grade kid, you know, and it's like I don't know the answer. I'm just going to make it up, right? And that's the risk. They start to hallucinate because they don't know. Sometimes they basically just kind of put stuff together. So, we have to be very careful as to like also, and this comes back to us as a as a population or species, like, and this is not just LM, but this is for all information that's generated, all generically put that in context of like it's a probabistic answer, right? How much of it are you going to trust based on your own knowledge? Yeah. And if you're hyper confident in the answer and it's wrong, then it's on you, not on the engine. It's a trust but verify trust but verify system for sure. What I mean, yeah. And the whole AI, I mean, I do want to pivot off AI at some point. We could spend all afternoon on this, but like for me, again, Chad GBT and some of these are things that we just know about like I mean like we don't know what Russia's working on behind the scenes. We don't know what China's working on behind the scenes or even our our own government is working on, you know, for their own. This is what's public. Yeah. This is what's public. I mean, and how's it going to be weaponized? I mean just with I mean now all the deep fakes and everything they're just get those are getting better like every day. I mean like it just how are we going to know what's real and what's not like so you know I think there was that letter that came out a whole bunch of like very famous people scientists stop progress on AI and it made no sense because for exactly that reason the bad actors were not going to we're not going to listen to right they were still going to go ahead and and even if you stop for 6 months and and like there'll still be a spike of like deep fake coming out there that's the it's to me it's very comparable to like not to be political but like the gun argument right it's like once the cat is out of back once the guns have been sold into the world like in the US you've got millions of guns out there it's very hard to to take it back to take it back once AI is into the universe and in this open source world the answer then comes and it's and it's also true for even the that example is is is the it has always come back to us as a society and how to manage it and I still think we have to re-educate ourselves absolutely right on as to how to trust but verify yeah that's where we're lacking and that's also true for you know managing you firearms, right? So, it has to do it's trust but verified, but it's education on us as to like, wait, wait, wait, wait, I just got a link that said this thing and it matches my exactly what I'm thinking about. So, it has to be true. No, follow where that source comes from. Yeah. Oh, it's a weird it's a weird URL. So, yeah, then obviously it's not a real thing. So, yeah, follow that. Is anyone else outside your bubble thinking that way? Right. And so, so that's how it is. And for I will say this example, right? So if there's a doctor out there that's using chat GPT to make diagnostics or patients, find another doctor. Yeah. Because like he could be it could simplify things, but actually like it's probably not accurate as well. if he's only using that and no doctor really is only using I think that's the biggest key because I I one of the case studies I used at at hivesell was a uh I think it was for breast cancer but you know a doctor like whatever groups of doctors would have a 70 to 80% chance of identifying early stage breast cancer from you know MRIs and cat scans and stuff uh not cat scans but um and then like the ML model that they had by itself would get like it was slightly less like 60 or 70% % but then the doctor plus the AI model was exceeded I think 90%. Yeah. And so like that is the power of this right that 10 10 to 20% delta is literally hundreds or millions of people's lives that get saved because of correct significant. So he also isn't just [clears throat] using only the only the yeah it's a tool it's a tool you still have to basically rely on your own kind of and the experts around you. So it's that trust but verify. It's I mean it's it's it's what they hammered on all of us in math, right? Know the bounds of your of your formulas. So, back to like society, the issue has never been uh I think AI will just exponentially grow into something that will be very powerful and we'll use it and it will have some negative impacts as well. That's obvious. That's with all technology natural natural and uh those impacts will be will have some dire consequences as well. But it it comes down to us I think to re-educate ourselves as to how to handle it. trying to stop it is impossible. It's already out there. It reminds me very much of like the early days of the internet like Napster and and things like that, right? Like this is new. It's exciting, but it's the wild west. And over time, guess what? [ __ ] gets figured out. Like we iron things out. We always hopefully always do. But uh no, like I saw uh I think it was a tweet yesterday about this law firm that has now been uh sanctioned by a US judge because in their uh documents which were come to find out written by some at least partially written by GPT it cited three court cases that don't exist. I've seen this. I've seen this exact exactly right. Yeah. Like that's that's they should get better lawyers first off. Yeah. That's that's a good example. You shouldn't you find a new lawyer because like Yeah. That's just said, oh, we pulled up some information and we didn't verify it and we went out there. Yeah. Yeah. A much lower risk uh kind of use case that I've also seen was Colin and I were doing something on it. We were trying to get it to analyze some kind of data set. And of course, it's not made for analyzing statistical data, but it was like, yeah, just send me it. We he asked it, how can I get you, you know, can I upload this data or whatever? And he was like, no, you can just send it to me. And it gave whatever, chatgbtgmail.com. And I was like, is this real? Are we going to be able to send it like a CSV and it'll pull the data in? And of course not. Of course not. Yeah. Yeah. It [laughter] did. And I was like, are you sure I can email you something? I was like, no, I'm a natural language model. And it's just like, what happens? Yeah. Exactly. Well, what if someone did or what if that was, you know, was hacked somewhere? Yeah. So, yeah, it's just insane. Um so to kind of pivot back to you know again traditional data bricks you know not data bricks in two years when you can do train your LLMs on it and do everything there but like you know say people watching this like can you give them some are there some uh case studies or use cases high use cases that I mean and names if you're able to or not but like you know say the end to end like you know data bricks experience like how people this problem implemented data bricks and then they got this result which led to x y or z. Yeah. So, uh I mean there's there's lots of it, right? If you want to think about you know the industry itself uh you know we have um hundreds I mean recently I think it's over 500 like energy customers around the world globally at different stages of it and essentially you know like the the the the use cases the challenges could range from depending on on what those clients do whether it's oil and gas renewables utilities you know but but the way we solve it is essentially the same for every single client right and it isn't into it usually starts with us the ones that we work better with that are more successful is the ones that focus on landing data in our platform in the data engineering phase on and we basically do the landing data so the way delta works is it has this medallion structure for for data so it has bronze silver and gold so all the raw the metadata and information will land in in bronze and then basically you have curation goes into silver and then basically gold is your business insight so that's usually the format we've seen for most successful uh clients is landing data, curating data and then basically creating this ETL pipeline. So the most successful clients we've had usually arrive are solve basically managing these kind of like ETL platform. Then we do the machine learning right as as it comes out of it. So so what are people using like as a new customer they're coming in they were using X Y and Z tools and now they can just use this one tool. What are some of examples of like you know kind of the what does it meal? Yeah. Yeah. So, so normally it would be things like um um PI data is kind of a traditional one. So they're usually trying to get data from either turbine, a piece of edge, some system out there. Exactly. And they want to land data and they find that the system that they have in place don't actually allow them to do analytics on there. So they've been landing data historians. the historian well you have to have historians to save data but they're not they were never designed for data analytics right so they had to save data on some environment and pull it into another environment and look at it there so with data bricks you can land data directly into delta and get that analytics in those in within those two or three stages you know and then basically get some idea of it which means you can get some visualization we even with like DBSQL of like how some piece of equipment is performing in near real time not real time because we an analytics platform form and you know we focus on we're very very fast uh but we don't sell ourselves as this you know you know I think that's a whole argument in of itself is what is real time data what does that mean to you near real time is what most people actually mean it's near real time and we need to be pretty and I think everyone needs to be the the honesty is it is near real time it's pretty fast so spark is batch generally right I mean like it's batch and streaming batch and streaming data coming into it and uh so data bricks can get batch and streaming into it we even have things like uh structured streaming so we can actually you know optimize the fact that in case that stream actually breaks down we can concatenate it again and merge it into a place right place. So we have this kind of technology in place to help you kind of manage that that batch flow. So in terms of like use cases, you know, we solve things like uh asset management, predictive management, uh predictive maintenance, we're working with customers towards condition based maintenance is going to help you that cost, supply chain, inventory management, um managing AMI data. So there's a whole range of these kind of use cases, but it fundamentally starts with getting data into into data inside format so you can get quick visualization, quick analytics on there, and then you scale up. Yeah. in getting all of the data into one usable place, right? Well, let let's talk about that. Should you get all data in one place? And the answer is well, it depends on your use case, right? Exactly. So like even with like machine data like if you can do just store the analytics version of it, right? You could actually get the bulk of that data which isn't that valuable for analytics later on. Yeah. Right. So you can you can minimize your cost from that as well. So there's some there's some ways you can help customers. We have helped customers kind of like pipeline clean up that architecture to reduce their cost. Correct. But if I have unstructured data and structured data and I know I need to use it, data bricks is a great place to be able get it all into one spot, get it ETL into what I want it and how I like it. Most of our customers have uh unstructured data coming in depending on on what the the media is and then of course we have a lot of like batch structured data coming in as well depending on on what they want to store here. So it could be a combination of operational data and financial data being stored of these things and curated and basically working with it. What's what's one of the more kind of exciting or interesting projects you've worked on in the energy space that you can talk about? Oh uh again don't have to be in detail but just generally speaking. Oh yeah. There was a um there was a VP from uh it was a it was I don't know if it's public or not yet but it's a super large operator and basically they were talking about uh how a single algorithm helped them reduce the the the cost for CO2 right and essentially it was data bricks working on them but they didn't start with actually trying to solve this this basically uh carbon offset problem they actually started with a supply chain an inventory problem. That's what the algorithm was designed for, right? So, as you know that, you know, inventory that's actually out, you know, inventory is not the most exciting uh thing to think about is actually thing. But in an offshore environment, you know, it's it's a million dollars of NPT per day for missing piece of equipment. So, vendors and operators would actually over supply inventory just to so they could prevent that problem, right? So, actually, and in order for you to kind of find out where the inventory is was taking 48 hours to actually process. Yeah. So we went from 48 hours down to an hour. That's a lunch break. So that's that's all the hours, right? So they were able to find out where the piece of equipment was there and they were able to, you know, you know, basically run the operation. So there's very little delay. So that's saving millions of dollars actually in it. No, that's that's so my before I got into energy, I was in uh well, not in before I got in oil and gas, I was still in energy. I was uh industrial manufacturing for wind turbine blades. Yeah. And you know industrial manufacturing or industrial engineering is really really focused on like optimizing every single piece of the manufacturing just process, right? Yeah. So I got big into 5S and six sigma and conbon and just in time and all these all those uh very interesting things. And then immediately from that, I went to work as a frack engineer and when prices were high and like I remember being on location many times and looking out the window and seeing like 10 sand trucks just parked charging to merge because something happened in the field. We got shut down. No one the information wasn't communicated to those trucks. So guess what? They came out on their schedule and now we're being charged out the ass for absolutely nothing that could have absolutely been prevented. But the risk of them not being right. Exactly. is much better is much worse, right? But then how do you balance off that thing and they actually the answer is data, right? It's actually you can do demand forecasting, you can make the predictions out there. Now the the client that actually made that quote about you know how we reduce the cost for transportation, it was the same algorithm, the same process that they used to scale across all their use cases, right? They use data bricks to actually do that. Kless pipelines reduce the communication kind of gap between, you know, handing off between data engineers, data scientists and stakeholders. So the projects and all the models were able to to be developed faster and that was really it and they scaled out. They were never design they weren't trying to solve it but actually what they found was because they were so efficient in supply chain management they weren't buying more inventory which means that their CO2 footprint was actually reduced because they supplying it. So there's a knock on effect of it. So you get the the ESG check box plus the the CFO isn't pissed off that you're over orderering on inventory. Exactly. So they they but we have some really cool examples like uh in the in the utility space as well. It wasn't it's not been yeah it was you know uh there was a client in in in North America which was trying to likeuling of rivers and their substations. So it's a it's auling issue ingesting weather pattern data coming into it because they had turbines going as well. Weather impacts a lot of decisions. So we were able to ingest the weather pattern data. We were able to then make uh predictions as to the demand for energy. Uh and then because we because of the way our ecosystem work, we actually brought a partner to kind of help us solve it because while we are very good at what we do, we don't have the answer to everything. So we can bring a partner to help you actually help us kind of solve these problems. They did and that's the the kind of ethos we like working with. We love working with our ecosystem. We love working with our partners. And that's uh that's where we how we successful. take notes all the proprietary format uh oil and gas tech companies that don't want to partner with anybody or anything like that. It's the it's the path to success. That's what everybody wants. Yeah. Yes. Well, I mean I think one thing I want to get in maybe more in the weeds as a developer type, but like um but I mean data bricks kind of handles all like because I mean before I say with the data lake you could do parquet and all stuff but you had to like set your partitions like on your blob storage and it kind of manage all that yourself like the data bricks platform as I kind of handles a lot of that for you right as far as optimizing the partitions you know. On the on the control side, we have a I mean the the notebook itself is a control and a control plane and the data plane and essentially on the control plane you control all of that. You can just basically launch a cluster and you can set up what kind of nodes you want to work on depending if it's multi node or single node depending on your your gravitas and you can basically just launch that and start working very quickly. Yeah, you can just focus on the the business logic. You f you focus fundamentally on actually what you that you care about which is you can spin up a notebook pretty quickly and you can actually start going in. And now that we're kind of working, we're also moving towards serless itself, you know, that makes it a little bit faster, more cost effective. So yeah, and this is almost a parallel between what we were just talking about on the physical side with like the just, you know, more just in time using what you need when you need it. You know, this is a kind of cloud versus on-prem thing, right? Whereas before with onrem, well, every now and then I need to use a supercomputer. So I need to buy like way way over provision the compute. Yeah. Exactly. And then Yeah. Exactly. And then what do you do with that hardware when it's done and the hardware basically becomes obsolete over a few years? If you needed more, you have to wait a significant amount of time to get pretty kind to like how we used to do things in the past because that's how it was done. And then I mean 10 years from now, we'll be doing things slightly differently. Back and say what were we doing back then? What were we doing back then? You know, and and I I actually think my prediction for the future will be that I think all the clouds will basically become like we'll be it shouldn't matter what cloud you're using. It should be like a you know you should be able to get access. It's basically the internet, right? It's a internet for data. It shouldn't really matter where you are because as a consumer it's like I'm here so I want to use the most powerful cost effective entity that exists in this bubble in this space because I'm just physically here. So it shouldn't really matter where that's because it changes so rapidly too. I mean like well and that's what that's where like a lot of my like blockchain interest comes from is this idea around like distributed compute and things like that, right? Like there's some interesting kind of companies out there, but one of them specifically, you know, you get paid in their token or whatever by allowing them to essentially run I can run it on my I have a node running on my laptop right now, but you just spin up the this software on your computer and now if somebody wants to watch video on this service, the video will buffer at the closest node on the network to their physical location. Yeah. Right. And just things like that where it's like if you think about, you know, if Apple had some kind of distributed compute uh SDK on all of their iPhones and you could then use everyone's iPhone as a giant super distributed computer like wasn't Amazon doing that trying to do with with their uh or it was more about sharing the network but like with their Echo devices like now there's the neighborhood thing or whatever and it's likely if you don't disable it they can basically share with anyone walking by who has also has Alexa X Like Comcast does that now with their Yeah, Wi-Fi. Xfinity Wi-Fi, right? Like I was I I didn't realize it until I was a customer and when I was sitting in my living room and I saw the Xfinity thing come, I was like, "Oh, so my router also puts out next Wi-Fi." And I was like, "This actually makes a ton of sense." Does AT&T do that? I'm curious. I don't They may. I'm not I'm not aware. I mean, yeah, cuz XT does that. Yeah, but it's it's a great concept, but you've got all this comput power. Why not use it? It's all around you network in your neighborhood. It's a mesh network. Yeah, exactly. and and that's really where we'll see things going and so yeah so I guess a mesh network of clouds exactly irrespective but then again it comes down to like well how do you once you get the network going how do you manage the complexity of of the data that's actually processing and this is where I still believe the open status open source of the the lake screen will work right now we are you know on three clouds I don't know what the future will will be for that so yeah maybe it will be a mesh network stuff I don't know that's just a prediction that's not yet no there's I mean it to me it's one of those things it's like man there's so many possibilities but it's also like there's also a lot of ways that could get really wrong. [laughter] Yeah. Yeah. Because you can yeah you can you can share data and you can you know comp that's just the nature I think of I'm being quite blas but that's just the nature of technology. It's very early in a lot of it, right? It's just it's it's fun to always play that game of what does this look like 10 years. But what uh I wanted to ask you so one of my questions actually is make a a bold prediction for uh oh for for uh tech in the energy space in five or 10 years. But I think that's actually a really good one. Yeah, I think the mesh network for cloud might be something. What is something that you know just either personally or with work whether it's a software tool open source a GitHub repo guy on Twitter what's something that you've recently found or gotten into that you're kind of excited about that some people may or may not know like our last guest talked about a Python library for drilling that he thought was really interesting. I mean, right now I've been I've been mostly focusing on LLMs, right? Because that's really been the only topic that everyone wants to talk about and that's that's that's I mean that's great, but I mean yeah, I've been trying to figure out like how to like learn how to do better and looking at the the different types of engines out there, modeling tools out there. So LLMs, I guess that would be something to think about. What uh prompting outside of uh GPT? Have you used any of the bard or actually just got invited to uh [ __ ] that's going to drive me crazy but have you used any of the other ones? Yeah, but I should I should be getting to some of those stuff here. So I won't put you on the spot and ask you what your favorite cloud is because it's you know precarious. Oh, that's easy. It's a lenticula. It's one shaped like UFOs. You see them out in North Texas. Touche. Yes. Well done. Well done. Keeping the dad jokes going. I love that. [laughter] That's what I'm here. I'm here for my hat on cumulus. I paid attention. Yeah, you definitely told that line very well. Yeah, there we go. Congratulations. What uh we'll jump into the kind of speed round with some of the stuff because I think we got yeah about 10 minutes left. What um what is your kind of go-to or favorite either new tool that you found or just existing kind of open- source tool? Delta, man. Delta is the way to go. Yeah, exactly. It's the core for what we're doing and it's where we're seeing success. Even though Delta sharing, that's a good answer. Actually, look at Delta sharing. That that would be something to think about. Okay. Yeah, that's that's pretty cool. That's us. What about board game or video game? Oh, uh that's easy. God of War 2. You play that? I haven't, but I've I've seen the uh the trailer. And how old is your kid? Uh four and six. So you will you'll weep when it's over. Yeah, you'll every man weeps, dude, when they play that game. Yeah, that's uh the first one was great. This is better. No, I just actually just recently jailbroke my kids Wii. Uh please don't shut me down, Nintendo. But I'm having a blast just down like playing old Wii games with them. Although that's I don't recommend that for if you have really young kids because that's a exercise in frustration with a three-year-old trying to teach them how to use a Wii controller. No, no, no. It's up here. No, it's down there. Swing of the ball. Swing it. Yeah. It's like No. Yeah. [laughter] Exactly. He just hands it to me. You do it. I'm like, "Okay." So, uh, yeah. Play God of War 2, man. Like, I mean, like we I expl conversation with my wife all the time about how she's like, "Why are you playing games?" like because it's a multi-billion dollar industry and they're now more powerful and eventually these video games will become the way we see movies going forward. Yeah. You had what The Last of Us uh really good TV show that was based off of video games. Video game. Yeah, exactly. Great game. That was a great Yeah, great game. I did not like the second one as much as I like the first one. Yeah. Uh but yeah, it's a great game and became a great great show. Uh I really like God of War. I was surprised the thing, you know, forget the action can't go to war and the playability, but the way they build the characters like, you know, it just says, "Wow, that's that's pretty good, right?" Yeah. It Well, and it's I feel like it's unique now because I feel like a lot of what I would call kind of classic successful games have gotten into more just like how do we pull more money out of people because whether it's skins or etc. But like the gameplay itself is very light compared to like the stories and the campaigns of five t years ago even. And so it's nice to see games that actually where you can tell the devs gave a [ __ ] and put a lot of thought. There's there's games out there. There's stories out there. Uh I think uh my first great example is like so Red Dead Redemption 2 made $700 million in its first week. I was like it's insane. That's an insane amount of money. Yeah, it's ridiculous. first week. Red Dead was awesome, though, because it was Grand Theft Auto Cowboys. The cowboys. [laughter] Have you played the second one? I think I did. I don't remember the ending, but it's pretty pretty like a real cowboy ending. But yeah, but like like you you like the characters, you enjoy the thing. And ultimately, I still think that's how movies will be interactive, right? You know, because like now with AI and stuff generating, you know, voices and actions and graphics, we can do stuff. And then that rolls into AR VR stuff too, right? Like in the future as well, like all that stuff. Uh, have you tried the I mean, I'm not trying to make make a picture, but the new PlayStation VR2s. Mhm. So, that's easily the best VR I've ever seen, even commercially. Yeah. Yeah. Because you don't get the headaches, you don't get the motion, but the quality is is there. That's something we went to. There's Inredies, which is like a I know. It's a high-end gaming place. Yeah. Yeah. And they've got all kinds of really interesting VR stuff there and it but I the first time I did it I got off of it and I was like oh my god like the the sick the motion sickness that you get after you only play for like 10 minutes per it's a fortune. Yeah. Yeah. But this stuff is at your home. So I've been playing for like an you can play at hour intervals because you get physically tired. Right. You know because it's it's very active. So you don't get you don't get sick. That's cool. Yeah. That's that tells you where we're heading towards. Yeah. Well, I saw F Meta just released their newest one for like four like it's the cost of a console now, which is wild. Yep. And uh then eventually in a few years it'll be commercialized and cheaper. Well, that's the thing. Wireless and then you had to have like a $3,000 gaming setup just to play. Well, look how cheap TVs are now. I mean, like I can, you know, I put a TV outside, it breaks down, I'm just going to go I'll go grab another TCL 4K from Target and throw it up there for 200 bucks. I don't know if you guys like remember growing up with one TV and if that broke like you you had to get a guy to come by and yeah [laughter] so that yeah the days over like you know that's that's who we are now right we'll just yeah is a 75 65 75 now like less than a thousand bucks oh yeah I mean like I can get less than 500 less I mean I got like a 50 something inch throw out back for under $300 like it's 4K I mean like I'm not a video file but I mean it's good enough for me or anyone else who's sitting out there watching just blow your mind. And it's all wireless, too. I can just go slap it like a like a frame on a wall. Just go a TV. You don't even need any like devices and stuff in there to connect to it. So, yeah, it's all Roku. So, it all plays in my account and Wi-Fi and we're good. Yeah. Shout out to Roku. If you want to sponsor, please let me know. I love Roku. I love you Roku. Yeah, it's pretty good. Yeah, that kind of thing. You an Apple TV guy or what? Uh, so that be that's because I think they make the best they make the best shows I've seen so far. Best quality shows. I love Apples. Apple TV. Yes. Shout out. So Ted Lasso saved Apple TV. If there's no Ted Lasso, there'll be no Apple TV. That's true. Like uh I was like really enjoying uh For all mankind, which is awesome. That's a really cool show. I started it. I haven't gotten really really great show. And then there's a whole bunch out there. Yeah. Apple. What is the the one that my wife and I just watched? Um Silo. No, it's the therapy one. Oh, yes. It's made by the Ted Lasso guy. Yeah. What is it called? It's called uh Shrinking. shrinking. Yeah, highly recommend that. I think I think I heard someone recommend that to us. Great series. And and FYI, we don't get paid for the saying. [laughter] Don't get paid for this yet. Uh but if uh Brett Gian wants to sponsor something, he he can come by. Yeah, cuz he's the Yeah. He's [snorts] the writer for that show and he plays uh he plays Roy Kent. Okay. I heard he was writer and he decided like, "No, I need to play this character for Roy Kent." I didn't know that. Yeah. So, he's the he's the he's the chief writer on um he's the producer and writer for uh Shrinking and for Ted Lass. Yeah, that's cool. All right. Yeah, definitely check out. He's making good stuff. Yeah. Uh it's a pretty somber it's a pretty dark like the there are definitely darker parts of it, but it's also just I like how real it is too, right? It's a great Yeah, it's a great show. Yeah. Yeah. So, I think Apple TV makes really good shows. I don't know how they do it and maybe that has to do with the quality of writing and the quality of shows and the actors and uh probably has to do with they have more money than half of the world [laughter] buy the right people. Yeah. We've said it before they can buy every uh professional sports team in America with their cash. Would they cash? Just the cash. Not a not a loan against their value, but just the cash they have on like they So they could buy [laughter] like like they could buy like the Lakers and they could buy maybe, you know, uh Yankees and the Yankees and B say, "Okay, I want you guys play. I don't want you guys playing anymore." Yeah. [laughter]

Just like crush the fan bases. That would be the ultimate troll. That would be incredible, actually. Well, like rig roll. Yeah. If Bezos or Elon were just like, I hate this team. I'm going to buy I'm gonna buy them off and say sell off all their players for nothing. Oh [laughter] my god. Like they could just That reminds me of the Texans, but we won't get into that. [clears throat] Yeah. Well, [laughter] uh go one or two more. Yeah. Um so outside of what you can visualize in within data bricks, what's your favorite visualization tool? Uh Spotify. Okay. Yeah. Yeah. I mean, but I mean because it's neutral, it's pretty cool actually. It's pretty cool tool. Yeah, it to me it's one of the easier to learn like get just Yeah, I think it is and it isn't. But I mean like I guess I'm also Yeah, but I mean Yeah, I mean I love it. I mean we use it. I mean it's just because again I can do my R or Python in it and you know or it's super flexible. It's flexible. It's easy to use. Also doesn't get in trouble with any of the cloud partners for not picking their Yeah. their specific technology. Yeah. Uh what what's a favorite book or an influential book that uh you know you you like either personally andor on the tech side? Oh [ __ ] [snorts] uh uh I was going to recommend like a data bricks book but no I mean uh oh this one stumped me one sting. So my my tech one is like crossing the chasm because I think it's such a you're not the only one that brought that up. Yeah it's a really good book. Yeah. You not read it? No. Oh, you got to read. You're not the only one that brought that up. Uh, yeah. Because it was written in the like early 2000s, maybe, but it's still so incredibly spot-on today. All it is is it's more psychology almost, right? Yeah. It's the it's the psychology about technology adoption. So, it doesn't matter what type of technology it is, but it talks about, you know, you've got your bell curve of users, right? And that far left side is the early adopters, the tinkerers that just like new [ __ ] right? like it doesn't have to work, but these are the guys that are going to go, you know, build the arduinos in their in their house. And then you get into the next part of the bow curve which is like the visionary which is more the kind of techsavvy business user or just in uh work user that understands the problems that the company faces but then also kind of can speak some of the technology language and then can kind of see okay this thing plus this thing in my business would solve this problem. But then there's the whole idea is crossing the chasm is after that visionary is what they call them. There's this chasm which then gets you into that 65% of the bell curve which is the early adopters, the late adopters and then the tail end is the lag. But it's it's fascinating because it it just gives you it gave me when I first read it when Bobby and I were working together when we first started doing our web app, but it gave me this understanding of like I don't need to go please every single user out of the gate or like add every single feature. the right people, right? You find the right people. It talks about, you know, they call it the beach head or bowling alley methodology of like develop one feature that works really well for a specific user base, then ask them what the like then discover what the next one is and then they all end up compounding on each other to the point that now you have a legitimate product, app, etc. that will appeal to that 70% of the the market that you're trying to get into instead of going in with just one little thing that solves one problem because it then breaks down the psychology behind each of those user types. Yeah. And how they make business decisions, how they make buying decisions, the thing like you know that first primary user thinks of things as in risk, right? So moving away from another platform is harder than it would be for a tinkerer or visionary, but it's a great one. Yeah. Okay. This was fun, man. I really enjoyed it. I mean, thank you so much for coming. Covered a lot. Great to Yeah, I know. We could We're definitely We could be here for a while. Yeah. We're going to do another one probably longer and have some folks in to just do like a round table around like AI and just different topics. Yeah. In fact, my my colleagues live in Houston. We spoke about this too. Get get one of them as well. Yeah, for sure. Yeah, they they're really good. Especially those either one of those two would be great. Mike, how uh how can people find you or learn more about I'm on LinkedIn like most people are. That's usually where it is. Yeah. And then is it databicks.com? Databicks.com. Yeah. Thanks again, man. Yeah, man. Yeah. Thank you so much. Thanks everybody. That's it. We'll see youall next time.