Technology Explorations in Data & AI

Emil Krause and Jonny Daenen explore the claim that speech-to-text makes you three times as fast. Emil shows the functionalities and benefits of SuperWhisper and Whisper Assistant, two innovative speech-to-text tools designed to enhance productivity. 

They discuss the installation process, user experience, and the accuracy of these tools. Emil shares insights on how these tools can streamline workflows, particularly for those who frequently interact with AI, and emphasizes the importance of context in dictation. It turns out that SuperWhisper gives much better accuracy when it comes to technical terms.

Resources:
  • Whisper Assistant: https://marketplace.visualstudio.com/items?itemName=MartinOpenSky.whisper-assistant
  • SuperWhisper: https://superwhisper.com/
  • Click here to watch a video of this episode.
  • Full playlist: https://www.youtube.com/playlist?list=PLJ_da7qdfL80rA7byzC_CmyrfJWjcCTnb
Note: This video is not sponsored or affiliated with Whisper Assistant or SuperWhisper.

Chapters:
  • (00:00) - Intro: Whisper Assistant vs SuperWhisper
  • (03:17) - Demo: Whisper Assistant
  • (07:27) - Demo: SuperWhisper
  • (10:24) - File tagging
  • (12:01) - Personal take: which one for you?
  • (14:17) - Installation & setup
  • (17:09) - Final thoughts & recommendations

Data & AI: Technology Explorations is a biweekly show from Dataminded. Each episode a Dataminded engineer demos a tool or technique worth knowing about -- working code, honest takes, no hype.

Music by Aleksandr Karabanov from Pixabay

Creators and Guests

Host
Jonny Daenen
Head of Knowledge at Dataminded
Guest
Emil Krause
Data Engineer at Dataminded

What is Technology Explorations in Data & AI?

Deep dives and practical demos on the technologies shaping modern data and AI development. Join the Dataminded team as we explore, unbox, and critically review the latest tools, from building AI agents and RAG systems to optimizing cloud costs and accelerating data pipelines. We cut through the hype to show you what actually works in real data engineering practice, complete with demo code!

Hi everyone, welcome to technology explorations at Dataminded. I'm Jonny, knowledge lead here at Dataminded, and today I have with me Emil, and we're going to talk to you about a technology called SuperWhisper and Whisper Assistant, which can help you increase your productivity so that you can talk instead of typing. In this session, we'll show you how to install it, how to use it, and also some pitfalls, and we'll end up with some next steps.

We won't go yet into security aspects, but we'll hope to give you an initial view into what this technology can do for you. All right. Hi, Emil. How are you doing today? Hi, doing good.

Thank you. You're going to talk to us about SuperWhisper and Whisper Assistant. I haven't seen this in action yet myself, so it's going to be new to me as well. What is it, in a few words? And why did you start looking into that? So these are text-to-speech tools.

They let you transcribe whatever you say into text. And I was interested in this, just looking for other ways to increase my productivity. So we have a lot of bash aliases. We have a lot of shortcuts. We find ways to streamline our processes. But I think, especially with AI, a big chunk of the day is you typing.

And talking and arguing with AI. So I thought that if this is such a big chunk of the day, and typing is slow, and I don't generally like typing, I don't know if people enjoy typing as such. So maybe that's something we could cut out and replace it with something that's more efficient. So this led me to the exploration of speech-to-text tools.

Yeah. So it's more than just, it's not coding with text, it's basically sort of dictation, and it's a dictation tool that should allow you to become more productive. Exactly. And also I was hoping to, because when we speak, it's easier to speak than type, I thought that I would give more context because I would have longer messages to the AI, which would in turn reduce the number of iterations I need to get to my result.

Because sometimes we give instructions, but you look back at your instructions and yeah, I really didn't give enough context. So I got something in corresponding quality back. Yeah. So I also hope to shorten this feedback loop or, yeah, and narrowing in onto the answers with AI thanks to this.

Yeah. And they claim, I quickly looked at their website, they claim it's like three times productivity. Is this like a general statistic? When you speak, you're faster than typing? Yeah.

You're right to be skeptical. I also was skeptical because it's a very prominent marketing claim. So first thing I did was to double check this. And it is really true. It's actually between three and I think three times, 3.8 times faster when you compare words spoken per minute to words typed per minute for an average typist.

Okay. This would be quite a productivity boost if we can get those speeds. So I'm actually curious what it looks like. So I propose we dive in. Yes. Let's start with Whisper Assistant.

So it's a free tool. So that's already very interesting. And it's a VS Code extension. So you simply install it in VS Code or Cursor and you will see this little thing appear here in the bottom right corner, Whisper. Yeah.

And when you press this, it will start listening to your speech and it will transcribe it. All right. So I have a couple of challenging commands here that we can try with Whisper Assistant. And then we can see, I think there are two dimensions. The first is the speed of the transcription and the second is the accuracy of the transcription. And the second is the speed of the transcription.

Because if we save so much time but wait 20 seconds for it to render, that's no longer three times faster. So let's try. Can you build these dbt models? I want you to coalesce all the columns with nulls in the last sync date table. And pivot this table and influence the monotone. Fine.

Fine. Fine. infer the schema remembering it's a parquet file. Can you find the lastingdate.sql file in my repo?

What VPCs are my RDS instances on? So now I click. So you gave it quite a few different commands in one go? I thought to just do it all at once and just to be more efficient and we can just compare.

So let me just new line separate commands. So it actually did really well on dbt. Previously no matter what I did it put dvd.

So I'm very positively surprised with this. This it never got right. It never does coalesce. It's always call us. And what did you say there?

Call us. Yes call us. Neat. Yes. So yeah difficult word. It doesn't know this word.

I think it's pronounced coalesce. No? I'm not sure actually. Maybe that's the reason it doesn't capture it. Okay but anyway that's something to try afterwards.

Maybe. And this it also doesn't get right. I said unpivot but it's unpivot. And for the schema it's a parquet file.

Yes. Parquet of course. Parquet. Yeah that's good enough. And it's also the acronyms that got correct.

Yeah and so it does capture a bit of technical jargon but not everything. Yeah I think it understands what an acronym is and it just transcribes it. It doesn't understand what it is but it's clearly missing.

A lot of technical jargon from this field but possibly also from other science fields. Indeed like DBT is specifically indeed an acronym and I always get corrected it should be lowercase.

My colleagues always tell me you need to write it lowercase so it is actually not understanding what DBT is or it doesn't have the technical jargon context it seems. Yeah. So like remembering this is a free model I think this is pretty good.

Okay this is a free version and so from my understanding Emil so you have this cursor which is a fork of VS Code in essence and you're now entering text you put your cursor in that prompting field and you dictate it a bunch of things you press stop and that session is then translated into text.

So in this case it's primarily a dictation tool right? Yes. Yes exactly. So as you said it also goes via Docker image that I have running locally for it to work and yes it's a good dictation tool correct.

Okay nice. So if you like as a benchmark we can also now try SuperWhisper the paid version to compare all these commands and see how it's doing.

Okay yeah and SuperWhisper is it related to this whisper you have here is it just a paid version or is it something completely different with a similar name? It's completely different with a similar name.

Okay. So for SuperWhisper it integrates into anything where you have an open text field and blinking cursor. So now what I'm doing I'm going to use a shortcut of option space. It's going to open SuperWhisper and then it's going to insert text into this active field.

Okay. So I'm going to say the same commands now with SuperWhisper the paid version. Okay. Unpivot this table and infer the schema remembering it's a parquet file. Can you find lastsyncdate.sql file in my repo?

What VPCs are my RDS instances on? Oh, yeah. So it's better, right? It got the lower casing right already. That's good. The dbt and the coalesce indeed.

Yeah, so like you see, it doesn't know what the actual name is, but kind of did the was it camel camel case? Yeah, it basically understood like this is a technical term and it put it. Exactly. And, and you also have Yeah, I mean, it's basically perfect. You have unpivot you have parquet you have dot SQL dot wasn't did you have a dot as a word here?

Yeah. So you see, so it's, it's really, it's very smart. Yeah. Okay, so in this case, this seems like to be a system wide tool that you installed you spawn it using a hotkey, you dictate as well. And then when you're done, it pasted in your active field that is in focus.

So you could use this beyond VS Code, I assume. Absolutely. You can use it for to write slack messages, email messages, literally anything where, where you have typing inputs. Yeah. Okay, this is nice.

And so you could feed this into your prompt, you could write comments to it. But I suppose again, code will be challenging to write using your voice, right? Because this is different with all the special characters we are using typically. All right, very nice. And what I could also demo is file type.

Yeah. Yeah. So I can do a little bit of file tagging, because the first thing when I was using this, I realized that, you know, right now it's very, it's very easy, you know, you can just find your files. But how do I use this tagging functionality when I'm speaking?

And the answer is that you cannot, that's, that's not possible. But it's very curious to see if it will still figure out via fuzzy search, what file I'm actually referring to on my file system. So you can try the few commands here. And and as well, let's start with with persistent. What hooks are included in the pre commit config dot YAML file?

Let's have this transcript just for later. So you see, it's a little bit more complicated than you might think. So you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that you can see that In this case, the Cursor AI understands it's about a file and it will do the effort itself.

Exactly. So it did fire. I mean, it was almost a perfect match, but it found three of them because I have three projects. So it is not a big hurdle that you cannot tag files. Okay, very nice.

And are there things that you noticed that you were amazed about when you use it? So let's say you use it and you're like, okay, this is really cool or things that you were not really happy with? Yeah, I was really amazed by the accuracy of SuperWhisper.

I thought it's really impressive. Sometimes when I stutter or I misspeak in a sentence and it actually doesn't even record it, it understands I was misspeaking and correcting myself. So you actually get this full, intense, intended sentence that you meant to say without you kind of starting a half word and not finishing it.

It was really amazing. So you mean you can self-correct during when you speak and then it only catches the final result? Yeah, exactly. If you self-correct mid-word or if you got the wrong thing and then it knows you made a mistake and you self-corrected yourself and it only shows the corrected version.

Yeah. And I mean, my experience is overwhelmingly positive. There are some hiccups with the Whisper Assistant, but again, it's a free version. So I think we shouldn't be that demanding. But overall, overwhelmingly positive.

Yeah. And if you would use this yourself in a day-to-day fashion, which of these two would you prefer? Is it good enough with the Whisper that you have locally running? For day-to-day tasks? Or would you then still opt for a paid version?

Yeah, it's a good question. I think probably the free version would be enough for me. Because even if it makes a small mistake, you can always correct it. It doesn't send it automatically to your AI. So you can correct I'm pivot to unpivot and then it's not such a big deal.

I don't think I write that many emails. Or that many Slack messages to warrant using SuperWhisper. And it's great integration possibilities. Okay. Let's maybe go to a few more deep dive questions. My first question would be, how do I install and run this?

Yes. So with SuperWhisper, it's very easy because it's just a MacBook application. So you go, you download it, you double click and you're done. So there is not... Not much to show. With Whisper Assistant, it's a little bit more tricky.

Because first of all, you download the extension. So that's quite easy. But it's still not going to work if you just hit this button. Then you have instructions on the GitHub page. And there are essentially two steps you need to take. First step, you need to have some Homebrew package installed.

And then the second step, is that you need to have a Docker container running. So you need to run this command. And from that point on, the extension is going to work. With no further actions required. And so every time you boot your Mac or your device, you will need to spawn the Docker.

You start a Docker image as a container. And very often I speak for a minute and I hit it and it says, is your Docker running? I'm like, yep. Nope. I just wasted that minute speaking to it.

So yeah, you have to remember. So that means in the end, your VS Code or your Cursor is buffering all the speech. And then that's being done by the extension, I assume. Then it's being sent off to the container, which transcribes it and it's fed back and then pasted into your input field. Yes.

Okay. But this is fully local. So that means it works when you're offline. And also nothing is, I assume, sent somewhere else to the cloud or to some really weird service or something. Yeah, correct.

There is an option to have remote transcription here if you add an API key, but that's not something I export. Okay. But it does mention this. And I don't think we give any guarantees on this package being secure or anything. No, this is...

The philosophy is that it's local. You can inspect all the code, I assume. And you don't... You don't depend on a service and it's free. Could you also maybe show the website of SuperWhisper? So you download it here.

Actually, see, it's also for your phone. That could be interesting. Yeah. So yeah, you just click here and it's just a seamless installation. And then you have it here where you can customize. Okay.

Okay. Maybe to some conclusions, what are your takeaways on these tools, on SuperWhisper and Whisper Assistant? So my takeaway is that possibly most people who work in the correct environment should use them, especially if they talk to AI a lot, which I think a lot of people do now.

And I think it's... It's... It's a huge productivity boost that even if you pay $9 a month and you save 20 minutes a day, that's already definitely a good investment. So I think...

I think it's something I think more people should use and I should use more as well. And it's a bit different field where you can seek productivity increases. And... And... And yeah.

I think it's... It's the way to go. And I recommend. Even though I haven't fully embraced it, I recommend. Okay. From what I've seen, my takeaway is essentially you have the local version, which is not super accurate, but it's locally hosted.

So it gives you a bit more, well, feeling of control when your data doesn't travel anywhere, essentially. At the same time, you see SuperWhisper, which is a paid option, and gives you higher accuracy. It gives you system-wide integration. And it gives you more options towards integration with LLMs.

So my preference would go to that because it's more system-wide. We didn't talk about the price, or did we? Well, you mentioned it's in between the lines, I think. What is the price of SuperWhisper? So it's advertised at $850 per month. And I actually think maybe it's worth paying just so we have more...

more of a motivation to adopt it. As it is with any free tool or anything you get for free, generally, the perceived value is lower, irrespective of its objective value. So I think if you're into this, anything like this, this could be cool.

Just pay for it and see if you manage to form a habit easier just because you have some skin in the game or money in the game. All right. Thanks a lot for showing us this. And thanks for sharing this technology, Emil. It was a first look.

We didn't go too deep into, as I said, security or enterprise-y things. It's basically an initial look into the technology. Thanks a lot for explaining this. And I hope to see some new technologies from you soon. Yeah, for sure. Thank you.

All right. Thanks, everybody, for watching. And I hope to see you in the next video. Bye-bye.