1
00:00:01,083 --> 00:00:04,958
You open up your laptop and it's time to check the overnight data pipelines.

2
00:00:04,958 --> 00:00:08,000
You open up a browser, you log in and check the pipelines.

3
00:00:08,000 --> 00:00:12,583
You open up another tab, you log in again, different environment, same thing.

4
00:00:12,875 --> 00:00:18,541
12 environments, dev production, that's 24 environments every single morning.

5
00:00:18,625 --> 00:00:28,125
My colleague Jan got tired of that, so he built a terminal user interface called Flowers
that allows him to manage all of his environments at once, from his keyboard, no mouse, no

6
00:00:28,125 --> 00:00:30,208
browser, no nonsense.

7
00:00:30,250 --> 00:00:36,083
And in this episode, Jan is going to show us how that works and how you can build
something like this yourself in Rust.

8
00:00:36,416 --> 00:00:41,208
I am Jonny, knowledge theory at Dataminded, and welcome to Technology Explorations.

9
00:00:49,708 --> 00:00:54,791
Hi everyone, today we'll have a look at text user interfaces or terminal user interfaces.

10
00:00:54,791 --> 00:00:56,166
For that, I've invited Jan.

11
00:00:56,166 --> 00:00:56,833
Welcome Jan.

12
00:00:56,833 --> 00:00:57,458
Thank you, Jonny.

13
00:00:57,458 --> 00:01:00,666
Could you tell us a bit more about yourself and your role at Dataminded?

14
00:01:00,666 --> 00:01:04,583
I'm working here at Dataminded as a data engineer.

15
00:01:04,833 --> 00:01:11,458
I'm a team lead currently at one of our clients and I'm also heading uh our academy.

16
00:01:11,500 --> 00:01:14,750
Okay Jan, so you have been building a terminal user interface.

17
00:01:14,750 --> 00:01:17,083
Maybe show us what you built

18
00:01:17,083 --> 00:01:17,458
Sure.

19
00:01:17,458 --> 00:01:23,458
I'm in my terminal and in my case the tui is called Flowrs and you can start Flowrs by
just typing Flowrs.

20
00:01:23,875 --> 00:01:33,208
The name Flowrs is a pun on Apache airflow and flow Flowrs and Because it is written in
rust a lot of rust projects.

21
00:01:33,208 --> 00:01:37,916
They have this RS suffix So that's where the name comes from Flowrs

22
00:01:37,916 --> 00:01:41,958
And so for our viewers, Airflow is an orchestrator for data engineers.

23
00:01:41,958 --> 00:01:44,875
Could we maybe have a look at what the UI looks like normally?

24
00:01:44,875 --> 00:01:45,333
Sure,

25
00:01:45,333 --> 00:01:47,250
So this is the Airflow UI.

26
00:01:47,250 --> 00:01:54,333
And in Airflow, you have your DAGs, your directed acyclic graphs, which represent your
workflow, your data pipelines.

27
00:01:54,333 --> 00:01:58,083
And you can inspect them and see how they are running, what they are doing.

28
00:01:58,083 --> 00:02:01,291
So here you have an overview of all of your DAG runs.

29
00:02:01,291 --> 00:02:01,791
Each...

30
00:02:01,791 --> 00:02:03,791
vertical line is a DAG run.

31
00:02:03,791 --> 00:02:07,333
Each square here is a task your pipeline.

32
00:02:07,333 --> 00:02:15,208
You can also visualize it as this DAG, as this directed acyclic graph of all these
operations that will get scheduled by Airflow.

33
00:02:15,208 --> 00:02:19,583
And so Airflow is basically the babysitter of these DAGs, of these pipelines.

34
00:02:19,583 --> 00:02:23,583
And as you can already see, to navigate in this UI,

35
00:02:23,583 --> 00:02:25,375
First of all, there's a lot going on.

36
00:02:25,375 --> 00:02:27,041
There's a lot of things that I can click.

37
00:02:27,041 --> 00:02:30,250
There's a lot of different buttons and tabs.

38
00:02:30,250 --> 00:02:39,500
What you've now seen was one Airflow environment, one Airflow instance, and that's still
manageable if you need to navigate through the DAGs and through all of the pipelines to

39
00:02:39,500 --> 00:02:45,250
see after a night of batch runs, which pipelines have failed, where do you need to take
action.

40
00:02:45,250 --> 00:02:53,541
But if you have 10 or tens of these environments, it quite quickly becomes a hassle to do
this through the UI by just clicking.

41
00:02:53,541 --> 00:02:56,750
And I got sick and tired of doing that, and that's why I built the Tui.

42
00:02:57,166 --> 00:02:57,583
Yeah.

43
00:02:57,583 --> 00:03:03,500
So you hate clicking around and jumping across all these environments because it takes a
lot of time.

44
00:03:03,875 --> 00:03:06,666
And as a data engineer, you manage multiple ones of these.

45
00:03:06,666 --> 00:03:13,500
Like in this case, we see 12 environments, and you have a development production, you need
to check all of them.

46
00:03:13,500 --> 00:03:15,833
So that would require 12 tabs being opened.

47
00:03:15,833 --> 00:03:20,625
And then even in those environments, you manage multiple DAGs, lots of work.

48
00:03:20,625 --> 00:03:26,250
lots of clicking, of exactly, lots of digging deeper to figure out what happened.

49
00:03:26,250 --> 00:03:27,791
And I wanted to make this easier.

50
00:03:27,791 --> 00:03:32,750
Yeah, so you built your own terminal user interface called Flowrs to navigate it.

51
00:03:32,750 --> 00:03:36,916
So could you show us around a bit in the terminal user interface on how you use it?

52
00:03:36,916 --> 00:03:37,791
Yeah, sure.

53
00:03:38,000 --> 00:03:39,833
So let's drill down on an environment.

54
00:03:39,833 --> 00:03:44,250
So when you go into an environment, you have, first of all, the list of DAGs.

55
00:03:44,250 --> 00:03:51,458
So these are your pipelines, your workflows, and then you can navigate through the DAGs
just using your keyboard.

56
00:03:51,458 --> 00:03:53,125
can use VIM key bindings.

57
00:03:53,125 --> 00:03:56,375
So meaning J is going down, K is going up.

58
00:03:56,375 --> 00:04:03,916
You can also jump to the top of the table by using gg, or you can go to the bottom with
capital G.

59
00:04:03,916 --> 00:04:12,000
And you can also filter if you just quickly want to find a DAG that you're interested in
by using the slash command and then you see a filter popping up.

60
00:04:12,000 --> 00:04:18,041
at the bottom and then you can look for your favorite DAG and in my case I want to have a
look at this DAG.

61
00:04:18,041 --> 00:04:23,333
If you then select the DAG, you get to see all of the DAG runs of the DAG.

62
00:04:23,333 --> 00:04:29,208
So a DAG run is an instantiation of a DAG and DAG is this concept of a workflow or
pipeline.

63
00:04:29,208 --> 00:04:31,125
DAG run is a run of this pipeline.

64
00:04:31,125 --> 00:04:33,000
You get some information about this DAG.

65
00:04:33,000 --> 00:04:33,791
When did it run?

66
00:04:33,791 --> 00:04:35,125
How long did it take?

67
00:04:35,125 --> 00:04:35,750
There's

68
00:04:35,750 --> 00:04:40,000
a line gauge to show how long it took in comparison to other DAG runs.

69
00:04:40,000 --> 00:04:48,250
And if you then drill down, you can see all of the individual tasks of your DAG run to see
how long did they take, did they run successfully.

70
00:04:48,250 --> 00:04:56,333
And then even at the run or the task level, you can drill down and see all of the logs of
that specific run.

71
00:04:56,333 --> 00:04:58,000
I also see a follow mode here.

72
00:04:58,000 --> 00:05:04,375
So you would be able to have a trail and follow along when the pod is running in this
case, where the task is running.

73
00:05:04,375 --> 00:05:12,916
Exactly, So it basically continuously polls for new logs and then it will follow along so
you don't have to do anything.

74
00:05:12,916 --> 00:05:22,916
while you did the drill down, I saw these tabs at Every time you went a bit deeper and you
go from the top level where you have all your instances to your DAGs, to the runs, to the

75
00:05:22,916 --> 00:05:24,416
tasks, to the logs.

76
00:05:24,458 --> 00:05:28,375
And how do you effectively do use this in your day to day?

77
00:05:28,375 --> 00:05:36,083
In my case, what I typically do, there's a few workflows, but we typically get alerted of
important pipelines when they fail.

78
00:05:36,083 --> 00:05:38,666
This is happening in our alerting system.

79
00:05:38,666 --> 00:05:49,208
It's pager duty, And then I just look at the name of the DAG in the alert and I navigate
to the right environment and I start searching for that DAG to see uh it failed.

80
00:05:49,208 --> 00:05:52,083
And then I drill down to see what failed, why did it fail?

81
00:05:52,083 --> 00:05:57,250
And as I mentioned before, you get the application logs for a failed task instance.

82
00:05:57,250 --> 00:05:58,083
So I can easily

83
00:05:58,083 --> 00:06:00,083
figure out what went wrong and try to fix it.

84
00:06:00,083 --> 00:06:00,500
Yeah.

85
00:06:00,500 --> 00:06:03,375
your trigger is an error from a pipeline.

86
00:06:03,375 --> 00:06:05,291
You get an alert, there's something wrong.

87
00:06:05,291 --> 00:06:11,375
And this allows you to immediately dive deep into that error without opening a browser,
opening the right environment.

88
00:06:11,375 --> 00:06:13,583
So it saves you a bit of time, I guess.

89
00:06:13,583 --> 00:06:14,541
Yeah, it does.

90
00:06:14,541 --> 00:06:16,625
can also, if I know which DAG it is,

91
00:06:16,625 --> 00:06:21,500
Here there's a bunch of successful tasks, but I know that there's also a couple of failed
ones.

92
00:06:21,500 --> 00:06:23,833
And then I can also filter on the state.

93
00:06:23,833 --> 00:06:27,166
You also see hopefully that there's like this autocomplete.

94
00:06:27,166 --> 00:06:32,375
It will also cycle if I start typing failed, it already knows.

95
00:06:32,375 --> 00:06:36,125
And then I just have an overview of the failed tasks of my...

96
00:06:36,125 --> 00:06:44,000
DAG run in this case and then I can have a look at the logs and see okay here's some a dbt
job is running and one of the tests actually failed

97
00:06:44,000 --> 00:06:44,416
Yeah.

98
00:06:44,416 --> 00:06:48,958
And also these things like this auto-complete, you built that yourself in this UI.

99
00:06:48,958 --> 00:06:53,291
yeah, the autocomplete is basically a state machine.

100
00:06:53,291 --> 00:06:59,208
It generates the possible attributes of a DAG run or of a task that you can filter on.

101
00:06:59,791 --> 00:07:00,416
very nice.

102
00:07:00,416 --> 00:07:03,750
I think this looks quite slick actually, this UI.

103
00:07:04,208 --> 00:07:06,166
Anything else you'd like to show

104
00:07:06,625 --> 00:07:11,500
Actually, yes, on the UI side, Jonny, I recently added something based on a request of a
user.

105
00:07:11,500 --> 00:07:22,125
So we all know, and you also know that any self-respecting developer uses a dark mode
terminal, but there are some people that have maybe some color issues, color blindness,

106
00:07:22,125 --> 00:07:26,500
and they sometimes prefer a light mode terminal.

107
00:07:26,500 --> 00:07:35,666
So when you would open Flowrs in a light mode terminal, looks at your terminal, tries to
figure out are you in a light mode terminal or in a dark mode terminal.

108
00:07:35,666 --> 00:07:37,375
uh

109
00:07:37,375 --> 00:07:40,958
Because terminals, do they pass on the information in the environment?

110
00:07:40,958 --> 00:07:45,958
well, I cannot give you the full details because I used a dependency to do this.

111
00:07:45,958 --> 00:07:51,500
There is a crate called Terminal ColorSaurus and I like the name already, but that does it
for you.

112
00:07:51,500 --> 00:08:01,083
And if you look at the readme of the project, it seems like it's people really knowing
what they're doing and really knowing terminals inside out, looking at specific escape

113
00:08:01,083 --> 00:08:02,916
sequences and C codes.

114
00:08:02,916 --> 00:08:04,291
It just works in my case.

115
00:08:04,291 --> 00:08:07,416
It correctly can figure out that this is a light mode terminal.

116
00:08:07,416 --> 00:08:11,166
So the extra feature is light mode where we used to see dark modes.

117
00:08:11,166 --> 00:08:13,708
You have a light mode as a new feature.

118
00:08:13,708 --> 00:08:14,500
exactly.

119
00:08:14,500 --> 00:08:22,916
And not only light modes, you can also, because of course we have to over engineer
everything, you can also select these Catppucin themes.

120
00:08:23,125 --> 00:08:24,708
like latte, frappe, macchiato.

121
00:08:24,708 --> 00:08:28,375
And they each have a slightly different color scheme.

122
00:08:28,375 --> 00:08:29,958
And you can also enable

123
00:08:30,000 --> 00:08:41,958
uh Flower supports both Airflow V2 and V3, which is something that was quite useful for us
because we have many of these environments and we were migrating these environments from

124
00:08:41,958 --> 00:08:43,250
V2 to V3.

125
00:08:43,250 --> 00:08:46,125
But we still wanted to manage our DAGs in the same way, right?

126
00:08:46,125 --> 00:08:50,208
We still wanted to navigate through our DAGs, find the failed task instances.

127
00:08:50,208 --> 00:08:54,916
And it kind of looks the same for both V2 and V3 Airflow.

128
00:08:55,166 --> 00:08:55,416
So

129
00:08:55,416 --> 00:08:59,125
So you provide more UI stability than Airflow itself at this point.

130
00:08:59,208 --> 00:09:01,541
Yes, but also less features.

131
00:09:01,541 --> 00:09:04,500
In the Airflow UI, apparently now you can play Doom.

132
00:09:04,500 --> 00:09:06,625
That's something you cannot do with Flowrs.

133
00:09:06,625 --> 00:09:07,541
nice not yet at least.

134
00:09:07,541 --> 00:09:12,000
I wanted to show this escape hatch maybe not everything you want is visible

135
00:09:12,000 --> 00:09:13,375
within Flowrs.

136
00:09:13,458 --> 00:09:22,958
So you can press O on any object on the DAG, on the DAG run or on task instance, and it
will open the Airflow UI of that environment.

137
00:09:22,958 --> 00:09:29,666
you can easily, if you need more features or more graphical features, you can easily dive
in.

138
00:09:29,666 --> 00:09:30,166
Nice.

139
00:09:30,166 --> 00:09:33,791
if you press V, it shows you the code of the DAG and you can

140
00:09:33,791 --> 00:09:45,041
one last thing I would like to show, which I think is quite cool, is if you have a bunch
of failed DAGs, for example, and you want to all mark them as successful, you typically

141
00:09:45,041 --> 00:09:48,416
would do this if there's nothing really actionable about your failed DAG.

142
00:09:48,416 --> 00:09:56,625
Maybe your DAG has failed because there was a timeout or something, but you know that it's
fixed now, and so you just want to mark your DAG as successful.

143
00:09:56,916 --> 00:10:06,125
You can press shift V, you go into visual mode, just like in vim, you can navigate and
that basically selects a bunch of these, in this case, DAG runs.

144
00:10:06,125 --> 00:10:11,916
And you can then press the button to mark them as success, failed I want to mark them as
successful.

145
00:10:11,916 --> 00:10:18,250
So in terms of managing such a large workload or multiple environments, this is a lot
faster than going to the UI.

146
00:10:18,666 --> 00:10:21,250
I see tuis popping up everywhere.

147
00:10:21,250 --> 00:10:22,833
Where is this coming from?

148
00:10:22,833 --> 00:10:25,250
that's a good question.

149
00:10:25,250 --> 00:10:30,458
terminal user interfaces were the OG interfaces of our computers, right?

150
00:10:30,458 --> 00:10:35,250
You used to have only a terminal and some text on it.

151
00:10:35,250 --> 00:10:38,791
This is a VT100, one of the first tele typewriters.

152
00:10:38,791 --> 00:10:40,958
That's what you got, black screen and some text.

153
00:10:40,958 --> 00:10:45,208
And that used to be more than enough for a lot of use cases.

154
00:10:45,208 --> 00:10:48,166
You also have an example of MS-DOS.

155
00:10:49,000 --> 00:10:57,041
Also the first games, I really like this kind of trivia, the first games were also just
text-based games And I think as a developer,

156
00:10:57,041 --> 00:10:59,625
Yeah, you kind of like working in a terminal.

157
00:10:59,625 --> 00:11:02,291
You feel efficient, you feel fast.

158
00:11:02,291 --> 00:11:08,416
If you have to navigate through a file system, I think most developers would say they
prefer doing that than clicking around

159
00:11:08,416 --> 00:11:11,583
You have this element of speed, of being efficient.

160
00:11:11,583 --> 00:11:12,875
it's a lot less.

161
00:11:12,875 --> 00:11:17,208
convoluted and a lot less busy than the typical web UI.

162
00:11:17,208 --> 00:11:22,583
Although of course, if people properly design web UIs, they can also be simple and
focused.

163
00:11:22,958 --> 00:11:25,833
And then there's also these retro aesthetics.

164
00:11:25,833 --> 00:11:27,875
It's more of a personal thing, but I like it.

165
00:11:27,875 --> 00:11:30,833
and it's also available on many systems, right?

166
00:11:30,833 --> 00:11:35,208
Like there's always like a bash terminal available that you can use.

167
00:11:35,500 --> 00:11:36,083
Indeed.

168
00:11:36,083 --> 00:11:43,416
Where I also think it shines is in for these kinds of applications where you have a big
surface and like Airflow is a good example.

169
00:11:43,416 --> 00:11:48,583
You have a lot of things you can do with Airflow with the UI.

170
00:11:48,583 --> 00:11:52,166
There's also an Airflow CLI, which is also quite a big CLI.

171
00:11:52,166 --> 00:11:53,166
You can do a lot of things.

172
00:11:53,166 --> 00:11:57,125
What is then the difference between CLIs and TUIs according to you?

173
00:11:57,125 --> 00:11:59,625
When would you opt for each of them?

174
00:11:59,625 --> 00:12:01,125
a very good question.

175
00:12:01,125 --> 00:12:07,041
And also it can be tied to this new wave of agentic AI and agents doing things for us.

176
00:12:07,041 --> 00:12:10,875
A tui is a very visual thing, right?

177
00:12:10,875 --> 00:12:14,500
We humans are very visual creatures, right?

178
00:12:14,500 --> 00:12:17,416
And we easily spot small changes in uh

179
00:12:17,416 --> 00:12:23,666
in a visual thing, reading a bunch of text, which is typically what comes out of a shell
command that takes time for us.

180
00:12:23,666 --> 00:12:25,083
We don't read that fast.

181
00:12:25,083 --> 00:12:27,916
Now agents do read very fast, right?

182
00:12:27,916 --> 00:12:34,833
So you have all of these agent skills and a lot of agent skills wrap around the CLI, they
use CLIs.

183
00:12:34,833 --> 00:12:37,958
And I really think that's a beautiful marriage between the two.

184
00:12:37,958 --> 00:12:42,041
Agents can read super fast, the CLIs just give text back.

185
00:12:42,041 --> 00:12:46,666
But as a human, you are, I think, more efficient by just having something visual.

186
00:12:46,666 --> 00:12:56,333
In the case of Airflow, it's way easier to spot one red dot on your screen to show that
something has failed, and then read the output of one CLI command.

187
00:12:56,333 --> 00:13:01,833
Yeah, so your Flowrs TUI does it mean it's not as suited for Agents to use?

188
00:13:01,833 --> 00:13:03,625
Yeah, maybe it could be used.

189
00:13:03,625 --> 00:13:06,250
I'm not sure if it just will be very efficient.

190
00:13:07,083 --> 00:13:10,500
And so what is then the reason you designed Flowrs?

191
00:13:10,500 --> 00:13:14,875
Because now we have agents, they can talk to CLIs, you could just ask a question.

192
00:13:14,875 --> 00:13:20,083
Do you still have the need for a speed up that you cannot achieve with agents?

193
00:13:20,416 --> 00:13:31,000
Well, first of all, I started developing Flowrs before there were agents and it was born
out of frustration with having to click in the UI and also curiosity.

194
00:13:31,000 --> 00:13:33,583
I like to know how things tick.

195
00:13:33,583 --> 00:13:35,583
Like how does it actually work?

196
00:13:35,583 --> 00:13:37,791
What does it do?

197
00:13:37,791 --> 00:13:42,458
So I started building this before the whole agentic revolution,

198
00:13:42,458 --> 00:13:46,333
agents already replacing Flowrs in my day-to-day workflow?

199
00:13:46,333 --> 00:13:51,458
Not yet, but I also don't really have a good reason or argumentation of why they couldn't.

200
00:13:51,458 --> 00:14:00,041
But what I think you're still faster when doing it on your keyboard because if you know
your Flowrs then you can easily jump into things that you know.

201
00:14:00,041 --> 00:14:02,875
So it's like second nature to you, seems.

202
00:14:04,000 --> 00:14:06,291
You still win of the AI in this case.

203
00:14:06,375 --> 00:14:09,041
Still out competing the bots.

204
00:14:09,041 --> 00:14:09,916
Yes, that's true.

205
00:14:09,916 --> 00:14:17,666
But if cost is not of a concern, because in the end you can indeed tell an agent, hey,
here's the airflow CLI, figure it out.

206
00:14:17,666 --> 00:14:22,708
And it might take half an hour to figure it out and burn a bunch of tokens to do it.

207
00:14:22,708 --> 00:14:24,958
that might be a trade off you're willing to make.

208
00:14:24,958 --> 00:14:26,458
how do you build such a system?

209
00:14:26,458 --> 00:14:27,875
Where do you start?

210
00:14:27,916 --> 00:14:34,458
Yeah, so when I started building Flowrs, I started first looking for frameworks that
people are using to build these tuis.

211
00:14:34,458 --> 00:14:36,666
I quickly found these three.

212
00:14:36,666 --> 00:14:45,875
You have Bubble Tea, Textual and Ratatui and each are written in a different language
which is also quite nice if you're very familiar with a specific language With a colleague

213
00:14:45,875 --> 00:14:47,958
we experimented a bit with Bubble Tea.

214
00:14:47,958 --> 00:14:48,958
It's written in Go.

215
00:14:48,958 --> 00:14:50,750
It's very easy to use.

216
00:14:50,750 --> 00:14:52,708
It's very elegant as well.

217
00:14:52,708 --> 00:14:59,833
You also, by default, get a nice UI if you just take the default templates of Bubble Tea.

218
00:14:59,833 --> 00:15:01,333
But I didn't choose Bubble Tea.

219
00:15:01,333 --> 00:15:04,333
I thought, well, let's do something more challenging.

220
00:15:04,333 --> 00:15:08,666
And I wanted also to improve my Rust knowledge So that's why I went with Ratatui.

221
00:15:08,666 --> 00:15:11,208
Textual, I think it's also quite good.

222
00:15:11,208 --> 00:15:19,250
despite Python having this connotation of not being a very performant language, I think
textual UIs can be also very interactive.

223
00:15:19,250 --> 00:15:20,708
and very performant.

224
00:15:20,750 --> 00:15:24,291
and what is the reason for you that you're so hyped about Rust?

225
00:15:24,291 --> 00:15:25,250
I hear this a lot.

226
00:15:25,250 --> 00:15:26,625
I haven't looked into Rust myself.

227
00:15:26,625 --> 00:15:29,375
It's still somewhere on the backlog for me.

228
00:15:29,375 --> 00:15:37,000
I think one of the main reasons is that it's fun to get hyped about stuff, In the end,
it's just a programming language, they're all Turing complete.

229
00:15:37,000 --> 00:15:39,500
But it's fun to get hyped about things.

230
00:15:39,500 --> 00:15:44,708
And the other reason is that I like the expressiveness of the type system.

231
00:15:44,708 --> 00:15:54,375
I come from a scientific background, physics and mathematics, and I always liked the
rigidity of proofs of logic.

232
00:15:54,375 --> 00:15:57,458
And then with Python, it's kind of the opposite.

233
00:15:57,458 --> 00:16:00,666
You can write a lot of things and nobody will complain.

234
00:16:00,666 --> 00:16:08,458
And at some point they might crash and burn in production and you will look at the
wreckage and then see, ah, yeah, okay, that went wrong.

235
00:16:08,458 --> 00:16:09,958
I need to fix this in this way.

236
00:16:09,958 --> 00:16:12,166
With Rust, you have

237
00:16:12,208 --> 00:16:13,458
kind of the opposite.

238
00:16:13,458 --> 00:16:21,125
You are developing your application and for the first, in my case it was many days, the
damn thing doesn't even compile.

239
00:16:21,125 --> 00:16:23,250
Some people say the compiler shouts at you.

240
00:16:23,250 --> 00:16:31,625
I now look at it as the compiler teaches you why what you wrote doesn't make sense, is not
fault tolerant, has some issues with it.

241
00:16:31,625 --> 00:16:36,333
So I like the expressiveness of the type system and the guarantees that it provides.

242
00:16:36,333 --> 00:16:42,083
Okay, so it's a static compiled language, I assume, and you get a lot more safety.

243
00:16:42,083 --> 00:16:42,416
yeah.

244
00:16:42,416 --> 00:16:46,083
And one of the selling points and the tagline is always blazingly fast.

245
00:16:46,083 --> 00:16:50,041
You could do a benchmark with some other tuis and you would probably find that it's
faster.

246
00:16:50,041 --> 00:16:55,166
All right, so you have these different frameworks, but under the hood, they're all
actually quite similar.

247
00:16:55,166 --> 00:16:58,541
In the end, it's just one big event loop, and it just...

248
00:16:58,541 --> 00:17:00,166
does three different things.

249
00:17:00,166 --> 00:17:08,416
It listens for input events, when you press a key on your keyboard, or you get a response
from an API call.

250
00:17:08,416 --> 00:17:11,166
And then based on the event, typically updates some state.

251
00:17:11,166 --> 00:17:16,125
And then after the state has been updated, there's a render phase and you display them on
the screen.

252
00:17:16,125 --> 00:17:18,750
And for the rendering phase, you often have...

253
00:17:18,750 --> 00:17:23,833
built-in widgets like a table, a bar graph, and that just repeats in an infinite loop.

254
00:17:23,833 --> 00:17:32,583
So most of these frameworks give you either built-in widgets that you can easily configure
yourself or you can create your own widgets and they typically have a few functions that

255
00:17:32,583 --> 00:17:33,500
you need to implement.

256
00:17:33,500 --> 00:17:41,625
So an example of some of these widgets, this is like the demo or the showcase project of
Ratatui.

257
00:17:41,625 --> 00:17:44,250
So you can create these kind of tabs with it.

258
00:17:44,250 --> 00:17:48,333
You can have maps, you can have sparklines, gauges, whatever you want.

259
00:17:48,333 --> 00:17:53,458
This is really good for in the movies to put on screen and show something really cool is happening,
right?

260
00:17:53,458 --> 00:17:57,958
This is the basic gist of it.

261
00:17:57,958 --> 00:18:00,791
You have your application, which is basically a loop

262
00:18:00,791 --> 00:18:07,041
which first draws or renders to the terminal, and then it starts processing events.

263
00:18:07,041 --> 00:18:08,250
See if something has changed.

264
00:18:08,250 --> 00:18:12,791
In this case, the only thing that can happen is you press a key and then it will break.

265
00:18:12,791 --> 00:18:15,041
And then basically your app will shut down.

266
00:18:15,041 --> 00:18:22,125
But what you would do is here, put all of the logic, like if people press the J key or the
down key, that means you

267
00:18:22,125 --> 00:18:23,916
to go to the next item in a table.

268
00:18:23,916 --> 00:18:38,250
em If there's a tick event, like this clock ticking every 200 milliseconds, for every 2000
milliseconds, make an API call to the Airflow REST API to get the latest DAGs.

269
00:18:38,250 --> 00:18:44,583
You update your state with that, and then the next loop starts basically, terminal.draw,
and it will render.

270
00:18:44,833 --> 00:18:48,250
actually my main takeaway message is go forth and mulTUIply.

271
00:18:48,250 --> 00:18:51,583
It's very easy to build your own Tui and to wrap around it.

272
00:18:51,583 --> 00:18:54,000
And how long were you working on this?

273
00:18:54,000 --> 00:18:55,000
that's a good question.

274
00:18:55,000 --> 00:18:58,750
in the beginning, of course, I didn't know the language, so I had to learn Rust.

275
00:18:58,750 --> 00:19:01,083
I made a lot of stupid rookie mistakes

276
00:19:01,083 --> 00:19:02,250
So I think it took me.

277
00:19:02,250 --> 00:19:08,666
on and off about a year to get like a first UI where I could just see the DAGs and the DAG
runs in the task instances.

278
00:19:08,666 --> 00:19:19,333
But then this filtering, for example, with the state machine, this theming stuff that
became a lot easier because I feel comfortable with the language and we have agents to

279
00:19:19,333 --> 00:19:19,750
help us.

280
00:19:19,750 --> 00:19:20,458
Okay, nice.

281
00:19:20,458 --> 00:19:22,625
And so people can actually use this, right?

282
00:19:22,625 --> 00:19:25,250
Can they brew install Flowrs or how does that work?

283
00:19:25,250 --> 00:19:27,250
Actually that's also a nice thing.

284
00:19:27,250 --> 00:19:33,750
there was a contribution from somebody from the community last week and they added it to
the brew core taps.

285
00:19:33,750 --> 00:19:37,125
So now anybody can just install brew install Flowrs.

286
00:19:37,125 --> 00:19:41,541
So when you open Flowrs for the first time, actually don't see that much, right?

287
00:19:41,541 --> 00:19:47,666
Because works with airflow instances and you need to tell Flowrs where to find those
airflow instances.

288
00:19:47,666 --> 00:19:52,708
Airflow is an open source project, which means you can just host it yourself.

289
00:19:52,708 --> 00:19:57,875
can basically deploy it on any cloud environment on AWS, for example, with some EC2
machines.

290
00:19:57,875 --> 00:20:02,250
But you also have a bunch of managed services.

291
00:20:02,333 --> 00:20:05,708
So actually before you would even uh open Flowrs, you would do something

292
00:20:05,708 --> 00:20:11,916
like Flowrs, config enable -m for managed service and then you can type a managed service.

293
00:20:11,916 --> 00:20:15,041
There are a bunch of managed services as I already mentioned.

294
00:20:15,041 --> 00:20:16,833
There's conveyor which is

295
00:20:16,833 --> 00:20:25,708
Dataminded's own managed airflow offering, but you also have MWAA, which is managed
Workflows for Apache Airflow on the AWS cloud.

296
00:20:25,708 --> 00:20:27,458
You have Google Cloud Composer.

297
00:20:27,458 --> 00:20:34,625
You have Astronomer, which we all know now because of the CEO news story.

298
00:20:34,625 --> 00:20:38,250
um managed service.

299
00:20:38,250 --> 00:20:40,750
have and then it generates a config for you.

300
00:20:40,750 --> 00:20:43,125
Exactly, so you enable it.

301
00:20:43,125 --> 00:20:55,041
In this case, it was already enabled and there is a config file If it doesn't exist, it
will get created and it basically contains some configuration that the TUI manages for

302
00:20:55,041 --> 00:20:55,541
you.

303
00:20:55,541 --> 00:20:58,416
And then you can start Flowrs and it will actually.

304
00:20:58,416 --> 00:20:59,583
uh

305
00:20:59,625 --> 00:21:01,666
find all of the Airflow environments.

306
00:21:01,666 --> 00:21:05,250
If you don't want to use a managed service, you just have your own Airflow instance.

307
00:21:05,250 --> 00:21:08,750
You can also just provide a host name and credentials.

308
00:21:08,750 --> 00:21:17,958
If you use basic authentication like a username and a password or OAuth 2 with the JWT
token, those are all also valid options.

309
00:21:17,958 --> 00:21:21,791
Also in this case you have conveyor, but conveyor manages multiple of these environments.

310
00:21:21,791 --> 00:21:24,666
So it also discovers even the environments that you have.

311
00:21:24,666 --> 00:21:27,000
Cause that depends on the managed service, right?

312
00:21:27,000 --> 00:21:27,541
Exactly.

313
00:21:27,541 --> 00:21:40,666
And on MWAA so on Amazon, it also auto-discovers the environments that you have within
your region, On GCP, you can also select a region and a GCP project, and it will also

314
00:21:40,666 --> 00:21:44,416
auto-discover the composer environments that you have available.

315
00:21:44,416 --> 00:21:46,208
oh Sure.

316
00:21:46,208 --> 00:21:47,458
So here's the repo.

317
00:21:47,458 --> 00:21:48,250
Yeah, exactly.

318
00:21:48,250 --> 00:21:50,666
143 stars!

319
00:21:50,666 --> 00:21:52,666
Exactly and of course it's self-starred.

320
00:21:52,666 --> 00:22:01,958
So this already existed since 2023 and I was mainly using it myself, And I was constantly
poking some colleagues like, hey, have you tried Flowrs already?

321
00:22:01,958 --> 00:22:04,291
I know that you have to work with 10 airflow environments.

322
00:22:04,291 --> 00:22:05,750
Maybe you want to try Flowrs.

323
00:22:05,750 --> 00:22:08,916
And then at some point I got an email

324
00:22:08,916 --> 00:22:13,083
Mentioning like, your TUI is the TUI of the week.

325
00:22:13,083 --> 00:22:19,333
And then also the maintainer of the Ratatui project also shared it on his LinkedIn, I
think.

326
00:22:19,333 --> 00:22:21,083
And also some more people found it.

327
00:22:21,083 --> 00:22:25,166
And I recently posted it in the Apache Airflow community Slack.

328
00:22:25,166 --> 00:22:28,583
So there's some community interaction now, which I actually like.

329
00:22:28,583 --> 00:22:30,250
It's also a first for me.

330
00:22:30,291 --> 00:22:33,375
I built some stuff, but usually just for myself.

331
00:22:33,375 --> 00:22:36,375
What did you learn about this whole TUI process

332
00:22:36,375 --> 00:22:40,875
I learned that I really like Rust as a programming language.

333
00:22:40,875 --> 00:22:45,750
I also learned that it's fun to just sometimes take a peek under the cover.

334
00:22:45,750 --> 00:22:51,541
We all use these applications, figuring out how they tick is a fun and rewarding
experience on its own.

335
00:22:51,541 --> 00:23:00,833
I've also learned that agentic AI, especially the last couple of months, can really speed
up the frequency at which you can push out new features.

336
00:23:00,833 --> 00:23:10,833
And I also learned that you sometimes also need to push people, like you have to be the
annoying marketeer in order for people to start using your app.

337
00:23:10,833 --> 00:23:12,291
Even if you think it's amazing.

338
00:23:12,291 --> 00:23:14,250
I mean, I use it on a daily basis.

339
00:23:14,250 --> 00:23:15,416
I like using it.

340
00:23:15,416 --> 00:23:18,833
Just building it and making it public on GitHub.

341
00:23:18,833 --> 00:23:22,458
That doesn't result in people finding it and actually using it.

342
00:23:22,458 --> 00:23:26,791
You have to push it out and go after your audience yourself a bit.

343
00:23:27,083 --> 00:23:27,875
Yeah, okay.

344
00:23:27,875 --> 00:23:35,416
Yeah, I think it's a very nice tool you showed us, So thanks a lot for sharing what TUIs
are, how you build them, and put them in the market.

345
00:23:35,416 --> 00:23:37,291
if you're using Airflow, check out the tool of Jan.

346
00:23:37,291 --> 00:23:40,500
We'll share the link in the description and in the comments.

347
00:23:40,500 --> 00:23:43,166
So Jan, thanks a lot for explaining this.

348
00:23:43,166 --> 00:23:44,875
Thank you everybody for watching.

349
00:23:44,875 --> 00:23:47,083
Check out Jan's tool and we'll see you next time.

350
00:23:47,083 --> 00:23:47,750
Bye bye!

351
00:23:47,750 --> 00:23:48,375
Bye!