WEBVTT NOTE This file was generated by Descript 00:00:05.606 --> 00:00:11.216 Raphaël: Hey folks, and welcome to the small tech podcast from Ephemere Creative. 00:00:11.636 --> 00:00:12.806 I'm your host Raph. 00:00:12.896 --> 00:00:17.636 And today we're going to talk about data and dashboards. 00:00:18.136 --> 00:00:22.606 If you like the work that we do, I'd really appreciate it if you hit that like 00:00:22.606 --> 00:00:28.606 and subscribe button on YouTube, or if you subscribe in your podcast app of choice. 00:00:28.966 --> 00:00:30.676 Every little bit helps. 00:00:30.926 --> 00:00:34.436 We're a tiny team and we could definitely use the support. 00:00:34.936 --> 00:00:40.666 So today I want to talk about three different types of dashboards and 00:00:40.666 --> 00:00:45.406 data and how we build them, why we build them, the things that we use 00:00:45.406 --> 00:00:50.836 to build them, and they roughly break down into three different categories. 00:00:51.166 --> 00:00:58.466 So it's analytics data that you generally will use off the shelf, SaaS 00:00:58.546 --> 00:01:01.816 analytics tools to build dashboards for. 00:01:02.366 --> 00:01:08.786 The other one is when you use BI tools to create dashboards that 00:01:08.786 --> 00:01:13.926 include analytics data and internal data operations or application data. 00:01:14.526 --> 00:01:20.226 And finally the last one is application dashboards, things that you want 00:01:20.226 --> 00:01:22.746 users to interact with and visualize. 00:01:23.246 --> 00:01:27.376 So for off the shelf analytics we're generally talking about things like 00:01:27.406 --> 00:01:30.376 Google analytics or Mixpanel or Amplitude. 00:01:30.716 --> 00:01:35.046 These are the things that you're going to use primarily internally, there 00:01:35.046 --> 00:01:42.266 are maybe a couple weird edge cases where you could embed graphs from these 00:01:42.266 --> 00:01:47.626 different tools into a public facing user interface, but for the most part, this 00:01:47.626 --> 00:01:52.186 is for your team to understand how people are interacting with a product, with a 00:01:52.186 --> 00:01:54.406 website, something along those lines. 00:01:54.906 --> 00:01:59.406 These tools are usually really easy to get started with you add a snippet 00:01:59.406 --> 00:02:01.656 to your website or your application. 00:02:01.656 --> 00:02:05.826 Sometimes you need a developer to integrate with your 00:02:05.826 --> 00:02:08.106 application code more tightly. 00:02:08.356 --> 00:02:13.396 Maybe identifying users and submitting events with metadata about those events. 00:02:13.786 --> 00:02:19.726 But it usually doesn't take a whole lot of time to get started and they usually 00:02:19.726 --> 00:02:25.816 have some mechanism to say, all right, I want this event plotted out over time. 00:02:25.866 --> 00:02:28.266 Maybe grouped by certain things. 00:02:28.576 --> 00:02:29.566 So you want. 00:02:29.846 --> 00:02:33.356 Users of a certain type who were clicking a given button. 00:02:33.386 --> 00:02:37.436 You want to know, you want to compare these different user groups over time 00:02:37.436 --> 00:02:42.626 and see if there's any trends you want to see if there's one that clicks 00:02:42.626 --> 00:02:44.366 the button more often than the other. 00:02:44.836 --> 00:02:48.526 That's a great reason to set up those kinds of dashboards, where you can just 00:02:48.526 --> 00:02:53.056 quickly get an overview of how your application is performing, how your 00:02:53.056 --> 00:02:57.696 features are performing and understand your product and your user base. 00:02:57.946 --> 00:03:01.216 So next let's talk about BI data and BI tools. 00:03:01.216 --> 00:03:02.896 Business intelligence. 00:03:02.946 --> 00:03:09.216 These tools vary in their structure and usage and that's because BI business 00:03:09.216 --> 00:03:13.686 intelligence, that can mean a lot of things, that can include the analytics 00:03:13.686 --> 00:03:19.986 data from the previous section, but it usually also includes more internal data. 00:03:20.516 --> 00:03:24.236 Because it's not just about your application or your website, it's 00:03:24.236 --> 00:03:25.736 about your business as a whole. 00:03:25.766 --> 00:03:31.316 You want to understand how, for example, usage of your application 00:03:31.316 --> 00:03:33.116 translates into dollars. 00:03:33.146 --> 00:03:36.926 And if you're going to do that, you either need to find a good way to track dollars 00:03:36.926 --> 00:03:40.726 in your application, but that doesn't always make sense so you want to bring 00:03:40.726 --> 00:03:45.076 in data from, for example, a payment processor like Stripe or Braintree. 00:03:45.476 --> 00:03:50.756 You probably also want to understand how your features correlate 00:03:50.756 --> 00:03:52.646 with your work internally. 00:03:52.646 --> 00:03:57.356 So you might want to bring in data from an application like Assana or click 00:03:57.356 --> 00:03:59.846 up where you manage your productivity. 00:04:00.296 --> 00:04:05.126 And you can start tying all of these data together into a single data 00:04:05.126 --> 00:04:09.146 warehouse where you can visualize and understand the correlations between 00:04:09.146 --> 00:04:13.976 these different types of data to get a better sense of what's happening, that 00:04:13.976 --> 00:04:16.856 leads to more dollars or fewer dollars. 00:04:16.856 --> 00:04:20.936 And are you actually using your resources efficiently as a business? 00:04:21.436 --> 00:04:25.336 So the tools that you might want to use to build out these kinds of dashboards 00:04:25.366 --> 00:04:27.346 are well, you'll need a database. 00:04:27.406 --> 00:04:29.056 You'll need a place to store your data. 00:04:29.336 --> 00:04:30.386 We generally. 00:04:30.436 --> 00:04:33.076 For the scale of projects that we work on. 00:04:33.106 --> 00:04:37.616 And given that this is the small tech podcast, we find that a Postgres 00:04:37.636 --> 00:04:42.616 database is usually fine, but there are tools that are specifically set 00:04:42.616 --> 00:04:46.486 up for this kind of data, warehousing to ingest huge chunks of data. 00:04:46.766 --> 00:04:53.256 You can think of tools like Snowflake or Google's big query, or aWS Redshift. 00:04:53.286 --> 00:04:55.146 There's plenty of them out there. 00:04:55.646 --> 00:04:59.666 So I talked about how you might want to bring in data from tools like 00:04:59.666 --> 00:05:01.926 Stripe Braintree, Asana, ClickUp. 00:05:02.246 --> 00:05:04.976 You can even bring in data from your marketing tools, like 00:05:04.976 --> 00:05:07.826 your Google ad words, AdSense 00:05:08.126 --> 00:05:11.096 Facebook ads, LinkedIn ads, all of this other stuff. 00:05:11.366 --> 00:05:16.136 And to do that, you will either need to hire some engineers to deal 00:05:16.136 --> 00:05:19.976 with it, or probably more sensibly, you'll want to use some tools that 00:05:20.006 --> 00:05:21.896 already provide that functionality. 00:05:22.426 --> 00:05:25.606 We have some experience setting up a tool called FiveTran, which is 00:05:25.606 --> 00:05:28.906 pretty neat, pretty, fairly priced connects with a lot of stuff. 00:05:29.146 --> 00:05:34.396 Basically allows you to connect to your different tools, point it to a database, 00:05:34.426 --> 00:05:38.326 and it takes care of normalizing all of your data, making sure that it's 00:05:38.326 --> 00:05:43.216 in sync, making sure that everything is updated on a regular basis. 00:05:43.276 --> 00:05:46.876 And yeah, it makes it really easy to pump all of your data into one 00:05:46.876 --> 00:05:51.252 database that you can then use to build your custom BI dashboards. 00:05:51.592 --> 00:05:54.302 There's a neat open source tool called AirByte as well. 00:05:54.671 --> 00:05:58.631 We like to use Metabase to actually produce that dashboard. 00:05:58.961 --> 00:06:01.181 It's just a really neat tool. 00:06:01.481 --> 00:06:05.981 Because it's open source we just deploy it onto our own servers 00:06:06.371 --> 00:06:08.681 and use that to build dashboards. 00:06:08.751 --> 00:06:14.031 But there are other alternatives, like QuickSight or Google Data Studio or 00:06:14.031 --> 00:06:16.761 Tableau or plenty of others out there. 00:06:16.851 --> 00:06:19.611 Some of them can get pretty pricey pretty quick. 00:06:19.851 --> 00:06:25.571 Google data studio I find is a neat product in that you have 00:06:25.571 --> 00:06:29.141 access to it if you're already using the Google workspace tools. 00:06:29.531 --> 00:06:34.131 And I don't actually know what it costs because whenever I've used it, I 00:06:34.131 --> 00:06:37.341 haven't run into any pricing questions. 00:06:37.731 --> 00:06:40.791 I, so I get the sense that you don't need to pay for it. 00:06:41.121 --> 00:06:45.171 Maybe there are advanced features that you do need to pay for, but yeah, for 00:06:45.171 --> 00:06:46.761 that reason, it's a pretty neat one. 00:06:47.261 --> 00:06:48.341 Just before I forget. 00:06:48.391 --> 00:06:52.111 Another thing that we like to bring into these types of databases and 00:06:52.111 --> 00:06:54.511 these dashboards is accounting. 00:06:54.541 --> 00:06:55.051 And. 00:06:55.551 --> 00:06:56.691 Invoicing data. 00:06:56.871 --> 00:06:58.641 We use zero for that sort of stuff. 00:06:58.701 --> 00:07:01.611 And I found it pretty practical to have that kind of data 00:07:01.671 --> 00:07:03.411 in this type of dashboard. 00:07:03.911 --> 00:07:04.211 Okay. 00:07:04.211 --> 00:07:07.601 With these BI tools, everything we've talked about so far 00:07:07.601 --> 00:07:09.041 is really internal data. 00:07:09.071 --> 00:07:13.151 It's stuff that you want to understand to get an overview of your business. 00:07:13.541 --> 00:07:18.611 But the thing about them is that you can visualize any kind of data with them. 00:07:18.641 --> 00:07:21.881 You can visualize your internal application data. 00:07:22.241 --> 00:07:24.701 You can visualize your analytics data. 00:07:24.701 --> 00:07:26.231 If it is in a database. 00:07:26.321 --> 00:07:29.591 You might get it in there using a tool like segment.com. 00:07:30.091 --> 00:07:34.321 And if you do have that data in there, there are some cases where 00:07:34.321 --> 00:07:39.751 you want to display that data back to a user of an application. 00:07:40.141 --> 00:07:46.081 Now tools like Metabase do provide a way of filtering that data so that 00:07:46.081 --> 00:07:53.131 you can just provide a dashboard that is for a given user, for 00:07:53.131 --> 00:07:55.231 example, or given organization. 00:07:55.231 --> 00:08:00.661 So you can actually use Metabase and some of these other tools to embed analytics 00:08:00.751 --> 00:08:03.001 into your user facing application. 00:08:03.391 --> 00:08:06.001 That being said, you can't. 00:08:06.631 --> 00:08:10.191 Customize how they are displayed to the extent that you could building 00:08:10.221 --> 00:08:12.861 custom tools and custom dashboards. 00:08:13.361 --> 00:08:17.561 So when I talk about building custom tools and dashboards, essentially 00:08:17.561 --> 00:08:21.821 what I mean is you're not using off the shelf tools for visualization. 00:08:22.331 --> 00:08:25.091 And there are a couple of reasons why you might want to do this. 00:08:25.451 --> 00:08:28.661 Hey, you might need to have a really custom user interface, or 00:08:28.661 --> 00:08:33.311 it might need to be tightly coupled with an existing user interface. 00:08:33.581 --> 00:08:39.071 Let's say you have a bunch of sensor data from an IOT platform and you 00:08:39.071 --> 00:08:44.441 have a custom platform where your users can manage those sensors. 00:08:44.441 --> 00:08:46.181 They can interact with the data. 00:08:46.211 --> 00:08:48.731 They can see real-time updates, that sort of thing. 00:08:49.181 --> 00:08:52.931 And you want to show them the data and allow them to graph it and 00:08:52.931 --> 00:08:54.791 visualize it and manipulate it. 00:08:54.791 --> 00:08:55.871 Maybe annotate it. 00:08:56.261 --> 00:09:02.891 All of that functionality is stuff that you might be able to get 30, 40% 00:09:02.891 --> 00:09:08.201 of the way there with the BI tools, but it will never be as smooth as if 00:09:08.201 --> 00:09:11.921 you build it out yourself and really integrate it deeply into the code that 00:09:11.921 --> 00:09:13.811 you're building for your platform. 00:09:14.441 --> 00:09:16.991 So that's exactly what we did for one of our clients. 00:09:16.991 --> 00:09:22.061 They have sensors that are out in the field, monitoring various metrics for 00:09:22.061 --> 00:09:26.171 a given use case, and they needed their clients to be able to interact with this 00:09:26.171 --> 00:09:30.041 data, to annotate it, to say at a given time, this thing was happening and we 00:09:30.041 --> 00:09:33.581 can see that there's these different changes in the different metrics. 00:09:33.661 --> 00:09:37.051 So to do that kind of dashboard, you're generally going to want 00:09:37.051 --> 00:09:39.001 to use a charting library. 00:09:39.061 --> 00:09:43.581 You could build something yourself but generally you would use an existing 00:09:43.611 --> 00:09:48.501 charting library because building that from scratch is going to be a lot of work. 00:09:48.811 --> 00:09:54.661 So we've used one in the past called am charts, which is really customizable, 00:09:54.661 --> 00:09:57.761 flexible, easy to use and very powerful. 00:09:57.761 --> 00:09:59.441 You can do a lot of stuff with it. 00:09:59.651 --> 00:10:05.291 We've also built charts and dashboards on react, native, using a Victory 00:10:05.661 --> 00:10:08.271 chart's victory pie and victory. 00:10:08.361 --> 00:10:11.301 I forget what the other victory charting libraries are called, but 00:10:11.301 --> 00:10:15.351 it's all under the banner of victory and they've been pretty great for us. 00:10:15.851 --> 00:10:21.431 They usually still offer you a variety of user controls, animation, that sort of 00:10:21.431 --> 00:10:23.531 thing that you can tweak to your use case. 00:10:23.831 --> 00:10:28.071 And the thing with them is that when you're building your own dashboarding 00:10:28.071 --> 00:10:33.141 system, you can really customize things to your user specific case. 00:10:33.351 --> 00:10:38.001 You can really make it smooth and flow very well for that user. 00:10:38.591 --> 00:10:41.861 The trade-off of course is the time it takes to get to your first 00:10:41.861 --> 00:10:45.401 graph and the cost of developing these types of dashboards. 00:10:45.581 --> 00:10:50.441 So if you really need things to be smooth, tight, and designed for a very specific 00:10:50.441 --> 00:10:52.811 use case, you might want to go this path. 00:10:52.991 --> 00:10:57.881 If you don't, then you might want to look at BI tools and an existing 00:10:57.881 --> 00:11:00.221 sort of off the shelf platforms. 00:11:00.721 --> 00:11:04.531 Before we wrap it up, I will talk about a couple other tools 00:11:04.531 --> 00:11:05.641 that you might want to consider. 00:11:05.671 --> 00:11:08.341 If you're going to build your own custom dashboards. 00:11:08.581 --> 00:11:12.811 If you're going to build around time series data, there are a variety 00:11:12.811 --> 00:11:14.791 of time series databases out there. 00:11:15.001 --> 00:11:18.171 But the one that we found most practical in this small tech 00:11:18.171 --> 00:11:20.391 context is called Timescale. 00:11:20.541 --> 00:11:24.861 And the reason that it's practical is because it's just a Postgres extension. 00:11:25.071 --> 00:11:29.151 So if you're already building an application on a Postgres database, it's 00:11:29.151 --> 00:11:31.701 really easy to get started with Timescale. 00:11:31.941 --> 00:11:35.931 You can integrate it with tools like Hasura, you can use other libraries 00:11:35.931 --> 00:11:41.031 like Cube JS to make your data available for your dashboarding system. 00:11:41.531 --> 00:11:45.041 It makes it really easy to build around that type of data. 00:11:45.221 --> 00:11:47.501 Of course, you don't have to use timescale. 00:11:47.501 --> 00:11:52.061 You can just use straight up Postgres, you can use Mongo, you can use basically 00:11:52.061 --> 00:11:55.181 whatever database you want, because a lot of the tools out there are 00:11:55.181 --> 00:11:57.101 built to work with a variety of them. 00:11:57.281 --> 00:12:00.791 But specifically for time series data like sensor readings from an 00:12:00.791 --> 00:12:05.861 IOT device, timescale is a really great option to get started with. 00:12:06.161 --> 00:12:08.261 Alrighty folks, thanks for listening. 00:12:08.761 --> 00:12:13.401 If you enjoy this stuff please like and subscribe on YouTube and make sure to 00:12:13.401 --> 00:12:16.731 subscribe in your podcast app of choice. 00:12:17.231 --> 00:12:18.611 And leave us a review. 00:12:18.611 --> 00:12:20.861 It would be so helpful for us. 00:12:21.011 --> 00:12:25.361 We'd love to hear about why you like the podcast to what you want 00:12:25.361 --> 00:12:26.981 more of what you want less of. 00:12:26.981 --> 00:12:31.151 Is there anything that you'd like to talk about with me on the podcast? 00:12:31.231 --> 00:12:36.091 I'm really into the idea of talking to you about the things you have learned 00:12:36.091 --> 00:12:38.821 as you build a small tech business. 00:12:39.321 --> 00:12:43.011 Also make sure to subscribe to our newsletter. 00:12:43.511 --> 00:12:47.171 We are going to be sending you tons of great info about how 00:12:47.171 --> 00:12:49.361 to build small tech products. 00:12:49.861 --> 00:12:54.871 Everything from videos, blog posts, and even the podcast episodes 00:12:54.871 --> 00:12:58.711 that you might have missed, but surely you didn't miss any right. 00:12:59.211 --> 00:13:05.541 So go to smalltechpodcast.com and click the newsletter button in the navbar. 00:13:05.841 --> 00:13:11.441 That'll take you to the right spot, and we'll see you in your inbox. 00:13:11.941 --> 00:13:16.741 That's it for this week's episode and we all want to do some good in the world. 00:13:16.771 --> 00:13:19.621 So go out there and build something. 00:13:19.621 --> 00:13:20.611 Good folks. 00:13:20.671 --> 00:13:21.271 See ya.