A podcast about the intersection of technology and sustainability.
Whether it's software-as-a-service platforms helping to stir up the circular economy, platforms helping artists build economically sustainable practices, or hardware helping to monitor and change emissions... we talk to all kinds of people in and around the intersection of technology and building a sustainable future.
🗣️ Got feedback? Want to be a part of this? Contact us!
hello@sustainabletechpodcast.com
🚀 Subscribe to the podcast:
https://sustainabletechpodcast.com/
Raphaël: Hey folks, and welcome to the
small tech podcast from Ephemere Creative.
I'm your host Raph.
And today we're going to talk
about data and dashboards.
If you like the work that we do, I'd
really appreciate it if you hit that like
and subscribe button on YouTube, or if you
subscribe in your podcast app of choice.
Every little bit helps.
We're a tiny team and we could
definitely use the support.
So today I want to talk about three
different types of dashboards and
data and how we build them, why we
build them, the things that we use
to build them, and they roughly break
down into three different categories.
So it's analytics data that you
generally will use off the shelf, SaaS
analytics tools to build dashboards for.
The other one is when you use BI
tools to create dashboards that
include analytics data and internal
data operations or application data.
And finally the last one is application
dashboards, things that you want
users to interact with and visualize.
So for off the shelf analytics we're
generally talking about things like
Google analytics or Mixpanel or Amplitude.
These are the things that you're going
to use primarily internally, there
are maybe a couple weird edge cases
where you could embed graphs from these
different tools into a public facing user
interface, but for the most part, this
is for your team to understand how people
are interacting with a product, with a
website, something along those lines.
These tools are usually really easy
to get started with you add a snippet
to your website or your application.
Sometimes you need a developer
to integrate with your
application code more tightly.
Maybe identifying users and submitting
events with metadata about those events.
But it usually doesn't take a whole lot
of time to get started and they usually
have some mechanism to say, all right,
I want this event plotted out over time.
Maybe grouped by certain things.
So you want.
Users of a certain type who
were clicking a given button.
You want to know, you want to compare
these different user groups over time
and see if there's any trends you
want to see if there's one that clicks
the button more often than the other.
That's a great reason to set up those
kinds of dashboards, where you can just
quickly get an overview of how your
application is performing, how your
features are performing and understand
your product and your user base.
So next let's talk about
BI data and BI tools.
Business intelligence.
These tools vary in their structure and
usage and that's because BI business
intelligence, that can mean a lot of
things, that can include the analytics
data from the previous section, but it
usually also includes more internal data.
Because it's not just about your
application or your website, it's
about your business as a whole.
You want to understand how, for
example, usage of your application
translates into dollars.
And if you're going to do that, you either
need to find a good way to track dollars
in your application, but that doesn't
always make sense so you want to bring
in data from, for example, a payment
processor like Stripe or Braintree.
You probably also want to understand
how your features correlate
with your work internally.
So you might want to bring in data from
an application like Assana or click
up where you manage your productivity.
And you can start tying all of these
data together into a single data
warehouse where you can visualize and
understand the correlations between
these different types of data to get a
better sense of what's happening, that
leads to more dollars or fewer dollars.
And are you actually using your
resources efficiently as a business?
So the tools that you might want to use
to build out these kinds of dashboards
are well, you'll need a database.
You'll need a place to store your data.
We generally.
For the scale of projects that we work on.
And given that this is the small
tech podcast, we find that a Postgres
database is usually fine, but there
are tools that are specifically set
up for this kind of data, warehousing
to ingest huge chunks of data.
You can think of tools like Snowflake
or Google's big query, or aWS Redshift.
There's plenty of them out there.
So I talked about how you might want
to bring in data from tools like
Stripe Braintree, Asana, ClickUp.
You can even bring in data
from your marketing tools, like
your Google ad words, AdSense
Facebook ads, LinkedIn ads,
all of this other stuff.
And to do that, you will either
need to hire some engineers to deal
with it, or probably more sensibly,
you'll want to use some tools that
already provide that functionality.
We have some experience setting up
a tool called FiveTran, which is
pretty neat, pretty, fairly priced
connects with a lot of stuff.
Basically allows you to connect to your
different tools, point it to a database,
and it takes care of normalizing all
of your data, making sure that it's
in sync, making sure that everything
is updated on a regular basis.
And yeah, it makes it really easy
to pump all of your data into one
database that you can then use to
build your custom BI dashboards.
There's a neat open source
tool called AirByte as well.
We like to use Metabase to
actually produce that dashboard.
It's just a really neat tool.
Because it's open source we just
deploy it onto our own servers
and use that to build dashboards.
But there are other alternatives, like
QuickSight or Google Data Studio or
Tableau or plenty of others out there.
Some of them can get
pretty pricey pretty quick.
Google data studio I find is a
neat product in that you have
access to it if you're already
using the Google workspace tools.
And I don't actually know what it
costs because whenever I've used it, I
haven't run into any pricing questions.
I, so I get the sense that
you don't need to pay for it.
Maybe there are advanced features that
you do need to pay for, but yeah, for
that reason, it's a pretty neat one.
Just before I forget.
Another thing that we like to bring
into these types of databases and
these dashboards is accounting.
And.
Invoicing data.
We use zero for that sort of stuff.
And I found it pretty practical
to have that kind of data
in this type of dashboard.
Okay.
With these BI tools, everything
we've talked about so far
is really internal data.
It's stuff that you want to understand
to get an overview of your business.
But the thing about them is that you can
visualize any kind of data with them.
You can visualize your
internal application data.
You can visualize your analytics data.
If it is in a database.
You might get it in there
using a tool like segment.com.
And if you do have that data in
there, there are some cases where
you want to display that data
back to a user of an application.
Now tools like Metabase do provide
a way of filtering that data so that
you can just provide a dashboard
that is for a given user, for
example, or given organization.
So you can actually use Metabase and some
of these other tools to embed analytics
into your user facing application.
That being said, you can't.
Customize how they are displayed to
the extent that you could building
custom tools and custom dashboards.
So when I talk about building custom
tools and dashboards, essentially
what I mean is you're not using off
the shelf tools for visualization.
And there are a couple of reasons
why you might want to do this.
Hey, you might need to have a
really custom user interface, or
it might need to be tightly coupled
with an existing user interface.
Let's say you have a bunch of sensor
data from an IOT platform and you
have a custom platform where your
users can manage those sensors.
They can interact with the data.
They can see real-time
updates, that sort of thing.
And you want to show them the data
and allow them to graph it and
visualize it and manipulate it.
Maybe annotate it.
All of that functionality is stuff
that you might be able to get 30, 40%
of the way there with the BI tools,
but it will never be as smooth as if
you build it out yourself and really
integrate it deeply into the code that
you're building for your platform.
So that's exactly what we
did for one of our clients.
They have sensors that are out in the
field, monitoring various metrics for
a given use case, and they needed their
clients to be able to interact with this
data, to annotate it, to say at a given
time, this thing was happening and we
can see that there's these different
changes in the different metrics.
So to do that kind of dashboard,
you're generally going to want
to use a charting library.
You could build something yourself but
generally you would use an existing
charting library because building that
from scratch is going to be a lot of work.
So we've used one in the past called am
charts, which is really customizable,
flexible, easy to use and very powerful.
You can do a lot of stuff with it.
We've also built charts and dashboards
on react, native, using a Victory
chart's victory pie and victory.
I forget what the other victory
charting libraries are called, but
it's all under the banner of victory
and they've been pretty great for us.
They usually still offer you a variety
of user controls, animation, that sort of
thing that you can tweak to your use case.
And the thing with them is that when
you're building your own dashboarding
system, you can really customize
things to your user specific case.
You can really make it smooth
and flow very well for that user.
The trade-off of course is the
time it takes to get to your first
graph and the cost of developing
these types of dashboards.
So if you really need things to be smooth,
tight, and designed for a very specific
use case, you might want to go this path.
If you don't, then you might want
to look at BI tools and an existing
sort of off the shelf platforms.
Before we wrap it up, I will
talk about a couple other tools
that you might want to consider.
If you're going to build
your own custom dashboards.
If you're going to build around time
series data, there are a variety
of time series databases out there.
But the one that we found most
practical in this small tech
context is called Timescale.
And the reason that it's practical is
because it's just a Postgres extension.
So if you're already building an
application on a Postgres database, it's
really easy to get started with Timescale.
You can integrate it with tools like
Hasura, you can use other libraries
like Cube JS to make your data
available for your dashboarding system.
It makes it really easy to
build around that type of data.
Of course, you don't
have to use timescale.
You can just use straight up Postgres,
you can use Mongo, you can use basically
whatever database you want, because
a lot of the tools out there are
built to work with a variety of them.
But specifically for time series
data like sensor readings from an
IOT device, timescale is a really
great option to get started with.
Alrighty folks, thanks for listening.
If you enjoy this stuff please like and
subscribe on YouTube and make sure to
subscribe in your podcast app of choice.
And leave us a review.
It would be so helpful for us.
We'd love to hear about why you
like the podcast to what you want
more of what you want less of.
Is there anything that you'd like to
talk about with me on the podcast?
I'm really into the idea of talking to
you about the things you have learned
as you build a small tech business.
Also make sure to subscribe
to our newsletter.
We are going to be sending you
tons of great info about how
to build small tech products.
Everything from videos, blog posts,
and even the podcast episodes
that you might have missed, but
surely you didn't miss any right.
So go to smalltechpodcast.com and click
the newsletter button in the navbar.
That'll take you to the right spot,
and we'll see you in your inbox.
That's it for this week's episode and we
all want to do some good in the world.
So go out there and build something.
Good folks.
See ya.