Pushing to Production

In the first episode of Pushing to Production, we interview Jamie Robinson the founder and CEO of Mashoom.

Show Notes

We talked to the founder of Mashoom a company that specializes in helping businesses with all of their tech needs, the range of solutions is amazing and Jamie goes into detail how development and servers are run.

You can learn more at
- https://www.mashoom.co.uk/
- https://twitter.com/JamieTheMashMan

Mashoom is currently hosted with AWS running on Linux servers and making use of RDS MySQL aurora DB and S3 file storage.


What is Pushing to Production?

The podcast were we talk to founders and product managers about how the technical side of business. How do they keep the production lights on

Intro: Hello, everyone, Simon here from snapshooter.com, and you are listening to the Pushing to Production Podcast. On this podcast, we focus on the purely technical side of running a project, learning about how startups go from code to production.

Simon Bennett: Hi, everyone, welcome to the first episode of Pushing to Production; my guest today is Jamie Robinson from; I've forgotten how to pronounce the name of your business already, Mashroom, is it?

Jamie Robinson: Mashoom.

Simon Bennett: Mashoom, which is a business solution for you said small to medium businesses, sort of technical solution.

Jamie Robinson: That's the one.

Simon Bennett: Could you go into a bit about that?

Jamie Robinson: So, instead of online tools for small to medium businesses, we basically, we've grown for a long time now by various companies coming up to me and saying, we have a problem, whether it be file storage, doesn't work for them, or we do inventory tracking, a bit of HR, that kind of thing. And we have built the platform based on what people have asked us for, so code-wise, we have to build incredibly flexibly and we reuse coding components an awful lot, and obviously, from that, it bubbles up to what our users see, they're very flexible modules and components, which fit a range of use cases.

Simon Bennett: Okay, does that mean you deal mostly with sort of physical companies who don't have any technical skills or in-house?

Jamie Robinson: Yeah, mostly that's where we sit down, we sort of pride ourselves on being quite a friendly company, one to sort of pick up the phone to, and have a chat to, and we talk about that. So, yeah, where we've had the most success, let's say, is companies that really, they're sort of fun, we call them not tech-enabled, some people in the company are, some of them aren't, well, we cater to them all. And yes, it's the case of saying what's possible with tech, what's not possible with tech, and taking them through that process.

Simon Bennett: Interesting, sounds a little difficult to have a solution for all, you have such a generic solution.

Jamie Robinson: On a technical basis that's the fun, for me, so a lot of SAS providers for better or worse, come out with a product and then find the people that will fit the product, where we have a very flexible set of components, and we allow, as I always say when we're chatting on LinkedIn and what have you, let's just have a chat, see what you're having difficulties with and we will fill in the bridge between us and you if it's possible. We're also very happy to say, look, we're not the right people for you if you tried X, Y, and Z, so the coding that goes into being able to make that happen is a lot of fun.

Simon Bennett: Interesting, yeah, we'll see, yeah. Here at SnapShooter, we run a completely different ballpark, but we basically offer fixed-seven solutions to many users instead of tailoring everybody. So, how do you, maybe you could get slightly into the technical side of how you build such a sort of abstracted solution for so many different businesses and yeah, a little curious what Tech Stack you're running and infrastructure and stuff?

Jamie Robinson: Yeah, but if I do, but I say the basic Tech Stack for those interested, we're all on open-source Tech Stack, so say as of Mashoom 7, which is about to be released this evening or the day after, would you believe?

Simon Bennett: Oh, good luck.

Jamie Robinson: They have coincided quite nicely [alongside our work 3:30], so we're going to be my MySQL on the back, going up to an Apache with PHP on the top or on the derp size, that's a GraphQL API taking through to a react front-end. So, that's our user side, so it's sort of slightly the Facebook stack, but without the weird stuff that Facebook does these days, the simplified Facebook.

Simon Bennett: I haven't looked at Facebook's language for a long time.

Jamie Robinson: So, it's actually quite interesting, they found PHP too slow for dealing with billions of requests, so they now translate it into hack, which I don't think anyone else has followed them. And I certainly don't plan to anytime soon; it's a weird sort of how to make PHP, not PHPs.

Simon Bennett: Yeah, I had a quick look at it when it first came out and it was still compatible, but as soon as it sort of became its own language and.

Jamie Robinson: Yeah, it's actually funny because what GraphQL is, because GraphQL only has one endpoint as it were, one core place where it all stems from, it's actually easier to translate that into hack because the biggest thing with hack is they actually made PHP have an endpoint rather than just opening up the file and running it because it's all compiled. So, it's possible we could do it, no, I don't really care.

Simon Bennett: Yes, I wouldn't.

Jamie Robinson: It's keeping; we've got enough problems as it is, keep things simple. So yeah, that's our Tech Stack, so I guess one of the most difficult things I find when customers come up to us and say, can we do X, Y and Z, is the data storage, would you believe making that flexible? Because in the database you have a certain number of columns in your database and they have a certain number of column names and you really don't want to be adding new column names for each customer that's a really nasty place to be.

So, we have come up with what I've sort of labeled the Fuzzy Data Engine, which allows, essentially, to turn the database, and turn it through 90 degrees so the keys in the database are variable and that allows us to, say we're just about to come up with the CRM solution, say you take a person in the CRM. What you can do is, the user can add fields to that person to collect different bits of data, and coding side as well, I can write in new fields that everyone's going to have and that fits into a database, and it also boils down to a hierarchical structure as well.

So, you can have contact with a person who has two layers and then et cetera, et cetera, so it means that you can do what us coders know as objects inheritance. You can do your keys in the database, which is one of the technologies that underpins all of our modules and allows us to offer features quite flexibly, and allows our users to set it up as they want to.

Simon Bennett: Do you have any performance worries around that or has it been baked in the sort of a first principle? The,

Jamie Robinson: The short answer is, no, not really, so we have customers with, yeah, they're probably up to 600,000 now, but certainly, last time I checked 500,000 data points and they can knock out reports that come down in Excel, I think the report takes about eight seconds to run that kind of thing.

Simon Bennett: Oh, okay.

Jamie Robinson: So, it's one of those things that once you get a database doing what it's meant to do and you get code that is, let's say, not doing anything stupid, the data comes in and it flows through to what you're doing, there isn't sort of some churn somewhere for no reason. I think you would call the performance acceptable.

Simon Bennett: Yes.

Jamie Robinson: And with modern servers and modern CPEs and those sorts of things, we certainly haven't found a need to get any more impressive in data science about it than what we're doing.

Simon Bennett: Where do you host at the moment?

Jamie Robinson: So, we're all in AWS, is our hosting provider, from top to bottom basically, everything's in there.

Simon Bennett: Okay. Do you use their managed databases as well?

Jamie Robinson: Yeah, so we're running an Aurora cluster, so we have two databases in our cluster, I believe, and read then write.

Simon Bennett: Okay, yeah, we run in the same way, but we have two replicas to get us that three availability zone.

Jamie Robinson: Yeah.

Simon Bennett: Well, I've got two.

Jamie Robinson: Yeah, we haven't got the third one.

Simon Bennett: No, it gets pricey to be honest.

Jamie Robinson: Yeah, yeah, it does.

Simon Bennett: Yeah.

Jamie Robinson: I think AWS is a beautiful business model where they say, look at all these shiny things you could have and you go, yeah, if you don't.

Simon Bennett: I think the only good thing about Aurora compared to mySQL in that regard, is that you can actually use the replicas, whereas with mySQL if you were to have cross-regional, those things just sit there doing nothing.

Jamie Robinson: Yeah, they.

Simon Bennett: Tonnes of money just waiting for the day for it all to go wrong, so, yeah.

Jamie Robinson: Yeah. So, another bit of kick we've got under the hood is we've got a custom database controller if you want to call it that, and something called the database millions, wrote them a long time ago. And what they do is they use, if it's just a read statement and it's not under a transaction, those sorts of things they could use the read endpoint, there's a special read endpoint and the write endpoint. Using the transaction writing, we can do the write endpoint, so from an application layer, we can split the data quite nicely, which sell us really well, we haven't had a database issue, I think full stop, we had one in the very early days, but I don't think that was something.

Simon Bennett: Yeah, fair enough. Well, that probably leads into a good question; do you have any horror stories around some DevOps and the service sides and stuff like that?

Jamie Robinson: I think the memories I have most fondly from the early days of Mashoom, it was a hobby, I was doing it through, sort of to help a university project. So, when I was learning how to do things properly it's safe to say we had some quite interesting whoopsies, we had at that point, I didn't know what Git was, so code, yeah. So, this is a mechanical engineer learning how to code here, it was fun, no one was paying for this, by the way, no one in the early days. So, I was uploading files individually, and that went about as well as we now know it's expected to go, I also wasn't using transactions when doing database, multiple coils together and it was a complex thing, our design module, that's what it's now called.

So, it would get halfway through, the script would break, and then you had a database, which is out of sync with each other, and if anyone manages the database, that's generally where bad things start to happen.

Simon Bennett: Why would the code stop halfway through, that's?

Jamie Robinson: Yeah.

Simon Bennett: Yeah, that'll never happen.

Jamie Robinson: So, we ended up in some, so the thing we started this database with was a spreadsheet of design part numbers if you've ever sort of gone that way, so we kept on, sort of we ran the database or ran what wasn't called Mashoom at the time, all this for a week. It broke in all sorts of ways, I cleared the database out, re-uploaded the part list again, did it again, two weeks broke again, so the early days of just not knowing how to manage these things or how to error manage is what it turned into.

Simon Bennett: Yeah, very much learning there, that's interesting. The uploading of individual files makes me laugh, back in 2013 I had a job and they hadn't adopted any standards, even Git, yeah, Git was definitely a thing, I was trying to recommend it to them, it was all FTP. And you'd have to in the office, declare then open, I've downloaded this file to modify can no one else please deal with the control of these, and it was absolutely a nightmare

Jamie Robinson: FileZilla or Cyberdoc?

Simon Bennett: FileZilla.

Jamie Robinson: It was FileZilla until that broke and then Cyberdoc took over.

Simon Bennett: Yes, yeah, yeah, yeah.

Jamie Robinson: Absolutely, and I was running on that time, I wasn't running on AWS either, I was running on Fasthosts, if you know them, possibly.

Simon Bennett: Yes, yeah, yeah.

Jamie Robinson: And their servers were not meant to be doing what I was getting them to do then put that way, so they were falling over and those sorts of things, so that was fun.

Simon Bennett: Cool. So, I have to ask this question because obviously, SnapShooter is a backup company, what are your current backup strategies?

Jamie Robinson: So, we do the AWS, so some sets of things, we've got retention of 30 days on our AWS database where a snapshot being done date and we only have two places of storage in Mashoom, which is a single database and our S3 file storage. And it's one of those things; I have put the question out to our customers and sort of said do you want another form of backup? Do you want it to go to your servers? Do you want these sorts of things? And when you take them through, I respond to what my customers ask for, what we take them through is sort of what AWS is capable of, and they sort of go look if a bomb drops an Island, we'll have problems as well kind of thing.

So, they sort of go, that's okay, what we would really like to, and we'll certainly be on your door if it happens, but if a customer comes to us and says, we want our data backed up to a second destination, then we will write the option and enable it so that accounts can have that done, and that's basically the way we grow the platform. If somebody said, please could we have, we write the option in for people to have that, and then some other people go, huh, haven't thought of that, thanks very much, so we're in that place with it, we have what I think is a fairly resilient setup, but if people want an even more setup we'll set them up.

Simon Bennett: So, obviously, this isn't the sell the SnapShooter podcast, but yeah, yeah, it always depends how you draw the line and potentially, in the future, do you want to back up your data to a different provider, why you want to be completely out of AWS? But your point about AWS, if you're an S3, you're in three different data centers in that particular region, and they're separated by I think Islands, there's, when I looked it up recently, 60 kilometers apart from each other, so that's actually three bombs that need to go off for your customers' data to be lost from S3, so.

Jamie Robinson: I haven't recently looked up how big a nuclear bomb is, but.

Simon Bennett: Yes, that's true, that's true, that's very true. Yes, and she started having bigger problems at that point.

Jamie Robinson: No, I think there would be.

Simon Bennett: I was thinking more terrorist level attack, not national, nationwide.

Jamie Robinson: Oh yeah, I guess, taking a proverbial pair of scissors to the day's connection would be the biggest worry, but that's sort of where we are, you have to be practical about these things and that's where we put it. I think there is one thing we do come into quite a lot is about, we generally digitize businesses, which aren't very digitized, that's generally where we do most of our time, not all the cases, but mostly, and there is this sort of hang on, you're taking my data and you're putting it in the cloud.

Simon Bennett: Oh, I see, yeah.

Jamie Robinson: Because as techies, as people that are happy, confident with this technology, it's totally cloud safe, fine, whatever. But to people that have their server sat over there with a cup of coffee resting on it, but we're happy that that's a manageable risk, et cetera, and so forth, that is a very strange thing.

Simon Bennett: Yeah, personally, because the idea of running a server at home for core infrastructure is terrifying, I would never consider doing that.

Jamie Robinson: I think it's terrified to us that have never done it, I think to the guys that were putting together servers, I think I've seen it on your Twitter actually, you've done that kind of thing, but for me at least, I've got Synology Drive and I can set it up to do various things, but the idea of me knocking it over in a party and that being the company gone.

Simon Bennett: Exactly, yeah. So, I have [some storage 15:37] which I mostly use for like, they back up each other as part of continuous testing of SnapShooter's sort of process, and also it's a lot quicker to test the server that's just in the garage than in the cloud. But actually running any infrastructure, to give a bit of mind, in a garage without redundant power supply and climate control and all these things, security, is just a disaster, so I can see all these things are things you can sell to your users, right?

Jamie Robinson: No, we've got a direct resilience PDF, which sort of says what does Amazon do for us? And a lot of people go, ooh, and if you take it as marketing alone that's worth its space.

Simon Bennett: That's interesting. Yeah, so I went to a local data center to have a tour recently, oh, a few years ago, and they were showing me the check-in location where the security guard sits and how much bulletproof glass he sat behind. And I thought, I can't ever imagine someone bringing a rifle out and the knowledge to get into a data center, but I guess that's the solution, lots of security.

Jamie Robinson: It's one of those things you don't hear it on the news at all of people breaking into data centers, I'm really happy that people don't, and I'm really happy that precautions are in place and those sorts of things, but you sort of imagine somebody running in and going, I've got a hard drive, oh, what do I do with this?

Simon Bennett: Yeah.

Jamie Robinson: Where do I go?

Simon Bennett: Yeah.

Jamie Robinson: There's probably a movie and it's somebody, the groffs running in with the AK 47 and some probability.

Simon Bennett: I can already imagine the movie where they just plug a USB drive into a random server and start hacking, but.

Jamie Robinson: Just have it down, mate.

Simon Bennett: Yeah, have it down, yeah. Cool. Well, I think we have covered everything I really wanted to talk about, what are your costs roughly at AWS at the moment unless you want to [keep that to yourself, what is your size 17:41]?

Jamie Robinson: At the moment we're at about 250 pounds a month, roughly.

Simon Bennett: Okay.

Jamie Robinson: So, relatively low.

Simon Bennett: Yeah, it seems pretty reasonable, really?

Jamie Robinson: Yeah, yeah, in the grand scheme of things relatively low, we run things very efficiently, basically.

Simon Bennett: I imagine most of that cost is towards RDS.

Jamie Robinson: Yeah, and again, we run small RDS instances compared to what AWS does, we run small EC2 instances that still have more than enough firepower to do what we need them to do. I think it's the case because we have all, I have written a huge amount of the application stack, the database side, all of those sorts of things, we have full control over how the application handles. When you get a lot of open-source components doing your connections and those sorts of things, you start losing control of, there are two connections in the same call and those sorts of things, you suddenly start going whoop and the amount of power that your application requires just goes through the roof and you're not entirely sure why.

So, we've done quite well to mean that we handle a decent, when it happens for us it's like Friday afternoons, absolute hell on earth for us because suddenly everyone's uploading their files, backing up the weekend, sharing the report that we send, and yaddy, yaddy, yar, it all goes off on Friday afternoon we found. And we handle that fine, the first time you see it spike, you go, hmm, and then we're sort of fine from there, and it's going to get better with Mashoom service as we leverage a bit more of handing off the big things to AWS, so they can handle the workload. So, no, we're quite proud of keeping it simple, I think that's what we would say.

Simon Bennett: Brilliant, okay. And that's probably a good place to wrap up, so where could people find out more about you?

Jamie Robinson: Mashoom.co.uk is nice and straightforward and you can find us on Twitter, there's a contact form on our website and all those sorts of things, so yeah, come find us.

Simon Bennett: Brilliant.

Jamie Robinson: I say it's sort of personally; it'd be nice to have a chat. I always feel with these things that you say, come and find us so you can give us money and those sorts of things, why not come find us, so you could just have a chat, middle of a pandemic, it's just nice kind of having a chat, to be honest.

Simon Bennett: Oh, fair enough, okay, well, we'll put your [Inaudible 20:14] link there. Cool, well, thank you very much.

Jamie Robinson: All right, thanks very much though.

Simon Bennett: Cheers.

Outro: Thank you for listening to the Pushing to Production Podcast from snapshooter.com. SnapShooter is a backup service for all of your online servers, databases, and applications and the whole service snapshots are providers like DigitalOcean, AWS, and Vultr. As well as direct backups from any provider to your given storage of choice, S3 Back, there is Google Drive; the list goes on, check us out today. We have a limited free plan or all plans come with a 14 day free trial with no card required, thanks for listening and tune in next week.