03 - Commerce Orchestration (with Mateusz Woźniak)

Show Notes

In this podcast we dive deep into data orchestration in Composable Commerce. Why it is important, why you need it and why ‘just a GraphQL layer’ won’t work for complex use cases.

What is The Commerce Composers?

Welcome to the Composable Commerce Podcast powered by Deity (Acquired by Ultra Commerce), the leading platform for Composable Commerce. In this podcast we explore the world of Composable Commerce: What is it? How does it work? And most importantly, how will it help businesses grow? We talk with online merchants, agencies and tech companies about their experience in Composable Commerce, including some of the biggest retailers in the world. So, do you want to know everything about it? Please hit the subscribe button so you won’t miss an episode.

00:00:18 Jamie: Hello everyone, and welcome to a new episode of our Composing Commerce podcast. Today we have Mathias Wozniak as a guest, and we're going to talk about composable commerce and more specifically about the real middleware layer. What is the difference between just GraphQL layers and what we do, which is a little step further where we dive into commerce orchestration. What is commerce orchestration? What makes it special and why do you need it in a composable commerce architecture? So Mathias, welcome to our podcast. Welcome, Jamie. Thanks for having me here. So, Mathias, can you tell us a little bit about yourself, about your background? Yeah, sure. I'm actually a developer for more than ten years. Twelve years initially mostly focused on the frontend, but right now I'm full stack and also with kind of DevOps experience. And here at IT, I'm the lead developer and I'm responsible for our commerce composer and also for the cloud platform, which is part of our ecosystem. Yeah, you talk about an ecosystem, and I think that immediately hits the button of where we are with composable commerce. We don't deliver just a platform. And composable commerce is not just a platform, it's an ecosystem. So part of that ecosystem, we have front ends. We have backend services where we use partners, for example, BigCommerce. For orders, we can use Algolia for search. We can use tools for sending unpacking, we can use for payments, etc., etc. to bring all that data together you need a layer. And that layer is traditionally called a middleware layer, where you bring all the data and bring it between the different services and to the front end. So this has been going on for a while, and we see some people in the market coming up and offering these kind of layers. The big difference is that you can have a layer which just brings that data back and forth. And usually today I think they're using GraphQL in most cases. But what makes it more important. And I think what makes it really special, what we are offering. And also in my opinion, what you need to do real good composable commerce is that you can control that data, and for that you need something more special. But can you explain what you need for that? Yeah, I think you partially mentioned, but there are two layers. Not so long ago, the middleware was the solution and the middleware was doing everything. Right. Now there are two parts of that together. We can call that middleware, because it's kind of the middle ground between front end and all the backend systems. But there are two layers. First layer is just the API itself that is exposed to the client. So this is the web browser. This is for example mobile application. And the second layer which is actually orchestrating what happens behind the API itself because it's of course, not only the call from the frontend to the API and the response sends it back, but there is a lot of things happening behind the API itself where the business logic happens between all the other systems. And that's the second part. And I think this is the much more important part, because the API itself can be very simple. You can of course get the data, send the request from the frontend, send the request to the multiple backends, put it all together and that's it. But this is just a gateway. It's not orchestration, it's just gateway to the other APIs. But in a more complex situation where we have the orchestration, there is a separate layer which actually orchestrates, in the end, what happens between all those systems behind the gate? Yeah. And I think to make it a little bit more understandable for our listeners, you know, what does a commerce orchestration do? I mean, why not just push one API, you know, the data to the other one? I think we can use an example, which we have in our case, Jimmy brings weather using very complex promotions with the promotion engine talent one and would love to connect that and get that in the order lines of BigCommerce. Can you explain a little bit of what was the case there and why we needed commerce orchestration for that? Yes. With that particular case, there is a few examples when the actual orchestration comes in as a handy tool, because again, it's not just getting the response from the third party server. In Jimmy Briggs case, this is a very clear picture of how those pieces work together. Because, for example, one of the requirements was that when customer orders beer, because this is the alcohol delivery service, you can score points. So for example, every ten orders you get one order for free or delivery for free, or there is a bunch, literally a bunch of other rules that apply to not only to the orders, but also to the interactions that customers do. For example, if you add five items of particular product, you get one item for free and so on. And this is when the orchestration comes in. Because you cannot just ask this service, get me the customer's card and service big. Get me the customer's promotions because those proportions are much more complex. Those proportions comes from the scenarios that the user goes through, and each customer goes through that journey. And because of the particular actions that the customer does, the promotion engine applies specific things and those things are applied between the actual platform. So it's not on the front end. In case of Jimmy brings, it was title one, which is the promotion engine of choice. Title one tracks all the operations that customer does during the usual shopping session. So whenever you add something to cart, remove something from cart, change quantity and so on. Apply a coupon. All those interactions are also fit through the composer to the title on one engine. And based on that title on one, does the decision whether particular customer in this particular time with this particular state of the cart can have a promotion applied? So this is not only the interaction between the customer and one of the systems, but those systems interact with themselves. And this is where the composer comes in again, because it actually orchestrates all those events between the platforms and makes sure that everything is synced and served to the end customer in a proper way. So, Matt, what would have happened if I wouldn't have a composer layer if I would just start building using a GraphQL engine in between? What would happen if I would do this? Yeah. Good question. In the end, you would have to build probably the integration between those systems. So you'd have to manually, for example, integrate in this case BigCommerce with talin one, which would be very time consuming. I guess second approach would be to build the layer by yourself. And of course you can do that. It's not a problem. The case is that it's not going to be reusable, and you will have the implementation tied to specific products. Of course, there are upsides of that because you can focus only on what you need. But in the end, if you will want to add another system to that, then the complexity grows a lot. Because if you have a composer, then you build just the logic or put the business logic in that composer, and this is the place which takes care about those orchestrations. But if you don't have that place, you build a kind of mesh between all the systems. So you have to communicate every system with every single one. Yeah, the complexity grows to the level that it's going to be very hard to maintain. And of course, there are a lot of other aspects like performance, scalability and so on, because it's not only about connecting those pieces, but making sure that those pieces work together under high load when there are peaks in the customer's views and so on. Yeah. I mean, and that is exactly what composable commerce is about. So you answer the question, as would I would expect it if you do not have a good composing layer and a business orchestration layer or whatever that orchestration layer, you're going to build a mess, you're going to build a mesh literally, but also a mess. You're gonna have a big spaghetti of code and you're back to the monolith again. So you're breaking down all the benefits of composable commerce. And we see now some vendors in the market, and I think just delivering a GraphQL layer from backend services to front end. And I think this can work well for simple shops. Some products from one side Search from the other side. You're done. But as soon as you're going to hit a little bit complexity. Some systems that have to work together or react to each other, you're gonna really need an orchestration layer. So how does that orchestration work? Matt, I know that, you know, we are providing that in a code way so developers can can work with that. We have it fully open source. So as soon as you start working with us, you have access to that piece and you can start working with that. But I think more importantly is, is that we are there's a bit of a scope for the listeners of our podcast. We are soon going to release this in a no code, low code environment. But can you say a bit about that? How is that going to look like? Yes, there are a couple of things that actually realized that functionality. One thing is that we have the integrations with the third party systems. So this is one thing. And those integrations have a set of features that are, let's say shared or standard across e-commerce landscape. So those are things like adding to cart, removing from card and so on. And those things have event handlers and so on from the technical point of view, so you can hook to those events and implement your own logic. This is one thing. This is a very simple one. Second thing is that you can extend the implementation because you get basically libraries. So you can use those libraries and extend them and build your own functionality. If there is something outside of the scope that we deliver. And usually this because every single business has its own cases, that you have to adapt the solution to those cases. In that case, you can simply build on top of what we deliver and then you have total freedom. You can basically implement anything that you want. All those things that you implement are implemented from the technical point of view as a plugins or extensions or modules that are multiple levels from the code perspective. And those plugins extensions are loaded in the runtime into the composer and basically realizing the business logic. So this is more complex case when you have to have resources to develop those things. But as I mentioned, usually every single business has its own quirks and its own features that must be implemented. It's not like everyone has exactly the same platform and exactly the same processes, so that actually this is the part that allows you to implement your own custom business logic up to your needs. Yeah, that is a massive feature because I'm sure that, you know, a lot of people who dive into composable commerce, they see the first layer where they see, okay, we have frontends, we need better performance there. We have different services that we would like to combine. Then they do some research. They come across products and platforms like ours, among others obviously. And then they're going to see, hey, what I'm going to need to do that. And part of that is fancy dashboarding and, you know, your opportunity to have that freedom. But I think a very important part of that is does this product actually, you know, can handle my business in a composable way. Can we build out our business without adding complexity? And, you know, I think that is happening for a lot of people out there now who start with a GraphQL layer and then having to add a lot of complexity to that just to get it to work. And then in the end, your end product, you still end up with a spaghetti kind of code that only works for that particular situation. But as soon as a new vendor comes up, a new service or some new product feed or whatever data that is happening, you know you're going to have an issue because you cannot start building in that mass again, and it's going to slow you down. One of the real benefits of composable commerce is the time to market, to get new solutions out there, to build a massive operation that can handle peak traffic and all this kind of stuff, but still keep it not complex, keep it easy with an overview. And for that, you need a layer that actually takes care of all that orchestration between the services. You don't want to have that built in to one. You need a layer for that. So today on a code level, people can access that very soon. You can access that in a really drag and drop situation. Uh, for low code, no code environment for still tech savvy people. You have to understand your processes. But I think that is gonna really change what we're doing here. So, Matt, anyone who is starting composable commerce. How would you recommend them from a developer point of view to do? How would you start Composable Commerce project? Yeah, I mean, we take the decision. We want to go composable commerce because we have a certain level of complexity here. You know what to do. Yeah. First thing is not even from the technical point of view, but getting the sense of what actually you need, what platforms, what the interactions are between those platforms, what the business systems, business processes goes behind. So first you really have to know what you want to build, because in old fashioned way, you usually had a one system with multiple plugins. And those plugins were doing some things. And the problem is that in many cases, those plugins were simply installed enabled because someone needed a particular thing. And that's basically it. And you don't have the grasp of all the processes that actually are needed in your business. So first thing is getting the requirements. So you know what actually you need. And then let's of course go with the example of having our composer involved into the process. Then you actually install the extensions for those systems. If the extensions are available, then you can use our implementations and extend those. If you don't, you get our documentation and our support. You can implement your own. Of course, there will be always situation that you have to have some kind of custom code, because the bigger businesses usually have their in-house systems like stock management. Basically whatever system they are using in-house that's built in-house. This is also one of the benefits of the composable commerce, because you can reuse the systems that you already have and you are using those, so you are building on top of them, not replacing them with something else. When the time comes, then you decide that you want to replace it. Of course you can do it, but the most optimal path would be to reuse what you have and build on top of that, instead of rebuilding an entire ecosystem from the very beginning. So having in mind that you already know what you have, what you need, what the business processes are behind you, do the basic setup of the platform of the composer with the extensions that you will be using with those third party systems, and you start building the interactions between them in the composer itself, the interactions, as you mentioned right now, it's a requirement to write the code. So you basically put the business requirements and code those, but in the future it's going to be just a simple. We will of course leave the option to write in the code, because usually if the interactions are quite complex then it's oftentimes very easy to implement that in code. And it's quite easier to maintain those. Cool. I think yeah, I think that's it. I mentioned and I love this topic you mentioned okay, internal systems. And I get this question when I talk with people who are looking at composable commerce. You know, the businesses have been building and been growing and they have internal systems could be some hacked open source version of whatever platform could be something that they started build from scratch when they started to build this. And let's say that's about ten years ago or seven years ago, maybe twelve, maybe fifteen years ago. They obviously they didn't think about in the future, I'm gonna need, you know, something GraphQL. I mean, it didn't even exist then, but, you know, something's going to happen there or in the future. I'm going to need it in a composable way. They started building it. You know, what we call now? More legacy tech? I mean, some of these systems are built in. Net, you know, we see this kind of stuff happening when we look at these older internal systems or, you know, these systems, they work well. You don't want to replace them immediately because they work, but you want to be able to use them in the more modern, you know, let's say out of the box stacks that are being offered today. You cannot use that kind of technology there. You're gonna have to change your old technology to make that work. Please, listeners, don't kill me if I call your system a legacy or old. But what I mean is, you know, the the ones that have not been built with GraphQL and APIs and commerce composing in mind, the ones that have been working for a long time and are very robust and very scalable, these kind of services we want to use in a more dynamic environment. Ready to go platforms do not allow us to integrate that kind of data. So that is where in the world of composable commerce, you need that commerce orchestration layer where you can have all that data in and map it to this digital environment. Matt, can you explain a bit how that works and how do you do that? So you have that, let's say that machine taking care of all your prices, and then you have your website where you want to update the prices every 10s or ten minutes. Yes. This is kind of pretty similar to the thing that I've mentioned. So yeah, we are actually working right now on an interesting case because we are building the integration with the system that exposes only Soap API. So everything that Soap was a standard a couple of years ago. Right now, of course, there are systems that expose Soap, but it's not as popular I would say right now. There are different APIs. Uh, usually it's the Rest API that is exposed by the platform. But yeah, this is actually very nice case for us to validate how we can work with the totally different stack. And in that case, we have the Soap API that our orchestrator communicates with. So from the technical point of view, we built an extension to our composer that communicates with that API and exposes its own API to all the other systems that need it. So basically when customer starts the ID verification, we trigger an event, and that event is received by the extension that communicates with the ID verification system. The extension takes care about entire communication and gets back the result and broadcast that result. So the actual client of that API can consume it. So the nice thing is that in old fashioned approach, you would have to implement the integration within the particular system. Let's say if you use Magento and you have to have that there, you have to implement that inside the Magento, Magento, whatever, whatever backend you use, that would be it. But in the modern system, in the composable approach, you don't integrate the systems between themselves. You integrate them via the composer itself. So there is a customer that comes with the request. Composer takes care about that route. That request to the particular extension and extension takes care about the communication with the Soap system. Soap system processes returns the data, and then the response gets back to the customer via orchestrator. And in between there is a lot of other things, because when your ID is verified, we can also trigger a lot of other actions. For example some kind of promotion because customer is now verified. So we want to encourage them to do the verification. So once the verification is done, they get some kind of promotion. We can of course track that for our analytics purposes and so on and so on. And everything is connected together on the composer. That sounds like, let's call it a mothership control center of all your business kind. Yeah, kind of it is. It is. Yeah. And imagine a room with a lot of buttons now. So last question, Matt, can true composable commerce, and especially for a little bit more complex business, exist without an orchestration layer? It could I would risk saying it could, but it will be so complex and not scalable, so I wouldn't go that way. Of course, you can build everything. If you have resources, you can build everything. But the problem is that it's. Yeah, but then it's not composable commerce. Kind of true. Yeah, yeah. The case is that in the end, you know, in the end, you want to have the system that works under high load. Because one thing is that you have the system that communicates with many systems, and that's fine. But if there is a high load, all those systems must work together and the composer itself must do its work. So it must be scalable without limits, let's say. So this is also part of that solution. It's not only putting things together and okay, we've done that. It works. And when there is a high load everything goes down and there is like no response to the customers at all. So it's not the way to do it. The way is to first have that orchestrator that handles everything between all the other systems and have it implemented in the way and prepared, deployed in the way that it's scalable and the high load doesn't affect it from the performance point of view and from operational point of view. So, yeah, I think you know my answer in the end, if this is a big system, it needs to have some kind of orchestration. And that orchestration must be implemented and deployed in a way that it's scalable. Yeah. Of course. I mean, that is true composable commerce. It's about flexibility, about freedom, about scalability, about building your business and of course, growing your business. So everyone, I hope Mathias and I today could educate you a little bit more about orchestration. And why is that important when you are looking at composable commerce? If you come across composable commerce and you would like to try that, always reach out to us. We're happy to have a conversation with you, but also be critical to what people are offering you. Is it just a GraphQL API layer pulling in some data, or is it truly giving you the freedom and the flexibility and the control over your data? And for those who do not know, Mathias Wosniak was one of our very first people involved. He's built many versions of what we are doing, including also versions where there's no orchestration layer. And then, you know, we hit the scalability issue. So we built the orchestration layer. We've been there, we've done that. And we would love to help you to not make the same mistakes. Last but not least, Wosniak is very, very interesting name. Uh, I always love to say that because I'm a Big Apple fan. Uh, Steve Jobs has his wosniak. We have our Wozniak. And trust us, this brain is great and very lovely to work with. Thank you. Mathias. Thanks, Jamie. Let's hope we help some people. And. Yeah. On to the next podcast. Yeah, thank you very much. Bye. Thank you, everyone, for listening. Goodbye.