Open Source initiative:
https://opensource.orgTranscript (Provided by Otter.ai) (Please reach out with corrections):
George Stocker
0:00 Hi, I'm George Stocker. And this is the build better software podcast, a podcast, where we talk about the issues that will help software teams build better software. Today, I'm answering the question, should your team adopt that open source library?
Now for the purposes of this episode, I'm going to use the Open Source Definition provided by the Open Source Initiative. At https://opensource.org.
There are two main divisions, there are copyleft licenses, which require that derivative works, use the same license as the original work. And then there are permissive open source licenses. And that's anything else that's not a copyleft license. A permissive license basically allows you to do anything you want with the software. Again, I'm not a lawyer. This is not legal advice. This is just my understanding of it. So back to the question, should your team adopt that open source library?
First question we want to ask is, is the license that you is going to help us or hurt us. That is, is the license of the project copyleft license, which means that we probably need to redistribute what we're doing if we use that library in the same license form, or is it a permissive license? Will it basically allow us to do whatever we want with the software? That's the first question you have to answer. And that's a legal question. I can't answer it for you.
The second question is, is that open source library central to the problem domain your software operates in? Or is it 10 gentle to it? Here's what I mean by that. If you are a data processing company, then you will pay very close attention to how your sorting and searching algorithms work. You may even adopt your own ETL processes for data warehousing. That's to be expected after all great data processing company. That's what you do. Now, if you're a web designer, company and you design websites for other businesses, then you're probably not going to build your own web framework. Because that's not how you make your money. You make your money by giving someone a finished product, not by putting your time and money into making a web framework. If the software that you're trying to adopt as tangential to your problem domain, you're more likely to decide you either want to buy it or use an open source library than you are to say, we want to develop this functionality in house ourselves, then to say we want to develop this functionality in house ourselves.
The next question you're going to ask is, Will adopting this library helped my team solve the problem it's trying to solve faster? Here's what I mean by that. All software, whether you build it or whether you buy it, or whether you adopt an open source library has an adoption timeframe. You have to take that into account when you're deciding whether or not to adopt a library or framework. For instance, when we were adopting RabbitMQ as our message bus We had to adopt it and understand what it did with each and every language that we use to interface with it. So it had drivers written in Perl and C# in TypeScript. And what we had to do is everybody that taught to RabbitMQ, had to invest that time into learning those API's. Now, in some cases, that'll be a very trivial amount of time. After all the software has good documentation, or it has very self explanatory use cases. Other times, it won't be, and you can't just pay that time for the person implementing it. It's everybody on the team that's going to be touching that open source library or framework. They all have to worry about this.
And that gets to the next problem.
What is the worldview of the open source library you're trying to adopt versus what is your applications worldview? If you're adopting something like SQLite as a database engine, it has a very strict understanding of the world. It believes that it's going to be a single consumer database that is, that might be one connection to the database. And it will run on a local or a situation where only one person will try to talk to it at a time. That doesn't mean it can't handle bigger workloads it can. But it means that it was designed around the idea of a self contained database isolated from the larger world. Contrast that with something like SQL server or Postgres, and its idea of the world is, yeah, there's going to be a lot of people connecting to me at once, and they're all going to want data. How do we handle that? Every open source framework or library has a worldview, and you have to understand its world view, to understand how much trouble or how easy it will be to integrate into your application.
George Stocker
4:52 The next question you have to ask is, how often is this open source library updated? What is its traffic look like? If it's on GitHub, how many forks does it have? How many open issues does it have? How many closed issues does it have? How many pull requests are pending? How long have they been pending? For, you want to know the answer all these questions because it's going to determine whether this is a hobby project, or whether this is an actual serious project that you will be able to rely on without having to pick up the pieces yourself. Just like my own hobbies, I get to them when I get to them. I haven't played golf in almost three years now, just haven't been able to get around to it. Now, if golf are my job, I'd be playing every day no matter what open source software runs the gamut from Hobby to business. And before you adopt a library, you have to know where it fits. If an open source library isn't updated very often, or if it has open issues, then you'll have to fix those issues if you decide to adopt that library, which means forking the project. There's a second part to updates. Once you've adopted an open source library into your system. You now have to keep up to date with its security releases with its minor releases with its major releases. Hopefully it's following something like semantic versioning where you can tell Just by looking at a version number, whether or not a change would be breaking or not. If you're going to adopt a library, you have to fit its release cadence and its update cadence along with your own. Miss gets further complicated if you're releasing your software as a distributor, as a distributable, to your users, in maintaining open source software that you've adopted into your project, you also have to worry about forking and releasing changes that may not be upstream of you. For instance, you're using a data grid, you find an error in that data grid, you fix it, you've released an upstream patch, but you don't want to have to wait for the library that you're using to be updated for them to release a new version. So you go ahead and you fork it, you make the change in your forked version, and then you use that version. That's something that you're going to find yourself doing. Often in open source software, the longer you use it. You need to plan for that and you need to understand it can happen What to do when it does happen. And in fact, if you're going to adopt an open source project, look at how it's built and look at how it's deployed. Those are two areas where you're going to find trouble when you need to fork or make changes in the software for your releases that aren't going to be accepted in the upstream release. Overall, with open source software, there's a certain feeling of you've adopted it, you get to maintain it. Now, whether or not that's legitimate is up to each project that you've adopted. It's different for each one. In some projects, changes are adopted very quickly, and they're fixed very quickly. In other projects, it could take weeks or even months for changes to be adopted. That means that whenever you adopt an open source library, there's a high likelihood that you will have to fork it and make changes in your fork. Your team needs to be prepared for that. The next step is giving back does your company Allow you on the company's time to give back to open source. Now in the example I just gave where you found a bug in a library and you fixed it, well, you've got to give that change back in order for it to be included in a release. To do so, there could be copyright issues. In other words, the code that you wrote for your employer on their time, as a work for hire is different than you making the change on your own time on your own computer. Without doing it for your employer for pay. You need to clear with your company's legal division. If you have one, what code you are allowed to submit back to an open source project and whether or not you're even able to submit code back to an open source project. This is a legal question, talk to your company's lawyer. But if you're able to give back, that's a very good thing. It allows us to keep it open source software going reinvigorates the author of the open source library that they have people that want to help out and it looks Good for your company too. Now one of the most difficult parts about adopting any piece of software is understanding it through its documentation. Before you adopt a library, you're going to see what kind of documentation it has, whether that's on the libraries on site, or whether that's on Google
George Stocker
9:19 Stack Overflow, blog posts, anywhere, you want to see what people are writing about it, so that you can better understand whether or not a problem that you have with it will be solved by someone else. With open source library, you can assume that there will always be a problem when you adopt it. Always. The trick is, is can that problem be solved before you have to solve it? Has someone else solved it and fixed it? Or has someone else even seen it? The smaller a community is, the fewer number of people who have written about it, the fewer number of people who've probably seen the problems you've seen, and it's always a consideration when you're deciding Whether or not you want to adopt an open source library is, what's the chance someone else in my tech stack is having the same problem with the library, and will found a solution to it, or will have written a solution to it. Before I get to that problem, it does go without saying that every problem that you find in software affects your ability to deliver your product or the problem that you're trying to solve. So it's always something to keep in mind. Now on the subject of community, you want to look for an active GitHub community. If the project's on GitHub, you want to look for Stack Overflow questions and answers about it. And in general, you want to see a lot of Google hits when you Google problem with library acts. Now, one of the problems we don't talk about enough in open source but it happens because of the way the open sources set up is project abandonment. Rarely, a project author will come out and say I'm no longer maintaining this. Good luck. Have fun with it. More often than not, it just sits There with no one answering questions, pull requests not getting accepted, and issues not being responded to, or no commits to the project for a given length of time. And you need to take it into consideration with every library. If I use this library, will it go away? Will the author stop supporting it? If the author stops supporting it? Who will step in? Is there a community that may step in? Is there not? Am I willing to step up and lead a community around this project? If the author abandons it?
George Stocker
11:33 Now you don't have to answer Yes, in order to adopt an open source library. But you do have to keep that in mind. Because if the author abandons it, and you need to make changes, you're now on the hook to make those changes, even if it means forking it, making the changes and not releasing them upstream. Now, we talked a little bit about legal considerations before but your primary legal considerations are, does this software's license. Let me do what I need to do with it. And does it affect my financial interests as a couple Money? And am I allowed to give back to this project? Through my company's own legal policies? Will there be copyright issues? If I fork this project on company time, make a fix to it on company time, and submit that fixed on company time. There's always also a consideration that a project may change its license in the middle of its lifecycle. MongoDB that this, Elasticsearch has done this, and a few other big name open source projects have done this. And they've done it because cloud companies that shall remain unnamed, swoop in, fork the project, release a commercial version of it, and then take all the bread crumbs for themselves. Now this affects everybody that uses that software, you now have to determine Are we going to pay for this new license? Or are we just going to use the other older license and not accept any new changes? So these are everything that you have to consider if you want to adopt an open source library. Now on the other side of that You can build it or you can buy it not going to cover buy it here, that's pretty much a financial question is does your company have the resources to pay for a library? Or to pay for a framework? And then do they have the resources to keep paying for it? After you build it yourself? You don't have any of the issues that I talked about before the civvie their documentation or just their community exist? Or do they abandon it? Or do I have to give back to it? Or, you know, how often do their updates come? Or what is their software's worldview? No, I mean, this is all centered around you and what you need as a company. If you build it yourself, you're now responsible for it. anything goes wrong, you're on the hook to fix it. Every developer from hither to Yon in your company, will now have to become an expert on whatever this thing is. It's also going to take longer than you think to build it, especially if it's tangential to the problem that your team operates in and the problem Your software tries to solve, the closer it is to your core competency, the faster to go farther away it is, the harder it will be. If you want new features, you're going to have to be the one to add them. There's no magic feature fairy that's just going to add features to the software that you build. No, you get to add it. And there's now a documentation cost. You built it, now you get to document it, then if you don't document it, you have to run institutional knowledge to understand how to use it. And if you're relying on institutional knowledge, that's people and people leave people win the lottery. And people forget. There's also the idea that you may not actually have the expertise needed to solve this problem. Think about ORMS and the object relational mapper has been around for at least 20 years. One time I worked with an employer where they created their own object relational mapper. Now they weren't experts in database design and an object oriented programming, but it seemed like an easy process. To solve Sure, I can write my own object relational mapper. And they did, or the following 6, 7, 8 years, the entire team had to deal with this framework that didn't have wide adoption had no adoption outside of the company had no documentation, no new features other than if someone on the team spent the time to create those features. And there was nothing portable about the object relational mapper, they couldn't take their knowledge of it somewhere else. It created lock in, in the worst possible way. It locked employees in to a dead end technology that they were sad to use every day. Now, this is just one example of not having the expertise required to solve the problem. When the team needed something new that the software couldn't do. Well, that wasn't taken into account when it was developed. And then therefore, they ended up using another ORM anyway, because the software couldn't do what they want them to do. And that's going to happen. If you're building software. That's not part of your core competency. You're going to miss fundamentals in that software, and your team will have to pay for it in some form or fashion. This isn't to say you can't write your own software that you can't write your own ORM or your own web framework, to really need to ask yourself, are you going to get the value out of it for how much it's going to cost you? Now, all this is going to depend on your business contexts, your social context, your team's context, the technical context the software operates in and your customers needs. I can't answer all those questions for you. But they're questions that you'll need to answer in order to be able to decide whether or not you want to adopt an open source library, or you want to build it yourself. Now, that's all for this week. I'm George Stocker, and I hope you'll join me again next time on the build better software podcast.
Transcribed by https://otter.ai