Show Notes
About the Guest
Dan spent 12 years in the military as a fighter jet mechanic before transitioning to a career in technology as a Software/DevOps Engineer/Manager. He's now the Chief Architect at the National Association of Insurance Commissioners. He's leading the technical and cultural transformation for the NAIC, a non-profit focused on consumer protection in the insurance industry. Dan is also an organizer of the DevOps KC Meetup and the DevOpsDays KC conference.
Links Referenced:
Transcript
Mike Julian: Running infrastructure at scale is hard. It's messy, it's complicated, and it has a tendency to go sideways in the middle of the night. Rather than talk about the idealized versions of things, we're going to talk about the rough edges. We're going to talk about what it's really like running infrastructure at scale. Welcome to the Real World DevOps podcast. I'm your host, Mike Julian, editor and analyst for
Monitoring Weekly and author of O’Reilly's
Practical Monitoring.
Mike Julian: Alright folks, I've got a question. How do you know when your users are running into showstopping bugs? When they complain about you on Twitter? Maybe they're nice enough to open a support ticket? You know most people won't even bother telling your support about bugs. They'll just suffer through it all instead and God, don't even get me started about Twitter. Great teams are actually proactive about this. They have processes and tools in place to detect bugs in real time, well before they're frustrating all the customers. Teams from companies such as Twilio, Instacart and CircleCI rely on
Rollbar for exactly this. Rollbar provides your entire team with a real-time feed of application errors and automatically collects all the relevant data presenting it to you in a nice and easy readable format. Just imagine — no more grappling logs and trying to piece together what happened. Rollbar even provides you with an exact stack trace, linked right into your code base. Any request parameters, browser operating system and affected users, so you can easily reproduce the issue all in one application. To sweeten the pot, Rollbar has a special offer for everyone. Visit
rollbar.com/realworlddevops. Sign up and Rollbar will give you $100 to donate to an open source project of your choice through
OpenCollective.com.
Mike Julian: Hi folks. I'm Mike Julian, your host for the Real World DevOps podcast. I'm here with Dan Barker and the chief architect for the National Association of Insurance Commissioners. Welcome to the show Dan.
Dan Barker: Hi Mike, it's great to be here.
Mike Julian: So National Association of Insurance Commissioners, it's like the four least interesting words in one title I've ever heard. What in the world do you do?
Dan Barker: We thought if we combine them that it would be more interesting, that may not have had the intended effect. So the NAIC is a nonprofit, about 150 years old. We kind of got our start organizing events for insurance regulators, so primarily the chief insurance regulators for each state and territory and we organize events to get them together. And we also created model law. Over time, as technology advanced, we started to host some centralized technologies within the NAIC and providing a lot of the back-end applications that regulators use. We also offer something called
Insure U, where you can go and learn about insurance, all kinds of insurance, I'm sure that it will get flooded now.
Mike Julian: Because we're all chomping at the bit about insurance.
Dan Barker: Yeah. And so we still do a lot of the event planning, taking regulators and kind of informing them on technology. There's a big movement in Insure-Tech with tons of investment right now. And so we're trying to make sure everyone is staying up to date with things like blockchain, AI, a lot of the data-focused stuff particularly around unconscious bias and the data and how to clean that out. So we're a pretty diverse group and we have a lot of kind of different focuses but one of those is the technology side and I'm the chief architect and leading up a lot of the technological transformation as we move to Amazon Web Services, moving everything to the cloud and moving to some more open source tools and trying to move towards a DevOps culture.
Mike Julian: That sounds like a pretty fascinating situation.
Dan Barker: Yeah, it's really exciting. We have all kinds of new tools and new things. We're moving towards ... This company has done a pretty good job of standard blocking and tackling actually, what you might be surprised at in a nonprofit.
Mike Julian: Right.
Dan Barker: One of the only companies I've ever even heard of that had one version of Java and that has been-
Mike Julian: That's pretty impressive.
Dan Barker: Yes, I continue to ask in every meeting if that's true, and-
Mike Julian: Are you sure?
Dan Barker: Yeah. I'm sure there's one somewhere around here. But yeah, for all of our applications, they're all on the same version of Java and it's up to date to you know, like Java 4.
Mike Julian That's pretty impressive, congratulations.
Dan Barker: Yeah, so it's a great base to start on. And it's been a great journey so far. I've been here for a year and the CTO that kind of came in to start this off has been here for, I think about three years. So it's about the length of the transformation so far.
Mike Julian: Okay. So we're talking about transformation here. You actually gave a talk about this at DevOps Enterprise Summit in Vegas earlier this year, or 2018. What was the problem that started this entire transformation process? What were you trying to solve?
Dan Barker: So, we had several different opportunities that we were looking to kind of utilize moving forward. So we have this infrastructure that has been ... We're a nonprofit, so we don't have a ton of funding and it's been a bit challenging. So we haven't been able to move as efficiently because we haven't optimized a lot of our technological systems, much of what we have has been more by request, kind of an IT department within a company, non-technical company. And so we're trying to move towards being more of a technology company which requires a little bit more proactive planning. So one of those areas we're trying to gain some efficiencies, gain some efficiencies across all the different teams. So trying to standardize a lot of our [inaudible
00:06:30] methodologies, standardizing our development techniques. We also have a lot of silos, so we were siloed not only in our operations and development sides, being siloed but we also had each development area being siloed. So we have three main areas and they all do everything differently, which is very challenging, especially when you try to move someone across teams. As priorities shift, it becomes very hard for them to pick up on what is going on there. And they really form different cultures, different processes, and they all use different technologies.
Dan Barker: We also needed to improve our technology. So I talked about Ensure-Tech coming. And the regulators and their staff are expecting more from us on the technology side to help them better regulate more technologically advanced companies, particularly companies that are now getting into blockchain and AI. And we need to advance all of our technology and our training and understanding to a point where we can explain to them and hopefully help audit some of the algorithms that'll be used in the future and validate that the data doesn't have any implicit bias in it. That they aren't noticing or that the company hasn't noticed, which has been pretty common to this point, that it's something that hasn't been looked at enough in most companies and most of it is unintentional. It just happens to be that the data is formed in a way that there's unconscious bias. So we need to accelerate all of our technological capabilities to deliver on those types of features that will help protect consumers of insurance, which is a long-term buy. So you're not going to know if it's going well until it's too late. So we're trying to protect against that.
Mike Julian: Right. So if we were talking about, what was the whole business case behind this, it's really that second one. The second group of explanation is the business reason of why you started down this process of your main stakeholders. The insurance commissioners are looking at the market seeing a whole bunch of changes and realizing they're not ready, from a technical perspective, to handle those changes.
Dan Barker: Right.
Mike Julian: So they're looking to you for that support and that increased capability
and flexibility to handle the major shifts in the market that they're seeing.
Dan Barker: Right, yeah. The main driver is definitely that we are able to offer faster response to demands by regulators and higher quality products for them. And notice I never said anything about saving money.
Mike Julian: Right.
Dan Barker: And that is not even an expectation of ours as we move to the cloud, which is something important, that we've often focused on saving money, but now we're focused more on delivering high quality products.
Mike Julian: That's absolutely interesting. I want to dig into that a bit. Because every DevOps transformation we tend to see, as well as every cloud migration is either explicitly or implicitly focused on cost savings. We have all these companies out there with their own data centers, and now they're looking at cloud services and realizing, "Hey, we can save a boatload of money by moving." The reality is that they rarely do, they always end up costing more, but they generally come out ahead.
Dan Barker: Yeah, exactly.
Mike Julian: It's interesting to me that you've kind of skipped that whole thing and said, "No, we're just not going to focus on saving money here. We're looking for the increased capability."
Dan Barker: Yeah, so that's something that we discussed as well, about how we were going to focus on that or choose not to focus on that and it's something that was discussed. As soon as I got here, I was supposed to read and edit and I did a document called State Ahead, that we released shortly after I arrived here and Scott Morris, the CTO here had written most of that for the technology side and in there it never mentions anything about saving money. And the point of that was to make sure that people weren't focused on trying to save money, because we thought that we would save money, but we didn't want to make it the priority, the focus because we could make a lot of things, a lot of caveats, a lot of choices that are maybe negative to save money that negatively impact our quality or speed of product development. And so State Ahead is something that you can find out on the
naic.org website. It's our three years strategic plan. It's been really cool to work for a nonprofit because they ... And a particular nonprofit like this where the board is a little bit more in flux. So they've had a lot of problems of every year the board changes because of the way it's cycles. I won't bother explaining here. But anyway, it changes every year. And the membership changes regularly because people get re-elected or they term limit out or they have-
Mike Julian: Does it roll all at once every year or is it staggered?
Dan Barker: No, it basically ... Yes, so I guess I'll explain it here.
Mike Julian: But my whole point there is are you losing all of your stakeholders every single year?
Dan Barker: No. You're losing the top one pops off.
Mike Julian: Okay.
Dan Barker: It's basically a queue.
Mike Julian: You have an organizational queue, congratulations.
Dan Barker: We have our board at the queue. So it's a first-in first-out queue and there's five of them. And so every year we get a new president and a new vice president. Our CEO and COO are static, but we do have that change and we have membership changes as well who approve all these things. So the reason we wanted State Ahead is because we wanted a three-year plan that everyone in the membership was bought into. And so that no matter who got into the board, it was very likely that they had signed off on the three-year plan so that we could commit to that. Because a lot of times people would come in, they would have their own initiatives for their own state or wherever they were at, or whatever the reasoning was, they had their own initiative. And so they would focus on that for a year, and then we do the next year and the next year and the next year. And so very few of the projects were we able to extend beyond just one year. And so this was really great to have that and then also all the budgets and everything, all the proposals I write up are public. So you can actually go and comment on them if you'd like to. All of our technology proposals are out there. They're available for comment for a certain number of days and then they are voted on after all the comments are addressed and we'll usually address comments if you have them.
Mike Julian: That's pretty cool, also a little nerve-wracking.
Dan Barker: Yeah, well, I was really happy that my first cloud one didn't get any comments, because I was a little nervous. It was the first one I'd ever done. But I think the next one after that had gotten some really good comments. So that's really helpful to help shape the direction of the insurance industry.
Mike Julian: So was this report how you actually got started on this whole process, or did you start somewhere else?
Dan Barker: Well, so that's kind of the end of one journey.
Mike Julian: Okay.
Dan Barker: Or the culmination of one journey maybe. And so what happened is ... So Scott Morris was here for a couple years before I got here, and he kind of started on the culture side and he really wanted to build a conviction in leadership so that we didn't sway after a few months, things don't ... Public companies, it's very common to have three to six month window, and then if it doesn't show massive improvement, then it's gone.
Mike Julian: Right.
Dan Barker: And this is not a feature improvement to an app, that we can show value coming back. This is largely a paying off debt maintenance site move. That should show improvement but it'll take time to show. So he wanted to build that conviction. So he took a lot of the leadership to Amazon, to their data center, took them to partner companies, the leadership team and the technology area went to partner companies to discuss what they've done with cloud and how they move with blockchain and other technologies. So we really wanted to build some shared experiences, so that we all had the same vernacular kind of to look back on.
Dan Barker: We did an initial assessment as well. And we used that throughout the entire transformation. We did it with AWS, and it was focused, it was AWS, an AWS partner, and it was very focused on those technologies, but we've used it for a lot of the same verbiage and language so that we had a common lexicon moving forward. And that has been really helpful, even though we aren't necessarily using all of the recommendations, we have something to look back on, that's a common place. He also went ahead and encouraged everyone to read the Phoenix project. And so that went really well. Everyone really enjoyed that book and kind of understood why we're doing what we're doing and the focus initially was all about culture and has really been about culture, the entire time. We're doing technological things, but we know that we have to have a solid base in the culture before we can execute on those technological areas effectively.
Mike Julian: What is culture media in this context? What were you trying to change? What were you trying to pay attention to?
Dan Barker: So, I mean, this place has a really good culture. I don't know what it was like when Scott arrived, but when I arrived, I was shocked at how nice people were and how open people were to different ideas and to kind of shining light into areas. And what we focused on is really a culture of continuous learning, trying to encourage people to proactively go out and learn on their own. This is a nonprofit, a larger enterprise and it's right next to government. So it had a bit of a top down structure for a long time. And a lot of people were used to ... I'm sure they've been in trouble for doing things they weren't told to do.
Mike Julian: Right.
Dan Barker: And so they had a bit of a defensive mechanism of kind of like, "We’ll, wait until we're told what we're supposed to do and then we'll go to that." And so they've very quickly, once given room, kind of opening up space so that they can move on their own and solve their own problems. That was the big piece, kind of empowering everyone, so they can solve their own issues, their own problems in their area, rather than having someone have to say, "Hey, this is how you're going to do it, and this is what you need to do." More just giving direction and additional context. We've done that in many ways, so we do lunch and learns. Those are really hard to keep populated at other companies that I've been at. It takes a great deal of effort. We haven't had a single vendor come in and speak, I don't believe anyway. We may have had some consultants, but pretty much everybody has been internal.
Mike Julian: That's awesome.
Dan Barker: And we've had it every other week, but I used to do it once a month, and it was hard to staff the vendors a lot of times. So it's been pretty amazing and there have been talks on all kinds of things, work-related, not work related and really impressive to see everyone share-
Mike Julian: How do you keep that so populated? What's the secret to success with that?
Dan Barker: I think some of it is just the inherent culture here that people want to share and want to help. That might be a side effect of working for a nonprofit that's focused on insurance, that includes things like health care and other things about helping people. So I don't know if that's part of it. Definitely the person in charge that Gail McDaniels, he really gets out there and tries to gin up interest in coming and speaking. So he's done a great job running that piece. We've also done a tableau vizathon, a city-wide tableau vizathon is what they call them.
Mike Julian: Okay.
Dan Barker: I'm not a tableau person, but that was really well attended. And we had basically a hackathon and we really helped kind of get our folks involved in the community more and we get all of our money, all of the nonprofit’s money comes from consumers, it comes through the companies that pay us based off of, I don't know some algorithm that other people know in the company. I have no idea how they get the money. So this comes from consumers, and my goal when I got here was, I wanted to give back as much as possible. So some of these city-wide events that we've done, we've done meetups and other things, was something I wanted to focus on, just like our adoption of more open-source tools has largely been, so that we can give back more to the community since they're helping fund us.
Dan Barker: So that's been a big thing. We also have done a hackathon. So we did this during the AWS re:Invent Conference. And I think we may have a hard time getting people to go to re:Invent because it was such a successful event here. We watched the keynotes live as a group and we actually had a Slack channel open, so that people at the conference and in Kansas City could live chat about what was happening during the keynotes. And then we had a hackathon that had, I think it was over 40 participants. We only have about 260 engineering staff. And a lot of people, I think another 20 or 40, or something like that were in Las Vegas at re:Invent. So it was a pretty good attendance just for the hackathon. And we had a lot of cool things that people produced, some that I'm probably not allowed to mention.
Mike Julian: These are always the most interesting ones.
Dan Barker: Yeah. So it was a great time, everyone really seemed to love it. And we also had internal people present. We had AWS and GitLab come and present for us and we had a couple other people present on culture. We had an internal panel. I mean, it was really just a ton of sharing for I think it was three days long. We rented out a local theater. I mean, this was ... The amazing point or the amazing thing about all of this wasn't even necessarily the event, which was amazing. But it was how the event was planned, which was all in a Slack channel. So basically a guy named Dennis Wilson who's the Director of Technology over on the NIPR side, which is the National Insurance Producer Registry, which is a wholly owned subsidiary, a lot of explanation. Go watch my talk notes. I'll talk a little bit more about it.
Mike Julian: That talk will be on the show notes.
Dan Barker: Yeah, okay, great. So he just kind of mentioned it and had an idea of watching the keynotes, and then somehow that snowballed into a hackathon and all these other people coming in and talking and renting out this theater, and having all these GitLab training sessions. And it was pretty amazing to have people just come into the Slack channel as they heard about it or said, "Oh, well, I think I can help with that." And they would just come in, kind of read the history, grab a task off the task list and start going. I mean it was wild to just sit there and watch and not really get involved anymore.
Mike Julian: Yeah.
Dan Barker: Kind of like ideas stuff.
Mike Julian: That's pretty cool. To me I see a pretty direct line between the ... There's something you mentioned earlier where when you first got there people ... Or before you joined NAIC people were very, "I will wait to be told what to do." But since you've been there that's not been the case at all people are jumping to do whatever they think is necessary to do. That culture change, whatever prompted that, wherever it got started to me is a pretty direct line. So from there to what you're talking about, of people feel empowered to do what they think is necessary or do what they think is interesting.
Dan Barker: Yeah, definitely. And I think it's all just really giving people opportunity and allowing them that opportunity to succeed or fail, and to not judge whether or not they succeed or fail, but to judge that they've taken the opportunity. There are times where, whether I'm predicting it right or wrong, I may have seen that, "Well, this probably isn't going to work well." But it's better to let it happen and let it succeed or fail than say, "Well, I don't think this is going to work." Which is, "Stop it now." Because you're automatically not empowering them, which is by definition, you're not empowering them.
Mike Julian: Yeah, but it's exactly.
Dan Barker: …protect them from possible failures. And it's like, "I may have done the same thing, but this is a different context as well." So when I started, I was very clear that I may have done similar things at other companies, but the context here is different. And that's going to change a lot of decisions, and I will not be the ... Very quickly we will make decisions where I have never been here either.
Mike Julian: Yeah.
Dan Barker: I've never been to this situation before either, so you're going to have to help out.
Mike Julian: Yeah, absolutely. So kind of switching gears a little bit, you mentioned how you got started with this, with your CTO, working with the executive teams and taking them to partners, taking them to Amazon data centers, basically getting executive buy-in. Once all that got started, and was underway, how did you get everyone else on board? What did you do for the rest of your management? What about the engineers, maybe people outside of engineering but are kind of on the ground, doing the work? What did you do to get everyone on the same page?
Dan Barker: Yeah. So part of that was the State Ahead document. But we even got started before that with taking an application to the cloud. So we took one of our smaller application, but something pretty important and we move that up into AWS and we use Lambda and we found some pain points in there with Java.
Mike Julian: As one does.
Dan Barker: It’s like the older Java apps spinning up on Lambdas. But it was a good experience, and it's still running up in production. Now we've obviously made modifications and updated it, but we wanted to get a win. And then we also used a bunch of new technologies for another app with MongoDB and some other more NoSQL type systems and that was a big hit because we were able to get it done very quickly, replace a old and fairly complex system and do it in half the time and half the budget of the original project.
Mike Julian: That's pretty awesome.
Dan Barker: Yeah. Which was amazing.
Mike Julian: I bet everyone loved that one.
Dan Barker: Yeah, definitely. Everyone loves that, even if it's not about money.
Mike Julian: Right.
Dan Barker: People love saving money.
Mike Julian: Yeah.
Dan Barker: So that was a huge success. And so we did those with a small team, engaged team. We tried to pick people who would really step out of their boundaries and help wherever they could. And we then just repeat that over and over again. What I've learned is that if you're not tired of saying it, then you probably haven't said it enough. And so I try to make sure that I'm fully tired of saying everything and congratulating people and telling people what a good job this company has done so far and particularly those two were kind of trophies that we can hold up and say, "We've done this already, we don't have to worry about this. This is something that everyone can achieve.” We also offer a ton of opportunities for learning. We'll basically pay for whatever courses you want to take or if you want to have a subscription to any of the online learning systems we'll provide those, we buy books like crazy. Had an order of, I got a little worried. So I listed all the books that I liked and gave links for them on Amazon and it was over $1,000 in books.
Mike Julian: Wow.
Dan Barker: Things I could remember kind of off my head and they bought them.
Mike Julian: That's incredible.
Dan Barker: Yeah, and I was like, "Wow. You just bought $1,000 in books. The for-profit companies I've worked for would never do that." So it was really great. We have our own little library. We also have an actual formal library, which is an interesting component of the kind of the cultural transformation journey. They're actually researchers up there, because we have a research center and we have model laws that we create. And you can basically ask them to do any research and they'll just go do it and send you back all this amazing stuff.
Mike Julian: That's pretty cool.
Dan Barker: They do pretty much everything now. But one of the people, Erin Campbell, jumped on to Slack before we ever told anybody about it. We were just kind of beta testing it and I guess she heard through the grapevine, how to get on it. I don't even know how she knew how to get on it. But somehow she showed up there, created a book club channel and some other channels, and then started ordering books that people were talking about and then taking them down to their desk and giving them to them when they were in then really engaging with everybody which was an awesome thing to have, when you have someone who's not in the technical area, actually in the library, which you don't usually think of — I mean it's like insurance.
Mike Julian: Right.
Dan Barker: Most people don't get that excited about the library, I'm sure there are people yelling at me on their radio now.
Mike Julian: Absolutely.
Dan Barker: Although they're probably whispering, “Let's be honest.” So she got on and really engaged and that was a great thing to be able to show people on the technical side that, "Look, there's someone from the library getting on here and fully engaging." And another piece that we were able to show, particularly developers but also project managers and management throughout the company, is that I did some of the docs on our internal communication standards for the IT group — and I basically copied most of those from GitLab, but don't tell them. They're all in their public handbook. And so I had the head of HR go in and help us make sure that they were all in agreement with it and he actually submitted a commit back to GitLab, the HR director.
Mike Julian: That's incredible.
Dan Barker: It is shocking, right? And so then you-
Mike Julian: Yeah, that's amazing.
Dan Barker: Yeah. So I went and helped him through it and stuff and we talked through everything, but he did everything. I never typed anything for him. And so it was great to hold that up, as kind of like a collaboration with someone outside of the IT group and that they're engaging in these systems that we're using and that we find to be easier systems to interact with because that kind of stuff, we can put it into a web page, we can put it into Confluence and all these other tools through automation. Once it's in a ... like git-type repo. So they thought that was pretty cool. And to have someone who's on the HR side interacting with our systems was a nice achievement.
Mike Julian: Yeah, that's great.
Dan Barker: And we've also had engagement, I mean, this company is really amazing. We've also had engagement with our lawyers. So they've been always very fast to respond and always do a great job of reviewing all the standard stuff you'd expect IT lawyers to be reviewing. However, they also engage with us early on our ... Nexus IQ Server engagement with Sonatype and coming in and trying to understand how they're going to work in this new system. They never complain about something being new or us trying to get them into other systems. They've been fully on board with new open-source program that we're introducing. So we can open source more of our internal stuff and have more contributions to open-source. They've been really strong advocates. So it's pretty amazing to have so many people on board with the transformation across every area of the company, really.
Mike Julian: Yeah that's great. So all that sounds amazing, but surely there's been some stuff that hasn't gone as well as you'd like.
Dan Barker: Yeah. So there always are, right?
Mike Julian: Right.
Dan Barker: So a lot of times with these things, certainly everyone wants you to predict the future of how long things will take. So you're always going to be behind schedule because no matter what you say, something happens. We've taken down our kubernetes clusters that run in GitLab or that [inaudible
00:35:00] the runners. That was a bit of a hard impact but we're not ... We've done a good job of introducing blameless postmortems and the blameless culture so that people don't ... You know, we don't want them to feel bad if something happens. It was a honest mistake, it was a growing pain of having to swap out some fundamentals of the AMI that cause the overall crash of the kubernetes system. And so that was a little bit of a setback and we've had multiple things like that. Networking is really hard between [inaudible
00:35:38] in AWS in the data center, and we've had some struggles of ... There's the silo mentality of, "I am network." Or, "I am database and I don't know anything about the other things," has been a bit of a struggle. We have one person on the kind of the platform team building out a lot of the platform components, who is a database administrator and has learned a ton of other stuff now. But it's funny because that person will answer questions about monitoring, but then will get upset if someone wants to do database stuff because database stuff is just too complex.
Mike Julian: Yeah, interesting.
Dan Barker: Yeah. And it was interesting podcast last week or the last podcast on this series talking about databases actually, because it's very pertinent to a lot of the things that we're doing. We've had some connectivity issues and we've questioned, “Do we have any settings anywhere limiting connections?” And we've had this persistent issue of single connections are clearly being limited somewhere because they just hit a flat plateau and just stay there, and you can run 1000 of them and they'll increase linearly. But every one of them has the same limit. But we haven't been able to find it. And we were told very sternly, that there were no limits in the system anywhere. And we're still searching that one out, we're still trying to make sure that when we move more of our apps up there, that we are going to be able to connect our databases on premises, because we're not prepared to move them. At the same time we feel like that's going to be too big of a bang, right? To do all at once. So we're trying to do the apps and then the databases. We also have ... I mean, it's a normal enterprise, so a lot of apps talk to other apps' databases. So you gotta wean those off during that process as well or at least understand the connections. And we've had other things, so open communication like Slack is hard. I sometimes kind of suck at biting my tongue. And so I've made mistakes and I've said things I shouldn't have in public forums, and I know that others have argued and things. And so it's been really nice to be in a company where people forgive you rather than holding things against you because you said something when you're upset and in Slack, it's permanent.
Mike Julian: Right.
Dan Barker: You can delete it but I think that it's better-
Mike Julian: Someone saw it.
Dan Barker: Yeah, someone saw it, and I think it's better to leave it, to show that, "Look, you can recover from a failure like that." That's a failure of ... It's a different kind of failure than the technical failures. We often talk about DevOps, but I think it equally is important because we all have hard days and we have hard weeks or months or years.
Mike Julian: Right.
Dan Barker: And stressful situations that come up where you accidentally say something or type something that you wish you hadn't.
Mike Julian: Yeah.
Dan Barker: So it's a very forgiving culture. And that's a personal failure for me. But we've also had some of the technological failures and we've had a lot of questions on QA, what's going to happen to QA? And MBAs and other types of jobs like that. And for the most part, I've tried to share articles or other information but not specify a direction, because I want the leaders to emerge in those areas organically and then lead those teams through that.
Mike Julian: So all this has been pretty awesome, is really fascinating stuff. For those people that are listening that are also working in a similar organization or looking to start or perhaps in the middle of their DevOps transformation, what advice would you give them?
Dan Barker: So I guess the most important thing that I've learned is patience. You have to be patient. These are usually long transformations. Oh, they're really never-ending, I don't if you achieve the perfect culture and then you don't have to do anything. You'll repeat the same thing over and over again to the same people sometimes. And the way you say it one time will be the time they get it, just by changing up how you say it, or how you display it. And that takes patience to not ... Especially when they are like, "Hey, I totally understand this” and you're kind of happy, but you have that little happy frustration.
Mike Julian: Right.
Dan Barker: [crosstalk
00:40:55]. And so I try to place the blame on myself in those situations of, I need to figure out how to say it differently every time rather than just saying it the same way; and the change needs to be organic, but supported from the top. So we can't force anything into the culture, we can create opportunities like Slack, or like GitLab, where there's more open communication and it's easier to interact with each other. We can provide those platforms somewhat by force, although when I provided Slack, it was adopted in an insane way. We didn't expect we had to quickly buy more license because we were hitting our license limit very quickly. And then we had to offer training and a rollout plan which we hadn't planned on doing. It was just going to be our IT department.
Dan Barker: And then the business unit started getting involved and legal and everybody… It's a real battle of maintaining patients throughout the long haul and then trying to play catch-up when the dominoes start to fall. The same thing happened with GitLab and so I would say be patient, offer as many opportunities for learning or collaboration as possible and let people choose whether they want to do something, whether or not you think they'll succeed or fail. Just let them do it. Even spending several weeks on something that you think is going to fail or may not be as big of a value to the company might spur tons of innovation in the future from that person. So you often have to kind of let them ... Give them some goals to achieve. You can't just let people go do whatever they want, without any type of limit, but usually we'll try to have some goals but then some free time to do extra stuff and letting people go and lead those initiatives and not have it just, "Well, you're not a manager so you can't lead." So that's kind of the core of what I would do. But all of that is [inaudible
00:43:28] patience.
Mike Julian: Right.
Dan Barker: I was in the military for 12 years. So I learned patience.
Mike Julian: Oh, yes. Well, thank you so much for joining me. This has been great. Where can people find out more about you and your work?
Dan Barker: So all of my information is on
danbarker.codes — that's C-O-D-E-S. And you can also check out
naic.org for all additional information around NAIC and our initiative and get our State Ahead document, any of our fiscal budget proposals. And then all my information is on
danbarker.codes, all my presentations, where I'll be speaking next.
Mike Julian: That's awesome. Well, thank you so much for joining me.
Dan Barker: Yeah, thank you very much, Mike. I appreciate it.
Mike Julian: Thank you to our dear listeners for listening to the Real World DevOps podcast. If you want to stay up to date on the latest episodes, you can find us at
realworlddevops.com and on iTunes, Google Play or wherever it is you get your podcasts. I'll see you folks in the next episode.