Matt & Jason discuss how Stackify does DevOps when we don’t even have an IT operations team.
The key question is… can you do DevOps without an ops team??
The goal of DevOps is to get developers more involved in deployments and monitoring production applications. Even without an operations team, Stackify still does DevOps to monitor their large production environment on Microsoft Azure.
How does DevOps relate to small development teams? Stackify doesn’t have an IT operations team. Do we still have a DevOps culture? Matt & Jason discuss how the Stackify development team monitors and manages Microsoft Azure.
The role IT operations is changing but will never go away. Companies are leveraging things like site reliability engineering to give the developers more and more control of the operations needed around their mission critical applications.
Software development and IT operations as a whole seem to be shifting away from larger enterprise IT operations groups to more distributed teams. Individual teams within a business unit or associated with a specific project, are able to build their apps, deploy them to the cloud, and monitor them all on their own, thanks to products like Retrace.
| Matt Watson: | Welcome, this is Matt Watson with Stackify. I’m here today with Jason Taylor as my co-host on the Stackify Developer Things podcast. Since everybody’s talking about DevOps, I figured we should do an episode about DevOps. What do you think? |
| Jason Taylor: | Let’s do an episode about DevOps. |
| Matt Watson: | First of all, what are you drinking? |
| Jason Taylor: | I’m going to call this DevHops. |
| Matt Watson: | Is it really hoppy? |
| Jason Taylor: | It is fairly hoppy. I think we’ve talked maybe before about, we’ve got this wonderful team here at Stackify. One of the best things about our team is we have a number of people who homebrew beer. |
| Matt Watson: | That’s a good culture, DevHops. |
| Jason Taylor: | DevHops, we’ve built a culture around DevHops, which is funny because also DevHops includes a culture of yeast- |
| Matt Watson: | Oh, perfect. |
| Jason Taylor: | … as one of the main ingredients, so there’s a lot of culture going on today and this one specifically is a beautiful rye saison brewed by the head of our client tools. |
| Matt Watson: | You know way too much about beer. My question for you is, if you don’t have an ops team, do you still do DevOps? |
| Jason Taylor: | If you don’t have an ops team, do you still do DevOps? Who’s taking care of your production applications? |
| Matt Watson: | Well, I guess it’s the developers. |
| Jason Taylor: | Then, you’re doing DevOps. |
| Matt Watson: | But how can there be a culture of DevOps if there’s no ops? |
| Jason Taylor: | You know, sometimes culture just happens organically. It doesn’t have to be planned, it doesn’t have to be built. You just naturally start building a practice or doing the things that need to be done around running your company, running your code, running your platform. |
| Matt Watson: | We’ve got eight developers, give or take and we manage probably a couple hundred servers, all in the cloud, but we don’t have a single system administrator. |
| Jason Taylor: | Not a one. |
| Matt Watson: | If DevOps is a culture and it’s a culture of operations and development working together … |
| Jason Taylor: | Well, it doesn’t necessarily have to be a culture of two different capacities working together. It’s a culture, it’s a practice, it’s the things that you do with the resources you have. In our case, we run all of our infrastructure as code. Our developers are responsible for that, so even though it’s not a development team, an operations team working together, it is managing our operations as a function and an extension of development. |
| Matt Watson: | I think the goal of DevOps is getting the development team more involved in operations, right? |
| Jason Taylor: | Absolutely. |
| Matt Watson: | Be that deployment, monitoring, troubleshooting, all of those things. We definitely do those things. |
| Jason Taylor: | Absolutely. |
| Matt Watson: | We do DevOps with no ops. |
| Jason Taylor: | Yeah, that’s the question. Is it DevOps or NoOps? I’d say it’s not NoOps because I know we spend a lot of time there. We’ve got a big platform, like you said there, a couple hundred servers, thousands of databases. We’ve got tons and tons of data coming in everyday, terabytes worth of data and this team does spend probably about 50% of its time handling the care of and the growth of and the triaging of issues in that production system. That’s our operations. That’s our operational engineering capacity is making sure that we’re keeping the lights on and keeping that system running all the time. |
| Matt Watson: | I mean it seems like to me we’re doing DevOps with no ops and our development team is simply doing those things. Now, I’ve wrote many blog posts about DevOps before and I think it’s important to point out that I think IT operations is always going to exist. We’re a small company and we have 20 people and luckily, we’re in the cloud and we don’t have physical servers. We don’t have storage, we don’t have firewall and switches and all those sort of things. We also don’t have a lot of desktops to support. We don’t have a lot of those things and traditionally IT supports a lot of those things and IT that supports a phone system or desktops is obviously different than IT that supports production servers and applications, right? I think IT is always going to exist in a lot of those capacities, especially in larger companies. In a smaller company like ours, we all kind of wear a lot of hats and you are also in charge of the phone system, right? |
| Jason Taylor: | Yeah, I am in charge of the phone system. I think we have a phone. Do we have a phone? I think there’s a phone. |
| Matt Watson: | We have a few phones and arguably, I don’t even know why we have them, might as well use our cell phones at this point. I think the critical role that’s always going to exist outside of what development does, if development is focused on deploying our applications, making sure the deployments go smoothly and then making sure applications work perfectly in production, there’s also some other things to deal with, right, like security, compliance. How do you think those things fit into this? |
| Jason Taylor: | Overall, I think there’s a shift and probably two different things happening. One is that traditional operations, what I refer to as operational engineering that care and feeding of production software systems that sort of thing is pushing towards development because it’s all infrastructure as code and developers manage that. They’re responsible for monitoring and patching and updating, and you just mentioned there, we don’t have firewall and switches and those sort of things, but we do. We just don’t deal with them. It’s not physical hardware that we have. It’s a part of a platform as a service that’s part of infrastructure as code. That allows us to focus on the things that are really important, our applications, our code and how that’s running. |
| I think there’s that shift of not needing traditional operations to stand up networks and data centers and make sure that we can scale and we’ve got VM, or capacity available and we’ve got new servers available. Then, there’s the other side of IT, which is the traditional, desktop deployment and phones and kind of all those things. What do we want to call that? The digital transformation part of workplace technology. | |
| As a part of that digital transformation of workplace technology, what you’re also seeing is, let’s talk about your example of desktops. You’re a large company and you add 100 employees a year, 1000 employees a year, whatever and they get a new laptop. Somebody’s got to take the time to set them up in Active Directory and install their software and image the machine and all of that good stuff, but though services like Google’s G Suite and Office 365 and all these different things that’s becoming software as a service as well. You can take a brand new Windows 10 imaged laptop and hand it to somebody and all they have to do is log in with their Live account that’s tied to Azure Active Directory and then, they’re automatically logged in to Office 365. You can do the same thing with the G Suite. | |
| Matt Watson: | For the record, my new laptop I just got is not on the company domain. |
| Jason Taylor: | I’m going to let that one slide because I might be in the same boat. |
| Matt Watson: | I think there goes to the same question of I don’t even know why I would put it on the domain. We don’t have a single server, we don’t have a file server. We don’t even have any local assets for me to connect to, to authenticate to. |
| Jason Taylor: | Let’s look at that. There’s a couple reasons why you have a domain. Part of it’s security and access control and that sort of thing. For us, all of those sort of things have lifted and shifted to the cloud as well. The assets that we have to secure are Azure cloud deployments. That’s all behind Azure Active Directory. In order to get to that I have to go, log in. We don’t really have anything on our network here locally that matters in that regard. |
| Matt Watson: | Can we take this old server out to the parking lot and go like, Office Space style on it with some sledge hammers? |
| Jason Taylor: | You know, probably, someday, but not until I’m really sure we can do that because it’s been around- |
| Matt Watson: | Do you need to drink a couple more DevHops for that? |
| Jason Taylor: | Yeah, because it’s been around for a little while. |
| Matt Watson: | I think the key here is that a lot of those IT functions are always going to exist and no doubt, big corporations have, let’s say you have a call center, you have 1000 employees, they all have desktops, they lock them down. They have all sorts of different issues, but none of those issues really relate to the software development team and the products and that side of it. That’s a completely different core piece of it and these days, what we see more and more with our customers and we talk to Gartner and people like that is the development teams more and more have complete control. The development teams write the software, they deploy the software. A lot of times, they’re deploying it to the cloud or private cloud and they own it from start to finish. They don’t need any additional help. They control their own destiny. |
| Jason Taylor: | Who better to own it? When you’re developing code and you debug and you run into an issue, do you go ask your network admin, why you’re having this error, why are you having this issue? No, developers develop software and developers troubleshoot software and developers fix problems of software. |
| Matt Watson: | It forces them to own it, right? |
| Jason Taylor: | Absolutely. |
| Matt Watson: | Traditionally the problem we’ve had and I think that was kind of the root of DevOps is the finger-pointing. The system administrator says, “Oh, the server is fine, CPU is low, your app is on, it’s not a problem with the hardware.” Of course, the developers are like, “My code is perfect, I’ve never written a bad line of code in my career, it’s a problem with your shitty hardware,” and so goes the argument. Now all of sudden that’s gone, like the developer owns it and the server actually is completely irrelevant because it gets deleted and recreated every time you do a deployment or what have you, like they become disposable assets, which is what’s cool about it. |
| Jason Taylor: | I was having this conversation with our sales team the other day that we had a new client come on and part of the reason they came to Stackify is that they were having some of that internal argument. They were in the cloud, but they were having this massive scale to the point that they’re paying tens of thousands of dollars more per month and they think they should be. They aren’t sure why and there’s a lot of finger pointing going on and people saying, “Well, no you have to just scale the hardware, we’ve got a hardware problem.” My response to that and I’m glad they came to us and they found Retrace and they were able to run that on their servers and see, “Oh yeah, we’ve got a software problem here, we have code that’s not performing well.” My kind of internal response to that, what I’m thinking the whole time is, “Look at the hardware we have available now today, massive amounts of hardware, we can write code that handles millions of requests per hour on pretty commodity hardware. That the scale of hardware that they were running, like Azure D13 instances and they were having performance issues at 100 requests per minute. That’s not a hardware problem. That is absolutely, absolutely a software problem. |
| Matt Watson: | No offense to those listening or that company, but that was shitty code. |
| Jason Taylor: | It’s shitty code. |
| Matt Watson: | It was shitty code. |
| Jason Taylor: | Hardware is agnostic that way. Hardware does what hardware does and it does what the software tells it to do. |
| Matt Watson: | That’s one of the great things about our products, Stackify’s products, like Retrace can help cut through that very quickly and say, “Oh no, its database queries are stupid slow.” There’s no reason to point any fingers. It’s database queries, here’s the queries that are slow, here’s what’s calling the queries, go fix it. |
| Jason Taylor: | Absolutely. |
| Matt Watson: | That’s where we got with them, right? |
| Jason Taylor: | Mm-hmm (affirmative). |
| Matt Watson: | It was database queries I think that were the problem. That’s what’s great about tools like ours is they’re really designed for that DevOps mentality of solving those problems. If you’re a larger company that does have IT operations, we can help give the developers the insights they need, all the data they need, all the code level performance stuff, but also the errors, the logs, the metrics everything, without giving them root, administrator access, unless they have MacBooks, which have like no root password I guess anymore. |
| Jason Taylor: | Yeah, I hear that’s an issue. |
| Matt Watson: | Yeah, I think that’s an issue, but on their servers hopefully, with Linux everything and Windows, so what’s great is it gives developers everything they need and they can be so much more productive and eliminate the finger-pointing. |
| Jason Taylor: | That’s not to say that there will ever be hardware issues, but if there are, you need to be able to quickly determine, “Do I have a software issue or do I have a hardware issue.” Yeah, absolutely that’s the sort of tools we try to get people. |
| Matt Watson: | Or a cloud issue. |
| Jason Taylor: | Or a cloud issue. |
| Matt Watson: | We had issues, we’re in Azure and it seems like every month there’s some sort of issue in Azure and sometimes they’re big, sometimes they’re small. Sometimes, they’re in a specific region. Sometimes, it’s only specific part and we had an issue not too long ago that was Azure network related. That ended up causing problems with SQL and Redis, Azure Table Storage, all of these things. |
| Jason Taylor: | Yeah and I think we talked about that a little bit on our last podcast. That’s something that in more traditional days of a dev team and an ops team. That sort of issue would lead to a red herring that people are chasing, “Oh, it’s SQL, SQL’s down.” Everyone’s going to start blaming, “Oh well, it can’t keep up with capacity,” or there’s some other hardware issue and that’s a SQL DBA problem. Then, you look at the broader picture and you see, “Oh, well no, I’ve got these other services that are also going down,” and it all comes down to some like low-level TCP issue. Hello, we have a network problem, which I suppose you could say is a hardware problem, but having all of that data available to show you that absolutely this is the level on which the problem exists. Is it code or is it something beyond my control, is it infrastructure of some part? |
| Matt Watson: | We talk about DevOps here and I think there are certain job responsibilities that have to be done that are never a good use of developer’s time and some of those things are related to disaster recovery, security, compliance, IT audit. If you’re in an industry that has to deal with that and the processes and all that stuff that goes with it, so I think depending on the type of company, the size of the company, the industry, all of those things are still going to exist. I mean today, Jason, you’re the CTO and a lot of those responsibilities end up falling to you. In larger companies, some people may have like a chief security officer or something even, right? |
| Jason Taylor: | Let me make an argument there that in this kind of shift towards software eating the world, which I think you and I know, have spoken about before that everything now is a software issue, whether it’s security or IT audit or compliance or any of the sort of things you listed, at the root of all that is software moving data through systems. There may be, depending on the size and type of the organization, a different person responsible for that data, but at the end, there’s some software developers somewhere, who are responsible for how that data moves through the systems and so who better to help with these issues and work in these areas than software developers. If you’re right now going through compliance for the new GDPR Legislation, software developers know where all that data lives, which let’s face it is all over the place. You’re going to need software developers to pull that together into one place for then who is ever responsible for the reporting and carrying and feeding of that. No matter what you look at in all these different operational aspects, you’ve got software driving the concerns. |
| Matt Watson: | Even Azure and AWS and these platforms also helped a lot with some of these issues, like makes easier to do disaster recovery. They help do a lot of the backups or automatically ship them to a different region or availability zone and you have things that like I think Azure and AWS both have like a security center that provide best practices around, if you’ve done different security things. All these are also helping with some of these issues, especially for smaller companies like maybe like us that aren’t super, super sophisticated compared to like some giant bank would be at least. I think all of that is definitely changing and getting easier for us, which I definitely welcome. |
| The part of this that’s really interesting is the ability to deploy an application has radically changed. We’ve talked about this before how it used to be we monitor servers and everything was about servers and now, it’s more about the applications. It seems like containers are going to eat the world. It’s like the servers are so completely irrelevant anymore. It just seems, what we see more and more is the development teams are owning all of this from start to finish and they’re using containers and just going to Azure or AWS or they’re using Kubernetes internally, whatever they’re doing and just click a couple buttons, run a couple command lines, things and boom, they’re deploying their app. | |
| The problem is they still need a lot of application monitoring. I mean at the end of the day, the server part of it has disappeared or a simpler, but supporting these apps is still complex. Most of our stuff is all deployed in platform-as-a-service type stuff. We use Azure App Services, Azure Web Roles and even Azure Functions and all these things, but stuff still breaks. It’s not perfect. We still have performance problems. We still write shitty code sometimes. | |
| Jason Taylor: | The thing about that is whether things are containerized or not and elastically scalable or not, you have the same problem of … 14, 15 years ago when I had apps that were living on these three servers that took us three months to set up and get everything running right and then, we added another one. We went, “Oh wait, what was all the stuff we had to do to get our [inaudible 00:19:45] around this?” If you had a problem, you had the same surface area of like, “Okay, how do I diagnosed this probably, what’s the data I need and how do I get it,” and all that. |
| Matt Watson: | Well that was the route of configuration management. |
| Jason Taylor: | That’s the route of configuration. |
| Matt Watson: | [inaudible 00:19:59] and things like that. You go back, like it was making sure the configurations of those servers were all the same. |
| Jason Taylor: | Yeah, right, but then you still have the same problem when you’re trying to troubleshoot a problem there, whether it took [inaudible 00:20:10] to make sure the configuration management is the same between all of them or it’s a container, where it’s all configured in code and you just deploy. It’s the same problem. Then, now how do I get access to this data to troubleshoot? I’m going to pick for a minute here on .NET Core. I think we’re going to have a big problem with us on .NET Core because one of the wonderful things about it is you can deploy on Linux now, low costs. You can easily do Docker images and scale up rapidly .NET Core on Linux. How many .NET developers do you think have a super, super firm grasp on Linux OS to where they can get in there and quickly like diagnose all these problems that they’re having because it’s not whether you SUSE on Windows, so no user interface on a Docker image. There’s no [inaudible 00:21:01]. |
| Matt Watson: | What’s a command line? |
| Jason Taylor: | What’s the command line? |
| Matt Watson: | What’s SSH? |
| Jason Taylor: | What’s SSH? |
| Matt Watson: | I ain’t use that stuff before. |
| Jason Taylor: | These are all the problems that .NET developers are going to have as they shift to this brave new world, which I’m totally behind. I think it’s awesome, but it’s so much more black box. Developers are going to need more powerful tools all the time and more data and insights coming up. This is something that we hear about every day from our clients, who are moving to .NET Core on Linux to get cost savings and massive scalability and all this. That’s all great in theory until you write some horrible code and deploy it. Then, you can’t figure out where it is. |
| Matt Watson: | Right, which is why they need tools like ours. |
| Jason Taylor: | Which is why they need tools like this. |
| Matt Watson: | The performance of the applications and troubleshooting them is still real important and even more so with Docker. I believe traditionally with a lot of Docker containers, you don’t even write a lot of data to the container itself. I mean a lot of that is locked down. |
| Jason Taylor: | Well, in any elastically scalable solution, you should be, right? |
| Matt Watson: | Yeah, you got to get that data off the machine. |
| Jason Taylor: | Your data needs to be going to a persistent store somewhere. |
| Matt Watson: | Yeah, I mean that’s where tools like this become even more critical. Potentially, there is no log file to write log messages to. The containers, all these things, potentially you have a lot of them too and they are spinning up and spinning down. Those log files might- |
| Jason Taylor: | It’s not durable. |
| Matt Watson: | Yeah, I mean, so that the point is that data has got to get off of those machines and you’ve got to be able to send it to products like ours that can aggregate all the logs and the errors and the metrics and the performance data because those things are coming and going. They’re very transient, which is awesome, but has this whole new set of problems, right? |
| Jason Taylor: | Yes, absolutely. |
| Matt Watson: | I mean think about AWS Lambda and Azure Functions, like we’re going serverless now and who the hell knows where our code is even running? It’s running somewhere in the cloud and it’s not in our server. |
| Jason Taylor: | At least with the Docker image, doctor image I can get access to it. It might just be command line, but I can get access to, I can SSH in. I can get what I need to as long as it [inaudible 00:23:16] running. With an Azure Function, it lives somewhere for the duration of the time it took to execute this method. Then, it goes away and if it runs again, it might be somewhere completely different. |
| Matt Watson: | Another thing I saw that AWS announced this week is they’re coming out with a new like serverless database model. Did you see that? |
| Jason Taylor: | I didn’t see that. |
| Matt Watson: | Basically, there’re some other providers that do this already, I think Google’s BigQuery is like this, where you just pay like per query or per the sources it takes to run the query and Cosmos DB from Azure is [inaudible 00:23:49]. Amazon was doing it with, I think it’s like MySQL or MariaDB on top of that. It’s actually more of a traditional- |
| Jason Taylor: | Relational. |
| Matt Watson: | … relational database, but basically you run as many queries as you want, do whatever you want and it automatically like scales up and scales down your database and does like crazy provisioning or something. It’s just kind of crazy world we live in. |
| Jason Taylor: | Yeah, I mean I would have given my left arm for that 12-15 years ago when we were spending $3 million a year on [inaudible 00:24:20] for our SQL and we got to the point that Dell and HP and anyone selling big systems said, “We can’t give you a system with any more hardware.” |
| Matt Watson: | Yeah, I think even for Stackify, every one of our client has database, so we have over a 1000 databases we have to deal with and something like that could be really useful for us. The sharding and all that junk that we’ve had to do, how much easier would that have been maybe with a system this. |
| Jason Taylor: | I’d still make the argument that if you can avoid meeting massive, massive vertical scale, avoid that. The horizontal scale, it’s the Ferrari versus a fleet of Hondas. Go with the fleet of Honda model if you can, but absolutely, there are so many people out there now, who are, maybe they aren’t like us, maybe they aren’t in multi-tenant SAS model and they’ve got a deep investment into their kind of single data source model that their application uses. It grows and it grows and grows that they need those sort of capabilities. I think that’s awesome. |
| Matt Watson: | We’re getting to the point, where there’s so many different things out there that becomes hard to figure out what to use. No matter what the things are, you got to figure how to monitor them and that’s all new set of challenges and that’s truly the DevOps challenge. I think we can agree that DevOps is more about the developers having ownership of all these things, even if there is no operations team. I mean it’s DevOps without ops, but it’s still DevOps. It’s the developers are still taking ownership in the deployments and the monitoring side of it, wouldn’t you say? |
| Jason Taylor: | Yeah, absolutely and that’s why I keep referring to and do internally here at Stackify as well as operational engineering, it’s an extension of the engineering that instead of doing new value creation, create new features and new products for your company, it’s the arm of engineering that is making sure you can keep the lights on. |
| Matt Watson: | Google calls that … Another name for that and it could be [inaudible 00:26:36] than perhaps how you envision is like site reliability engineer. That’s another big term or performance engineer. There’s different names for these things and there could be little different flavors and people implement them different ways, but it’s definitely it all comes back to the developers owning these things and having more ownership and oversight of all of these things. |
| Jason Taylor: | Yeah, I think that’s very true. I think DevOps gets used a little bit of a catch-all term that the way I’ve seen it, it breaks down into a couple different categories. It’s operational engineering, it’s site reliability engineering, it’s automation engineering, but all of it centers around doing it in a development orchestration model. |
| Matt Watson: | The development team is involved in some way, shape or form. All of these things are different based on the size of the company. I mean if you’re a little start-up and you’ve got a couple developers from command line or from Visual Studio or something, it’s right click, publish, you’re done, things are so simple and obviously, all the way to a bigger company that has a whole lot of automation around deployments and the steps and how you get through multiple environments and automated testing that occurs and big, complicated pipelines. |
| Jason Taylor: | We’re seeing that every day in our own customers, who may go from a small company with 20 employees to a Fortune 100 or a Fortune 500 company that at the end of the day, they all have the same issue. We wrote some code, we deployed it and now, we have an issue with it. Who’s going to be responsible for that? No matter who’s responsible, they aren’t able to quickly, rapidly solve the problem until they get data that developers need to look at it and say, “Here’s how we solve it.” |
| Matt Watson: | How do you think that DevOps will continue to change in the future? |
| Jason Taylor: | Well that’s a good question. I think a lot of the things that were seen in the space now around standardization of containers and deployment and all that sort of thing is really going to push it forward that a lot more of this is going to become standardized, maybe some frameworks around it that everybody uses because right now, there’s a thousand tools out there. It’s a Swiss Army knife of things, plus some homegrown stuff and all that and I think that’s one thing that I’ve seen come out of the container movement around … Docker is a container system and your Docker Swarm and Kubernetes and some of the other orchestration frameworks is that we’re going to see standardization that helps people build practices around this. I think we’re going to continue to see that grow like crazy and remove some of the kind of wild-west of it. |
| I think you’re going to see cloud providers, especially adopt tools like Retrace as a core part of their platforms for their customers to use. They always have these insights and they always have this data. | |
| Matt Watson: | I was talking to a client today that one of their big issues was like containers and they were working with … They do custom software devolvement for other companies. He was talking about how everybody wants to move to containers, but a lot of people, they don’t understand that potentially it changes how they write their software and how they architect it as well. I think for some of these companies out there, they still have a [inaudible 00:30:29] ways to go to wrap their head around like how they deploy their software and all that stuff in a little different fashion, in a little different form. I think internally that’s still something we have to work through is we have all these apps that we’ve built and we’re sitting here, trying to figure out, “Well, do we rewrite them or change them in some way to get them, so they can work in containers.” |
| Jason Taylor: | Well, plus, not only that but there’s the maintenance of that. A really good example is we are working on a project right now to do a pretty significant refactoring of the subsystem, a Retrace that is going to require a very sophisticated way in which we deploy that to not have any downtime that our engineers as they’re writing this code and making these changes, they have to think about, “How are we going to deploy this without any downtime and have a seamless transition of data,” and all these things. At the core of that is a lot of knowledge around how the system runs and scale and the different DevOps pieces of it and how that’s going to work. |
| Matt Watson: | Yeah, when you’re at the scale of a company like ours, so you’ve got 20 servers that all process data off of queues or something like that, it’s not as simple as you shut one down and turn one on and like flip the switch, like you got 20 of them. It’s not like you can have 10 running the old code and 10 running the new code and the whole process of that is complicated. I guess we’re lucky to have those issues, but it’s a whole new set of issues that you have to deal with and back to our point earlier, then you have to collect all the data off of all those things and monitor, measure them and track all the things down. |
| Jason Taylor: | I think the other thing that comes out of this isn’t necessarily technology-based things, but it’s a practice-based things and it’s how people build their teams is that you’re going to see teams start to segment into portions that drive new value and portions that are responsible for uptime and maintenance and warranty work. You’ll see practices pop up and be potentially feeder systems for different teams. It’s doing maintenance and warranty work on live site and site reliability and operations is a really great entry point for junior to mid-level developers come into an organization and learn how a system runs and operates. Then, move forward into doing new feature development. I think this will continue to drive how teams organize and shape and run themselves as well. |
| Matt Watson: | Yeah, absolutely, I mean that’s something we continue to battle with internally as a company that’s growing fast. I mentioned earlier like the operational engineering side of it and there’s so many more things that developers have to know though as well as if they’re going to be also monitoring production that means they have to know a little more than a normal developer will. That’s also a little bit of the challenge here is the skillset is changing slightly. |
| Jason Taylor: | I think it’s making better developers. Think back to when you first worked with ASP.NET, you’re probably junior dev, hadn’t been working a lot and you cracked something open, you wrote a few lines of code and you hit F5 and poof, a website. You didn’t have any understanding of necessarily how the server model worked underneath that and how IS was serving that up and what the IS pipeline looked like and what the ASP.NET pipeline looked like. Now, in having to manage DevOps that’s a broad general term, developers are learning a lot more about the subsystems that run the platforms. I think that can lead to a lot of good that leads to a lot of efficiency and developers writing better code when they understand how it’s being served up. |
| Matt Watson: | Well, they are, but they aren’t. I mean, in some ways that’s like the argument of you can’t be a software developer if you don’t know assembler or something right, but because of- |
| Jason Taylor: | I know assembler. |
| Matt Watson: | Yeah, you copy and pasted some [inaudible 00:34:40] once. |
| Jason Taylor: | Yeah, there’s also something wrong with me- |
| Matt Watson: | Probably yeah. |
| Jason Taylor: | … that I know little bit of assembler. |
| Matt Watson: | It’s the same thing with like containers and platform-as-a-service, you don’t necessarily have to know the underpinnings of some of that stuff, like it’s abstracted away, but you still got to understand maybe how to work with Docker, how to work with Kubernetes, how to deploy, how to get down and dirty with some more kind of systems management and command-line tools and different things like that. The developers have to know to write the code, but there is a little bit of that kind of systems management stuff they have to learn. I’ll be the first one to admit like I am not an expert at how to log into Linux and what those commands even are. I’m definitely more of your traditional Microsoft developer that would much rather right-click and hit publish, but- |
| Jason Taylor: | I think that’s rapidly changing though. |
| Matt Watson: | It is rapidly changing. |
| Jason Taylor: | I think it’s rapidly changing. That’s kind of exciting to see and certainly, yeah, a low super, super low level operating system stuff, developers might still be left a little behind on and in the dark, but I think ultimately, what we’re shifting from is I produced some code and I compile some code. Now, I’m going to throw it over the wall to somebody else to handle and I’m never going to know anything about how it gets deployed or how it runs. That certainly changes and I think just knowing and understanding this is how it’s going to be deployed and this is how it’s going to run, this is how it has to scale and being a part of that is bringing better developers. |
| Matt Watson: | Yeah, I think the other theme that follows along with that is having smaller development teams, individual teams that own a specific sub-segment of the product and micro services plays a little bit to that, where if you’re a big team, you’ve got a 100 developers on one monolithic application, it’s so much more difficult to deal with that kind of stuff. As these applications are getting broken down into smaller pieces, it’s much easier for those smaller teams to own it from start to finish because it’s not 100 developers that then are trying to okay draw straws, okay which of the 100 is going to do the deployment tonight? It’s all these individual pieces, so that 100 developers might be 20 teams or something that own their little piece of the pie. I think that’s one of the advantages of micro services or that type of mentality. |
| Jason Taylor: | I think that is a wonderful topic for another podcast- |
| Matt Watson: | Yeah, absolutely. |
| Jason Taylor: | … is micro services. |
| Matt Watson: | Well, I guess let’s wrap this up and I think the key takeaway here is DevOps is more about the developers being involved in deployment and production, monitoring and troubleshooting, even if there is no operations. It’s still DevOps, it’s DevOps without ops. |
| Jason Taylor: | It’s DevOps without ops, it’s, yeah. |
| Matt Watson: | I mean, at least for some companies, it can be. I don’t know if you call that NoOps. I think as we discussed, there will always be things that operations will do, but they could actually be less about the product and the platform side of it of the company and the deployment side of it and more about security and compliance and other things that are separate. |
| Jason Taylor: | I think it leads to a better allocation of resources and having the right owners for the right things. |
| Matt Watson: | Right, all right, well thank you for listening to our podcast today. Of course, you can subscribe on iTunes and a bunch of different places and always on our website at Stackify.com. Thank you guys. |
[adinserter block=”33″]
Stackify's APM tools are used by thousands of .NET, Java, PHP, Node.js, Python, & Ruby developers all over the world.
Explore Retrace's product features to learn more.
If you would like to be a guest contributor to the Stackify blog please reach out to [email protected]
