"There's a lot of talk about 10x engineers. If you really want to be a 10x engineer, spend your time making 10 of your colleagues twice as good. That's the kind of approach that needs to be taken to scale."
Docker: A Lot Changed in a Year
Sinclair ZX81, Arduino, Raspberry Pi, OpenELEC Project, QCon, RabbitMQ, Docker, WeaveWorks, Netflix, Computer Sciences Corporations (CSC), Audit Paradox, Cloud Foundry, Google, Credit Suisse, Weblogic, GitHub, Ansible, Jenkins
Marc Andreessen, Alexis Richardson, Simon Wardly, Joshua Corman, Nick Selby, Derek Collison, Adrian Colyer
Chris Swan is interested in your education. He has spent his life doing challenging, interesting technology projects. Along the way he has been a champion of teaching others through documentation and presentations, helping define what enterprise level DevOps looks like, concentrating on the collaboration aspects between teams as much as the technology itself.
In this "Innovator's Journey” profile, we speak with Chris about how he got started with technology as a child, the process of discovery in his journey and how he’d like to encourage the technology industry to incorporate teaching and learning into the development process.
Mark Miller: As a child, were you interested in technology?
Chris Swan: Oh absolutely. I started out soldering together bits and bobs of electronics and my dad used to go and buy components and we would make circuits from radio shack guides and that kind of thing and then I suppose the break into computers was when Sinclair launched the ZX81. We got hold of 1 of those.
I've got a brilliant photograph of my brother and I sat at the keyboard of that ZX81 with a cassette recorder that was just about bigger than the computer itself. We used to sit there and type in programs from magazines and of course they never worked which is how we learnt to do debugging.
Mark Miller: How old were you then?
Chris Swan: I would have been 10.
Mark Miller: Did you go to school later for it? Did you go to university?
Chris Swan: I leaned towards electronics engineering. I think deep down I'm still a hardware guy. I see software as a necessary evil to make hardware do what you want it to do. That necessary evil has grown and grown and grown. I totally agree with Marc Andreessen’s point of view of software eating the world. That's why I spend almost the entirety of my time doing things with software.
I'm intrigued these days by things like field programmable gate arrays and the fact that they seem to be coming around again into the mainstream. I keep my eye on the hardware stuff.
Then you've got all these hobbies things like Raspberry Pi, and Arduino. I like to try and find some time to tinker with those as well.
Mark Miller: You've been doing the Raspberry Pi thing for a while.
Chris Swan: Yeah, well as soon as the thing launched, I got my order in and they launched them through two partners, and of course they didn't coordinate. With the hacker mentality in mind, I placed an order with both partners.
I had one of those things, sort of earmarked to replace an aging digital media player that I had in the house, and that's what got me involved in the OpenELEC Project. The other was earmarked for just general experimentation, and making things.
Mark Miller: You did a lot work with QCon too, what was your role there?
Chris Swan: I got involved in the very first QCon, which was here in London in 2007. I did a talk there on the financial services track. In fact the track host for that was Alexis Richardson, who was founder of the company I worked in last, and was most famous for being the co-founder of RabbitMQ. Those now in the Docker community are kind of keeping a keen eye on what he's doing with Weaveworks.
I stayed involved with that community, and it was a few years back, Floyd approached me. I'd been back at QCon as a speaker, and he said "How would you like to come in on the program committee and host a track?".
The model we ran that year was every track host was a member of the program committee, so we had a huge program committee. That was an interesting experiment, but I think it made it a little bit unwieldy. We then backed off to a program committee where the model is to go and find track hosts. I've generally got into tracks around internet of things, containers, security, and emerging fintech as we've done those over the last few years.
Mark Miller: That sounds about the time that DevOps was starting to crank up. Do you remember the first time you heard about DevOps?
Chris Swan: I'm not sure I do actually. I think this term came along, and it made so much sense to me, it was like it had always been there.
I think if I come and use my own thought model for what DevOps is ... it's a label for these emerging tools and practices that we see from organizations that have reached a greater design maturity, they've moved to design through operations.
We look at those practices, and they've been going on for some time. I think my college Simon Wardley makes these observations that if you go back to the financial services in the late 90s, they were sort of doing DevOps, they didn't call it DevOps.
We can now ask ourselves "How did it stop happening?", and it seems to be at least time coincident with some of the sort of large mergers that look place in financial services, and that seems to have inflicted more process, and more hierarchical bureaucracy upon IT. It seems to have erected the wall that then started to happen between Dev and Ops, which we're now actively seeking to break down to get to DevOps.
Mark Miller: That's been a consistent answer when I've been talking to people. DevOps when you think of it, is just the correct way to do it. It just happens to have a title on it now.
Chris Swan: Absolutely. I think part of the problem was with Waterfall.
We had a Waterfall mentality for how we build software, and we developed a waterfall mentality that corresponded to that on the operation side. Then of course the agile manifesto came along. That meant that the developers could throw stuff at the operations wall even faster than ever, and get even more frustrated with how hard it was to get stuff into production.
I think one way of looking at what's happened with DevOps is that it's been the natural consequence of Agile development methodologies. There's been a need for Agile infrastructure, the merging set of practices then to make the infrastructure agile, and make the infrastructure an extension of what's going on in the dev side.
Mark Miller: Along that line, what is your motivation for working in the DevOps community? Do you introduce yourself as a DevOps person? How do you associate yourself with the community?
Chris Swan: I think there's two ways I'm associating with that community at the moment.
The first is talking about the larger business context of agility. One of the reason we need DevOps is the companies that win in today's environment are the ones that are able to adapt quicker than their competitors. We see now organizations, and Netflix is a great example of this ... where the executives in those organizations have expressively gone about building an organizational structure and culture that is designed to be adaptive, and designed to find what's going on in their customer environment, measure those customer needs and desires, and then build products and services to satisfy them.
I think that this comes out more broadly; this is how any organization actually wins at business in the modern environment. If we're going to have agile businesses, then we need agile software to support those businesses, and then comes back to these things that we're labeling as DevOps.
The other part is, in my new day job at CSC, where I've been for the last 4 months, I'm trying to drive a lot of automation into what is essentially a very traditional infrastructure organization. I'm taking this thing that I could label as infrastructure as code, which is really see it as the operational half of DevOps, and driving that into the things that we're doing for delivery to our customers.
Mark Miller: When you're talking about automation, where does security play into that?
Chris Swan: I very much believe in this idea of baking security in. It's something that I've been aware of for a long while is if you try to build security in just purely in code, then that causes a problem for you in terms auditability. I ended up calling this out as the Audit Paradox: rather than building security into a code, that we'd instead go off and bolt security on, and we would buy network security appliances, because they were easier to audit.
The heart of this issue here was if an individual developer was building security in, he would have to follow hundreds of pages of security guidelines and get all of those right. That was hard enough for the developer, but for somebody else to come on and check on that homework was an even more mammoth task. The auditability wasn't there. That was driving us towards doing things wrong. I always felt that bolt on was not answer.
What I'm seeing happening now, and it's happening in part through adoption of platforms, especially platforms as a service, and in part through some of the practices we're seeing in the DevOps world, especially around micro services architectures where security services get baked in… but also, design for failure. If I look at Netflix's Simian Army, they've got a security monkey in there to test the security aspects.
All of these things start to play together, in that instead of building security in, and making that hard to audit, and instead of having to bolt security on, and making it clumsy, we can effectively have security that's bolted in.
Really what's happening here is security experts are able to put the security where it is needed, and have a focus on that. The developers for the broader business application functionality are able to harness the security that's been bolted in there for them. Security of course remains a concern, but they don't have to become a full on security expert in order to participate in building a secure system.
Mark Miller: It's interesting. It sounds a lot like what Josh Corman was talking about in a sense with rugged software. Even some of the terminology you use I recognize. You and Josh must have been talking a bit.
Chris Swan: Absolutely. Josh and I have collaborated on a number of things ever since he was at 451, and took over for Nick Selby. We stay in sync, and I think the Rugged Manifesto that he's come up with, is one that really resounded for me. I think he's absolutely been driving the right kind of changes that we need to adopt in the industry.
The other thing that's changing here is the practices emerging from DevOps, and particularly around infrastructure as code is allowing us to deploy security control points and achieve security monitoring of those control points in ways, and at a scale that has never been achievable in the past.
If we look at something like firewalls ... while firewalls have come in hardware boxes, they've been expensive to buy, they've been expensive to deploy, and it's taken individuals with expensive skills that don't scale in order to install, configure, and maintain these things.
The shift I'm seeing in the industry at the moment is ... all of that stuff has become implemented in software, and all of that stuff has become controllable through APIs. Where it was impractical to have 10 firewalls, or 100 firewalls, or 1000 firewalls, you can now have a million firewalls, and you can manage them all through automation via their APIs. I think that's another part of SecDecOps, that's going to really transform our industry.
It's broader than that. We can see examples all over the place, but I think the two that I would call out the most are the move to platforms. I think there's great work going inside Cloud Foundry, and the distributions that go along with that.
More so, we can look at something like Apcera, and what they've done. Derek Collison and his teams focus on building a more secure and policy driven platform environment, I think shines a light on how platforms can help here.
If we look some of the leading practitioners ... Google is full of excellent examples, Netflix, too… many other leading e-comerce plays, and as a service plays ... security is part of their business model, and they have to get it right, and they have been getting it right, and we can see good case studies arising from that.
Essentially what's happened in each of those cases is that they've built their own platform, so they are not selling it as a Platform as a Service (PaaS) that somebody else can come along and use. Within those companies they've built their own platform, and it's just very opinionated to the as a service delivery to that particular organization.
Mark Miller: What skills should the next generation of DevOps practitioners be developing? What should they be working on?
Chris Swan: I think first and foremost is still collaboration. Effectively talking to people. The other is constant learning. I don't think anybody in this industry can learn a skill and stand still. I think instead you actually have to practice the skill of constantly learning new skills.
We could talk here at a point in time, and I could say "I think Go is going to be really important", and actually I do think GO is going to be really important, and I could say the language geek friends of mine seem to be all excited about Rust, so maybe that's worth looking at as well.
On pure dev side, both of those languages are very interesting, but actually the point is not go learn GO, or go learn Rust. The point is keep an eye on what's happening with languages and their frameworks, so you can pick up the right tools for the job when a particular situation comes along, and that applies equally in the infrastructure world.
Last QCon we had Adrian Colyer, and he did a brilliant keynote about computer science papers that he's read, and how the future is unevenly distributed, but it is already here. He brought out some examples of ... we can see new memory technologies, and storage technologies starting to come to the fore which are going to completely change our infrastructure deployment model.
So that stuff is worth keeping a track on, but you look at that and think, "Okay, storage is going to change, that's significant", but then you think "Okay, the database technologies that I use on top of my storage are going to change as well". If state management is changing, how does that affect the architecture of my overall system.
There is this cascading sequence of changes here. Ultimately the point is be adaptive to them. I talked about business agility. I think you end up here with a layered model of ... to support business agility, we need software agility. To support software agility we need practitioners who are agile with their own skills, constantly learning.
Mark Miller: When you look back on your career, what are you most proud of so far?
Chris Swan: I think as a career I'm most proud of the fact that I've been able to keep on doing interesting, and challenging things. If you would have asked me to pick out some single points out of that, I think I'd probably go back to the grid that we built when I was at Credit Suisse.
That was somewhat pioneering at the time. I know all of the banks were trying to do the same sort of thing, at the same sort of time. To be part of a group of people building systems at a scale that hadn't been done in the mainstream before was great. That thing was a platform as well, if you ask me what I'm most proud about, that's certainly one of the things.
What I'm more so torn up about was we build a platform then, and we did actually demonstrate that it could host generic workload, and achieve time to market, that was incredible for the time. We got a particular application from a conversation about how we do this in production in three weeks, when it used to take us three months to order a server. That opportunity wasn't as seized as it could have been.
I think there's always two sides to these coins, and there's things that you can be proud about, but there's also opportunities that went with that stuff, which is still lost.
Coming up to date, I'm proud of being a contributor for the Docker project. I tend to focus my efforts on documentation. I would make an argument that as a software engineering culture, we do put too much emphasis on coding, and not enough on documentation, and samples, and examples, and yet it's the documentations, and samples, and examples, that makes things easy for others to adopt, and helps them win.
Actually if I look at software that's been incredibly successful in my environment I think of examples of BEA Weblogic. Weblogic won out over its competition because it had better samples and example. I look back at all of these Java enterprise applications and you can see at the heart of them what started out as a Weblogic sample, or example, that then turned into a useful piece of business functionality that has subsequently supported millions, or even billions of transactions, and that kind of thing.
Mark Miller: As you think about your process for learning, and moving on, and other things. What do you want to be your legacy in the DevOps community?
Chris Swan: I think I'd like to leave behind a culture of learning. I just made the point about documentation, samples, and examples. If I can just a tiny bit change the culture of our industry to think that the documentations samples, and examples are important, and that other people could come along and learn, then that would be a great thing.
There's a lot of talk about 10x engineers, but I forget who tweeted a little while back, that if you really want to be a 10x engineer, spend your time making 10 of your colleagues twice as good. That's the kind of approach that needs to be taken to scale.
I've been doing a thing for the last few weeks where we're doing an infrastructure boot camp. Just getting people's feet wet with not just the tools like Git, and Github, and Ansible, and Jenkins, but also the methodologies that go along with that. I spend a lot of time focusing on fork and pull within Github because I think within many traditional organizations, the thing that puts the breaks on innovation is the hierarchical structure, and the bureaucracies that go along with that. People constantly having to ask permission.
The great thing about fork and pull is that a fork is an action that is much more beg forgiveness, but when you get to a pull request being merged, then you have actually asked permission. I think it starts to allow people within traditional organizational structures to be freer in innovation, and at the same time let the organization retain their degree of control and governance that it feels it needs to. The overarching point is "Help people to learn". That should be the lasting legacy.