Maturing DevOps in TD Bank

TD Bank is a large, 150-year-old bank with a global footprint, that is only continuing to grow in the US due to acquisition. While their history gives them extensive expertise - as we fast forward to today, about half of their customers are digital - a problem they couldn’t even conceive of having when they were founded. This emphasis on global growth with a simultaneous digital transformation has driven the need for plenty of organizational change.

A Look at Industry Trends

In order to understand what’s causing TD Bank to make these changes, consider some industry trends that impact TD's servicing platforms. Those include:

1. Customers expect platform availability at all times.
2. There’s a focus on targeting mobile today with an emphasis on AI tomorrow.
3. Monolithic approaches are disappearing.
4. There’s been a transition to secure, delegated access (e.g., oAuth)
5. On-premise footprints are shrinking.

The Impact on TD

Given these trends, let's look at the impact on TD. Continuous, ongoing self assessment is essential to the maturation of any organizational process.

First up, it's worth examining has been going well within the organization. There's been a focus on speed to market and first-class consideration of customer journeys through the application ecosystem. On a philosophical note, this means blurring the line between business and technology and considering the bank to be a technology company.

The specifics of what has been going well over the last few years include a number of points:

1. The governance and funding model includes a constant run rate for agile "pods" (small agile teams), design for resiliency, and a readiness test, which drives a focus on productivity increase and a transition to thinking about customer journeys.
2. A streamlined intake process, in which a single business owner channels requirements, leads to fluid tactical scope that accommodates customer needs and market conditions.
3. The platform-first focus has meant an emphasis on stability and dedicated pods. This led to platform enhancements that support new features and builds. Platform-first focus means building platforms first and understanding that features will follow from well considered platforms.
4. Incorporating an integrated support team has made monitoring and alerting transparent and led to a reduction in detection time. This, in turn, increases stability and creates "ecosystems" within their broader offering.
5. The establishment of a technology roadmap has not only clarified the present but helped TD keep an eye on the future, including what new sorts of philosophies and technologies might be necessary tomorrow.
6. Technology platform design has helped them engage solution architects earlier, in backlog refinement and design decisions.

Lessons From the Last Three Years

But the things going well haven't come without some hard-earned lessons from their journey over the last few years. Here are those lessons that they've learned from the changes they’ve had to make.

1. They’ve learned is to put stability first—to make it a part of their DNA.
2. They’ve implemented high availability principles and focused on how to fail safely and keep downtime for maintenance to zero. This includes designing with the assumption that failure will happen, thus putting unprecedented emphasis on being in a constant position to recover.
3. They’ve put a focus on engineering and only acquire the top talent, technically speaking. This means looking for specialists and emphasizing platform teams with SME knowledge.
4. They’ve also have had to make tweaks to their operational approach to microservices. If DevOps is immature at a company, microservices can cause serious problems.
5. They’ve learned that breaking silos is critically important for any organization looking to establish and mature a DevOps practice. This means destroying traditional organizational barriers and focusing on ecosystems, internal accountability, and even the establishment of communities of practice, like guilds.
6. And finally, they've learned some hard-won lessons about what DevOps truly involves. That includes an emphasis on automation and accuracy of processes and going fast securely. It also involves the establishment of more modern people practices, such as the aforementioned establishment of guilds and participation in agile release trains.

Looking Back on the Journey and Then Looking Ahead

When TD started this journey, they began with continuous integration and then followed up with continuous delivery and continuous testing over the course of the next year. The reason for this is that they realized it isn't possible to have a true transformation if you only improve the process without the tech. So the journey became one of parallel process and technical practice.

Starting in 2019, there's been a focus on continuous monitoring. And this is going to logically lead into new pipelines for strategic platform and the automation of database releases to round out the year. Looking down the road, impressive infrastructure automation will include secret management, firewall, and network automation.

As a result of these changes, TD has seen the lead time decreases, the ability to provision environments has become shorter, the ability to deploy quickly has increased (in the form of a 10–15% reduction in time from code commit to production), and their automated security is three times lower.

TD Bank gave a full discussion of their transformation at the 2019 Nexus User Conference. You watch their session here: