As , , , and have established themselves as the building blocks for modern application development, the need for a simple way to manage those resources for internal software developer teams has become more and more important.
At many elite engineering organizations—think Google, Netflix, and Amazon—internal developer platforms (IDPs) ease the operations load on their devops teams, while abstracting away unnecessary decisions for software developers.
Just as former president Barack Obama to ease his , developers working with a good internal developer platform can worry just about their code, a Git repository, and pushing to a platform that takes care of the underlying infrastructure.
What is an internal developer platform (IDP)?
Internal developer platforms are like snowflakes, in that no two are the same. Each platform varies from organization to organization depending on their stack, culture, code base, and tool set—which makes finding an agreed-upon definition somewhat difficult.
As Evan Bottcher, head of engineering at ThoughtWorks, , “Words are hard, it seems. ‘Platform’ is just about the most ambiguous term we could use for an approach that is super important for increasing delivery speed and efficiency at scale.”
Bottcher’s own definition—although he prefers the term “digital platform” to “internal platform”—is “a foundation of self-service APIs, tools, services, knowledge, and support that are arranged as a compelling internal product.”
on the topic.
A good internal developer platform should abstract away infrastructure decisions, enable self-service environment builds, integrate with existing continuous integration and delivery () and deployment processes, and assign role-based access controls, all without a developer ever having to learn YAML.
Stephen O’Grady, a principal analyst at Redmonk, says he has seen a “definite appetite for taking the requisite pieces in a tool chain and putting them together into a platform where the developer can come in and write an app, and all of the underlying complexity is abstracted,” over the past calendar year.
The benefits of an internal developer platform
The latest identified self-service internal platforms as one of three key elements that set mature enterprise devops organizations apart from their peers, alongside automated change management and integrated security.
A fully functioning internal platform should ease the complexity burden of modern software systems, speeding up software deployment cycles and creating more stable releases, as well as improving developer happiness and productivity, all while lowering the operational burden.
), where the vendor typically dictates how a developer should work, an internal developer platform is built on the tools and processes developer teams are already familiar with, but with a greater level of abstraction and consistency.
As the Google technologist Kelsey Hightower in 2017, “I’m convinced the majority of people managing infrastructure just want a PaaS. The only requirement: It has to be built by them.”
Many smaller organizations turn to a PaaS to get their engineering team up and running quickly—with popular choices including , which was acquired by Salesforce in 2010, or , , or the big public cloud vendors’ own tools—but often find these lack the flexibility to truly scale out.
Opting for the IDP approach does lead to the risk of engineers looking to reinvent the wheel when given the chance to build their own platform, or worse, trying to run like Amazon or Google, which is a recipe for distraction and disaster.
“Even when those big companies provide their solutions as open source software, they often encode all kinds of assumptions about the surrounding ecosystem of available products and the culture and needs of the engineers using the product that may not work well in your company,” Fournier wrote. “It is not good product management to say, ‘Google does it, therefore we should.’”
Puppet’s Kersten, an ex-Google SRE himself, sees things in a similar light: “We have seen numerous large organizations try to adopt the small autonomous teams’ model, which works at Amazon as they have hugely skilled developers, but everything there is built as a service; they aren’t as regulated or limited by legacy software. [For other organizations, though,] it creates chaos and organizational debt.”
Where to start with an internal developer platform
Moving from a monolithic deploy process to continuous delivery is a major cultural change, and not one to be underestimated. “I believe it’s actually not that hard to start a small organization with a ‘you build it, you run it’ mindset, but making the transition takes courage and continuity of vision,” Bottcher .
Kersten has seen too many companies try to rebrand their existing processes as an internal platform because it is the latest buzzword. “One of the biggest antipatterns we have seen is rebadging central IT to a platform team but not making the technical or cultural changes required. As the term gains popularity, we expect to see more of this,” he said.
Shifting to an internal developer platform or deciding to build one from scratch can also be a difficult thing to sell into a wider organization, from convincing management to make such a big shift to making developers comfortable with a new way of working that they don’t have total control over.
“Having the capability to understand users and make those cultural changes is the bigger problem,” Kersten said.
His and many other experts’ advice: Start small. Expert Thinking’s Whinn advised: “Stand up a center of excellence and identify use cases where the platform could be developed to make a real difference.” Even standing up a test environment for a single application and building out the required APIs from there can help a platform team get on the right path.
One way to think about this is as the thinnest viable platform, or “being explicit about what about the platform is important and ensuring it is no bigger than necessary,” said Matthew Skelton, one of the authors of , during the . “We need to make sure whatever we build is compelling to use, has strong developer experience, and treating users as customers who we need to speak to so we can understand their needs and meet them.”
Humanitec’s von Grünberg said, “It is the same if you build or buy: You have to start at the grassroots level.… We usually see organizations take a small group of their best engineers and ask them to be the glue across segregated tool chains. Then you start to centralize this around a common API that teams can work against and bring structure to that sea of unstructured tools.”
Copyright © 2021 IDG Communications, Inc.