According to recent studies, ∼94% of companies use Cloud services, and most of their data is stored in the cloud. It, therefore, isn’t surprising that cloud services are growing rapidly, as they offer organizations flexibility and the capability to develop new services, by provisioning infrastructure instantaneously, as needed. Cloud services are also considered cost-effective, as no initial investment is needed, and the payment is consumption-based.

The continuous transition of organizational infrastructure to the cloud (Cloud-Native Services) has seriously impacted the work of the DevOps and IT teams within organizations. DevOps teams play a pivotal role in supplying infrastructure, automation, and security solutions for Development teams working in a Cloud environment. They allow developers to work efficiently, quickly, and securely, while maintaining high service availability (to adhere to customer SLA) and scalability.

Over time, dependence on DevOps teams has increased, to the point where the DevOps function, which was intended to promote efficiency and speed – has itself become a bottleneck: many routine developer activities (e.g., adding a feature or fixing a bug) now require communication with the DevOps professionals; and many companies are currently reporting a shortage of the increasingly-critical DevOps resource, creating a real challenge to handle the growing scope of activities.

Seemingly, the DevOps bottleneck could be resolved by recruiting additional experienced DevOps engineers. But the relative scarcity of such engineers makes this option challenging, lengthy, and expensive – while the organizations require an immediate solution for the root problem: the growing dependence on the DevOps teams.

The proposed solution: A self-service platform

One of the more interesting solutions that currently enable Development teams to grow significantly and sustainably, is an Internal Development Platform, or IDP. IDPs allow developer teams to independently use cloud services, and to create, test, and upload applications and programs in the Cloud - all with no involvement of, or dependency on the DevOps team. An IDP platform is a collection of services (Cloud infrastructure, authentication, security, and others), under a single self-service interface for Development teams. IDPs can automatically enforce the company’s standards of development, security, and compliance in a centralized manner – which is a very valuable capability, considering the constantly changing nature of companies, regulations, and markets.

IDPs are enabling companies to manage development processes rapidly and flexibly, even with high growth and high scale. Companies like Twitter, GitHub, Google, and Spotify all have IDP platforms tailored to their needs – and these companies are greatly contributing to the technological community by developing relevant open-source tools (e.g., Spotify’s Backstage project)

Where to begin?

As usual, with tech solutions and in complex companies, you must consider the alternatives: build from scratch or subscribe to an existing platform (SaaS). Each alternative has its advantages and disadvantages. It’s up to you to decide what’s right for your organization.

Subscribing to a platform

The main advantage of subscribing to an existing platform (“off the shelf” product) is the almost immediate value you receive, as well as support and updates. Your company only needs to define its service characteristics, and you’re pretty much good to go.

The main disadvantages are the dependence on the platform provider, and the fact that the platform was not tailored to the exact needs of your company: the platform’s success will directly affect your company’s performance, and if the platform’s capabilities are limited, then the output of your development teams will be suboptimal. Upgrades and improvements can be requested but take time.

This alternative primarily suits small companies, incapable of developing a platform on their own – or companies that need a platform at very short notice.

Building a platform

The main advantage is that your company retains full control. Teams are free to choose the tools and services to be included in the platform, without limitations, and will be able to flexibly adapt the platform to the company’s specific needs.

The main disadvantages are the need to invest significant resources into building the platform, a longer time-to-market, internal ongoing maintenance, and the fact that most companies lack experience and expertise in building IDPs.

A self-built platform is preferable in the long run since it is precisely designed to address your company’s specific needs; it contributes to the Development team’s success and provides your company with the efficiency and flexibility needed to handle changes over time.

7 guidelines for building your own platform

1. Set goals: Create a list of your company’s requirements, goals, and users. Choose an appropriate team to be responsible for the platform’s development.
2. Synchronize the teams involved: The transition to platform-based work (vs working directly with DevOps engineers) is sharp, and the platform’s success depends on how well the internal teams communicate and understand (and address) each other’s needs. Hold a kick-off meeting, and ongoing synchronization meetings between the platform team and the other relevant teams to share progress and adjust based on the feedback.
3. Choose the tech tools: Remember, many solutions already exist, including some in your company. You can keep up with the variety of tools recommended by the CNCF and combine those with tools that are already in use in your company. Ensure the selected tools are compatible with your company’s current and future growth goals. Moreover, confirm that the open-source licensing model is compatible with the cloud and platform needs.
4. Have a uniform standard: To allow users to work independently, a separation layer must exist between the user (developer) and the infrastructure. The separation must be standardized and allow full use with no involvement of IT teams. The users must of course be trained to be able to play their part.
5. Move to software-based Cloud infrastructure management: The preferred method to manage Cloud infrastructure is through smart agents in a method known as infrastructure-as-software (IaS). There is another option used by companies known as Infrastructure-as-code (IaC), but it becomes far more complex in the advanced stages.
6. Establish a monitoring and control mechanism: Make sure to control and monitor resource usage on the platform, to prevent excessive or unnecessary use. This way, teams responsible for Cloud costs can monitor and set limitations.
7. Build a user portal: To provide developers with an optimal experience, establish a portal where new services can be independently created. This is the platform’s “front”, and will help to create a smoother, more intuitive, and more collaborative development process.

Introducing an IDP into your organization requires resources and time, but it is a unique and effective solution for any company using Cloud infrastructure, especially with the growing bottleneck of DevOps teams. While there is no one “silver bullet” for successful implementation, the above principles should help your organization to implement this innovative solution and advance toward resolving the bottleneck.

Written by Igor Koshchyts, cloud architect at Earnix,