Research firm IDC has recently published an infographic outlining the new requirements for modern software distribution and key guidance for organizations to meet future needs and overcome the bottlenecks in the next era of large-scale application delivery.
With the advent of DevOps and modern delivery practices, organizations are becoming better at rapidly developing and deploying new applications to better serve their customers and stay competitive in today’s digital age.
With the increased frequency of applications’ updates, a new order of challenges has emerged for large-scale software delivery in the enterprise: software distribution.
What is software distribution
Software distribution is a critical part of your delivery process. Once applications are developed, software binaries/artifacts and Bill of Materials (BOM) need to be securely and reliably distributed to the next stage in the SDLC.
The next stage could be your testing or integration stages in your pipeline, or distributing to the last-mile of deployment to production – across vastly distributed, complex runtime environments and infrastructure ‘edges’ and ‘things’.
Bottlenecks in modern software distribution
Artifacts distribution has become increasingly more challenging due to recent trends in modern application delivery.
Rapid development and deployment, infrastructure and edge/IoT explosion, and increased adoption of cloud-native and embedded apps — all slow down distribution due to network utilization, security, and complex topologies’ overhead.
- Rapid development: Developers are hard-at-work producing more and more new applications and continuously updating existing ones, to enhance them with new features, security patches, and more. An IDC report estimates that by 2024, we will see over half a billion brand new digital apps and services created and distributed. These are a lot of applications developed in an agile manner, with more frequent iterations – meaning more software binaries being produced and consumed than ever before.
- Increased deployment frequency: With the increased adoption of DevOps and Continuous Delivery practices, deployment frequency is on the rise across all organizations. Many enterprises are now deploying to production multiple times a day. Consider, too, that every Production deployment is preceded by numerous deployments to the lower environments.
- Infrastructure scale and Edge explosion: To deliver optimal digital experiences enterprises need to run applications closer to their consumers, globally. Applications are being deployed across increasingly more and more complex, hyper-scale, distributed environments – spanning core data centers, cloud/multicloud infrastructure, as well as the exponentially growing numbers of local Edges and embedded devices that are part of IoT efforts.
- Cloud-native and embedded applications: To further complicate things, the increased adoption of cloud-native modern applications, and the increase in embedded applications for IoT and the myriad of smart ‘Things’, mean that we’re producing applications that rely on compounded artifacts – such as container images and embedded software. These are much heavier to share and distribute across the network, since they include a lot more application dependencies and components in them. Consider that 80% of Edge workloads are container-based. This means that when deploying application updates, every single node in every single K8s cluster across all regions and Edge devices needs to first do a “Docker Pull” to download the binaries from the network, to start the deployment sequence. This creates network bottlenecks, as more weighty artifacts now need to be more frequently and reliably shared via the network and distributed for deployment across remote infrastructure edges (which often have limited bandwidth and compute resources).
The 3 consumption points for software distribution
Once developed, software artifacts need to be securely and quickly distributed to these three consumption points:
- Internal repositories – distribution to developers across global sites and CI servers for continued development and testing
- External/ecosystem repositories – for consumption by the developer ecosystem, partners and customers. For example, drivers, plugins, OSS components, and others.
- Machines – Runtime environments and infrastructure ‘edges’ and ‘things’ where applications are deployed.
Distribution speed impacts delivery speed
Failure to address distribution bottlenecks is a major risk to businesses. When your remote offices around the world can’t get their hands on the most recent builds fast enough for continued testing, when your infrastructure nodes can’t download Bill-of-Materials (BOM) of 1,000s GBs in order to start a release deployment, or when your customers and developer ecosystem can’t download your recent drivers, plugins or OSS in a reliable manner — productivity and release velocity suffer.
Success in digital transformation requires a scalable and robust Software Distribution mechanism for organizations to deliver innovative applications across data centers, hybrid/multicloud environments, Edges, and embedded devices, and “things”.
Written by JFrog CPO Dror Bereznitsky