Years of Workshops: Industry Trends in Infrastructure Automation

What is an envisioning workshop?

The automation journey continues to evolve for both technology and business needs, and WWT spends countless hours providing thought leadership and consulting, as well as developing and co-developing solutions for our customers around the globe. Our automation envisioning workshop affords us the opportunity to learn how our customers adopt automation in different ways, discover the challenges they face, and absorb insights from the outcomes that support the success of their organizations.

As we analyze data from customer engagements over the past several years, we continue to see the adoption of automation throughout our customer base increase both in scope and complexity, and we work to identify and understand trends in our customer base that align with, or perhaps contradict, current industry trends. We posted our initial findings in 2022, and we updated those findings with data collected from our subsequent workshops. We use this data to analyze the trends in our slice of the industry and predict what the future holds for the evolving automation market. The maturing of automation tools, ongoing industry consolidation, and the ever-present allure of Artificial Intelligence (AI) make a compelling roadmap for the innovation and evolution of automation that our customers seek.

We facilitate our workshops with the primary goal of helping our clients find and prioritize their most relevant use cases for automation, and we continue to use ideation as a foundational tool to capture diverse thoughts about the state of an organization, the known and unknown challenges they face, and the goals they share. These ideation exercises allow us to solicit input from everyone who participates in a workshop equally, avoiding the trap of a few voices in a room steering the opinions of others. We set a theme of "no bad ideas" to help promote innovation, stir healthy debate, and allow the best ideas to stand out organically.

We've analyzed the sentiment of thousands of individual ideas written on sticky notes using the thickest-tip markers we can find to thoughtfully encourage conciseness. We've categorized and ranked the most common challenges and goals to present you with this research as a snapshot of what WWT sees through the lens of our workshops.

Where we are today

When we begin our ideation sessions, before we offer any expertise on how to build organization-wide automation strategies and momentum, we use ideation tools to foster conversations amongst the participants. Before getting into specific use-case ideas, we like to zoom out and ask if you could do anything with infrastructure automation, what would that paradise look like? Maybe counterintuitively, when we ask the most open-ended questions, we tend to get the ideas for the basic, foundational items of infrastructure automation. The data from our workshops shows:

1. Device configuration and management is the most popular idea, and the common drivers for this category include the need for configuration consistency, faster and more efficient device provisioning, and an overall reduction in configuration errors. Even as many of our clients surge beyond these seemingly simple use cases for automation into more complex solutions, we can't overlook the basic need for quick, reliable delivery of core system configuration data.
2. Transitioning the use of automation into daily operations is also a valid and often-repeated goal. Using automation to open and resolve service requests for everyday tasks such as creating new accounts, provisioning storage and virtual machines, and updating network switch configurations are high-demand examples of how using self-service tools to expose automated workflows achieves the goal of giving end-consumers the services they need in near real-time. Additionally, our clients seek to use automation to achieve faster MTTR (Mean Time to Resolution) of system outages or service disruptions by introducing automated troubleshooting to help pinpoint root causes and escalate incidents to the correct teams. We see substantial demand for automated testing and the ability to integrate automation with observability solutions to help achieve these sorts of goals.
3. Lastly, code hygiene through code version control and lifecycle management is the foundation on which all good software solutions are built, especially if organizations want to scale their automation throughout their organization. Introducing standard code repository systems and thoughtful branching strategies using Git-based platforms is essential for effective collaboration amongst product or technology teams. Once a standard operation model for code control is available to an organization, automated pipelines for linting and formatting checks for basic code hygiene are available, and the basis for separating automation code from config data is within reach. Once an organization can model their configuration or architecture for their IT systems as code, automation becomes a force multiplier by delivering the intended state of a system to any hardware or software IT platform or platforms an organization chooses, regardless of the specific vendor or type of device.

In addition to the work we do to document and discuss the ideas participants have about how automation can help their organization, we specifically discover and debate the reasons participants believe they haven't been able to realize the automated reality they want to create.

1. The ability to keep up with the pace of technology changes is a common pain point that workshop participants describe in virtually every organization we work with. The number one blocker is usually not technology, but people. It is the fear of how technology changes will impact the people who support technology that produces such a struggle. Fighting a "this is the way things have always been done" attitude is, of course, not unique to the world of automation, although it is undoubtedly a front-and-center challenge to transforming the way IT organizations function.
2. Time is always a challenge because the IT staff within most organizations is already stretched about as far as it can be. Having the time to learn about and implement automation technologies is often perceived as the most limiting factor to achieving automation goals. Further, the many IT tools and platforms within organizations lead to siloed or fractured automation practices that damage the vision of an automation strategy panacea.
3. A key reason organizations struggle to build sustainable automation practices is a failure to prioritize identifying and evangelizing measurable value from their investments in their automation. We often run into teams who really want to automate, but their automation "why" is tied to feelings that they could do their jobs better or even that others in the industry are passing them by. The ability to view automation as a tool to enable business outcomes is something that WWT helps technology teams focus on as they concurrently work to solve technical challenges with automation tools.

Common goals and trends

Beyond the specific automation goals highlighted above, such as configuration management and version control, more complex use cases have emerged in many of our most recent workshops. The most common trends we see for automation use cases WWT helps deploy are:

1. Software Image Management (SWIM): The ability for organizations to have a common platform or method to manage images and software on disparate targets creates efficiency for teams involed in managing those lifecycles. Having a common SWIM workflow allows teams to enrich the SWIM process with automated ticket management and documentation. Another key benefit is the addition of automated, comprehensive pre-upgrade tests that allow IT engineers to verify the systems' state before software changes occur. This process informs "go" and no-go decisions, potentially saving time troubleshooting unrelated post-upgrade issues. Further, post-upgrade tests ensure that systems return online and behave as anticipated. This increased assurance allows for more upgrades during maintenance windows due to the confidence of operations. Here is an example of a SWIM solution I presented at Red Hat Summit and our Automation Meetup in 2024.
2. Network Source of Truth (SoT): The vast majority of the organizations we work with have an IT Service Management (ITSM) platform, usually ServiceNow, that serves as their de facto Configuration Management Database (CMDB). However, we find that organizations typically use the CMDB function of their ITSM predominantly for asset tracking, which, while necessary, does not adequately support the serve as a foundational component for Infrastructure as Code (IaC)-based automation frameworks. IaC frameworks require the capacity to store configuration elements of specific components and the ability to represent the architecture of entire interconnected systems and services. For example, we see many network operations and engineering teams adopt open-source tools like NetBox to store representations of architectures for complex networks.
3. Automated Testing and Compliance: As organizations implement automation frameworks and reliable IaC SoTs, they position themselves to adopt more proactive and operationally focused automation solutions. The ability for IT operations teams to compare the desired state with the actual running state of their environments allows automated testing frameworks to perform scheduled, change-based, and on-demand health checks that can integrate with ticketing and incident management systems. The result is an elegant model for automated IT operations that includes built-in proof of compliance and the ability to respond to certification and audit requests with minutes of work instead of weeks or months.

Where do we go from here?

As we examine the IT automation horizon, we see a few specific areas growing substantially over the next few years.

1. Event-Driven Architectures: As observability and monitoring platforms continue to mature, the data we can gather from networks or other IT systems is more reliable and granular. The concept of self-remediation is on the minds of many of our customers, and we see a growing demand to use information in IT environments to inform changes. It's not just about the ability to predict IT service expansion needs or speed troubleshooting; it's about being able to automatically remediate service disruptions based on large amounts of correlated data that matches known and sometimes unknown patterns. Red Hat's release of Event-Driven Ansible (EDA) has lowered the barrier to adopting closed-loop automation, and we are beginning to see more widespread adoption of this architectural pattern.
2. Artificial Intelligence: Of course, Artificial Intelligence is part of the automation landscape. We contend that AI has a limited capacity to produce value without pairing with automated services. We continue to see the adoption of code development assistants like GitHub Copilot and Ansible Lightspeed to simplify and standardize automated workflow development. Many new use cases for pairing AI with automation are maturing quickly, including advanced network telemetry and proactive system health inquiries that natively integrate with observability platforms. We expect AI solutions to continue to advance and proliferate at a rapid pace. (WWT is at the forefront of AI technologies as is visible in our AI Proving Ground.)
3. Metrics-Driven Automation: As organizations succeed with automation solutions and their automation teams and practices mature, the need to document that automation becomes more important. The need to measure the effects of automation across teams helps prioritize ongoing use case development efforts. It's critical for organizations to have the ability to demonstrate how automation impacts their value streams and organizational objectives to justify ongoing investment in automation. Organizations that can tie the value of automated solutions to their strategic outcomes have a sustainable model to achieve the vision of their ideal automated "paradise." These outcomes align with improved quality of life for an organization's employees while reducing costs by optimizing the use of assets and reducing downtime by delivering more reliable services.

In summary, although the automation landscape continues to evolve, the goals, drivers, and blockers remain similar to those we explored in 2022. The solutions have become more mature or more varied, but without tackling the foundational issues and starting small, the larger outcomes are impossible to achieve. Establishing a culture of constant improvement and iteration, establishing automation as part of the mindset for daily work, and making automation development a core tenant of the day-to-day life of operations and engineering teams is the sustainable path to automation paradise.