Back to Home

February 2024

From: Brian and Tobias

Subject: AI for infrastructure, not infra for AI

Hello all,

For months, we have written about “AI infra” – the tooling required to help AI cross the POC chasm and start driving value for businesses. We’ve analyzed this both in the context of low-level hardware and middleware tooling. AI Infra is overtly significant.

But what about AI for infra? Where in the infrastructure engineer’s workflow can AI supplement and help? Our view is that AI will accelerate all engineers, including infrastructure engineers and not just application developers. How and why will be distinct and necessitate new technologies and companies. Let’s dive in.

‍

Software development is being transformed:

Github Copilot revenue is north of $100M ARR just ~2 years after launch, and research shows that users work over 50% faster than non-users. Recently, we’ve seen pitches for AI tools catering to infra engineers, rather than application engineers, although traction is still very early. This shift is exciting, and has prompted us to dig deeper. We use a purposely broad definition of “infra” to include anything below the application layer, and we've chatted with DevOps, SRE, and security leaders in our network to get their perspective on where and how AI might impact their work.

Broadly speaking, there are two ways we’re seeing AI applied to infra use cases:

  1. Code gen & low code/no code (not exciting)
  2. Agents or copilots to do actual work (maybe exciting)

‍

Code gen & low code/no code

We’ve seen lots of “Copilot for X” pitches, where the X is some kind of infrastructure engineer or infra framework. The thinking is that for infrastructure-as-code, Kubernetes, YAML files, and more, engineers could use a Copilot equivalent. 

We see more hesitation from infra engineers than application engineers, in general, and there’s less data to train good models for infrastructure code than for application code. But, we see a Copilot experience as inevitable as models improve. This is already beginning to happen. See the below Reddit post from 7 months ago. No doubt infra engineers are using Copilot in more advanced ways now. Some have written about this more in-depth.

‍

‍

If there will be code gen tools for DevOps, we expect Github to be the winner. Wherever code lives in Github repos, Github will have an inherent advantage in building a code gen tool. We’re skeptical of approaches that are trying to build a “better copilot” than Github where Github maintains distribution and data advantages, which includes frameworks and environments in which infra engineers spend time.

A cousin of the infra code gen solutions is no code/low code tools to make provisioning infrastructure easier. This category shares a lot with code gen, the primary differences being the user (technical vs. non-technical) and the UI. They broadly solve the same problem of speeding up the infra dev cycle.

Here too, we have our skepticisms, not because abstraction at some level isn’t good, but because infra engineers really want to see auditable work. Their choices are so high stakes and fundamental to the software architecture of a business that there can’t be mistakes. As a result, too much abstraction simply won’t fly. Additionally, where abstractions will be possible, incumbents are well-positioned to integrate them. Who will shift the paradigm around provisioning infrastructure from “infrastructure as code” to “infrastructure as natural language”? Probably Hashicorp (although we’re surprised they haven’t started doing this already), or some other company that already owns the DevOps environment and mind share.

Done intelligently, low code/no code could be compelling. There are places to apply AI to infra engineering tasks like scheduling, A/B testing, Kubernetes configuration, and more where the interface could be low code. Our point of view, however, is that the real value there is driven by AI doing important, labor-intensive work, rather than the existence of a particular interface. This brings us to the next group of companies…

‍

Agents or copilots to do actual work

We’re excited to find AI agents or copilots that speed up or do work that is already being done today by a human being. Across every type of discipline, we think there is a chance to apply Sarah Tavel’s wisdom around selling the work, not software. This is no exception. Our general stance on Generative AI is that it will make the biggest impact augmenting or replacing human labor, improving output over time in ways we could never imagine, at the speed of machines.

So, the question becomes: where are these opportunities in the context of infrastructure engineering? A few we’ve come across recently include:

‍

There is no shortage of examples here, and the pervasive issue in evaluating AI copilots and agents is the sheer volume of them. There is so much noise, and so many technical founders claiming to be able to do borderline magical things with AI. We’re trying to be optimistic, but cautious; exuberant, but calculated.

To orient us in this way and to refine our lens, we’re using a framework to think through these opportunities. Right now, this is our checklist:

This set of criteria basically boils down to: is it possible to rebuild an important dev-focused solution using AI that actually does work? We think yes, and we’re on the hunt for some of these possibilities. Here are a few:

#1 Chip Design:

One area in this vein we’ve gotten excited about is AI for chip designers and hardware engineers. Through our experience with Etched and talking to other founders, we’ve become attuned to the sheer amount of manual, pencil-on-paper work that happens while designing chips, along with the length of feedback cycles and waiting at various steps in the process. A couple of the companies mentioned above are trying to leverage AI to solve these problems.

#2 Data Centers:

How data centers are designed and built and how networking is orchestrated involve a lot of manual grunt work where we think AI can help. These processes are archaic compared to traditional software development, and where there are old, inefficient processes, there are probably automation opportunities, specifically for Cloudflare or other edge platforms that exist closest to these customers.

#3 ETL:

Another exciting example is ETL. Building and managing connectors between apps, the work of relatively junior data engineers, should be something AI could do pretty well. Even more interesting are the possibilities you open up if you are able to pull this off – suddenly you have a tool to pair with your data engineer to help him or her work 10x faster. For scaling companies that don’t want to spend on Fivetran and don’t want to hire a team of data engineers to deploy Airbyte or some other open source tool, this could be a very attractive option. If you know any engineers who may want to explore this with us, please feel free to forward this note.

#4 Security Analyst:

Security also has great opportunities here. We’ve been increasingly looking at companies less focused on securing AI, but on using AI for existing security practices. Tasks like triaging threats and writing up intel reports – the stuff of junior security analysts – is low-hanging fruit for LLMs. We’ve also started evaluating opportunities in AppSec (this alone could be a standalone newsletter, which we’ll save for another day).

‍

There are countless other ideas, including many we’re starting to evaluate but are admittedly still learning about. In the vein of selling the work, jobs like QA testing, data analysis, and Kubernetes management/generation could be opportunities for AI to assist with automation. We’re excited to continue exploring these areas.

Today, the feasibility of automation to enable all of the above solutions is unclear, but we believe that will change over time. One important unlock is advanced modeling for time-series data. Nixtla’s TimeGPT and Google’s TimesFM models are leading examples. Richer time-series forecasting, combined with agentic capabilities dreamed up in some of the startups mentioned above, could be the path towards “doing work” of infra engineers. A mix of technologies, not just LLMs, could be required to cross the chasm from automating marginally helpful work to automating very helpful work. 

‍

We are excited about opportunities to apply AI to problems faced by non-application engineers. This is an overlooked area, but one that we believe has lots of future opportunities.

Thanks all for reading. And a very special thanks to Joseph Bironas, Michael Guarino, Zac Smith, Mark Ross, Abel Mathew, and more for thoughts and feedback that helped us write this piece.

‍

Until next time,

Brian and Tobias

‍