January 2023
From: Brian and Tobias
Subject: What's ahead for 2023
Happy new year. Hope everyone is having a productive start to 2023. We’re back with our January edition of our infrastructure newsletter, where we share reflections, trends spotted, opportunities to collaborate, and ways for this group to help us.
Forwarding this to a friend? Let us know their email address and we can make sure to add them to future sends.
We have three ideas to touch on this month: AI & hardware, fighting big cloud, and how to sell developer software in 2023. Let’s dive in.
AI & Hardware:
The last three months have seen an inspiring explosion of activity in artificial intelligence and machine learning. ChatGPT is perhaps the most public-facing, mass-market expression of what AI can do (even if users trust it a little bit too much). In parallel, companies like Jasper have demonstrated that AI can be a powerful wedge, perhaps the most powerful one since enterprises started migrating to the cloud in the late 1990s and early 2000s. AI is no longer something mystical that happens in the background to help companies optimize ad targeting or personalize shopping recommendations. The power of AI is now laid bare, both for builders and users. Opportunities abound, and they're up for grabs.
The implications of this democratization of AI are vast, and we’ve discussed a few of them in prior newsletters. However, one clear implication is the pressure it will put on hardware manufacturers. Anecdotally, lots of operators and investors we have spoken to have voiced concern over a GPU shortage, despite evidence that prices are declining after a sustained period of price increases. If GPU availability for AI/ML use cases is truly an issue, there are three possible solutions:
- Better chips
- Smaller models
- More efficient use of chips
Better chips are a constant pursuit, and Moore’s Law says that chip performance should improve steadily (doubling every two years). There is evidence to suggest that rate is slowing, or at least won’t keep pace with bi-annual doubling. There are also lots of companies pursuing technology to reduce the size of models so that they require less compute.
The third option around efficiency of GPU usage is less obvious but also potentially more interesting. We’ve heard investors talk about opportunities to make chips “multi-tenant.” ~40% of the GPU time you pay for on average is not spent on compute tasks, but rather on waiting for data to get to the compute resources. What if you could crunch numbers for one model while waiting for the data required for another, and vice-versa? This is just one idea, but we think there’s lots of room for creativity in and around hardware efficiency in an AI age.
Sticking it to Big Cloud:
When we talk to engineering leaders at scaling tech companies, the biggest thing they complain about is their cloud bill. For many years, it has been hard for executives to criticize or audit their cloud spend because it is inherently (a) opaque, (b) hard to compare with alternate solutions, and (c) absolutely critical to the operations of a business. As a result, the cloud bill has long been perceived as simply “the cost of doing business,” or that it’s expensive but “costs what it costs.” The numbers here don’t lie – around 75% of SMBs and enterprises reported spending at least $1.2M on cloud costs in 2022.
This point of view is seemingly shifting, perhaps because budgets are being cut in a recessionary environment and no line item is too sacred to spare. Ben Thompson recently wrote about how exposed cloud providers, especially AWS, are to tech companies and how this is likely to impact growth in 2023. We’ve written a lot about cloud optimization and FinOps, which seem like low-hanging fruit solutions to the cost problem. We continue to have questions about whether they are long-term, sustainable businesses or simply features that the cloud vendors will roll into their offering.
Another relevant theme here is “repatriation,” the move from cloud back to on-prem (or “edge”). This seems weird and un-progressive. Isn’t cloud the future? Didn’t we spend all that money and energy moving off of on-prem? Why would we move back?
There is a good case to be made that the cloud really isn’t right for everyone and an even easier case to be made that certain workloads that are currently run in the cloud shouldn’t be. We recently chatted with an MLOps leader at a large national grocer who wants to move lots of his AI workloads away from AWS onto an on-prem, in-store infrastructure for both cost and latency reasons. There is internal friction to make this happen because decision-makers get nervous about making changes to infrastructure, especially when that means self-managing. However, the value prop is compelling for certain use cases.
We think someone will figure out a way to make this relatively seamless for companies. That could be an incumbent hardware player like Intel that wants to strengthen their relationships with customers (which the cloud vendors have slowly but meaningfully chipped away at), or that could be a newer entrant. Perhaps it’s a combination of both. Regardless, we think repatriation for AI and ML workloads represents a pretty interesting opportunity. We’ll continue digging in here, and please let us know if you have any people in your network thinking about some of these same questions.
How to Sell Software in 2023:
We’re hearing time and again that there are basically two value propositions that are resonating for software buyers in 2023: (a) cost and (b) security. Even better if you have a story about both. We’re encouraging portfolio companies to turn efficiency, productivity, and or infrastructure-related value props into one centrally focused on cost. We’ve seen a pretty simple shift in messaging here make a surprising difference in some cases. It’s an obvious callout, but still important.
We’re also evaluating investments under this filter as well. If something doesn’t have a clear story around cost reduction or security/compliance improvements, we’re going to be hard pressed to move forward. We imagine this scrutiny will be the new normal, and that’s probably a good thing overall, forcing entrepreneurs and investors to sharply consider where startup value creation comes from.
As always, please feel free to share feedback, thoughts, and reactions that emerge from this distribution. And if there are others you think we should include, please introduce us!
Until next time,
Brian and Tobias