February 2023
From: Brian and Tobias
Subject: Is AI a fad or real platform shift?
Hello!
We invite you to take a breather from the SVB-induced doomsday vibes and think about the future. This month's newsletter is all about AI.
Forwarding this to a friend? Let us know their email address and we can make sure to add them to future sends.
We’re also hosting an event on Wednesday, 3/15 on infrastructure cost savings featuring Jordan Tigani (MotherDuck), Zac Smith (Packet / Equinix), Nitin Pillai (Dataminr), Kayla Taylor (Datadog), and Sam Weaver (Plural). Check out this page to learn more – we’d love to see you there!
Is this a platform shift?
Platform shifts create massive opportunities for investors and founders. But what is a platform shift? Ask five different VCs and you’ll likely get five different answers. Ask ChatGPT to define a platform shift, and you also get a pretty confusing answer:
“A change in the dominant technology or infrastructure that enables a particular industry or ecosystem to function.” Confusing.
We honestly don’t know if Generative AI is a platform shift. VCs are incentivized to believe a platform shift is coming, or at least to behave like it is in case that comes to be, but we really don’t know for sure.
Generally, we’re AI optimists. The primary reason is that the power of AI is accelerating at a ferocious pace and human-esque capabilities won’t stop emerging. The technological advancements happening are unquestionable.
At the same time, the traction from products utilizing this technology is staggering. Not only is the tech impressive, but it’s showing signs of commercial success. The extent of the commercial success remains to be seen, but early signs are promising.
With all that excitement, we are reminding ourselves of the very possible bear case, which goes something like this: Generative AI has created a wave of awe, but it will fade quickly as users tire of chatbots that produce predictable outputs. Like EV and Robotics, the last mile problem in AI will still thwart the miraculous – drug discovery, labor dislocations through automation of knowledge work, Her-style consumer experiences, and ultimately AGI. The corporate rush to insert AI Assistants into their product will be in the dustbin before the end of 2023 and the promise will move out for years. Fundamentally, probabilistic models about what humans might say based on past situations are still very different from how humans interact with the real, utterly unpredictable, present.
Tempered excitement is how we’re trying to approach all the startups we see in the space. There are reasons to believe we’re on the verge of the next big thing, but there’s also a realistic path to this falling a little flat. Obviously, we’re rooting for the former.
LLMOps:
Large Language Models elevate, not eliminate, the need for MLOps. While it's true that you can do WOW-inspiring things with ChatGPT API access, those experiences will quickly get trounced by more nuanced, vertical-specific use-cases. To autocomplete my email or write the first draft of this newsletter, in a consistent way, you need models that effectively train off our writing, in these formats, and GenAI still falls radically short of this.To build it right requires added data pipelines, training, tuning, inference, and more – all the stuff of MLOps. One CTO in our portfolio working on an MLOps business (that is increasingly accommodating LLM use cases) validated this line of thinking, saying, “LLMs just increase the need and urgency for good MLOps. Lots of existing solutions can be used for companies operating within an LLM framework.”
This is the layer where we’ve spent the most time. A bunch of companies recently raised big rounds focused on solving some portion of this question. Langchain, Dust, and Fixie AI are just a few examples. The infra layer for LLMs has become so pervasive and the category has heated up so much that VCs are starting to coin a new term for it – LLMOps. Memes are popping up like this:
As the name suggests, the rationale driving interest in this sub-category of broader LLM startups is that the traditional MLOps stack will be rendered useless for companies building applications on top of LLMs. Instead of having to collect data, label it, and train it, you can plug into an OpenAI API and have a model ready to go. There’s a lot that needs to happen from that point on to get the model to actually be useful, but the point is: the ML paradigm and workflow is different, and the tooling needs to reflect that. The folks at Unusual Ventures summed up the differences between MLOps and LLMOps nicely:
Although we’re excited about LLMOps and agree that new tooling will need to come online to support builders of generative apps, we also have a more measured approach. We recently spoke to a founder in the generative AI space who’s raised ~$16m. We asked him about infrastructure solutions and he was skeptical. His point of view is that we need to see way more application-layer businesses emerge in order to even know what the infrastructure layer should look like. Because of use case-specific requirements of applications, many apps are going to build bespoke infrastructure that isn’t relevant to other companies. Where the horizontal opportunities end up being biggest remains a big, unanswered question.
If history is any indication, perhaps we will see application-layer winners before we see infrastructure-layer winners. If we think back to cloud (a different platform shift), the biggest initial winners were apps (e.g. Salesforce, ServiceNow, Workday), while infrastructure winners are still being anointed (in fact, one just became the fastest company to $100M ARR and a $10B valuation). We think big companies will emerge at both layers of the stack, but that it’s too early to know where the real value will need to be created at the infrastructure layer.
Fine-Tuning:
One element of using LLMs that we’ve been studying a lot recently is fine-tuning. In order for these models to be really, deeply helpful for enterprises, they’ll need lots of fine tuning and customization that builds off the original LLM. Think about the Notion AI product. It’s been a bit of dud because it’s just not helpful enough for the specific use cases users want – to do lists, wikis, etc. The models they’re using need to be fine-tuned to fit these specific purposes.
In Reid Hoffman’s interview with Sam Altman from late in 2022, Sam says he thinks there will be big opportunities for companies that take an LLM and make it fit for specific industries (e.g. the model for medicine). Check out the ~3 minute mark. This will be an essential piece of the puzzle, and something some of the infra layer startups mentioned above are trying to help with.
If you know engineers deep in the weeds of the fine tuning process – prompt engineering, labeling, RLHF (reinforcement learning from human feedback) – we’d love to talk to them. We think this is an essential piece of the process to really understand because the biggest value of LLMs will only be unlocked when the models can become fit for specific purposes and companies. Any intros appreciated!
These are wildly exciting times. We can’t say that we know whether this wave of AI will fall flat on its face or truly be the next platform shift. Regardless, there is some awesome stuff happening out there – as VCs it’s what we wait for.
As always, please feel free to share feedback, thoughts, and reactions that emerge from this distribution. And if there are others you think we should include, please introduce us!
Until next time,
Brian and Tobias