Back to Home

April 2023

From: Brian and Tobias

Subject: Reflections on a week in the Bay

We spent last week in San Francisco, and we want to welcome a dozen or so new readers to our Monthly Infra Newsletter. 

The trip was exhilarating and edifying. We had 36 meetings in 4 days, and it is abundantly clear that SF is back! The petri dish of vision, hunger, and raw talent that creates the Valley is thriving. The bedrock of innovation for the tech industry is secure. We couldn’t be happier. Everyone everywhere will benefit. 

To state the obvious, GenAI is the only game in town. The overwhelming consensus is that a platform shift is underway, but it is wildly unclear how AI will evolve, how fast it will come, and who will benefit most. How to invest is mystifying.  

Inspired from our week in the Bay, this month’s newsletter takes a broad tour of what we’re looking for and where our guard is up. 

Below the Application Layer:

• Open Source LLMs are gaining momentum. This touches everyone, from chip companies to GenAI applications, to the rest of the playing field that includes tech giants like Meta, Alphabet, Amazon, Apple and Microsoft. There is reason to believe open source LLMs will reach parity with the closed-source originals, which means less value will accrue to the model layer than many previously thought (see Google’s leaked “We have no moat and neither does Open AI” memo or Zuckerberg’s comments on Open Source from Meta’s Q1 earnings call, specifically p. 10). The core reasons for this are the ability to reverse engineer LLMs with other LLMs and the vested interest from big tech companies (Alphabet, Meta, etc.) in ensuring that OpenAI/Microsoft doesn't control the market. The end result could be akin to what Google orchestrated with Microsoft, IBM and others with Kubernetes, i.e. push a common operating system for the cloud so AWS wouldn't totally dominate).
• Hardware and models need to pair harmoniously. Huge efficiency gains can come from marrying models with hardware, hence Cerebras’ move to release Cerebras-GPT. Google and FB both have powerhouse chips for machine learning, which gives them some leverage with NVIDIA, and Microsoft is developing AI-specific hardware.
• LLMs need MLOps. The Infra layer for LLMs will keep shifting, as model providers move up the stack and large corporations build out their own stacks. We’ve met with many rising LLM infra businesses that have some hobbyist and hacker adoption, only to find that larger-scale companies that are moving quickly with LLMs have built their own tooling or are looking for more trusted partners to navigate the LLM integration. We believe an appreciation for traditional MLOps is all the more important in this context because model drift, security, data management, productionalization, etc. are all capabilities offered by MLOps solutions and they are more important in an AI-first world. No doubt MLOps will be transformed, but it won’t be retired.
• Vector DBs are the talk of the town, but we question their staying power. The top vector DBs have now raised almost as much money as Mongo did in its entire history before it IPO’ed. Embeddings will continue to be essential and architectures that effectively house, update, secure, etc. are needed, but how hard will it be for incumbents to adapt? We expect incumbents to enter the market quickly, and we have heard from seasoned engineering leaders that embeddings storage is a feature that incumbent DB businesses could offer. Put another way, there is a risk that the technology becomes commoditized over time.
• OpenAI faces an inherent tension that will grow over time. Consumers and Microsoft are pulling the company in opposing directions. Microsoft wants security, a checkmate for Office, expansion on Azure, developer lockin with Github…maybe they’ll even figure Linkedin out! But none of this will touch what consumers want: rapid iteration, new capabilities, access to the internet, etc. It will be hard to be the foundation model to rule all foundation models and the most exciting consumer internet company in the world. 

‍

Cost of Software Going to Zero. From the calculus of build vs. buy decisions, to a dramatic increase in the volume of code able to be produced, to entirely new application possibilities, developer velocity is the most important aspect of GenAI today. 

• Developer velocity is the most powerful GenAI value prop today, by a long shot. We all know the stunning stats for CoPilot (we’re now at 61% of Javascript code being written by CoPilot in editors where CoPilot is installed), but this is just the beginning. Wolverine automatically finds and fixes bugs in your Python code. Our view is that this is far from being an existential threat for developers; instead this will make them all the more valuable and the volume of code is going to explode.  
• We can only glimpse what GenAI will do for Low Code / No Code. Existing players are quickly integrating GenAI, but we will see whether LLM-native experiences will leapfrog incumbents. Perhaps agent-centric approaches where you simply screenshot an app before being able to build a customized version of that app is the future. We heard fascinating reports of companies doing this type of work during internal hackathons to replace existing SaaS vendors.
• While autonomous agents have infinite promise, this area could disappoint in the short-term. We’re on high-alert for instances where manicured demos wow where there’s little adoption beyond hackers and hobbyists playing around. The barriers to entry are just so low, and the market is so hungry for the next thing.

‍

Defensibility is King at the Application Layer:

The application layer is where the rubber hits the road. True value needs to be created.

• We’ve heard that some of the biggest GenAI darlings are shrinking. They could still be phenomenal businesses, but what happens there will be seen as a barometer for other GenAI applications. It is easy to imagine lots of different types of GenAI applications growing super quickly in a few months and then getting squashed by a foundation model business releasing a superior product on top of its own model or an incumbent with existing distribution. General functions will be hard to win without a technology or distribution advantage, i.e. some form of defensibility. 
• There’s promise with domain-specific use cases but the competition is already stiff.  Examples include Harvey and Casetext in the legal field, which are both growing quickly and are applying AI to a very specific domain where there is an acute need to process and synthesize lots of text manually. Other examples include:some text
  • Numbers Station: Data stack automation and analytics. Big problem that requires access to customers’ infrastructure and building a unique UI, so there is potential for more defensibility.
  • Synesthia: Fully customized AI videos for L&D training. These can be lasting pieces of collateral that provide value over and over again once created. They are hyper-specific to different companies’ use cases.
  • EvenUp: Synthesizing medical documents and other files into demands for personal injury attorneys and victims. Because of the domain specificity and the unique data involved in these demands, there’s less likelihood of competition from other AI players.
• Expect a consumer comeback. CharacterAI, and most importantly, ChatGPT from OpenAI, have been rocketship consumer applications that have used GenAI to offer something truly new. Replika and Tome are others that come to mind. They will not be the last, and we think this might be the most exciting time to be investing in consumer in the last decade.
• Our current view is that application layer companies are interesting when a very strong problem is being solved and GenAI is a means to an end. They’re not interesting because GenAI is slapped on some marketing copy or if they are merely a wrapper on top of third-party technology. They’re interesting because of the problems they solve. Then, there needs to be a real defensibility story layered in, with a reason an application doesn’t get crushed in the long-run by a competitor with better technology or distribution. This requires complex workflows that increase switching costs, data moats, network effects, and all the boring business fundamentals that result in lasting moats. 

‍

Lastly, the first mover may have considerable disadvantages in this market. This is inning one. We’re still figuring out where real value will accrue, and other investors are too. We expect more hype-y companies to emerge over the coming months, with some public crash-and-burn examples and other unknown companies that build something big in relative silence.

‍

Our plan is to be on the field, work closely with companies in our portfolio already riding this wave, while trying to understand what’s really needed. 

‍

Until next time,

Brian & Tobias 

‍