The biggest challenge for Generative AI

Luminaries from Anthropic, Mosaic, DeepMind are all raising the flag

Recently, I attended LLM Avalanche, a mini-conference dedicated to large language models. The event attracted a veritable who's who of the AI technology field, featuring luminaries from Anthropic, Mosaic, DeepMind, and more. These weren't just ordinary attendees, but the technical wizards responsible for building the platforms that product teams around the world leverage to tap into this exciting technology.

Not surprisingly, after living through a decade of hyped-up expectations and tantalizing promises around AI (the kind that required a team of ML trained engineers), these seasoned technologists echoed a concern that’s plagued the engineering field since it started: the real challenge lies not in the technology itself, but in determining what are the valuable problems to solve with the technology. They urged the attendees to shift their focus from the thrill of novelty and experimentation to the practical question of value.

As someone who has been building products using AI over the past ten years, this message struck a chord.

Tinkering is required to understand what is possible from a technology, but substantive growth only materializes when a few of these seemingly quirky tinkering projects proves their worth and gain widespread adoption. This is no small feat, given the time it takes for an individual ‘experiment’ to mature and the inherent unpredictability of success. Currently, there are tens of thousands of such experiments spread across the different LLM platforms, with the platforms' strategy relying on the hope that a few of these experiments will hit the jackpot and fuel massive consumption. But we can’t know which experiments will turn into valuable products, and we can’t know how long it will take for those valuable experiments to mature into the products we will eventually consume. (For more on this concept, I recommend the excellent book by Simon Wardley which you can find here https://learnwardleymapping.com/book/)

This scenario positions these platform vendors akin to utility companies of things like electricity or cloud compute, laying down the infrastructure for potential high-volume consumption. However, the market for such consumption isn't quite there yet . . . it can’t be. Professionals in the field are still figuring out the full potential of these models and how to harness them in a manner that offers stability, predictability, control, and dependability.

It's reminiscent of the cloud market's infancy when AWS was brand new and a wave of early adopters jumped on the bandwagon. Fast-forward five or ten years, and cloud consumption grew exponentially when cloud-native companies like Netflix gained traction.

For today's potential consumers of large language models, there's a silver lining: the transition from self-building to employing these cost-effective utility services is underway. The future is calling a model hosted by a vendor, so start getting better at it now. Instantly, experimentation becomes viable on these platforms due to the dramatically reduced consumption costs to custom building these models, and with time, a small number of these explorations mature into valuable products . . . your experiment might turn into one! As the upfront cost for experimentation has plunged to zero, we are witnessing a cacophony of noise. Amid this tumult, some unique and compelling 'hit songs' of killer apps are bound to emerge.

From a provider's perspective, there's more certainty about the future of the market as a whole. People are coming to these new platforms. Which providers will become dominant is far from certain. Also uncertain is if there will be one or three dominant providers. But the maximum uncertainty lies on the product side – figuring out how to harness these large language models to extract their full value to humanity.

So what do you do as a product team now that these utility service LLMs have emerged? While we can’t know what product ideas will be successful, we can use product management techniques to speed up this whole process. We can’t predict the products that will be successful, but we can predict which teams might have a higher chance of success because of how they work when there is such a high level of uncertainty. Stay tuned for more in the next post.