When app layer companies first raised monster rounds, the justification was that the cash would be needed to train domain-specific models. The cash would be funnelled into GPUs to stand up an impenetrable moat, similar to OpenAI. This turned out to be an idiotic idea.

Training is one of the worst possible ways to embed domain knowledge into an AI system: 1/ Expensive 2/ Hallucination prone 3/ Not realtime 4/ Not personalized

RAG, memory and other types of in-context learning are far better techniques, and have virtually no CapEx by comparison. RAG is amazing because it is realtime, much less hallucination prone, can be realtime and can be personalized.

@sama talks about this as using foundation models as your layer of human reasoning, rather than trying to use them as a database.

For 2 years, startups and VCs subscribed to the herd delusion that if you weren't doing deep model training, you were an indefensible "GPT wrapper". The herd ran in the exact wrong direction--raising unnecessarily large rounds to develop worthless assets.

Now we hear from companies who developed expensive model assets who are shutting down and looking to get acquired--and their models are completely worthless.

BloombergGPT? Outperformed by GPT4 at finance tasks. Harvey's model? Now outperformed by over 10 generalist models in legal tasks (though overall, I think Harvey has adapted super well, and has a great product)

Moats are still buildable, for instance, we have incredible proprietary dataflow around contract market standards which we can query through RAG. But they don't require the CapEx we thought. Cash is not a moat at the app layer.

So now we are in a situation where many app layer companies are overcapitalized. This cash which was meant to have a very productive use: standing up an impenetrable cash moat--is now being directed into much less productive uses:

- Training worthless models - Overhiring - Market orders on talent, driving up costs to unsustainable levels - Market orders on customers, driving up CAC to unsustainable levels - Savings: interest rate won't be enough to justify valuation - Secondary: premature liquidity

This capital misallocation has been hidden under fantastic growth. These are still great businesses providing enormous value to customers. But even great businesses have a price that is too high.

So—we are taking a more conservative approach at Spellbook. We think that the upper range of valuations will correct downwards as the market digests this error, and we will be positioned well to grow through that.

I also wrote a bit about this distortion back when we launched in 2022. Our bias toward things that seem prestigious and hard blinded us from seeing what was pragmatic: https://blog.scottstevenson.ne...

Thanks to @MTemkin for a conversation that helped me make this thesis a little more crisp

@MTemkin Also... Even if you are fine tuning of LLMs: it turned out to be much less expensive and more efficient than people thought.

@scottastevenson Agreed, design + context engineering are sufficient, the only reason app layer companies might want to train models is to prepare for supply chain shock from model providers, which shouldn’t be a problem in the next decade with the amount of investment/competition upstream

@zeeitthru Yes. I do still think training has its place, but it's not as big a place as people thought 3 years ago, at the app layer. Can generally be helpful for reducing costs and increasing speed.

@scottastevenson Added benefit: I think not raising insane amount of money at insane valuation sends a very good signal to high quality engineers on the market.

@matt_ambrogi Yes--and very importantly protects employee stock option upside--which is something I think about a lot. Underwater stock options = terrible time

@scottastevenson You guys are just RAG with legal documents, no moat, cost of your software going to inference + some small amount

@mikert89 We have proprietary data--and many other proprietary aspects in our system. "Twitter is just CRUD, no moat" https://x.com/scottastevenson/...

@scottastevenson The moat is the evals and optimizations. Not just on the primary inputs and outputs, but on the rag and tool calling systems also. Raising to speed that up with domain experts and rlhf seems valuable. PO through dspy/gepa is the base.

@scottastevenson Woah; banger

@scottastevenson great thread - I think this is very well written but I probably really like it because it is always nice to see someone endorsing the approach your company is taking!