In the enterprise IT discourse, the evergreen “build-versus-buy” debate presumes you are going to be spending big either way. Do you spend a bunch to build exactly the system you need without paying for features you don’t care about? Or do you eat the markup buying more than you need with the confidence that the thing you buy is being professionally maintained by people who have learned from your competitors’ mistakes? Do you lock in prices for ongoing staff salaries or lock in prices for vendor service agreements? For BayLeaf, we kinda don’t do either.
We don’t spend big on BayLeaf. We don’t build much, and we don’t buy much either. We don’t train models, don’t build inference servers, don’t assemble physical hardware, don’t develop general-purpose chat interfaces, and don’t build coding-agent harnesses. Okay, we did make one coding-agent harness, Lathe, but we spun that out into a BayLeaf-independent project immediately after it was useful enough to be worth sharing. We do, however, contribute patches upstream to Open WebUI, OpenCode, and OpenChamber. We do use LLM inference through OpenRouter and Tinfoil, web context from Tavily, and sandboxes from Daytona. We do have a custom service running for the BayLeaf API, but did we make the right build versus buy decision there? A recent, uh, incident got me thinking.
Earlier today I realized a problem with the BayLeaf API that might have been caught if there were more than one person looking over the implementation. Users of the BayLeaf API (at least the part of it that wraps OpenRouter) are subject to a daily spending limit. However, users could effectively bypass that limit by repeatedly revoking and re-provisioning their BayLeaf API key. Users need to be able to rotate their keys in response to personal leaks, so it’s a legitimate usage pattern. My earlier implementation of the BayLeaf API specifically accounted for this exact potential abuse pattern, but I didn’t closely track the implementation for regressions. Later work to add Sealed inference accidentally changed the key lifecycle: re-provisioning a BayLeaf key caused the next request to mint a fresh back-end key, resetting the provider-side spending limit. There’s a more precise writeup of the incident on GitHub. What matters is that I made a mistake, it cost me some money (perhaps $300), and it got me thinking.
Of course the buggiest parts of BayLeaf are in the parts we built ourselves. But why did we need to build the BayLeaf API in the first place? Other campuses are deploying the free and open-source API Gateway LiteLLM, even the NSF-funded National Research Platform LLM Service. Should I have just used LiteLLM? This could mean less building, more reuse, and more opportunities to have local frictions turn into upstream contributions with global impact. I may yet decide to adopt LiteLLM, but it would currently run counter to a strategy to reduce total system complexity and institutional cost or environmental impacts.
Any institutional adopter of a gateway product, whether self-hosted like LiteLLM or managed like PortKey, is going to need to glue the LLM inference service to institutional identity and policy: people need a way to log in with their existing accounts. The trouble is that people need more than just LLM inference services to make widely productive use of generative AI technology. The BayLeaf API abstracts over the web fetch and web services provided by Tavily and the isolated code execution sandboxes provided by Daytona. We could have decided to build these versus buy them on a per-usage basis, but we’d still need them. The LiteLLM developers know this, and they have special-case support for web search integration and code interpreters. However, these are slight generalizations of the instantiations of those ideas present in ChatGPT, not the weird and wild versions we have. So, let’s presume any campus-scale adopter is going to build just enough custom glue to solve at least authentication and moderate extensibility (rather than getting lost in LiteLLM’s plugin system). For BayLeaf, that ease of extensibility bought us quick integration of E2EE inference providers, smoothed desktop harness onboarding, a place to document how to use coding agents with Canvas, etc.
It’s hard to speculate whether this kind of incident would have been just as likely if I had been using LiteLLM (or PortKey) instead of OpenRouter for the normal LLM inference path. But maybe?
My cool-headed editing agent, who didn’t just find $300 on the floor, wants to put it this way: “The local seam cannot be purchased away. Therefore, reduce its scope, reuse everything on either side of it, and recycle its inevitable failures into shared institutional memory.” Fine. In this case, recycling means turning the incident into automated lifecycle assertions and this public reflection. Zooming out, what else can we reduce, reuse, or recycle through some different combination of building, buying, or simply b-not engaging? I’ll be on the lookout.
Ah, because this whole incident started with me looking for where the biggest flow of BayLeaf’s spending was going: Stop using Claude Sonnet 5, UCSC folks, or just everyone. According to the DeepSWE benchmark, Sonnet 5 is everywhere dominated by other models, including open-weight models like yesterday’s (mid-sized and vision-capable) GLM-5.3-Flash. You could use this model to generate fewer output tokens over fewer total agent steps to get your thing done while, under reasonable assumptions, using less energy and spending less (of my) money to do it. I like giving people room to explore and learn for themselves, but I’d also like to build (or maybe even buy?) better tooling to gather evidence about irresponsible model choices. Right now I have personal vibes, stale reviews, and scattered benchmarks that aren’t aligned with campus interests. Deciding that a model choice is conclusively irresponsible requires political and institutional analysis. Does anyone want to try to sell us tools for that?