About a month ago, I wrote one of the first posts on this blog about delisting closed-weight models on the BayLeaf API while still serving requests for them: https://blog.bayleaf.dev/p/delisting-closed-weight-models
Today, I’m blocking access to closed-weight models in the BayLeaf API.
Why make such a big deal about whether you can download the model weights? I don’t think I want to try to convince you on this specific point today. Some of you might already be sold on it on the basis of relational reciprocity (giving back to the Web that made these models possible) or free-market economics (access to model weights enables an ecosystem where many different implementations of AI can thrive). Others might just read it as me trying to argue against the generic architectural pattern of software as a service, which (and you’re so smart) I seem to use all over the BayLeaf project, actually.
I’m making a big deal of it because I think some big deals ought to be made, with promises, compromises, and with robust and inspectable implementations. Let’s draw some lines, propose not to cross them, and see what living with that decision feels like (at least for a while). At this specific moment, I don’t think it is super important precisely where those lines get drawn.
Here are a few packages I’m imagining:
No closed-weight models (the one I’m implementing): Just don’t serve inference for models for which we can’t find an associated public weight repository (e.g. a HuggingFace link). This might lock us out of future proprietary models that make more efficient use of energy (proxied by inference cost in dollars) or human attention (proxied by number of agent steps to complete a task). However, because whether the models are downloadable isn’t intrinsically coupled to efficiency and other issues, the risk of potential, future, temporary regret seems low. Seems reasonable. Let’s do it. Done, even if it reverses one of my earlier deliberate choices to preserve access for research and comparison.
No data or money for inference to problematic providers (or whoever is suitably OpenAI/Google/Anthropic-like to attract your ire): I like the hit-em-where-it-hurts spirit of this one, but it gets tricky to implement. Can data flow back to OpenAI for the use of their open-weight GPT-OSS models? Can we use GPT-OSS from a non-OpenAI provider? Can money flow to Amazon, who pays Anthropic behind the scenes? What about how Anthropic pays SpaceXAI behind the scenes for Colossus? Can we still use Google Drive or even Google Search if we’re boycotting Gemini? If we effectively reward Google for good behavior by paying them for inference of open-weight (or other less-problematic category of) models through their Model-as-a-Service (MaaS) system, are we still feeding the beast? This confusion is what kept drawing me back to just blocking closed-weight models and seeing what it is like to live with that choice for a while.
Model–harness disaggregation: Microsoft once famously got in big trouble for bundling a browser too tightly with an operating system, but it was fine for Google to do the same a decade later, at least for a while. There’s currently tension around how strongly Google pushes to be the default search engine in beyond-Google browsers. Something analogous is happening with mobile operating systems and their associated (or importantly not-associated) app stores. In the generative AI space, I can foresee problematic power relations when models and harnesses are too tightly integrated. Currently, you can use non-OpenAI models inside of OpenAI’s Codex harness, and you can use Claude models in harnesses beyond Claude Code. But you can’t leverage your Claude subscription when you leave an Anthropic-provided harness. And the situation between Cursor (the harness), Cursor Composer (the model), and being acquired by SpaceXAI (who makes their own models and harnesses with different degrees of mandatory coupling) is rapidly changing as I write this. Rather than a wait-and-see, sure-hope-we-don’t-get-locked-in strategy, we could just commit to exercising the discipline of never running a harness that comes from the same organization that provides the model. BayLeaf encourages but does not enforce this usage pattern. Trying to live by this one myself has gotten me to mostly give up on Anthropic models, because I can’t access them in a cost-efficient way without leaving my agentic home in OpenCode and other open-source harnesses that I’ve contributed to. When model–harness antitrust action comes, I’ll be ready, but it doesn’t feel like a big issue for others.
Capped context windows: This one is trying to merge some kind of self-care practice with labor action by way of an inference server parameter tweak, but there is some logic to it. Context windows limit how far back in a conversation (or other agent activity trace) a model can reach when trying to decide how to respond. What happens if we cap context windows on supported inference endpoints at, say, sixteen thousand tokens? (This is maybe comparable to twenty-five pages of text.) This limits how deep of a relationship you can get into with your robot buddy. It limits how complex of a job your robot buddy can do after they let you go. It limits how lost you can get in a conversation before the session hits a hard stop. It also puts a nice low cap on your maximum per-request spend, far below the cost associated with those long-horizon agentic traces that many model-trainers like to brag about. It’s not a cap on the agent’s intelligence, maybe just on the scope of the application of that intelligence. Someone needs to test this to see if the effects on self-care or labor displacement point in the right direction, but it would be super cheap to implement.
Backing up to the title of this post, I’m blocking access to closed-weight models only a little bit because I care about reciprocity and healthy markets. I mostly want to get people thinking about specific, actionable demands and to have municipal-scale institutions adopt them (even if only temporarily) so we can all gain experience with what’s feasible and effective.
(Because my agents have the data at hand: “Models without published weights accounted for about 33% of BayLeaf’s OpenRouter inference spending in August. Claude Sonnet 5 alone accounted for 23% of total spending, and Claude Opus 4.8 accounted for another 7%. The remaining 3.5% went mostly to Grok, with smaller amounts spent on Gemini and GPT models.”)