You’ve probably been trained to shovel data into someone else’s furnace, and you’ve been trained to feel that this is the friendliest and most convenient way for things to work. Let’s challenge that, at least a little.
Let me illustrate with a story. A student collaborating with me on BayLeaf had a mildly frustrating experience that puts this situation into focus. The student has some PDF files: scans of handwritten lecture notes. They want to chat with an agent about them. They are trying out the OpenChamber desktop app supported by BayLeaf, but they are more familiar with Claude-branded AI interfaces.
They think to bring their data to the agent. So, naturally, they start a chat, press the little “+” button near the message box, navigate to their lecture notes folder, and select a few files. It almost works.
They see something that looks like this:
For those who (with great irony for this post) can’t read images, it says:
Opencode failed to send message with error: Upstream request failed: [unsupported_content_type] Messages may contain text, images and video; this model does not read audio or files.
It’s trying to say that the inference provider has rejected the request because it contains PDF files. To my knowledge, no LLM can natively perceive PDF files, but many can perceive their contents if the files are first rendered as text or images.
The result in the screenshot is a little embarrassing (and someone should fix it, a inference provider), but it isn’t ultimately a sign of a terrible problem. Fixing it would only make it easier for us to bring our data to the agents, when I want to do the reverse. When I try the same request with a different model, or a different provider for the same model, I get a much more useful response instead of an error:
I can’t take PDF input directly, but I can likely extract the text with a shell tool. Let me find the file first.
The agent then goes off to find the file on my computer that I tried to attach (using the filename as a clue), uses command-line tools to convert it into familiar text and image formats, reads those, and continues to chat about the contents of the file.
If I had successfully gotten the model to perceive the contents directly (perhaps with the provider quietly rendering the PDF as images, as if I had attached image files instead), I’d be worse off. The model could answer some questions about the file, but it wouldn’t be able to copy it, modify it, transform it, zoom in for detail, extract non-visual data from the binary, and so on. I wanted to talk to the agent about files on my computer. I didn’t actually want the file itself to be treated as if it were my speech: an immutable message from me, disconnected from the rest of my computer.
Maybe this distinction is hard to see for PDFs because we sometimes think of them as just fancy images. Although it’s uncommon, you can embed video, CAD models, or interactive elements in PDFs. So, think about software: there’s a huge difference between being able to run an app and having access to a screenshot of it. If you recommend that I try an app, you’d be frustrated if I turned around and reviewed it based on just a screenshot.
If you follow your consumer training and shovel your data, chunk by chunk, into a cloud service, you are doing unnecessary work for a limited benefit. However, if you invite the agent, which might or might not be powered by a cloud service, to work with your data (or executable software) on your computer, there’s more that can be done and less shoveling for you to do.
Let’s return to the student with the handwritten lecture notes. If they are using an agent to work with the files on their computer, they can ask that agent, perhaps just once, to transcribe the notes, saving the result in plain-text files on their computer. Other agents or tools can then scan these text files for keywords. The student’s operating system might index them automatically, making the notes easier to find even outside the agent that did the visual transcription.
When I write “bring your agents to the data, not the data to the agents,” I want you to notice that bringing your data to the agents almost immediately requires separating it from context, disconnecting some segment of it from the rest of what might help interpret it. The smartest agents can only do so much with that fragment (and some might keep a copy of it regardless).
More and more, I feel that “using AI” (in the 2026 sense) is increasingly just “using my personal computer.” I’m frustrated when using my personal computer gets distorted into using someone else’s data-gobbling subscription computer. Let me bring those agents to my data. Don’t ask me to chip off little slices of my data to feed your agents. My computer is right here.
—
Because I can’t help it, here’s what some narrow but effective fixes for the error in the screenshot might look like. On the inference provider side, when the request contains an unsupported media type, render the message about what was attempted to the model as if it were part of the user’s request and let the model respond (e.g. in text) as it sees fit. This let’s the larger agent notice, respond to, and potentially recover from the problem. Alternatively, the harness (OpenCode here) could be modified to detect media-type related request rejections and re-attempt to send message, now with the diagnostic information. A design where we quietly re-encode the offending media at lower and lower fidelity until the model accepts it without the user noticing: let’s not do that. Fail loudly (in text delivered to the model), and let the relevant stakeholders who have way more context for what’s truly intended attempt recovery for themselves.