A few months ago, I wrote a post reporting on the kinds of things people were doing with BayLeaf Chat. In the time since, I’ve come to feel that we should make this kind of not-actively-consensual data aggregation, analytic insight mining, and publication increasingly difficult to impossible.
Here’s a stack of anti-analysis strategies I’ve been applying since then to user activity data:
Don’t concentrate it
Don’t store it
Don’t handle it
In the current design of BayLeaf Chat, everyone’s data is organized into a common format and stored in a database. It’s encrypted at rest, inactive records are automatically purged, and there’s currently a single human (me) with the organizational authority and technical ability to access the data. “But we can open the box,” I think to myself. The cookies are right there, and I ate some to write the “What are people doing with BayLeaf Chat” post last August.
Instead of making it super convenient to use the data-aggregation machine, let’s put some effort into making it super convenient to use other machines, one per person. Let’s let people decide which app feels right for them, knowing each has a slightly different way of storing data.
A few weeks ago, I started rearranging BayLeaf services to expose managed apps on per-user sandboxes. On some level, this was a response to Meta’s Muse and OpenAI’s Dots: give each user’s agents a little container or virtual machine where they can store data and run code. On these sandboxes, we have one-click launchers for apps like OpenChamber and Nanobot. These claw-like apps are designed for individual users, and they record conversation data in sandbox-local files and databases that don’t get synchronized with a central archive.
BayLeaf has had per-user sandboxes for quite some time, so what’s new here is convenience of access and app setup. Making it easy for people to use a variety of individual-scale apps means people can step away from apps that use insight-mining-friendly data storage. This implements the “don’t concentrate it” strategy on two levels. First, storage is decentralized: the data is literally not stored in one central place. Second, storage is somewhat diversified in format: there’s no pressure for users to pre-organize their data into a common format for analytics.
As I write this, I’m thinking about how weak the “don’t concentrate it” strategy is. As the BayLeaf operator, I can access everyone’s sandbox (just as Meta and OpenAI can access their customers’ sandboxes). There aren’t too many different conversation data formats in use. My agents could probably almost completely re-centralize my users’ data in a few minutes. Those few minutes of friction matter, but we need to go deeper.
How can I get users to store their data on machines that I don’t already fully control? I can (and do) provide onboarding instructions and support for people transitioning from oh-so-convenient web-based agentic apps to apps that run on their own devices. “Own devices” here means things like your personal laptop, something I can’t log into. When these apps record your data in local files, it’s on your computer rather than one I’ve loaned you in the cloud.
This is part of how I implement the “don’t store it” strategy. One wrinkle here is that we need to make it so that nobody else stores a copy too. If your apps aren’t using on-device inference, they’re relying on someone else, probably somewhere in the cloud, to process your data for you. The deal you want with those providers is zero data retention (ZDR). If you’ve got that, then your laptop is indeed the only one storing your data. The BayLeaf API only integrates ZDR providers, so you’re good.
Well, the BayLeaf API currently only integrates ZDR providers. I don’t want to, but I could change that at any time. We could open the box of cookies with only a few minutes of friction, installing centralized logging. The “we” here could actually be someone else exploiting my credentials after some other data leak, adding these logging features even when I don’t want them to be added. Again, “don’t store it” isn’t enough if “just begin storing it now” is only a few minutes away.
Zero operator access (ZOA) is what we’ll need for the “don’t handle it” strategy. If I can’t handle your data, I (or the people who hack me) can’t flip a switch to begin storing and concentrating it. There are two main strategies I know of to achieve ZOA. One makes me do extra work, and the other makes you (or your laptop) do extra work.
I’ve written before about “military grade encryption” for inference services. This refers to LLM inference operators that use hardware-backed (rather than legally backed) attestation to verifiably run software that doesn’t store your data or handle it in ways that make it readable either by the inference provider or by me as the one routing your data to the inference provider. In this design, you are still handing your data to an external party for processing, but the friction they’d need to overcome to simply turn on data logging becomes enormous. Using these services (called “sealed” inference in the BayLeaf constellation, but “confidential” inference more broadly) is only a tiny bit less convenient than standard inference. However, you still need to trust us a little bit and accept a little bit of risk. That said, this post isn’t about you; it is more about me and making it not so easy for me to open the box and eat the data-analytics cookies.
The other way to get ZOA is to just have zero operators, or more precisely zero operators other than yourself and your own personal laptop. Hobbyists on the r/LocalLLaMA subreddit have been running individual-scale inference services for years now. While some have escalated in scale (r/HomeDataCenter), others on r/LowEndLocalAI are de-escalating: finding new uses for discarded, previous-generation GPUs, CPU-only laptops, gaming consoles, smartphones, and even (questionably) smart TVs.
You’d be mistaken to think that there’s no corporate interest in supporting down-scale or even backward-looking local inference setups. Within the last 24 hours, model-training company Liquid AI released their latest, bleeding-edge medium-scale language model: LFM2.5-24B-A2B. This model fits comfortably on phones and laptops with 16GB of memory and extrudes text at usable speeds. Getting it running on your laptop isn’t nearly as convenient as using all of the stuff I provide via BayLeaf, but you and a BayLeaf-backed helper agent could probably have it working in an hour of tinkering.
Stepping back, are the anti-analysis strategies I’m sketching here an expression of my personal hatred or dislike of analyzing user data? No, I actually have a lot of fun and learn a lot by analyzing user data. The issue is that I think we already do enough behind-the-scenes data analysis in the higher-education sphere. So, we should make sure to explore ways that don’t mindlessly sign us up for a whole lot more.
But, the ed-tech product manager inside you suggests, if we don’t mine everyone’s usage data, how are we going to figure out how they are using the products so that we can truly improve their user experiences? Use a social technology as old as AI itself: users’ groups. Support the growth of users’ groups, and take their input seriously.
I’m only recently starting down this route, but at least I’ve come up with something that has a delightfully ugly acronym: BayLeaf Users Groups, BLUGs. Bring something you’ve learned to do with BayLeaf or BayLeaf-like systems, get help trying something new, brainstorm community demands, etc. Don’t wait for us (or me, specifically) to mine your data for product design insights. Just talk in the open in your community.