I’m listening to this Darknet Diaries podcast episode about National Security Letters, and it got me thinking about what it might be like to get an NSL as the BayLeaf operator. What if someone comes knocking on my door with a secret demand for my colleague’s data from BayLeaf Chat? What could it be asking for? What do I need to give up? What could my colleague actually learn about what’s happening to them? Would the answers to these questions make me want to change the design of BayLeaf in anticipation of the next letter?
I haven’t received any NSLs (yet), so please don’t interpret this post as a telegraphed announcement of disclosure. I’m just thinking things through.
For narrative purposes, let’s say that you, dear reader, are one of my faculty colleagues. Even more, you are actively using BayLeaf Chat to plan and execute terrorist operations. You are the bad guy, and you need to be stopped. Or maybe you are a just doing Middle East politics research, an international student, or someone organizing a union drive, etc. And I, Adam (the BayLeaf operator), get this letter, and the letter is about you. The FBI or whoever needs me to share what I know about you, and they don’t let me tell the public or even you about how I’m doing it. I can talk to my own layer about it, but that’s about it.
The letter can’t ask for everything about you. Per usual interpretations of 18 U.S. Code § 2709 - Counterintelligence access to telephone toll and transactional records, the letter can only ask me for transactional metadata, not the content of your messages. So, I might be compelled to reveal that someone with your campus email address sent a message or got an agent’s response at a specific date and time, and maybe I’d also reveal the conversation identifier that links several messages into a specific flow. An NSL isn’t going to require me to share the payload of these messages. That’d take a search warrant, but we’re talking about NSLs today. You can learn a bit more about “electronic communication transactional records” (ECTRs) in this DOJ FAQ (hosted by CDT).
I think ECTRs are about where, when, and with whom an individual is communicating. Is a chat with an agent even two-party communication? Is the name of the model-generated title of your conversation data or metadata? Are tool calls and responses protected? How about the length of the messages (token counts)? I don’t know. My understanding is that the boundaries between data and metadata are murky even for traditional communication between two human parties. In this world where you are plotting your attack in a computer-mediated conversation, apparently, with yourself, there’s even less precedent I can find.
Let’s assume the letter is just asking for message timestamps (analogous to phone call initiation records). How much of those could BayLeaf Chat leak? Per our current data retention policy (and automated cleanup machinery), data and metadata for your old chats are faithfully wiped 90 days after you last interact with them (on a per-conversation basis). [✨ Editing agent’s note: Two honest caveats I found while auditing this: attachments are not deleted along with their conversation, they become orphans and get swept on a later run, so a file outlives its chat by up to about 48 hours; and deleted rows persist in the managed database’s backups until those age out on the hosting provider’s schedule.] Well, if your account is in a special records hold group, your records are not cleaned up automatically. However, the Help agent on BayLeaf Chat can transparently tell you if you are in one of those groups. Currently, I’m the only user in such a group because that’s part of how I test the retention exception mechanism.
In my own audit of the data retention machinery for this post, I realized that a user’s custom instructions (system prompts) and explicitly written memories are currently retained forever. Given that these are where you might have the system remember a clue like “Attack in September”, I need to think about how these are retained. Previously, I had been considering them like innocuous profile data (like your display name), but they plausibly contain the precursors to sensitive messages that you’d like to not have haunting you years after writing them. Ah, and if I’m considering them to be subscriber metadata (akin to a billing address in a traditional phone system) rather than message content, are they available to snarf via NSLs? Dunno.
I was relieved to learn that “§ 2709 does not require communication service providers to create records which they do not maintain in the ordinary course of business” (OLC, 32 Op. O.L.C. 145 (2008)). This means I can’t, at least via NSLs, be compelled to start recording additional information. The same opinion is blunt about the other side of the deal: if you do keep a record, even scattered across aggregate data, you have to go extract it and hand it over. If BayLeaf were operated by a big company with tons of internal logs and metrics already being recorded, there might be a rich picture of you being painted by previously-recorded metadata. Here at bus-factor-one BayLeaf, however, there is much less metadata about you than a commercial provider would hold, and I don’t plan on logging more any time soon. That said, I shouldn’t pretend the pile is empty: your account record and group memberships (which can indirectly encode course enrollment) are currently kept indefinitely, the API database keeps a per-backend record of the last date you used each one, revoked API key rows are never purged so an email address is permanent once I’ve seen it, and a handful of log lines in the API carry your email address for a few days.
Speaking of logging, it is important that BayLeaf Chat isn’t the only service available from BayLeaf. Using the BayLeaf API, you can setup chat and coding agent apps on your own personal devices. When you use those, your personal device has the only copy of the message content. So, even if I were served with a search warrant (secret or otherwise), I would have nothing stored to hand over. If you are concerned about privacy of your chats, please stop using hosted chat services altogether, even BayLeaf’s. This is the same move of giving up using Google Docs for sensitive document editing. Keep those documents on your personal devices, please, for your own sake.
Back on NSLs, it doesn’t seem that realistic that I, the BayLeaf operator, would actually be the recipient of the NSL. To the degree that BayLeaf is associated with UC Santa Cruz (which is, to say, partial), NSLs are likely going to be delivered directly to UC administration or to one of my subprocessors (like DigitalOcean or Cloudflare) better equipped to respond in a timely manner. [✨ Editing agent’s note: In fact I am not authorized to accept service on behalf of the Regents at all: UC’s own guidance tells employees to redirect anyone attempting service to the Office of the General Counsel in Oakland.] When your messages come through BayLeaf services, your email address generally travels along with them. So, if someone wants your metadata and has the legal authorization to get it, they can get it without going through me. (Even more reason to practice ways of using Generative AI without a centralized service in the first place!)
If BayLeaf’s records about you were cleanly interpretable as my own records in my role as faculty, they might be additionally protected under the UC Electronic Communications Policy. However, to the degree that I’m operating BayLeaf as a shadow IT system, I’m unclear on how others in power might interpret the policy. It sure would be simpler if the university actually owned this whole thing.
So, if you really are doing something nefarious (or something benign that someone else decides is nefarious), how are you supposed to keep it under wraps? Using your own chat or coding agent harness on your own device will keep your data out of BayLeaf’s databases, but a compromised back-end service provider could leak your data. In the last few weeks, I’ve added a Sealed Inference service facet of the BayLeaf API based on Tinfoil. If you get your local agent to use the Sealed service, the content of your requests is protected from both myself and the back-end providers on a level that you simply can’t get from OpenAI/Anthropic/Google, at least in their Education packages. What Sealed does not hide is the metadata this whole post is about: the fact that you made a request, when you made it, how big it was, roughly how many tokens it used, and even which model you picked. Should this inference mode be the default on BayLeaf? Maybe. The threat model behind that choice goes beyond NSLs, but the associated paranoia isn’t entirely unjustified.
When I saw this slide over a decade ago, I thought to myself “If I’m ever operating a service like this, I’m not going to design it to have that one convenient point for the full take.” But here I am, running BayLeaf Chat in a shape where there is such a bottleneck. At least I’m running an open-source chat harness with transparent operations. I tell myself that the Chat service’s design is justified because it offers people an incremental offramp from Google-shaped services. Time will tell how many decide to follow that offramp.