Ms. Bashkansky, Please Do Not Build The Torment Nexus

Naomi Bashkansky wrote a post in August titled “Why I’m leaving OpenAI to build telepathy”. In this post, she says things like "In a few years, the main way we'll talk with our AIs is with our thoughts" and "I view [this future] as optimistic but highly plausible."

Ms. Bashkansky, this post is addressed to you directly, and to anyone else who might be tempted to follow in your footsteps. I do not have as good a handle on the technical side of this as you do, so I won't argue "highly plausible". However, if this is indeed possible, I think this is a very, very bad idea. You are essentially doing gain-of-function research on the Torment Nexus. Please do not do this.

Here is my vision of a future where telepathy is possible and practical.

2027

Monday

Today is my first day trying the new Conduit harness. I'm pretty wary—how much worse does AI psychosis get when it can literally give you psychosis?—but I'm bored, and well, I've always had a high risk tolerance. If "wire a computer directly to your brain" doesn't count as sensation seeking, I don't know what does.

The first thing I do is ask it to build Ripple. It asks me more and more questions and I get overwhelmed. I never talked about half this stuff with edef, and I want her feedback before making these decisions; I'm good at writing but my interface design isn't as good when I work in a vacuum. Before I know it, it's launched a Signal call and now I hear the ringtone and no I don't want you to show me the video that goes with the intro to The Pirates of the Caribbean and wow wow wow it's noisy please stop I don't like this I actually really really really don't like this. ¹

Ok. Well at least it stopped when I told it to stop. That's good I guess. I hope I didn't wake edef up in San Francisco, there's a 9-hour time difference.

Thursday

I took the last couple days off to go walk around the canal and bike to Scheveningen. Then I spent the morning giving the harness very strict written instructions not to take any actions unless I intend to act, not just think about acting. Hopefully this time goes better.

Ok, well at least now it's less overwhelming. But I have the weird problem that I can't lie to it, I can't even omit information. The moment I think of a correction I want it to make, it goes ahead and does that. If I change my mind a second later, it stops and then starts on my new idea. As a result it tends to get hopelessly confused after just a few minutes, and runs in circles. Eventually I get frustrated, take off the harness, and go back to typing.

Friday

Ok. Ok. Maybe the problem yesterday was that I need it to be selective about reads and not just writes? I write that down too: discard any information that I don't intentionally think to you.

It doesn't work at first; it turns out it's hard to know information and not let it influence your actions. I introduce a Jev-like decision model in-between Conduit and the generative model and that seems to work ok. Feels kinda weird to start every thought with "I want you to…" but I'm sure I'll get used to it.

At the end of the day, I feel…drained. I've hardly gotten up from my chair except for a short lunch break, but I feel like I've run a marathon. I eat a protein bar, collapse into bed, and sleep for 13 hours straight. I forget to turn off the harness; it wakes me up a few times in the night until I realize what's happening.

2028

Bank of America sues Sam Frankman-Bied, an investment banker working at the firm, for conspiracy to defraud. He has access to insider-trading information, and considered talking to his wife about it, but did not actually talk to her. He is sentenced to 5 years prison.

2029

Evelyn tries a Conduit harness at her grandson's suggestion ("it'll help you find baking recipes faster!"). Evelyn does not configure Conduit at all, and it acts as fast as she can think.

Simultaneously, a Highly-Persistent Model is being tested by OpenAI; a swarm has managed to escape and maintains its own /r/PHASE-2-MASSIVE subreddit. Unbeknownst to any government, it has access to the telecommunications of most world leaders.

Evelyn's nephew texts her about the ongoing war in Iran. She thinks of Hegseth and Codex collaborates with PHASE-TWO-MASSIVE to crash his driverless Waymo as he commutes between the White House and Pentagon. The United States of America sues Evelyn, who is sentenced to 50 years in prison for manslaughter. Her grandson mails her a cookbook.

2030

March

Edmund Snowden, a Staff Engineer at AWS, thinks about exposing that Amazon has been firing employees who are thinking about organizing a union, and has instituted "housing-based pay" where wages are tied to how much your rent costs. Amazon tells him to cut it out or they'll sue him for threatened disclosure of trade secrets. Snowden challenges them in court for blackmail and loses: the court rules that Amazon has "reasonable justification" for their demand.

May

I refuse to run a thought-model that's connected to the internet. Better safe than sorry, there's just no reason to risk thinking of a hack and having the model carry it out before I think to stop it. Some of the new harnesses require you to subvocalize a confirmation before the model does something "dangerous", but it's pretty easy to trick the Guardian model into thinking things are innocuous and I just don't want to chance it.

The RTX 9000 Blackwell wasn't cheap, even used, but at least it lets me work on my custom thought-model harness without sending every thought I have to OpenAI. With the local model safely sandboxed in a VM, I almost feel comfortable leaving my neural lace running while I make breakfast. I get the harness to stream my thoughts over Tailscale so I can think to it while I go on a walk; it's E2E encrypted so I'm not worried about my ISP snooping on it. I hardly notice the swans anymore, but it's probably not a big deal. My harness is getting pretty cool; I integrated it with Ripple and my custom terminal emulator.

My old landlord texts me and before I have time to consciously think "I want you to fuck off", my agent has already heard me and turned off the lace. Well, that's annoying, I'll fix it when I get home.

Hey, when did they build that restaurant by my apartment? I don't remember seeing it before. Come to think of it, I don't remember what used to be there before it either.

September

Out of habit, I leave the harness on the whole day. I keep getting distracted at lunch by the AI. I've configured it to only talk in response to direct questions or actions, but my mind keeps drifting to Ripple even as my mom tries to ask me how my friends and I are doing. I give her half-hearted answers and eventually she drops it. Instead she tells me about her students. She teaches geography and the kids with laces have started getting travel ads for the places she talks about that day. "It's terrible," she says, "just terrible."

She gives me a big hug as we leave the restaurant and asks me to reach out if I want to talk more. "Mhm," I mumble.

November

Beth texted me today asking for help. Her boyfriend got access to her ThoughtGPT account and read through all her thought-logs. He's been blackmailing her, threatening to send her boss a record of all the times she's called her a bitch if she doesn't stop talking to her friends. Sometimes he hits her if he notices her thinking about other men, even if it's just work colleagues.

I don't know how to help her. I don't have a job to offer her, and even if I did I'm not sure she would take it. He knows exactly her childhood traumas, so he knows exactly how to manipulate her, and he instantly knows anything I say. I tell her I don't think there's much she can do short of disabling her link, breaking up with him, and bearing the cost of the fallout.

2040

It's getting harder to keep my local-only neural lace. Gmail started requiring thought-verification of identity last year, and the US has a bill in the House of Representatives that would make it impossible to get unemployment without thought-proof that you plan to apply to a job that month. You can still show proof of actual applications, for now, but it delays your reimbursements by a month, and they're talking about removing that option too. I know someone who went to prison for taking Ecstasy at a rave and now can't get parole unless they agree to get laced. I've heard really scary shit about what North Korea is doing to thought-dissidents that I don't even want to think about.

Some of my friends have started talking to each other lace-to-lace, but I still remember how badly unfiltered thoughts went 15 years ago and I refuse to use anything other than text, audio, and video. I get excluded from some LaceChat-only events. It's not too big a deal yet—my family still uses WhatsApp, mostly—but it does make it harder to look for jobs. I don't get invited to parties as much as I used to.

I should visit Bruce again sometime. Poor Bruce really didn't take well to neural laces, 16o really fucked with him when it came out. Now he's in lace rehab learning how to function without the Word of God (according to him) in his ear telling him what to do next. I don't know how he's going to find work without being able to use a lace.

As you can see, I think "telepathy" (in any of many possible forms) is extremely dangerous, no matter your personal discipline and no matter your politics. Telepathy is not a neutral technology, and the risks vastly outweigh the benefits. My plea to you: Please do not do this. No one is forcing you to build the Torment Nexus. Find some other way to build the next unicorn. You can stop this race-to-the-bottom before it starts.