Ben Mann Monthly May 2024
Mindbloom, Mexico, Manifest, camping, nuclear war, dreams
Listen to the speech I wrote for Manifest about our Responsible Scaling Policy! Thanks to Google for the text-to-speech.
Purpose
An index for my memory
A menu of topics for my next conversation with you
A faster way to share what I’m excited about without the barrier of writing a complete blog entry on it
A skimmable way to spread content I found valuable
Experiments and experiences
I took Mindbloom's 6-week course. After an intake session with an RN or NP, they mail you tablets. You can do the sessions at your own pace, and you can pick a non-credentialed guide that helps you prepare for and then integrate your experiences before and after the session. During the session, there's a menu of themed audio you can select from. There's music-only, 7 minutes of guidance, and fully guided. I tried the latter two options. During the first 7 minutes, you dissolve the tablets in your mouth and let it absorb through your mucous membranes before spitting out the rest. Then you "experience the session" for one hour. The last half-hour, you write in a physical journal. Half way through the course, Mindbloom introduced AI-powered speech to text-based integration and journaling in the app. I found it too overwhelming to interact with a machine in an altered state, so I stuck with paper and pen. Once the journaling is over they say not to drive or operate heavy machinery, but mostly I felt good and able to go about my life normally. Between sessions you talk to your NP again to adjust dose, usually upwards. They offered group-based integration sessions, but I didn't try it.
I felt like I got a lot out of these sessions. Independent of the medicine, I had time to think deeply about what matters to me, what gives me joy, what my priorities are, and generally to appreciate my life as it is. But I also noticed that it felt much easier for me to break free of the bad habits I'd formed lately, and replace them with new ones, which fits nicely with the theory of ketamine as neuroplasticity-inducing. In the last 6 weeks I've exercised more, eaten better, and generally felt less akrasia than usual. I feel more able to mentally and emotionally take a step back when I'd otherwise be caught in the moment, which is a mental motion that mindfulness meditation deeply roots, but which I haven't been doing for many years.
I'm grateful to my wife for supporting me by taking care of our daughter solo during these weekly 2-hour sessions. With a young kid it's hard to carve out that amount of time!
Overall, even if these effects are temporary, I got enough out of the program that I'm confident it was worth the time and money. As soon as I finished, there was an up-sell for 6 or 18 more sessions. I'm going to give it a few weeks to see how long the feeling of agency sticks with me, but I'll probably periodically reengage.
Euda goes to Mexico
We offered to pay my parents to go to Hawaii since they'd taken care of Euda while we went, but they said they'd miss her too much. So we offered for them to take her, and they agreed! Diana and I stayed home while they flew out. Everyone had a great time! Diana and I remembered what it was like to have proper downtime, staying out at a birthday bonfire at Ocean Beach until 9pm one night, and sleeping in on the weekend. Euda played on the beach, at the pool, and met lots of other babies.
To make sure they could get through customs easily, we filled out a Letter of Consent template provided by the embassy through the airline, got it notarized at UPS, and included her original birth certificate. But no one asked for it, probably was unnecessary.
When we picked her up from the airport, my parents said she'd learned all of Twinkle Twinkle. I couldn't believe it. She'd maxed out at maybe 3 word phrases before she left. How could she have learned a whole song? And yet when they said, "Euda, sing Twinkle Twinkle," she proceeded to recite all the lyrics, if haltingly and off-pitch. I was blown away. She certainly doesn't understand many of the words involved, and yet was able to remember them. Rhyme, meter, and melody must be that effective at sticking in her brain 🤯.
Camping trip
Diana planned a camping trip to Anthony Chabot east of Oakland. On the way over, we stopped for Korean Chinese food and ice cream, which were excellent. We did a short hike with Euda. We bought firewood for $10 at the entrance, then hung around the campsites for the rest of the day. The main attraction was attempting to build the fire without some easy fire-starter. All we had was a BIC lighter Diana had packed for her Jetboil. I managed to catch one of the big logs by burning a napkin, but then poked it too soon, knocked over the teepee I'd made, and smothered it. Then we tried again with some dry bark we'd collected and the visitor information they'd handed us when we entered. We were sure it wouldn't catch, but then it did! After a bit more poking and prodding we had a stable flame going, and cooked some foil-wrapped corn that ended up being some of the best corn I can remember eating, even unseasoned. Our clothes and everything we owned got covered in smokey smell after a while. Once the sun started setting, I tried to put the fire out by filling our camping pot with water and dumping it. Normally I would've used a little shovel to bury the flames, but I didn't have any. Despite four attempts, I couldn't stop it! After a minute or two the water vapor would boil off and the wood on the surface would reignite from the heat of the embers still burning at the base. We let it burn down until there wasn't enough fuel left to reignite. Patience!
We borrowed a 6-person tent from a coworker, and Diana bought a queen-sized camping mat. Euda rolled around all night, keeping us awake. Everyone needed a long nap the next day.
The next day when we came home, we threw everything including our backpacks into the wash. The first hot shower after a night in the woods is always the best.
Speech at Manifest
I gave a speech on how our Responsible Scaling Policy ties into AI forecasting, then took half an hour of QA. I learned a lot just writing the speech, since I hadn't made the time to engage with the material beforehand.
The crowd asked great questions, many of which I had to navigate carefully to avoid over-promising or revealing inside information. Press-training definitely helped with this. One example: "How often do you think about the fact that the alignment problem might be intractable?" My answer was something like, "If it is intractable, we should focus our efforts on proving that's the case, and explaining the gravity of the situation to those with power to make things better. But generally we take a portfolio approach to safety that accounts for the variance in possible futures." But I didn't answer the question itself, and in this case that seems good. Giving a numeric answer without additional background wouldn't really help the audience understand anything, or build their own model of why one number or another would make a difference.
There were so many people I respect and admire at Manifest! I wish I could've spent more time milling around, but I only had time to chat with a few before heading back to have dinner with the fam. I did enjoy Dwarkesh's talk, which he structured as a "let me use my interviewing skills on an entire audience at once." I couldn't help but offer my two cents a few times, and afterwards followed up with a few other audience members who I'd disagreed with. And there were lots of talks I wanted to attend that I'll likely dump transcripts and skim or throw into Claude.ai once they're released. I'll definitely go again next year!
I usually get what I call "performance blackout", where when I give a speech or take a difficult test, I can't remember few of the details of what happened afterwards. So I'm especially interested in watching the recording whenever it's released!
Dad life, month 19
Euda is much more fun to spend time with now. She can follow instructions, ask me to do things for her, can play by herself, make up strange games, and parrot pretty much anything I say up to a few words.
The next baby is coming sometime in the next two weeks 😱. I feel much more prepared this time. While I'm sure many things about the next one will be different, many of the skills will generalize, and there are many fewer unknown and known unknowns. I'm dreading the long period of boredom before the new one catches up to Euda's current capabilities, but at the same time, I'm excited to see how Euda will interact with the new baby, and be able to experience things through her eyes.
I'm planning to take two months of paternity leave. Last time, I came back earlier than planned since things were on fire at work and I felt uniquely suited to get things back on track; indeed it was the right choice in retrospect. But this time I'll be surprised if that happens again. Anthropic has grown massively in the last two years, so there should be enough people with a skill set similar to mine that they can do without me for a while. Anthropic offers 22 weeks, but I can't bear to be away for that long. I love the work too much 😊.
Life updates
📣 Published Responsible Scaling Policy update
🔬 Published breakthrough Interpretability paper
📖 Started reading God Save Texas
🤰 Due June 22
Content
5 point Likert ratings for “I would recommend this content to a friend”, sorted
Civil War 4/5
Civil war was a brutal spectacle of how people treat each other when filled with hate and armed with potent weapons. I felt more able to connect with the emotional toll, burden and message of the film the next day than I could while watching it. The death, suffering, dogged dedication to the mission and craft of the journalist protagonists made more sense as a story and artifact than the surface-level plot of the film.
I thought about the scene with the soldier in pink sunglasses pouring dead bodies from a truck into a mass grave. What was going through his head? How does one get to the point where one can kill so casually? It takes so much time, effort, care and everything else to raise a new person. For that to be thrown away in an instant is horrifying and enraging. And yet, I know it happens every day, somewhere in the world. Who knows? Perhaps things will be different in a world of machine gods. But that feels unlikely. There will simply be more powerful weapons for individual humans to wield. And yet, we have not seen nuclear war. Let's enjoy what we have while we have it! To me, this was the power of the film: to take what happens every day somewhere, and make it feel real for Americans and others who live in the safety and security of the West.
It seems like we are just beginning to understand a part of the purpose of dreams. This article cites recent research that dreams may be a sort of offline training method, in which individual muscle movements are calibrated against the resulting sensory data. The muscle twitch happens first, and the brain activity occurs after that. And of course, many experiences from throughout the day are replayed, which sounds just like an experience replay buffer, an essential part of some reinforcement learning algorithms.
It makes sense for this to continue throughout life because muscles are constantly getting stronger and weaker, tendons stiffer and looser, joints more or less frictiony, etc.
This may explain the twitching or kicking that gestating fetuses exhibit, and account for their ability to have any motor control at all when they emerge from the womb, rather than starting from scratch. This suggests the brain may be more blank-slate than previously considered, instead relying on these early self-training mechanisms to gain capabilities early.
I found this book very compellingly written; indeed the author also contributed to Tom Clancy's Jack Ryan TV series. It opens with a nuclear weapon detonating over Washington, and the rest of the book bouncing between understanding how such a scenario might actually unfold (who are the relevant people and decision makers, what systems and technologies would come into play, what would the political game board look like) and a series of history lessons relevant to the plot point at hand.
My wife has a better sense of North Korean history than I do. She thinks that a single madman would not be mad enough to achieve MADness. I hope she's right.
A couple of anecdotes stuck with me:
Nuclear secrets are some of the few that we've managed to truly protect. The amount of compartmentalization needed to make this happen is extreme, but it at least serves as an existence proof that with enough motivation it is possible.
The president would have ~minutes to make a decision in any nuclear scenario. This really drove home the point that everyone loses if nuclear war were to begin.
Our missile defense systems basically will not protect us against any real threats.
People bemoan the constant analogizing of nuclear weapons to powerful future AI systems, but I think it's by far the best we have. And similarly, although the shape of the threat is better known for nuclear weapons than AI, it's the same kind of game theory that must be based on almost no historical data to drive the debate. Only two bombs have ever been used against people, and from that we know quite a bit about how destructive they are. But we know nothing of how a modern scenario in which there are thousands of weapons on hair-trigger launch status might play out, except for what we can simulate on the tabletop.
Zack Weinersmith 80k hours podcast 3/5
I appreciate the contrarian take that for so many reasons, space is nowhere near the first frontier in which we'd deploy our best technology and resources to make humanity more resistant to eradication. It's so fantastically hard to get our fragile meat bodies to survive in such harsh environments as orbit, the moon, or Mars. He goes into eye-watering (because of how it shatters my space dreams) detail on why every little thing, from pooping, to reproduction, to merely avoiding irradiation will be a massive hurdle.
And yet, I can't help but feel he's missing the point in some important sense. Yes, it will be perhaps the hardest thing humans ever do to crawl out of our humble gravity well. But that doesn't seem like an argument that we won't do it. So another way to interpret the thesis of the book is, "to sell my book I need to have a thesis, and selling the book simply as a list of open science and engineering problems won't sell." I'll buy that one! On a long enough timescale, I'd say centuries at the most, we will go to the stars. Would Zach bet against that?
Gwern's The Quantum Thief trilogy review
Reading Gwern's review, I hypothesize that I'm the ML researcher he's referring to in his foreword, and will fully admit that I completely missed the deeper layers of meaning Gwern explores in his review. Kudos to him for giving it a second go! This makes me much more excited to read the other books in the trilogy through the lens of a conscious entity trying to break free from the bounds of its own identity. I wonder if, even knowing what to look for, I will still miss some layers from my ignorance of the source material from Lupin to deep game theory to Russian Cosmism.
Dwarkesh podcast - John Schulman
I found this level of "AGI is almost here" and "I haven't thought hard about how we make this go well" alarming.
This was a perfect opportunity to wax poetic about Responsible Scaling Policies and to mention the work they are surely already doing to that end, like some sort of responsible actor who has made public safety commitments.
I mentioned my concern to someone at Manifest and they said they'd talked to John after the podcast and he had more nuanced views. John's a smart and careful thinker, and my guess is that in the heat of the moment maybe didn't represent himself as well as he could, or there were things he couldn't say on the record. I hope so! Or perhaps he is like me heads down in the work, and didn't get a chance to read his own company's policies, instead hoping to let the experts specialize in those things and guide his work accordingly. Except I did study ours thanks to the invitation to speak at Manifest.
Whatever the case, I hope he was sufficiently caught off guard that he has subsequently put in the work, or plans to.
