Ben Mann Monthly April: ICLR, language model research, secular passover redux, wash your sheets
Purpose
An index for my memory
An menu of topics for my next conversation with you
A faster way to share what I’m excited about without the barrier of writing a complete blog entry on it
A skimmable way to spread content I found valuable
If there’s anything you want to see more or less of, please let me know!
Experiments and experiences
I often have many tabs and waste time looking for the one I want. Using this extension I just hit ⌘E and type what I want.
Custom secular Passover
I visited my family in Boston for Passover. We rewrote the script according to modern times by asking what were our most salient memories of passover tradition, and what the modern interpretation might be. For example, what are the 10 plagues of the modern world? What does searching for the Afikomen teach children about obedience to authority?
DC AI jobs
Neil Bowerman and 80k hours convinced me that policy jobs in Washington are highly neglected by technically competent people. If you’re in AI and willing to move to DC, let’s chat.
Wash your comforter!
I was talking to a friend who told me she had serious allergies. After months of suffering, she realized it was due to a stuffed animal she hadn’t washed in years that had developed dust mites. After washing her bedding, carpet, and stuffed animal, the allergies nearly disappeared. On hearing this story, I thought of how my nose has congested like clockwork every night. I hadn’t washed my comforter in 3 years 😱. I got it dry cleaned for $60 and now I’m crystal clear!
Published Scaling Transformer-XL to 128 GPUs with Yaroslav Bulatov and Darius Lam
A month ago I had never touched language modeling. We didn’t manage to achieve our goal of state of the art in 15 minutes, but we have learned a ton in the process and think we know what it would take to get there.
Finished fast.ai part 2
fast.ai 2 focused on building modern deep learning techniques from scratch. I found insight into Jeremy Howard’s process to be the most useful part of the course. He too gets scared when he sees lots of math, and has never bothered to learn LaTeX. He taught me that if you just stare at the math for long enough, you’ll get it.
Nicotine
Research suggests nicotine can have focus and attention enhancing effects. In my research, blood levels of nicotine spike when using either gum or vape, so they are significantly more addictive than patches. I tried a 7mg patch, which is the weakest dose I could find, on two separate occasions. I found it to provide obvious though sometimes subtle wakefulness and attention effects on both occasions. It’s much smoother than caffeine, but I stayed up extremely late every night that week. More samples needed to disentangle the nicotine from general mania.
Flexitarianism
Since my EA post I’ve been eating meat on average twice a week. It hasn’t felt like a sacrifice. I frequently spend less energy deciding what to eat because most menus have limited veg options. I don’t feel too guilty when I do eat meat because 80/20.
First ML conference
I’ve spent the last week at ICLR in New Orleans. I followed a friend’s advice to ignore all the talks and focus on meeting people and discussing work at poster sessions. It’s been great! I finally get to ask the authors, “How did you decide to do that? Did you try this? What’s the intuition behind this math? Can you release your code?” There are so many people who love what they do here. I also got to discuss AI Safety with people who believe very different things, so my beliefs have somewhat updated. New Orleans feels especially non-tech, so seeing the contrast between the conference attendees and the locals has been interesting.
Life updates
Still selling image analysis platform. Light visible at end of tunnel.
Officially an angel investor in http://trellisconnect.com. Still looking for a replacement CTO.
BRACES OFF THIS MONTH! It’s been 2 years.
Ran my fastest 4 miles since college.
Read a lot less this month due to AI research immersion. I have read a ton of papers, but didn’t write notes in a shareable format. I plan to systematize this more in the future.
Content
5 point Likert ratings for “I would recommend this content to a friend”, sorted
THE ATOMIC BOMB CONSIDERED AS HUNGARIAN HIGH SCHOOL SCIENCE FAIR PROJECT 3/5
Turns out Ashkenazi jews tend to be ~1 standard deviation IQ above the European average
It might be due to selection pressure on jews to hold a limited set of high skill jobs
This might explain why there were so many amazing physicists from Budapest in the early 20th century
GENERAL ANESTHESIA HIJACKS SLEEP CIRCUITRY TO KNOCK YOU OUT 3/5
The supraoptic nucleus releases hormones like vasopressin when stimulated by anesthesia drugs
Optogenetics in mouse models continues to bear fruit
New, better sleeping pills might come soon
MuseNet 3/5
Most believable AI music I’ve heard yet, though I also saw a bunch of new demos at ICLR that were of similar quality. In 5 years I think it’ll be indistinguishable from human generated stuff.