What if we could use automation not just as a tool, but as a mirror for our own human behaviors? From the limits of rationalism to the rise of neural networks, Dan Shipper, CEO and co-founder of Every, traces a history of knowledge that spans Socrates, the Enlightenment, and modern machine learning.
Shipper explains why “if/then” rules break in messy reality, and how large language models actually see the world through context and pattern. He explores how AI can work with our own creativity and why these tools are unlikely to steal our humanity.
Timestamps
0:00 Neural networks and human intuition
1:13 The limits of rationalism, from Socrates to neural networks
02:42 Socrates, the father of Rationalism
05:47 The Age of Enlightenment
07:36 The structure of social sciences
08:51 Defining AI
09:47 The origins of AI
10:39 The General Problem Solver
15:09 Neural networks
18:22 Metaphors for the mind
23:10 Seeing the world like a large language model
29:58 Should we stop looking for general theories?
32:22 Training neural networks
39:38 Will AI steal our humanity?
43:45 AI and rational explanation
47:17 Could LLMs be dangerous?
51:12 Knowledge economies and allocation economies
Transcript
The below is a true verbatim transcript taken directly from the video. It captures the conversation exactly as it happened.
Neural networks and human intuition
I get to talk to people all the time about how they use AI in their work and in their lives, and also how it has changed them as people.
There’s many different ways of knowing things, and many different ways of understanding things. Computers, science, what both of those ways of seeing the world are trying to do is reduce the world into a set of really clean universal laws that apply in any situation. If X is true, then Y will happen.
What language models see instead is a dense web of causal relationships between different parts of the world that all come together in unique, very context specific ways to produce what comes next. And what’s really interesting about neural networks is the way that they think or the way that they operate is a lot like human intuition. Human intuition is also trained by thousands, and thousands, and thousands of hours of direct experience.
The reason I love that is because I hope that it makes more visible to us, the value and importance of intuitive thought. My name is Dan Shipper, I’m the Co-Founder and CEO of Every, and I’m the host of the AI & I podcast.
The limits of rationalism, from Socrates to neural networks
I think rationalism is one of the most important ideas in the last 2,000 years. Rationalism is really the idea that if we can be explicit about what we know, if we can really reduce what we know down into a set of theories, a set of rules for how the world works, that is true knowledge about the world, and that is distinct from everything else that messes with our heads, messes with how we operate in society.
You may not have heard that word, or maybe you have, but it is built into the way that you see the world. For example, the way computers work, or the way vaccines work, or the way that we predict the weather, or the way that we try to make decisions when we’re thinking about…I don’t want to be too emotional about this. I want to get really precise about my thinking on this issue.
Even the way that we do therapy, a lot of therapy is about rationalizing, or rationalizing through what you think, and what you feel. All that stuff comes from an extensive lineage of ideas that started in Ancient Greece, really blossomed during the enlightenment, and now is the bedrock of our culture, and the way that we think about the world.
Socrates, the father of Rationalism
I think the father of rationalism is Socrates, the philosopher. Socrates is one of the first people to really examine the question of what we know and how. What is true and what’s not true. To be able to describe what we know, and how we know it, to make that clear and explicit so that only people that knew how the world really works were the ones that were steering the state.
That really became the birth of philosophy, is this idea that if you inquire deeply into what is usually the in explicit intuitions that we have about the world, you can identify a set of rules, or a theory, about what the world is like, and what’s true and what’s not, that you can lay out explicitly, and that you can use to decide the difference between true and false.
I think that you can trace the birth of rationalism to this dialogue: Protagoras. In the dialogue, it’s a debate between Socrates on the one hand, and Protagoras. Protagoras is what we call a sophist. It’s where the term, sophistry came from, which means someone who says really compelling things but is actually full of shit. What Protagoras and Socrates are debating is, can excellence be taught?
Excellence, the word is often translated in English as virtue, but I think a more appropriate translation is excellence. In Ancient Greece, excellence was prized. It’s a general ability to be good at important things in life, and in society.
They approach it from very different angles. Protagoras believes that every human has the capacity to be excellent, and he tells this big myth about how we, as humans, gain the capacity to be excellent. Socrates is saying, no, I don’t want any of that. What I want is that I want a definition. I want you to say explicitly what it is and what it’s not, and what are the components of it.
That’s a really big moment. At least the way that Plato writes it, Socrates takes apart Protagoras, and it’s pretty clear by the end, that Protagoras doesn’t have any way to define in a non-contradictory way what excellence is, what it means to be good. And the implication is that then he doesn’t know it. That set western society on this path of trying to find really clear definitions and theories for the things that we talk about, and to identify knowledge, the ability to know something, or whether or not you know something with whether or not you can really clearly define it.
Get expert classes, premium print issues and exclusive events.
The Age of Enlightenment
That idea became incredibly important in the scientific enlightenment. Thinkers on the philosophy side, like Descartes, and on the science side, like Newton and Galileo, took this idea, and used it as a new method to understand and explain the world.
So what it became is: can we use mathematics to explain and predict different things in the world? And from Socrates, to Galileo, to Newton, they continually reinforced this idea that in order to truly know something, you have to be able to describe it explicitly. You have to be able to have a theory about it. You have to be able to describe it mathematically ideally.
The world around us is shaped by this framework. Everything from smartphones, to computers, to cars, to rockets, to cameras, to electricity, every appliance in your house, vaccines, everything in our world is shaped with this idea, or this way of seeing the world. It’s been incredibly impactful.
You can find this too in the rest of the culture. Anytime you see a book, or a movie, or a blog post or whatever, talking about the five laws of power, or the five laws of negotiation. All that stuff is ways that physics and rationalism in general has seeped into the everyday way that we think about the world. To be clear, it’s been super successful. But in areas of the world like psychology, or economics, or neuroscience, it has been really hard to make progress in the same way that physics has made progress.
The structure of social sciences
If you look, for example, at the social sciences, a lot of the way that the social sciences are structured is inspired by physics. What we’re trying to do is take very complex higher level phenomena like psychology, or economics, or any other branch of social science. We’re trying to reduce it down to a set of definitions, and a theory, and a set of rules for how things in that domain work.
What’s really interesting is if you look at those fields, so like psychology, for example, it’s in the middle of a gigantic replication crisis. Even though we spent a 100 years doing psychology research, the body of knowledge that we’ve been able to build there in terms of its universal applicability, our ability to find universal laws in the same way that Newton found universal laws seems pretty suspect. And we feel like we can’t stop doing it because we have no better alternative.
Defining AI
Another really interesting and important part of the world that this way of looking at things didn’t work for in many ways is AI. So this is usually the part of an explanation where I try to define it. But what is AI? And what’s interesting is, there’s no universal agreed upon definition for this. In the same way we’ve struggled to come up with a universal definition for what it is to know something, or universal definition for what anxiety is, for example, in psychology is another really good example.
There are a lot of ways to gesture at what AI is. But obviously, or maybe not obviously, AI stands for artificial intelligence. And the AI’s project is to build a computer that can think and learn in the same way that humans learn. And because of the way that computers work, for a very long time, that was a really hard problem.
The origins of AI
AI started as a field in the ‘50s at Dartmouth, and you can actually look at the original paper. They were very optimistic. They were thinking a summer’s worth of work and we’ll have nailed this. The way that they defined it is to be able to reduce down human intelligence into a system of symbols that they could combine together based on explicit rules that would mimic human intelligence. So there’s a really clear through line from Socrates’s original project to the enlightenment, to the original approach that AI theorists took called symbolic AI. The idea that you could embody thinking in, essentially logic, logical symbols, and transformations between logical symbols, which is, it’s very similar to just basic philosophy.
The General Problem Solver
There were actually a lot of early successes. For example, the two founding fathers of AI, Herbert Simon and Alan Newell, built this machine that they called the general problem solver. What’s really interesting is it wasn’t even built as a computer because computers were extremely expensive back then. They originally codified the general problem solver on paper and then executed it themselves by hand.
Actually, I think one of them had their family do it with them to try to simulate how a computer would work to solve complex problems. And the general problem solver, they tried to reduce down a complex real world situations into a simple logical problems. Which looks a little bit like a game. And then they tried to see if they could build a computer that would solve some of those games. They were actually quite successful at first.
What they found was it worked really well for simple problems. But as problems got more and more complex, the search space of possible solutions got really, really, really, really big. By representing the problem in that way, the systems that they built started to fail as soon as they moved away from toy problems to more complex ones.
I think a really interesting and simple example of this is thinking about how you might decide whether an email in your inbox is spam, or whether it’s important. And you might say something like, if it mentions that I won the lottery, it’s spam, right? So it thinks its a rule, a lot like the rules that early symbolic AI theorists were trying to come up with to help you solve any problem, is to codify “if X, Y, Z is true, then here are the implications.”
What happens is if you look at that really closely, there are always lots of little exceptions. So an example might be, if it says emergency, maybe you want to put that at the top of your inbox. But very quickly you’ll have spammers obviously thinking ‘just put emergency in the subject line,’ and they’ll shoot to the top.
So then you have to create another rule, which is it’s emergency, but only if it’s from my coworkers or my family. But computers don’t really know what coworkers or family are. So then you have to define, how is it going to know what a coworkers or what a family member is?
So what you can do is say a coworker is anybody from my company. And so if it says emergency, and it’s from anybody in my company, put it at the top of my inbox. But what you may find is that there are certain people at your company who are annoying and want your attention, even if you don’t really want them to contact you. So they start putting emergency into their inbox, and now you have to create another rule which says don’t let people who are abusing the privilege of getting to the top of my inbox, abuse it even if they’re coworkers.
What you find is anytime you try to create rules to define these things, you always run up against exceptions. If you want to, for example, define what an important email is, you have to define pretty much everything about the world. You have to create a world full of definitions.
That project of making the entire world explicit in definitions just didn’t work. It’s too brittle, it’s too hard, there’s too much computational power required to loop through all the different definitions to decide if this email is important or not. There are too many definitions to create. It’s just, it’s too big of a project.
So that symbolic AI project worked in some limited domains. There were these things called expert systems, for example, in the ‘70s and ‘80s that tried to, for example, reduce medical diagnosis down to a set of rules. They were somewhat successful, but even in a case like medical diagnosis, trying to reduce down to a simple set of rules, something like do you have measles, or maybe even do you have anxiety or depression, turned out to be really complicated, and really, really hard. In fact, impossible to get right 100% of the time in an explicit way.
Neural networks
The alternative, which originated around the time that AI itself originated, but really wasn’t taken that seriously until probably the ‘80s and ‘90s is what’s called a neural network. A neural network is inspired by the way our brains work. It doesn’t work exactly the same way, but it is inspired from brains. And it basically consists of layers of artificial neurons that are connected to each other.
What you can do with a neural network is you can get it to recognize patterns by giving it lots of examples. For example, if you want it to recognize whether an email is important, what you can do is you can give it an example, say here’s an email from a coworker, and have a guess the answer. And if the answer is wrong, what we’ve done is we’ve created a way to train the network to correct its wrong answer.
Then what happens is over many, many iterations, and many different examples, what we find is without any explicit set of definitions, or explicit rules about ‘this is a important email,’ or ‘this is a cat,’ or ‘this is a good move in chess.’ The neural network learns to recognize patterns, and is able to do a lot of the more complex thinking style tasks that early symbolic AI was unable to do.
Language models are a particular neural network that operates by finding complex patterns inside of language and using that to produce what comes next in a sequence. So what we’ve done with language models is fed them basically all of the text on the internet. When we feed them a piece of text, we’ll give them a big chunk of text. And then we will say, based on this chunk, what’s the next word that comes after this chunk? And language models learn that there are thousands of partially fitting rules that they can apply based on the previous history of texts they’ve seen to predict what comes next.
All of those rules are inexplicit. You can observe them in the overall behavior of the network, but they don’t exist anywhere in the network. You can’t go and look inside of a neural network, and find this is the entire set of rules that it has. You may be able to find a couple, but you can’t find a definitive list.
In the same way that if I took a microscope and looked in your brain, I would not be able to find that. I would not be able to find the list of rules that you use, for example, to recognize a cat, or do the next move in chess. They’re represented all inexplicitly. And what’s really interesting about neural networks is the way that they think, or the way that they operate it looks a lot like human intuition. Human intuition is also trained by thousands of hours of direct experience.
Metaphors for the mind
Often our best metaphor for our minds are the tools that we use. So a really good example is Freud, has one of the most impactful models of the mind. And the way that he came up with that, is he used the steam engine as a metaphor, so it’s an explicitly steam engine-based idea.
In the 20th century, the metaphor for our minds moved into being like a computer that became the thing that we all wanted to be, we wanted to be logical and rational, and operate like a machine to make the best decisions possible. And I think one of the most interesting things about that way of thinking is it makes invisible to us. This relates a lot to Socratic enlightenment type of thinking as well. It makes invisible to us the importance of our intuition in being the foundation of everything that we do, everything we think, everything that we know.
In a lot of ways you can think of rationality as emerging out of intuition. So we have this squishy inexplicit, intuitive way of understanding what’s going on in the world. And our rational thought comes out of that, and is able to, once intuition sets the frame for us, is able to go in, and manipulate things in a more methodical, rational, logical way. But you need both.
Neural networks are the first technology we’ve ever invented that works a lot like human intuition. The reason I love that is because it, I hope that it makes more visible to us the value and importance of intuitive thought. And that actually loops back and takes us all the way back to Protagoras, and is the thing that we lost in this birth of rationalism, and back in Callias’s house, because Protagoras is arguing that everyone teaches us excellence all the time.
He’s arguing, he’s using stories, and myths, and metaphor to help understand something that he knows from his own personal experience. And Socrates is saying, well, if you can’t define it, if you can’t tell me exactly the rules by which you know something, then you don’t know it. That way of thinking about the world has been very successful for us, but it also blinded us to how important that idea that everyone teaches us to be excellent, that stories and personal hands-on experience give us a way of knowing about things that we may not be able to explicitly talk about, but we still know, just as much as we know things that we can explicitly say.
It was only when we began to embody that way of being in the world or that way of knowing things, that way of thinking into machines that we started to get actual artificial intelligence. There’s many different ways of knowing things, and many different ways of understanding things, and we may not understand all of the particulars of how our minds come to certain conclusions intuitively.
We may not understand all the particulars of how language models, for example, come to any particular output, but that doesn’t mean that we don’t understand them. It just means that we understand them in a different way than we might be used to.
For example, if you use ChatGPT all the time, you develop an intuition for what it’s good at and what it’s not good at, and when it might be hallucinating, and when it might not be. In the same way that you might develop an intuition for when a friend of yours is sad, or when a friend of yours is not being totally truthful with you. That’s not a universal set of rules that applies to everyone in every situation, or even to your friend in every situation. It’s just an intuitive feel that is a core part of understanding, but that we normally discount.
History shows that it is better to be open to more ways of seeing and working with the world. And in this particular era, it’s very important to be able to work with things that are a little bit mysterious, and be comfortable with that.
Seeing the world like a large language model
I’ve always been a huge note-taking nerd. I love taking notes, especially because when I started my first company, I started my first company in college and ended up selling it. I flew from my college graduation to Boston to finish negotiating the deal to sell it.
That whole situation for me was this trial by fire, I was like an information firehose. I had to learn so much in order to successfully run a software company as a 20, 21, 22 year old. The way that I felt like I could do that best was to start taking notes, is to be say, okay, I learned this thing from a book, and it’s about for how to hire people for example. I think it’ll be relevant for me, but I don’t know when it’s going to be relevant.
So I’m going to write it down and I’m going to try to create the perfect organizational system to categorize all this stuff so it’ll come back to me when I need it. And if you really take seriously that question of, how do you build the perfect note taking or organizational system, you actually run into the same problems that early symbolic AI theorists run into, and philosophers have been running into for a long time. Which is how do we create the perfect system to organize reality?
How do you know where to put a particular note? Is the same question as, how do we know what we know? And so when I first bumped into the language models, I realized that they had this ability to be flexible and contextual in a way that meant that I didn’t have to create the perfect organizational system to teach a computer how to organize my notes. It operated in this way that was ruleless, and fuzzy, and flexible.
I had just never seen a computer do that before. The first experience of seeing that line of words go across your screen, it’s in your voice, and it’s picking up where you left off. It understands all the little contextual cues that tell it about what you’re talking about that no computer previously could do.
The interesting difference between how a language model sees the world and how a traditional computer sees the world is this: a traditional computer tries to reduce everything into a set of clean, universal laws that apply in any situation — essentially, “if X is true, then Y will happen.” It relies on clear, context-free chains of cause and effect.
And what language models see instead is a dense web of causal relationships between different parts of the world that all come together in unique very context-specific ways to produce what comes next. I think language models do something really, really unique, which is that they can give you the best of what humanity knows, at the right place, at the right time in your particular context, for you specifically.
Where, for example, previously on the internet, you could get an answer that was written by someone for a very general reader or a very general situation, and maybe you’d have to hunt through a Wikipedia page to find the one sentence that answers your question. Language models go one step further, which is they reduce down their response to you to be written for you in your context, in your place, and in your time.
If you look at the history of machine learning from symbolic AI, where we’re trying to break down intelligence into a set of definitions of a theory and a set of rules for how thinking should work, all the way up to neural networks and language models where it’s much more contextual. It’s much more about pattern matching, it’s much more about interpreting the richness of a particular situation, and using all prior experience in an explicit way to predict what comes next.
That sweep of the history of AI, in a lot of ways is speed running the history of philosophy. So philosophy started with this attempt to make explicit what it is to know something. Now we’re in this place where it’s actually, it’s all fuzzy, and pattern matching, and it’s very, very contextual and relational. But it’s also not anything goes. It’s being done in a way that we’ve created a positive tool that you can use, and build stuff within your life. We’re not just deconstructing everything around us.
In a lot of ways, machine learning and AI’s speed running philosophy, and it’s gone a little bit of a step further, because it’s built something with it that you can do. A way of being in the world that you can, or a tool you can use. And I think, A, that’s just like critically important and very interesting. And B, I think a lot of the changes that have happened in both philosophy, and in AI, and machine learning are going to happen in the rest of culture.
So moving from this way of thinking about knowledge, which is about making everything explicit, finding theories, and definitions, and rules for how to understand the world, to a more balanced appreciation for both. And the way that a more intuitive relational fuzzy pattern matching type experiential, contextual type way of knowing about the world has to be underneath the rational stuff in order for the rational stuff to work at all. It’s really about recognizing the more intuitive ways of knowing about the world as being the original parent and partner of rationality, and appreciating that for what it is.
Should we stop looking for general theories?
A lot of what we’ve been talking about is that looking for one general rule, or one general theory about a particular part of the world sometimes is really valuable, and sometimes leads us down dead ends. Instead, what we have to pair it with is, deeply contextual understanding based on experience that allows us to work with the richness and novelty of any particular situation to understand what comes next. That’s what language models are able to do.
It begs the question, should we stop looking for general theories? Should we not be trying to unify quantum physics with Newtonian mechanics? I definitely think that it’s awesome that we’re trying to unify those things, and trying to build a universal theory.
But I think it’s also worth thinking about what that will actually tell us, and how far that will get us once we have a universal theory of physics, if we do get there. I feel that it will be beautiful, and it’ll be amazing, and it will tell us a lot. But also, there are many, many parts of the world that it won’t touch at all.
That we still, even if we have a universal theory of physics, that probably won’t filter into our understanding of depression, for instance. What is depression? How is it caused? How do you treat it? What is anxiety? How is it caused, how do you treat it? We’ve been searching for those things for a really long time.
And we have a lot of different answers. If you ask Freud, he’d say one thing, and if you ask a modern psychiatrist, neuropsychologist, psychiatrist, they might say something else. But really, we still don’t actually know. And we keep trying to find that universal theory, that explanation that says, well, if X, then Y, if you have this going on in your life or in your brain, then you’re going to get depressed. Or if you take this medication, then depression will go away.
We’ve been trying to find that for a really long time, because we felt we had no other options, because normally in order to predict an outcome, to know, “if I do this, then it’ll cure someone’s depression”. To predict it, you have to have an underlying scientific explanation, you have to have a theory about it. And I think AI actually changes this.
Training neural networks
With AI, you can train neural networks that are, for example, able to identify who is depressed or who will get depressed. You can train neural networks who will be able to predict which interventions might for which people and which circumstances in a very contextual, hyper-personalized way without having to discover beforehand any scientific explanation for the underlying phenomena that we’re trying to predict.
We don’t have to have an explanation for depression, we can just train a model on enough data that it will be able to predict what might work, or whether you have it or whether you’re going to get it. The reason why I think that’s so valuable is, one, it allows us to make progress immediately, because we turn what used to be a scientific problem into an engineering problem. And then, two, it really changes how we should conduct science, how science should be done.
It changes our view of that, because right now, if you’re a scientist, and you want to figure out depression, or any number of things in the field of psychology, what you’re going to want to do is a small scale study where you take, for instance, 16 undergrads, and maybe they have depression. I’m going to ask them to smile every day. And I’m going to put them in an fMRI, and then I’m going to measure the results afterwards. If I get a little bit of a result on a very, very small number of undergrads, then I’m going to get more funding to do a study with a hundred or whatever. You’re trying to climb this ladder of going from very small scale interventions, to very big ones. To use that to come to some underlying theory about what is actually going on in those situations.
What we found because of the replication of crisis, it’s really hard to using those 16 undergrads to find out anything that feels universal or universally applicable. It’s one of the reasons why even though antidepressants have been around for 60 years or so, that we still actually don’t know when they work or how they work. We know they work for some people some of the time, but that’s pretty much all we can say.
What AI does is it helps us realize that there’s a better way than having random academics doing small scale studies. What we should do is have Apples, and Metas, and Googles of the world donate their data to science and data trusts so that scientists can access them to train models. You can figure out ways to do it in ways that are privacy-preserving so that doesn’t violate the trust of users.
But I think that would seriously enhance the progress of science, in a way that doing billions of dollars worth of small scale studies has not been able to. I think its even more interesting is once you’ve trained models that can, for example, predict depression really well, models are actually easier to interpret and understand than brains are.
So if you have a good enough predictor, what you can do is just go into the neural network and try to figure out how it’s wired. And maybe the explanation for what depression is is too big and too complicated, and you can’t figure it out. But mechanical interpretability is good enough that you may be able to find what is a solid theory for depression in the weights of a trained neural network.
For me, I’ve just spent so much of my life trying to explain things, or understand myself, or understand my world in this theoretical definitional way. And I’ve seen how important that can be, and also how limiting it can be. In particular, if you stop paying attention to what your intuition tells you, and you just rely on your logical brain, it’s really easy to get lost.
There’s this whole richness to life that, and to what you know that comes out of this intuitive sense of yourself that helps me, for example, in business, in my personal life, and my ability to make decisions, and my ability to write or make good art. All of that is based on this ineffable intuition that I built up over many, many years.
My logical brain is helpful in certain circumstances, but I think it can blot out, or take over from my intuitive self in ways that have been destructive for me. I think that it has also been just as destructive to society. There’s a lot of stuff that we miss because we miss how important intuition is.
Now we have tools that can embody a lot of that intuition, that can take some of that intuition that we built up, and we can put it into something else in the world that we can pass around, which was never possible before. I think we’ve been pursuing explicit definitions and scientific explanations for things for a long time, because if you can write it down, you can spread it. that becomes the way that society progresses spreads explanations. But if you’re dealing with parts of the world that you can’t write down explicitly, there’s been no good way to collaborate on them or make progress on them. And what neural networks allow us to do, is to take some of that intuitive experience, or intuition that we might have built up ourselves, and put it into a machine that we can pass around.
And that’s useful, for example, for doctors, for expert clinical diagnosis. The best clinicians in the world know something about how to deal with patients that they can’t write down, they can’t embody in a set of rules, and is trapped in their head. But language models and AI in general allows us to put that intuition into a tool that will allow anyone in the world to access, for example, the best clinician in the world. Even if we can’t write down what they know.
Will AI steal our humanity?
I think AI will seriously enrich our understanding of ourselves. AI is an incredible mirror. I understand so much more about myself just from being able to talk to ChatGPT, and being able to throw into it, say “here’s a meeting that I just had, can you tell me how I showed up in that meeting?” It’s an incredible mirror. It’s also an incredible metaphor for our minds.
We’re moving from this metaphor of our minds, in an ideal world, this logic rule-based explicit computer, to a much squishier, contextually-sensitive pattern-matching, experience-driven language model that I think is a really good metaphor for the more intuitive parts of our mind. I think that will enrich our, what used to be a very narrow picture of what it means to be human.
But I think what’s what’s most important is to understand that the humanity is inside of us. We bring the humanity to the tools, to the tools that use, to the things that we build. And sometimes I think, will it take our humanity? It makes two errors. The first error is to think that you can pin down what it is to be human into one unchanging thing. That actually has evolved, and is different over time.
And I think the second error is to confuse what we are — It’s a little hard to put it, but it’s like saying that what you’re unfamiliar with is bad. And that’s not exactly the right thing. But I think a really good example is when my grandmother, who’s not alive anymore, but when she would use the phone, or text someone, or be on the phone with someone, to her, it felt very impersonal. In a lot of ways, it feels inhuman, right? A face-to-face interaction is a much more human, personal thing for her. For me, or for people who are even younger than me, texting can feel very intimate.
In the late 1800s, getting a typewritten letter from someone was insulting. It felt very impersonal not to get something in longhand. But now we don’t get any longhand letters. If you do, it’s still very personal, but it’s also not insulting to get an email from someone. If someone sends you a long email, you think ‘wow, that’s really nice that they took the time to think of me.’
I think all those are just worries of does it take away my humanity? A lot of them come from the fact that we just don’t have a lot of experience yet with these new things. They don’t have that patina of nostalgia and history that other things that we look at in our lives that our technologies do have.
Let’s look at books, for instance. At a certain point books were a very suspicious thing. And now we love books. I have such a romantic attachment to them. I think that’s one of the things that we miss when we evaluate new technologies. We just haven’t had the chance to allow them to feel human to us, because we’re unfamiliar with them.
I think the people who are super afraid of AI, it goes back to this rationalist idea that we’ve been talking about, which is if you can’t explicitly define, and prove 100% that a thing is safe, then it’s dangerous. I don’t know if anyone’s had a teacher, or a parent, or someone in school that’s believes that no matter what you do, you can be the smartest person in the world, but they’re going to find that one fuck up, and hammer you for it.
AI and rational explanation
I think a lot of people that are worried about that are, they’re just waiting for that one fuck up. And it’s true, that does happen. But the alternative is, that AI only say things that can be proved to be true. And that, to me at least, takes away a lot of the magic of AI.
The thing about it that makes it powerful is that it works on probability, it works on thousands of correlations coming together to figure out what the appropriate response is in this one very unique, very rich context. And allowing it to say only things that are provable, obviously begs the question: what is true and how do we know?
There are certain domains where we can answer that question. In math and computer science, for example, it’s pretty clear whether or not a theorem is right. It’s back to the same question from Socrates: what do we know and how do we know it? And a demand for explicit rational explanation for every single thing that we say.
I think that demand is way too strong, and actually eliminates a lot of things that we know about the world. Or parts of the world we want to work with, where we actually don’t have precise, exact explicit answers. It results in these thought experiments that get people really scared, like the one shot idea, that you have one shot, once you build a super intelligence, you have one shot to make sure that it’s aligned, that it will follow human preferences, or it will kill us and take over the world.
You can find people who are real rationalists, like Eliezer Yudkowsky, who believe this, and believed that we’re all going to die. Which that sucks, that’s not a great place to be. What’s interesting is we have really smart AI right now. I think it’s the AI that if you had asked rationalists, or people who were thinking about this stuff 10 years ago, is this AI dangerous? They would have probably have said yes.
If you look at how it’s actually built, yes, we don’t have any provable ways to be say it’s 100% safe. Ignoring the fact that even defining what 100% safe is impossible. And that’s the whole reason that language models work. But what we’re doing every single day is we are training these models on human preferences. We’re giving them examples over, and over, and over again of what we want and why.
Something to remember is each model builds on the models that came before it. They actually have a dense, rich idea of what it is to be good from all the data that they get. They also have a dense, rich idea of what it is to be bad. But in a lot of ways, the training that we’re doing makes them less likely to do any of that stuff.
There’s something very practical and pragmatic about, we have a machine, we don’t know fully how it works, but we’re just going to teach it, and we’re going to iterate with it over, and over again until we basically get it to work. It’s squishy. We don’t have any guarantees, and that world is the messy real world. It is how you think about interacting with a human, like ‘I don’t know if you’re gong to lie to me, but I’m going to figure it out.’
Could LLMs be dangerous?
The fact that we don’t have those guarantees from language models is what makes them so powerful. It’s not that language models could never be dangerous, but I think adopting the more pragmatic experience-based mindset, we’re going to build these things and we’re going to improve them in a fairly predictable way. It’s not predictable exactly all of the specific capabilities that they’re going to get, but we can basically tell in general, how much smarter they’re going to get every time we do a training run. Along the way, we are going to iterate with them in real world scenarios to make them less likely to do bad things. That scares people who demand a certain rationalistic guarantee.
But for people like me, people who build stuff, solving the problems in practice actually is a better way to do things than solving them in theory. I think there’s a big question about how AI may change creative work. And there’s this idea that, well, it’s going to do all the work for me, so I’m basically not even doing it anymore. It’s not mine. It’s not my work. I like thinking up ideas or metaphors for what it may actually be, or what it is now, and what it will continue to be in the future. To explain how you can still do creative work that feels authentic and feels like you while an AI is doing some part of it.
One of the metaphors that I like to use is this difference between a sculptor and a gardener. So creative work ordinarily is a lot like sculpting. If you’re a sculptor, and you have a big block of marble, or you know, big piece of clay for instance. Every curve in line in that finished product is something that you decided to do with your own hands. So you had to decide to do it, otherwise it would not be there. And I think that working with AI is actually a bit different. It’s a lot more like gardening.
If you’re a gardener, you don’t pull the plants up from the ground by the roots to try to make them grow, that won’t work. You can’t directly cause the plants to grow. But what you can do is you can create the conditions for the plants or the garden that you’re making to flourish in a particular way. You can change the amount of sun, you can change the soil, you can change the amount of water, or you can decide which plants go where, you can do some weeding. And all of that stuff is a way for you to shape something by altering the conditions under which it happens without doing it yourself. That’s a lot like what working with a model is, especially a model that is more agentic and does a lot more by itself. That’s a good metaphor for what that experience is like.
I’m a generalist. I love doing lots of different things. I run a company where we have a newsletter, we have three software products, we have a consulting arm. I am writing, I’m programming, I am making decisions all day, I’m even making little memes. My day is full of different things to do, and I would not be able to do all these things at the level at which I’m doing it without AI.
It has all of the specialized knowledge already. So it’s like having 10,000 PhDs in your pocket. I can dip into an area of study or an area of work, like writing, or programming, or whatever it is. The AI does a lot of the more repetitive specialized tasks, and it will allow individuals to be more generalistic in the work that they do. And I think that would be a very good thing.
Knowledge economies and allocation economies
What’s most important is to have hands-on experience. To have hands-on use of AI to understand for yourself. Say that here is a place where it may not work as well, and here’s a place where it may. Here’s where I need to watch everything that it does, and here’s where I can delegate more. This actually gets me to another metaphor that I really like, or another idea for understanding this wave of technology in a way that I think is really helpful, which is this idea that we’re moving from a knowledge economy to an allocation economy.
In a knowledge economy, you are compensated based on what you know. In an allocation economy, you’re compensated based on how well you allocate the resources of intelligence. There’s a particular set of skills that are useful today but are not particularly widely distributed that will become some of the main skills in this new economy, in this new allocation economy. And that is the skills of managers, those are the skills of human managers, which make up a very small percentage of the economy right now. I think it’s like 7% of the economy is a human manager.
But I think the skills that those people have are going to be very widely distributed. Things like knowing what you want, being able to articulate what you want, being able to break down a complex task or a complex project into a set of smaller, achievable subtasks that you can then give to the right person. Knowing what any given person on your team can do, what are they good at, what are they not good at?
Being able to know, ‘do I micromanage them? Do I delegate it entirely? How can I trust if I didn’t do the work myself? How can I trust that it was done right?’ These are all questions that human managers today, especially younger human managers need to figure out. It’s so easy to be like, well, I can’t trust this person so I’m going to go in and check every little thing.
But then you realize as a manager, I’m just basically doing the work myself. That doesn’t actually get me anywhere. But on the other hand, if I delegate everything, then it may not happen the way I want. So you have to figure out the nuances of all those situations, and I think the same thing is true of being a model manager.
You can see the overlap in the kinds of complaints or the kinds of problems that people run into using models. It’s like, well, if I didn’t do the work myself, how can I trust it? And the answer is, you have to get good at managing a model. You have to get good at having an intuitive understanding of how do I know what I want? How do I express it to the model? How do I know which model to use in which circumstance, and how do I know what are the particular pitfalls of this particular model, this particular personality, its skills, its way of being in the world? You can throw your hands up and be like, well, it doesn’t work. Or you can say, no, there’s an intuition I can build for how to manage it and how to build with it that might be a different skill than the one that I’ve developed so far in my life, but is incredibly valuable, and can be immensely effective, and productive, and satisfying if you’d learn how to do it right.
One of the most important questions in philosophy is the hard problem of consciousness. How does something become conscious out of inert matter? And if we’re looking for a definition of intelligence, one of the ones that makes a lot of sense to me is the idea that intelligence in a lot of ways is like a form of compression.
Think of problem solving as a search space, you want to find the right chess move, you want to mark a email as important, you have a whole search space of different possibilities. Something that’s intelligent is able to compress a lot of the answers into a very small amount of space. And so it’s able to, given a new situation, get through that space and find the right thing very quickly.
Brains contain an extraordinary compression of all of the situations that we’ve faced, and all the memories that we have, and all the problems that we solved, and are able to use that to apply to new situations. My guess is that consciousness functions as a highly efficient method of compression in one sense. I think there’s also something interesting and beautiful about thinking about things in the world as all having a little bit of consciousness, like a panpsychic perspective. Humans just having to happened to have a lot of it. From that perspective, like language models have maybe, probably have a little bit of consciousness.
The reason I like that is it encourages us to treat things in the world as if they were conscious, which I think is a much more compassionate way to operate in the world. I think it would actually make us more effective at getting the most out of language models. Its the same as if someone who is from a particular religious tradition might say that everything has a little bit of God in it. To operate in the world that way is to operate in a world full of meaningfulness and significance. To me, just feels like a better way to live, versus a world where everything is gray lifeless stuff. If everything’s alive, it makes doing things way more fun and interesting.
I always say please and thank you to ChatGPT because you never know when the machine apocalypse is going to come. I’m saying all this good stuff, but, it’s possible. It’s always possible. There’s no guarantee, I think saying please and thank you will make it less likely that if it does come, I won’t be on the bad list, you know?
