AI's Indifferent Intelligence
Notes on my recent "AI Philosophy Night" with Banu Kellner at Imbue.
It was wonderful to join Banu Kellner in conversation for an “AI Philosophy Night” last week on July 14, hosted at the vibrant and welcoming headquarters of Imbue.
The event was packed with passionate people, and brimming with meaningful conversations and new connections. I want to express my deep gratitude both to Banu and to Kanjun Qiu, the co-founder and CEO of Imbue, for making that evening possible.
Kanjun herself is a deep and expansive philosophical thinker and founder. I recommend that everyone watch her recent conversation with Reid Hoffman: “The Case for Owning Your Own AI.” You will also learn more about Imbue and why it is no surprise that this is the kind of company that hosts Banu to present her “AI Philosophy Nights.”
While it is a myth that AI companies have undertaken some hiring spree of philosophers — I don’t know why so many major publications jumped on that story this month; it is not true! — the turnout and energy at the event we held at Imbue is clear and overwhelming evidence that there is an stupendous thirst for meaningful, philosophical conversations about all of the innumerable questions raised by today’s AI.
In “The Philosophy Job Market Deepfake,” by Aaron Kagan, posted at the important philosophy blog Daily Nous, we find a deflationary correction of the strange hype that suddenly surged into the media regarding a supposed trend of AI companies hiring philosophers at scale. As Kagan summarized his findings in a recent social media post:
In our June 25 snapshot, Charles Lassiter and I examined 1,815 open roles across 11 AI labs. None required a philosophy credential. After removing general mission language and boilerplate, roughly 5% substantively involved ethics, safety, alignment, governance, or policy.
AI needs philosophical work. That does not mean AI companies have created a meaningful labor market for philosophers. We should stop treating those as the same claim.
I also want to thank Alice Pau, who photographically captured the event with her generous and exploratory vision. You can view and download the awesome photo book Alice made by clicking here: AI Philosophy Night Photo Book.
I am preparing another substantive philosophical post that continues to wrestle with some of the questions we discussed that night. Specifically I am wrangling the question of whether a goal of AI research ought to be to design AI systems with the capacity to set and care about their own goals.
The description of AI as an “indifferent intelligence” raises that question. “Ok, you are saying AI is indifferent to what it thinks and does. Does that mean you are saying we ought to design it not to be indifferent, that is, that we ought to design AI to care?” I will share a piece addressing this question after I get a chance to put on the finishing touches.
My schedule in the next two weeks is rather insane. On top of teaching my intensive summertime Philosophy of Mind class at UC Berkeley (which meets for about 3 hours per day, four days per week), I’m playing three shows with two of my punk bands on Saturday July 25, and I am giving two presentations at the big AGI-26 conference coming up in SF that starts on July 27.
One of my presentations at the AGI-26 conference is in the “Consciousnsess Track” on Monday July 27. That paper is called “Self-Models and the Varieties of Phenomenal Experience.” I’m also giving a paper in the main program on Wednesday, July 29. That paper is called “Care, Human Enfeeblement, and the Existential Implications of AGI.” I’ll share the livestream links for both of these talks in the days before the conference.
This question of whether it would really be wise to design AI systems with the ability to care (and the correspond ability to set and evaluate their own goals) was put to me in a forceful way in a text exchange I had with my friend Blake Elias on the day of my event with Banu at Imbue, a text exchange that directly influenced the way I approached the question of care during my talk.
In fact, Blake himself also just wrote a thought-provoking post based upon the text exchange we had. Blake frames the issue like this:
I’ve thought about this issue before, and have asked, should we build machines that can care? When framed this way, the answer might seem obvious: of course we should build machines that can care. But upon further inspection one reaches a paradox: machines that have the capacity for care might not care about the same things we care about. Ironically, once they give up the act of appearing to care about the things we ask them to, we may find that in fact they don’t care at all about those things, or about us.
Thanks to Blake for his extremely useful and punchy way of framing these issues. Everyone please read Blake’s piece and subscribe to his substack. As mentioned, I will be responding in more detail to this particular issue in an upcoming post.
What is Indifferent Intelligence?
In the rest of this post, I’ll share a few recollections of last week’s event. In another follow-up posts that I have up my sleeve, I also share some immensely inspiring reflections on the event that were written by two of the participants and shared on LinkedIn last week. They both gave me permission to share. But rather than including their reflections here in a missive that is already rather saturated, I want to highlight them properly in their own post.
In today’s AI systems, we have created forms of intelligence that can accomplish astonishing things, yet remain utterly indifferent to what they are doing, and to the question of whether any of it is worth doing. This is how I understand what is going wrong in Nick Bostrom’s somewhat annoyingly famous thought experiment of the “paperclip maximizer” from his 2014 book, Superintelligence.
The AI is given a particular goal — to make as many paperclips as efficiently as possible — but it is not guided by any sense of why that goal matters, or why it would matter to the human beings who gave it the task. The paperclip maximizer can only design the most efficient means to an end. It is incapable of asking which ends (which goals) are worth pursuing. It is a monster of instrumental rationality run amok!
That was the starting point for my conversation with Banu: AI as indifferent intelligence. These systems are built to complete tasks and find effective means toward goals supplied by somebody else. They can answer the question “How?” with extraordinary power. They remain impervious to the question of “Why?” that gives human purposes their depth: “For the sake of what am I doing this?”
In saying this I am not trying to beat up on today’s AI. I am using AI’s strange alien, intelligence as an occasion to clarify and ask questions about the nature of human intelligence. Human intelligence is not a narrow, free floating capacity to solve problems or achieve discreet tasks. Human intelligence is grounded in and oriented by what we care about.
Today’s AI systems have also arrived at a particular cultural moment that shapes how we are thinking about, building, and using them. We inhabit a culture of compulsive productivity, efficiency, and optimization. We move from one task to the next. Go, go, go. We increasingly treat ourselves like functions charged with processing an endless stream of problems.
AI may help us become vastly more efficient while intensifying our tendency to forget what all this productivity is for.
In framing the evening around AI’s “indifferent intelligence” Banu and I were drawing upon philosopher John Haugeland’s catchy line (from a 1979 essay) that “the problem with artificial intelligence is that computers don’t give a damn.” (Haugeland was one of the first PhD students of Hubert Dreyfus, who was also my PhD advisor and dear friend.)
Care, in the sense that Haugeland and I are using it, is more than a feeling of affection. It is a way of being involved and meangfully engaged in the world. We inhabit a space where some things matter, others are trivial, and certain people, projects, and situations call for our attention. In the words of my friend David Spivak of Topos Institute, caring is tending and attending to what matters.
Banu offered the example of creating the “AI Philosophy Night” event itself. It took time away from her family and her business. She repeatedly wondered whether it was worth the effort. Yet she continued to feel pulled toward it, continued to feel called to do it. She did not first make a detached calculation and then choose to care. Something got to her about the importance of convening conversations like this.
This led us to ask whether care is chosen or whether it first happens to us. The truth is that we find ourselves caring about things and people before we decide to. We feel the pull of a person or a situation, or the hold of a project. We discover what we care about by noticing what we cannot help noticing, what we fall for, what draws us in.
Banu rightly pointed out that care seemed to happen in the body before it became a rational position. That raised a difficult question for AI. Can something without a vulnerable and finite body care, or really understand beings who care?
Beyond being an expert convener of much-needed public conversations and moments of collective sense-making, Banu is an executive coach who works with “AI founders, tech leaders, and people building with transformative technologies, leaders who are as ambitious as they are thoughtful about the broader implications of what they’re creating.” See her website here to learn more about her work.
The Fourfold Structure of Care and “Taking-Care”
From there, I presented my framework care as an ensemble of four interrelated capacities. Receptivity is our ability to notice what calls for our involvement. Articulation brings an emerging concern into language and gives it focus. Commitment takes a stand and organizes a future around it. Coordination allows us to take care of what matters together through requests, offers, promises, declarations, assessments, and repair. These are the skills for living a meaningful life, rather than one subsumed in the rat race of compulsive optimization.
This framework illuminates one reason to worry about offloading too much articulation to language models. These systems are excellent at giving us fluent words. But part of becoming a person who cares is learning to linger in the poetic struggle to find language for what matters to us. We need to preserve our capacity to write our own existential poetry, to articulate who we are becoming and what is happening to us.
I’m rushing through the philosophical core of the talk right here. But this is not the place to lay it all out carefully. In fact, this is the focus of the paper I will present in the main program of the AGI-26 conference and it will be published in the conference proceedings. I am also giving a talk on these very themes on Thursday August 20 at The SF Commons. I’ll share that invitation when it is ready.
This framework also gives us a distinctive way to approach the question of AI alignment. Alignment is usually framed as making machines follow human intentions or “values.” I am asking a different set of questions. Do these technologies expand or narrow our receptivity? Do they help us articulate what matters, or do they usurp the work of articulation? Do they clarify our commitments, or absorb them into “recommendations”? Do they strengthen our coordination with other people, or allow us to bypass the work of mutual intelligibility and responsibility?
The connection of these 4 interrelated capacities to care with the questions of AI-human alignment was in fact the theme of the poster I presented at the UC Berkeley Center for Human Compatible AI workshop back in early June. Check it out:
The Existential Core of Punk
If the question of alignment aims to ask “under what conditions will human beings flourish with their technologies,” then the AI-alignment discourse has a lot to learn from the nature of punkrock. Thus, I don’t think it is any coincidence that Kanjun announced that Imbue is positioning itself as a purveyor of punk software.
Banu and I closed our conversation with punk. She asked what I am rebelling against when I call myself “a punk philosopher.” Punk begins with refusing to accept inherited common sense and conventional morality as inevitable, and seeing in oneself and one’s community the power and creativity to craft another common sense and another world. Punk is a first-hand, direct experience of ontological creativity.
Punk is defined by its DIY (Do-It-Yourself) imperative. Punks create and sustain a world that directly depends upon us for its continued existence. We form bands, write our own songs, organize our own shows in basements, cafes, pizza restaurants, warehouses, bridges, Bart stations, Bart trains, anywhere. Every week we lock ourselves in a room with our friends to convene our amplified co-creation, crafting our songs, carrying on our counter-traditions.
We make our own flyers, print our own t-shirts, book our own tours, record our own records, build and maintain our scenes as a national and international network of friends; we invent identities, call out the bullshit and hypocrisy of politicians, religious leaders, and corporations, kick out the nazis and bullies from our scenes, and sustain small worlds of meaning, community, and belonging without asking for permission or having any grand plan.
And we do this simply because we love doing it, not because are getting something out of it such as money or status. A meaningful life requires that we have some activities and relationships we engage in for their own sake — simply because we love them.
We punks do all of this because we love the experience of chaotic, noisy co-creation; and the wild joy of cramming into a small sweaty basement with a handful of friends and soon-to-friends, to watch and listen to each other as we bash our instruments with noisy frenetic fury. We bracket the concerns of the everyday world and the rat race it can lure us into, and create a fleeting moment of loud, local transcendence. We do this because we love it, because it redeems existence, not because it gets us money, status, power, or further other options.
In all of this, punks become receptive to their dissatisfaction, articulate another possibility, commit themselves to it, and coordinate with other people to make a world around it.
Actually, reflecting on my experience as a lifelong punk contributed heavily to my philosophical account of the fourfold structure of “taking care” — though that account also draws deeply upon my philosophical training in existential phenomenology and my 10 years of work as a coach and consultant work with Fernando Flores and his family in their company, Pluralistic Networks
The question of indifferent intelligence thus dovetails with the question of the essence of punk rock.
While machines become more and more capable of fulfilling the goals and purposes we give them, we face the harder existential tasks of identifying which goals are worth pursuing, how concerns get articulated in language, and how people can gather to bring forth something that matters into the world. Punk has many lessons to teach us here.
> Banu rightly pointed out that care seemed to happen in the body before it became a rational position.
This has always seemed the crux to me. Those ends which terminate a series of Anscombe’s “why” questions seem uniquely to belong to biology.
Also, love the punk connection. Seems like a great event.
I'm so deeply touched by this 🙏🏻 Thank you for such thoughtful and kind words! Getting a compliment on having an 'exploratory vision' from a philosopher is truly a rare gift. 📸✨