A Picture from the American Society for Existential Phenomenology conference in Bombay Beach.
Last time, I shared about the conference we had for ASEP (American Society for Existential Phenomenology) in the surreal little desert town of Bombay Beach, CA. This is by far the strangest “conference room” for a philosophy conference I’ve ever encountered: a long table placed in the top level of a double-decker storage container, with its sides cut out, revealing sweeping views of the stinky Salton Sean, the sumptuous surrounding hills, and the inexplicable installations, trailers, sculptures, trash heaps, and half-ruins that make up the town’s improvised and imposed public face.
Here the usual hotel-induced, aseptic separation between philosophy and world breaks down. Thinking happens under the pressure of a landscape and a human-scape that will not let you forget where you are: the literal and metaphorical edge of Western technological modernity. In that sense, it turned out to be an uncannily fitting place to gather and converse about finitude, care, and various dimensions of what it means to be human in an unsettled times.
Here is a picture of me and my friend, the Nietzsche scholar, Kaity Creasy taken by our friend, the phenomenology and Wittgenstein scholar, Dave Cerbone:
Now on to today’s theme:
Why Care Matters for AI: New Conceptual Foundations for AI Alignment
I’m enthused to share a talk I gave earlier this week to Professor Max Kleiman-Weiner’s research group at the Computational Minds and Machines Lab at the University of Washington. The talk is now available on YouTube: “Why Care Matters for AI: New Conceptual Foundations for AI Alignment.”
The talk draws on my collaborative writing with David Spivak of Topos institute, and is something of a sequel to another talk I gave last year that also presents some of that collaborative work, namely “Who Cares About Values?” (that phrase is from David).
The field of AI alignment is slowly entering a new era. More and more researchers—people like UC Berkeley’s Alison Gopnik, Brian Christian (author of The Alignment Problem), and Max himself—are bringing care and caretaking into the conversation about AI safety (see my earlier post on this emerging shift here).
This is an important opening. My talk steps into that opening and broadens it, bringing further philosophical clarity to questions that engineers and researchers are already beginning to ask. I come from a lineage of philosophers who have been thinking about AI and its assumptions and its implications, and what I wanted to do in this talk was to contextualize these issues in more depth, raise some questions, and give some concepts and distinctions that will be helpful in the design of research.
You can watch it here:
Brian Christian’s Framework for Human Care
In the talk, I launch from (and expand upon) the position presented in Brian Christian’s recent paper, Computational Frameworks for Human Care. There, Christian argues that incorporating a perspective of caretaking allows AI researchers to move beyond issues surrounding instruction-following and to design more complex models for human-machine interaction.
Central to Christian’s view is a framework that addresses the critical distinction between revealed preferences and higher purposes:
The Limitation of Revealed Preferences: A “revealed preference” assumes that an individual’s actions—such as scrolling on a phone for hours—demonstrate that the activity is a “good” for them. Christian’s work suggests that human reward systems are far more complex.
Wanting vs. Liking: Christian distinguishes between “wanting” (the immediate impulse) and “liking” (what I would call “aspiring”—what actually serves our well-being and flourishing as the person we aspire to be). A person may have a revealed preference for an activity they would actually rather not be doing.
Prioritizing Long-Term Well-Being: True care, in Christian’s view, goes beyond merely catering to a user’s momentary needs. It requires a richer understanding of human agency that prioritizes long-term well-being and higher-order aspirations over immediate, reward-seeking behavior.
By moving away from narrow models of agency, Christian’s perspective provides a foundation for designing AI systems that align with who we aspire to be, rather than simply what we are compelled to do in the moment.
Anyone interested in how AI impacts human autonomy needs to pay attention to this conversation and to the dynamics of human care. Here are a few elements of this that that I explore in the talk.
Autonomy Requires Care
Somebody’s sitting around and scrolling. Does this reveal their preference that scrolling is good for them, the “value” they give to scrolling? But we all know that just because we’re compelled to do something doesn’t mean it’s a motivation we endorse.
I may want to scroll on my phone on some level, but on another level I do not want this desire at all. I’d rather not be motivated in this way. Human agency is marked by this capacity to evaluate our motivations. The quality of our motivations matters to us. We live in a space of mattering, of distinguishing what is worthy and unworthy of our time. Instead of being a person who wants to scroll, I want to be a person who wants to engage in my philosophy of AI research. (Which is not to say that is how we normally act and do things; we often do fall into our immediate impulses).
The philosopher Harry Frankfurt thus distinguishes between first-order desires (I want to check my phone) and second-order desires (this desire sucks, I’d rather be motivated to do my research). A smoker might want another cigarette, while really not wanting to want that cigarette.
This suggests that true human autonomy isn’t just getting what you want. It’s the capacity to manifest in your actions what matters to you, to be guided and driven by what you love and care about. Autonomy comes from the ability to care about how you act, and to act in light of what you care about. Autonomy is the alignment of care and action.
In our current AI model, if you spend hours scrolling, the algorithm “aligns” with you by giving you more of it. But humans aren’t just bundles of impulses. We are creatures of caring and mattering.
If we want AI systems that support human autonomy, this requires understanding autonomy properly: as bound up with care. Care is how we relate to what matters. Through our care, we evaluate our motivations and set our aspirations.
We cannot expect current AI to care in the same way we do—care grows out of our embodiment, our mortality, our fragility, and our fundamental interdependence. What’s necessary is not that we make systems that care like we do. What’s necessary is that we make systems that are attuned to the fact that we care. Systems that can support our ability to care, not just our ability to be efficient or solve problems.
This is the new conceptual foundation that Christian’s work points toward: designing for human caring rather than for preference satisfaction. I say a lot more about all of this in the talk.
An Invitation
I’m sharing this talk because these questions matter for anyone thinking about and working on AI alignment or human-AI interaction. What assumptions about human agency are baked into the field of AI alignment? What would it mean to design for care rather than just for preference optimization? What does a sufficiently rich conception of autonomy and human-AI alignment require? This conversation is just beginning.
Thanks for reading and watching! Please leave a comment or question on the YouTube video if any occur to you.
Please give to our fundraiser for the Hubert Dreyfus Audio Archive Project!
See my summary of that project here. All of this thinking that I’m doing is happening in the space opened by Professor Dreyfus.
PS- I’m posting every two weeks rather than every week for the time being, since I’m traveling a lot, including to a big AI Safety Conference in Paris and an AI and philosophy in Austin (the Cosmos Institute’s “Symposium”). More about all of that in the future.
I appreciate the move from preference satisfaction to care—it’s a serious philosophical shift. Distinguishing what we want from what we endorse as worth wanting is essential if alignment is to mean more than behavioral reinforcement.
But the issue may run deeper than whether AI can distinguish first-order impulses from higher-order aspirations.
In increasingly mediated environments, is care positioned early enough in the causal chain to guide action at all?
The risk isn’t merely misrecognition of care. It’s that decision trajectories are pre-structured before reflective endorsement has the chance to interrupt or redirect them. Care then becomes retrospective—something we use to narrate or evaluate actions already unfolding, rather than something that shapes them at inception.
If that is right, designing for care cannot simply mean modeling higher-order preferences. It requires preserving interruptibility: structural space for experience to intervene before decisions harden into outcomes.
Without that, alignment risks becoming post hoc coherence—the system tracks our articulated aspirations while quietly constraining the field within which those aspirations operate.
So the deeper question for alignment may be: do our architectures still leave room for care to matter in time?
Love the work you are doing. It's from another philosophical angle, but I write about 'care as tension', or care as the structure that makes growth possible. This is also a big motivation for the Love & Philosophy project which I take a big part in. Maybe we could discuss your work together there some day if you think it would be interesting https://lovephilosophy.substack.com/ and https://www.youtube.com/@waymaking23