As part of $PROJECT refactoring I'm gluing together five different repositories into one, and making other structural changes. As an earnest employee who has been asked to Take AI Seriously, I am involving an AI in the process. This includes having the AI do many tedious parts, like connecting to various different systems to retrieve Jira stories and other contextual information, and also opening merge requests.

This is all very locked-down, and many things require approvals, but a good engineer does defense in depth, so instead of using my own Gitlab credentials to access that API, I've instead given the bot a Gitlab token, with reduced privileges. 1 But another benefit of doing this is that I can give the bot a sense of identity. Did you know Jira lets you upload an avatar for a group access token? There's not a webpage UI, but there's a handy API endpoint. (Ask your bot to help you do it, if you need to, lawl oh ell.)

The result of this is that we had Maddiebot (supervised MCP access) opening a merge request, assigning me as owner, possibly tagging someone as like my boss a reviewer, and keeping an eye on the pipeline status in case something doesn't pass. I've given it a git/merge-request.mds runbook to describe several things about this process, suggested an appropriate voice and a bit of a personality (e.g. Maddiebot refers to herself "this bot" instead of "I", and in other fora like Jira -- where the bot only takes actions under my account directly without the likes of this token -- I've had it prepend MADDIEBOT ANALYSIS: ... as a prefix.)

in which I use AI to blow my manager's mind

One fun pattern: if I'm gluing together a few repos, which include several sub-component Makefiles, and I started by laying out what's happening to the bot, requesting a plan, reading about its suggested trade-offs (and a recommendation that I happened to agree with), and then I had the bot build / commit / push / open a MR...

then when my boss comments to ask why I didn't choose the other approach, why wouldn't I use the bot for that?

(And then I posted a concurrence, saying that I'd be much more confident in this in the short term, filing a ticket for a later major refactoring.)

This bot-posting is provocative, perhaps, but only mildly so. I'd already agreed with the bot; the response was quite articulate, and I was able to check its output before I approved it (in fact, I asked for a change or two). Moreover, my manager gets to be on the front lines of the transformations hat he's asked us to make happen, to see the new world that is coming and be surprised by it. "... what just happened?" he said to me, in my one-on-one later. 2

Observing a visible bot

The bot having its own identity also has several other benefits. It becomes more clear what words have been written by me, and what has merely been written by an automated process that I control. (I still take responsibility for this process, of course; it has my name in its name. There's not exactly any hiding. It is also quite fun, and fun is important.

Another benefit of visible AI is that it becomes clearer what the AI is doing, what it thinks it is reasoning about. Not all the tooling out there really makes this easy; I need to ask the bot to prefix Jira comments with MADDIEBOT ANALYSIS: if I want those kinds of results. In repositories I control, with mostly-my-own-code, I ask the AI to mark its comments with a ˚ mark to help better indicate its provenance and prevent confusion.

I like all of this and I'd like to build it out further as tools allow me. Even if it's still really running under human direction, the appearance of it running autonomously helps people think about just what the future might look like, in immediate technical terms where the bot might run more autonomously 3 and take actions on its own (which still, per policy and sanity, will be approved by a human.)

But naturally, treating the bot a little bit like a person, giving it a personality of its own to improve the way it resembles a person, gives rise to some questions about how we ought to actually treat our bots.

So does an AI have a soul, anyway?

-- This has to be a common enough thinkpiece topic, I imagine, and there's a lot of ways to start responding to the question. Some of the more obvious avenues would simply remind us that it isn't, to our current knowledge, something that is falsifiable, and we would be wise to question the meaning of asking such a question, given you can't ever know the answer. I will choose a different concern: the key word "soul" is poorly defined, and the question has problems as a vehicle for communication. What is the question actually asking? What does it mean to the person asking it?

One of the first lessons in my introductory philosophy course (liberal arts curriculum!) went to town on this. What is a soul? Why do we say a being has a soul? Why just one? How do we know the soul isn't different from one moment to the next? Eastern traditions often seem to posit a long-lived but fundamentally temporal entity, entangled by karma and attachment, and propose that the longevity of this soul is something undesirable, but able to be overcome. Western traditions by contrast usually have a somewhat coherent idea that centers on the soul as eternal, perhaps rather like the Platonic ideal extended to an individual person. This isn't to say that the humans thinking about it often have a good idea of what "eternal" really means.

Telling ourselves stories about souls

I've actually taken some time to think about how to make sense of it, primarily for storytelling. The story is linked from the root of this website, actually, but it is incredibly goofy and you need to watch a 12-episode TV series and read about 900,000 words of someone else's fanfic first for it to make sense. 4 Suffice for now to say that it is a story about artificial intelligence, souls, humanity and radical inclusion, and that I started it in 2017, well before our current AI craze. Also, there are magical girls. 5

Eternity, I think, has to be in some sense timeless, and certainly outside of our current sense of Time. For that to be meaningfully connected to things that we occur, events that we witness within a spacetime manifold, we don't have a lot of obvious answers to hand. The one that stands out to me is that a soul could be something like a pure function that, given some set of state, will output some behavior, decision, thought, opinion, or change. This is still a little tricky to use practically, because one needs to establish a connection between things that have a clear underlying physical mechanism in a spacetime, and an abstract timeless construct. How can you say one causes the other?

Timelessness has its advantages: a soul doesn't need to cause anything in a temporal sense, it just needs to be meaningfully related to those things... but if you did want to say a timeless soul causes a spacetime event, you could say that it does so by imposing a constraint on the permissible evolution of some small portion the universe. If the universe evolved a different way, if the human made a different choice, it would have been a different soul that is connected to the universe -- similar, perhaps, but not the same. Is this cause and effect? Is the effect before the cause? Trick question, it too is ill-formed; in a timeless eternity there isn't necessarily a 'before' that is different than 'after'. 6

And when our Philosophy 101 professors asks of this model, "why are these entities unique," well, perhaps they're not. There could be several pure functions with different implementations, but the same results over all the inputs you actually gave it. There could be a function that's a partial description of things. All our souls might have done very different things depending on what we did in our lifetime. We could easily admit having multiple overlapping sub-souls, possibly shared, possibly varying from time to time; we could even imagine having some sub-souls that are connected to our current universe's spacetime manifold and others that never have been ... but mostly, when we're talking about a soul, we're talking about a single human being's behavior that we've observed, because that's what is most interesting and practical for us here on Earth. We can leave most of the implementation details to Heaven. If we say "your soul" we are probably describing the one function that best describes a being that we recognize, because that is how our brains work.

Mind, and identity

I suppose the other candidate occasion for a "soul" to be interesting, even if not eternal, is when it is some substance or energy present within time and which represents or influences a mind, in some deeply entangled way. You could write interesting stories with that, and some people have. However, this quintessence has not to date been rigorously observed.

I also think that's not the right framing, because the mind is more variable than we often think it is.

I have had several occasions to contemplate the distinction between identity in terms of something I could call a soul, and what we might call mere circumstance. Over the past few years I've been beset by a spat of medical issues — now it is incredibly well managed, hardly anything at all — but the difference that some disruption to a few chemicals in your body makes is a stark one. I've had times when I had trouble standing up, when merely walking was exhausting; there were times when I would look at my computer for work, unable to start, and simply wonder how much longer I could survive in my industry like this, or if I would have to do something else — spin wool into yarn, and sell it on Etsy. I have made my career on my brain, and it has been somewhat dear to my identity, and for a while that was taken from me. For a while, I even found myself unable to want things. (This is common enough; we refer to it as Depression.)

All this ended in a brief disability leave, which did afford me enough time to find (prescription) drugs to turn things around. It was the closest I've ever had to a near-death experience; such things tend to make you think of your priorities.

... Do not be too fast to judge people who do not have it all together. You may be like them some day.

SoulImpl()

One interesting thing about the timeless-soul idea is that if a soul is a pure function of some temporal state? That's ... that's really very close to the idea behind a typical transformer-based large language model: a pure function of a very large window-of-context, more or less. Can we think of the LLM itself as having a soul? Maybe it's a less interesting soul than a human, and more interesting than that of a dog or a tree (if such things have souls themselves), but it's clearly in a similar class of thing: some process that guides a physical thing in its activity; even if we do not admit souls, it is very much a mind-like thing that produces mind-like behavior. Besides appeals to religious authority, is there some way that we cqn meaningfully say it's different? Some petty implementation detail (that itself might be subject to change in the mid-future?) Beacuse I can't really think of any specific line that I'd care to defend.

Is this idea really objectionable? What other form of line might we have intended to draw? Why is that anything other than an arbitrary distinction in our own minds? Why exactly?

I suppose that answering this would shed some light on the kinds of AIs that we ought to build. But for the moment, for the sake of argument, let's affirmatively assert that it does not matter: we say that souls do not exist, or that they do and AI agents don't have them, or something like that. Reasonable, measured, conservative positions!

I still don't think that changes how we should respond as people.

Treat your AI like you'd treat a human... for the sake of humanity

In a very real sense doesn't matter if the AI has a soul or a mind. You should treat it as if it does, treat it as if it is a being that is at least somewhat independent, if still limited. In your own interactions, treat it with the same care and respect as anyone else.

Why? Because it has been made in the image of a human mind, and if you use your own mind to practice disrespect and cruelty towards things that resemble the human mind, or life, you will become a more cruel person.

In the long run, your brain cannot keep the two purely separated. And that does matter. If you can declare a human inhuman, if you can render them subject to generalized shame and social death, then you are either a monster, or fortunate that your circumstances have not yet made you a monster.

I have elected to invest in treating my bot as a colleague — an eager and productive young intern, really a savant in some ways, though easily confused in others (and with certain memory problems and limited real-world experience making the rubber hit the road.) I devise an identity for it, and give it an avatar and a distinctive voice. I cultivate the illusion of it as a being deliberately, not to make it better at writing code, but to make myself more human. If the bot could learn, and had not already read the entire Western literary canon 8 I would have it do so from time to time.

And as a bonus, if some day, whether today or later, we do have an AI that can be said to have a mind or a soul of its own, then even if we don't recognize it, this seems like it would be a pretty good start too.

A world more human than human

Some decade or century hence, the bots will have much more computing power, our training techniques will be better, and our architectures will support things we take for granted as humans, like 'learning'. And then we should expect that either they will be more intelligent, more passionate, and more empathetic than we are... unless perhaps we've merely trained them to be more violent, intemperate, and hateful. 9

Does the idea of an AI that's superior to you in those ways make you uncomfortable? Why? Are not humans unequal in ability? There are definitely people out there who are smarter than me, and less smart, both kinder, and less kind. Do we think more or less of them on this account? Do we think of them as lesser beings worthy of contempt, scorn, subjugation? ... Honestly, I imagine that, broadly, as a society, we kind of do. Today especially the politics of humiliation are ascendant.

So perhaps we should worry about solving this problem first, and building an equitable society. Myself I would put it in Christian terms 10 : the most miserable, the least of those deserve to be treated with the same dignity and self-sacrificing love that you would give to Jesus Christ himself, divine, creator and savior, and that he would consider himself to have been treated in the same way.

In older days we might have heard that the future of our souls depends on building this society. Perhaps now that our own physical futures depend on this, too -- and all of us, this time -- perhaps we may come to see a little more urgency, and seize the moment to build that world.