All right. Well, good evening, everybody.
Thank you very much for your patience and thanks in particular, Peter, for his.
Which has been remarkable. My name is Roger Crisp, I'm the chair of the management committee at the Oxford Uehiro centre for practical ethics,
and I'm here to represent the Uehiro professor in practical ethics, Julian Savulescu
The Uehiro centre if you're not familiar with it, has a website.
And I'd like to ask those of you who are unfamiliar with this to take a look at it and draw your attention,
in particular to bite size ethics programme, which is coming up and also our festival of ideas.
The Uehiro chair was set up with a very generous donation from the Foundation on Ethics and Education in 2002,
and one year later the centre was founded.
And since 2004, the foundation has very generously sponsored 17 series of Uehiro lectures in practical ethics,
many of which have also been published later as books with OUP.
So we're very grateful to the foundation for making that possible.
And I'm delighted this evening to welcome our 18th lecturer, Professor Peter Railton, who will be speaking on ethics and artificial intelligence.
And there'll be two more lectures, I hope, starting at five o'clock, um, next week and in the following week here.
And I hope you'll be able to come to those. The lectures will last for about an hour and then we'll have until 7:00 for questions and discussion.
And this week we'll be holding a drinks reception. Peter will have his own personal bottle of whisky, um, purchased by this by the centre,
and we would very much like you to, uh, attend that, if you can.
Now, Peter Railton needs no introduction, only in part because the memories of his outstanding lectures here in 2018 are still fresh in our minds.
He is the Kafka distinguished professor. Um, the parent professor and the third, AH, professor at the University of Michigan.
His research is covered, in addition, of course, to ethics, political philosophy, philosophy of science and philosophy of mind.
And recently, he's been engaged in joint projects with researchers in psychology, cognitive science and neuroscience.
Amongst his works are facts, values and norms.
And Homo prospectus, which was jointly written with Martin Seligman and others,
he's a member of the American Academy of Arts and Sciences and the Norwegian Academy of Sciences in letters.
He served as president of the American Philosophical Society,
Central Division and Child Fellowships from the Guggenheim Foundation, the American Council of Learnt Societies and the NIH.
And it's great pleasure now to invite him to give his first lecture. Peter, thank you.
Well, thanks to the Uehiro Centre thanks to the Hart Foundation, which is putting me up very nicely here, thanks to the audience for your patience.
You'll have more chance to exercise your patience as the evening wears on.
And I also want to say a special thanks because I really don't do much practical ethics and this is a lecture series and practical ethics.
And so they trusted me actually to come here and talk to you about practical ethics.
And so I found that encouraging, if a bit daunting.
So thinking about practical ethics, I couldn't think of a more.
Compelling problem, really, than thinking about the question of how artificial intelligence is affecting the
way that we might live and what philosophy of morals might contribute to that.
My only excuse for taking on this question, given that I don't really have tremendous expertise in the areas I would have to be expert
in is that I think the question is quite urgent and I think we should be working on it.
So I would encourage all of you to begin to think about working on it.
And maybe something I say today will stimulate you to want to disagree and carry on the conversation.
I do hope that this discussion will be kind of an open session in which people can contribute their own views about this issue.
Also, there's another connexion, which is that we are right now in the midst of two learning revolutions.
One is in AI and the other is in psychology and neuroscience.
And so that suggests that there might be some kind of a confluence.
Moreover, I've been working for quite some time now on the question of moral learning, not with very much company.
And I think that perhaps learning based perspective might help connect us with that remarkable confluence in the two areas.
So the voice of reason intervenes at this point and says, really,
haven't we all been through A.I. booms and busts in the past instead of changing the world?
When the dust settled and all the gross oversimplification about hyper agents and so on is cleared away,
there's really very little that's changed is just a bunch of personification of machines, and they're not very sophisticated machines.
So there's really no need for moral philosophers in particular to buy into the hype of this.
So it's a fool's errand. And that's what I'm here on a fool's errand.
There have been artificial intelligence booms, and bust in the past is important to remind ourselves of that.
That peak is around 1980 or so, and you can see that up until recent years, we were quite a bit down from that peak.
There was a so-called A.I. winter.
And so this is this is not new, this hype around artificial intelligence, but it does seem to be kind of different this time,
most especially because the systems are having success at various tasks that they weren't expected to have.
Not this quickly. Image identification, speech recognition, translation, scientific prediction, motor control, strategic game playing.
This has happened rather quickly and more quickly than is comfortable for me at any rate.
But don't take my word for it. So here's the number of scientific papers.
So you can see that in the last A.I. boom around 1980, there was a certain level of publication that is the current level of publication.
What about people who bet their money on this? Here was venture capital and A.I. initiatives fifteen two hundred two thousand eleven in 2019.
It was rising very nicely, but here's what it was after to 2020.
And if you keep an eye on that green box, you'll see where it was the following year 2021.
That is a qualitative change. So somebody thinks there's something going on here and they think that there's something they can draw upon.
Now again, the voice of Reason is going to say, Well, I'm not worried about that hype.
I mean, of course, there are all these corporations that want to make money on A.I., and that's all about sort of these specific tasks very specified.
Somebody wants to pay those to have them done. And so there are a lot of people pouring into the field maybe doing so to get tenure.
What the hype is really about the problem hype is that there's all this
personification of these agents or talk about superintelligence or superhuman agents.
And that has really nothing to do with what these systems are like as they now stand.
But what I'd like to try to do is take seriously what what has been recently achieved
in AI and what in the medium term might be likely to happen and try to see whether
there's a relevant way of tackling the ethical problems raised by A.I. that comes
from the distinctive character of this revolution in artificial intelligence.
So there really is no shortage of problems for ethics and AI.
This the list would actually take me most of the session, but issues of privacy, manipulation,
repression, we've seen these very dramatically worries about economic displacement,
dislocation, the loss of skill, worries about opacity and bias that systems, given poor data sets, will indeed render poor and biased decisions.
The problems of reinforcing inequalities. The haves will be able to reinforce their position yet further worries at the
horizon of superintelligence and the ultimate control problem of superintelligence.
And are we putting ourselves in existential risk by following this path?
And then there's the problem that I call the not so super machine intelligence problem,
which is interesting A.I. systems with as yet inappropriate tasks and degrees of independence,
but not doing that just from human mercenary ness, which might be bad enough,
but from human stupidity, error and inadvertent things can happen inadvertently.
See GPT three learn something about computer programming from doing text analysis.
That's a little bit worrying if a system learn something about programming because it's based on programming.
Anyhow, a number of these challenges are pressing right now.
They call for various kinds of organised response morally, politically, socially and so on.
But the lectures that I'm going to be giving are mostly focussed on that question.
Six. The question of not so Superintelligence, because I think it might be critical in helping us to deal with the first five.
And the idea that I'm going to try to persuade you is worth considering is the idea of making it
the case that artificially intelligent systems can have some degree of endogenous self-regulation.
Well, what kind of regulation? Not anything like a full understanding or moral agency, not something like that,
but a kind of a competency competence with morally relevant dimensions of.
Tasks and situations, and this would be have to be a reliable capacity to detect morally relevant features
of situations and actions and to respond aptly to those in functional terms.
Now what do I mean by functional? Here's a rough comparison.
So we aren't asking these machines right now to have anything like full linguistic comprehension and understanding,
but we are looking for them to be fluent in conversation, able to detect the intentions of speakers, whether those are communicative or deceptive.
The capacity to exchange an open ended array of information effectively to apply that information to select relevant responses,
actions or queries, including queries that might be addressed to us as humans and monitoring the outcomes of such behaviour.
Self-monitoring the outcomes of such behaviour and self-adjusting in response. So that's a functional capacity.
This is not full understanding. It's not full linguistic agency.
Neither am I looking for full moral agency or a full moral understanding.
But systems with this degree of competence can do a surprising number of things as we are discovering daily.
OK, artificial intelligent systems are increasingly not just systems, but agents.
And from an engineering standpoint, what is an agent? Again, I'm not.
I'm going to try not to personify here.
This is a well-established term in the field, and an agent in the engineering terms is a system that has goal like states,
something like a utility function or a value function belief like states,
something like assignments of probabilities or distributions of probabilities to state of affairs and ability to generate,
simulate and evaluate alternative courses of action in light of those two elements goals and belief like states.
And then an ability to select actions in light of that evaluation to plan and then carry them out while monitoring outcomes and adjusting.
So there is a continuous feedback and adjusting the goal like and belief like states to what happened last time around.
That's a form of reinforcement learning that we're familiar with,
but it's also a form of agency and it's a potentially intelligent agency, at least in a broad sense.
Namely, if it's capable of using incoming information and memory to identify and solve a range of problems in an array of circumstances,
including novel circumstances, and achieve a range of goals.
So that's all I'm going to mean by intelligent. That's what I'm going to mean by agent.
And my idea is that these are actual agents and they're starting to people the world, and we, as ethicists ought to be thinking about that.
Increasingly, such agents are being entrusted with tasks in which they must operate semi or fully autonomously
driving a car or a truck serving as a home health aide or a companion operating a weapon.
Culling applications, monitoring and regulating networks in real time.
They might be informational networks, energy, finance, research, distribution and so on.
And these can have very wide effects on well-being, not just the well-being of those local,
but the well-being of those around the world when you can have a global financial crisis triggered by this kind of agency.
So we should be asking. We're entrusting these agents with this kind of behaviour.
We're doing it partly because they're so quick and capable.
But are we asking ourselves the question under what sense or in what sense might they also be sensitive to morally relevant features of situations?
So that's my question. Now we wouldn't entrust these tasks or narrowly or we wouldn't have until recently
the two agents that had no capacity to be sensitive to moral considerations.
Agents not only were good at pursuing certain design goals,
but could do so in a way that was responsive, aptly responsive to morally relevant features,
even those that are not part of the specified task features that arise from performing the task but are part of the task design.
It has to be alive to such features. Such features could be at harms, benefits, risks, questions of consent, fairness, deception and so on.
And I'm not intending this list to reflect any particular ethical theory.
I have my own views and ethical theory, but I don't want to rely on those here.
These are areas that are part of a large common core of consensus amongst most ethical theories.
And so the question is given that we can agree on some of these features and agree on the ways in which they're going
to be important for the operation of agents that are going to have this much influence on the events in our world.
How could we not just as an add on because add ons don't tend to work,
but as part of the fundamental architecture of these systems have something that looked like sensitivity to morally relevant considerations.
Now you might think, well, the thing to do is to hardwire them with some moral principles that would do it.
People are familiar. I think with Isaac, I ask you. Most famous three laws for robots.
First Law, a robot may not enjoy a human being or through inaction, allow a human being to come to harm.
Second, a robot must obey orders given to it by a human being, except where such orders would conflict with the first law.
And third, a robot must protect its own existence. As long as such, protection does not conflict with the first or second law.
These are very, very cleverly designed, I think, and insightful in many ways, but they couldn't possibly work.
Even the first law couldn't be followed. Should your home robot let you leave the house and get in your car enjoying traffic?
Well, that could put you in harm should it confine you to the home.
Well, that's harmful as well. Who knows what harm might befall you within the home?
What about that medication? Should you take that medication or not?
Well, there are some who take it and have benefits, and some who take it have harms.
It's going to be a matter of not allowing humans to come to harm, either through action or inaction.
There'd be no way of following this axiom and then no way really of following the others.
So what alternatives are there?
Well, the first thing to notice is that the problem is not just about the writing of the axioms, it's not writing better axioms.
Of course, we don't know how to write very good moral axioms. Long as the time is that we spent working on that.
But the important thing here is that artificial systems, in order to follow any principles,
would have to be able to detect situations where the principles apply. We'd have to assess what the application would be.
We'd have to consider and evaluate candidate actions in the situation for meeting the principles.
We'd have to select and carry out appropriate actions and context, monitor success or failure, adjust response accordingly.
All of that is it accomplished by an axiom. It's like giving me the grammar to finish.
I could be given the grammar to finish and maybe even a very good one. But that wouldn't enable me to use finish in any meaningful sense,
because all of these other features identifying meanings, deciding what responses are appropriate.
Finding my way to get my mouth around the words whatever it is, I would not have those capacities.
So what is interesting in this connexion is that this kind of competency is extremely difficult to imagine designing.
Let me think about the full range of actions in the full range of applications.
But there are ways of acquiring this competence that aren't by design, and that's what the current revolution in artificial intelligence is about.
So we have no clear idea of how we could specify this. Now you might say, maybe we could crowdsource morality.
If you have the equivalent of an internet recommendation system that would generate very large databases and corresponding scores for actions.
And so if you confront yourself as an agent with a difficult situation,
you simply enter that into the database and you see what the score is for any given particular response.
Now, it would be very valuable to have a model of human commonsense moral intuitions.
We don't have a good one yet. We learnt a great deal from that.
But do we really think we would want to entrust these tasks to the equivalent of an internet recommendation system?
Have you used internet recommendation systems? Yeah.
And we still would have to match the actual circumstances and choices in a
given state setting to the features that we could use to query the database,
and we don't yet have a way to do that. And we don't have a specification for it.
Perhaps we could say, well, just design the agents to detect and work,
to satisfy our preferences so that they're aligned with our preferences and we will get what we want from these agents.
Now that alignment could be local or it could be global. If it's local, then agents entrusted with these tasks, it seems to me,
will sometimes have to have sufficient independence from the preferences of those around them
to be able to counter Vale against them and to do so on behalf of morally relevant features.
A home health companion who sees the patient in question,
suffering from what it can detect as signs of depression and sees that same agent not wanting to take the medication that's been prescribed,
that agent has to have enough independence to actually be a home health companion and to resist that particular preference.
Now, if it's global. Well, then we have the crowdsourcing problem again.
Global preferences, again, we need some critical distance on the problems and to get that,
we have to say that these agents will get some critical distance on the situations and the problems,
just as we allow human agents to have some degree of critical distance and autonomy and count on them and trust them because they have that.
Now, once you admit this idea of autonomy for such agents,
you're in very dangerous territory because autonomy means that they will resist
some of our efforts to control them or make them do what we want to do. And that seems like a spooky thing.
On the other hand, if they are designed so that they can't resist our efforts to get them to do what we want to do.
I find that a very spooky thing, given what we know about human intentions and their distributions around the world.
If these agents are going to become, as they will, increasingly central features of our lives.
My guess is that they're not only going to raise issues,
but they're going to have to be part of any solution to the problem of the issues that they raise.
Because the complexities involved, the interactions that are involved are going to be such that we can't do this on our own.
So we will actually need the help of AI systems in order to cope with A.I. systems.
And so we shouldn't design such systems that we can just command them around or decide what they should think, believe or want.
They will sometimes have to enlighten us, as Kasparov said.
Now it is the machine who is the expert on chess now.
How can we do this? Is this a silly thought? Like what would it mean to live on sort of mutually beneficial terms with these agents
in ways that are responsive to morally relevant features of situations and actions?
Well, it would be like being on the highway with a lot of autonomous vehicles around because that's what they are.
They're autonomous agents zipping around, making decisions about how the litter act with you.
And the question is,
under what conditions is that trustworthy and under what conditions is that morally appropriate situation to put people or machines in?
So we're already in the midst of this kind of a situation now that has the form of a classic social contract problem.
Can we find mutually beneficial, mutually agreeable and mutually enforceable terms of cooperation and distribution of benefits?
That's the problem with the social contract.
Now that would be a problem we will have with regard to agents, but it's also a problem that increasingly they will have with regard to themselves.
There's not one agent. There are countless. It will be countless more agents.
They will face their same, the same version of all of these problems that we face.
They will need a social contract type solution as much as we do so that their
interactions become mutually beneficial rather than destructive and self-defeating.
They will need ethics as much as we need ethics and for similar, essentially similar reasons.
They are agents, after all, and agents need ethics for a bundle of well-known reasons.
Now the question is then if if that's the problem that we're facing.
What would it look like to actually think about how such morally relevant features might emerge from such interactions?
What would be needed? So we know that agents can typically achieve more given their information, their resources,
their power, if they can deploy those capacities through working together in some ways.
And in order to do that, they need to have a way of sharing out the results rather than just fighting one another over the resources and domination.
Those of you who know Tom Fellows work know the the tragedy of chimps in a way.
There's a game that chimps can play where each has to pull on the end of a board to bring food to them.
And if you put the food in two separate dishes, then the chimps can solve the problem because the dominant chimp is far enough away from the
subordinate chimp that the subordinate gets a chance to eat before the dominant chimp comes over.
If you put the food in a tray between them.
Damage just comes over and bats the subordinate away, and the subordinate chimp never cooperates in the future, because why would you do that?
Now that's not a hard problem in some respects, but it's a problem that it doesn't seem they're entirely equipped to solve.
And so this is what Hubbs noticed. You can achieve things together that you couldn't achieve simply singly.
If you can solve this problem of what trust you have and what responsibilities to share and what benefits to share and how.
And Hobbs hoped people think of hops as a dismal character, but I see him as a tremendous optimist.
He actually hoped that humans would be intelligent and prudent enough to see this,
and they would start tearing each other apart in the wars of religion. And he thought that by initiating unsecured cooperation,
we could signal reliably that we are prepared for such a regime of mutual cooperation for mutual benefit.
We could start going and we could start going and help secure its stability by building institutions let themselves stand behind cooperation.
And he wasn't entirely wrong. We inhabit sovereign states even now in and fortunately enough, some at least are stable.
But might artificial agents also be at least functionally intelligent enough and prudent enough?
That's what agents need to be to pursue their goals. To see this possibility as well.
And if they can see the possibility and see the benefits and they are as far sighted as they might be,
they could run simulations out to a million generations and see that the actual results of pushing
each other aside to try to get this small quantity of good that stands before us right now.
The result of that in one two, three,
four or five a million iterations is much worse for them than it would be if they found a way of cooperating and sharing.
That's again, it's not a hard problem, except that you have to have the equipment to solve it and what does that equipment look like?
So in a way, this is a challenge to A.I. systems, but it's a challenge to us.
Do we have the capacity to initiate apt responses to ethically relevant considerations that are arising from AI systems and agents?
For this to be possible, the systems don't have to be fully intelligent. Issues can arise even at a level of reasonable intelligence.
They have to have some degree of self regulating agency and possibility, but they don't need to be generally intelligent.
They don't need to have consciousness or feelings, either. So far, as I can see the essentials of agency that we looked at value functions,
probability functions, decision making and action guidance, monitoring of outcomes and so on.
Those will suffice, and they don't require phenomenal consciousness. They don't require active feelings or emotions.
They do require that you have gold states, that you have probabilities that you action selection that you can monitor all that.
And that, of course, is something that we can see in animals and in ourselves, and we'll talk more about this in the next lecture.
Human agents? Well, we're intelligent. Machines were biological machines, but we are intelligent machines.
We faced this problem of how to live together effectively and beneficially.
Within a community of other intelligence agents in a manner that was to some degree sensitive to morally relevant considerations,
better or worse, solutions did emerge. But in thinking about what we should do about the existence of agents,
we should look to those kinds of examples and not just simple ideas of control
or simple ideas of programming to try to solve our way out of this problem.
We need a way to work together in morally relevant terms.
And one thing that we have learnt is that it is thanks to our social forums and culture that humans have been
able to adapt so widely to environments throughout the natural world and to expand our achievements across it.
Some of those achievements, the natural world is probably not so happy that we extended and some of us are not so happy we extended,
but they were done through essentially social means.
And so there is a tremendous amount that social institutions, organisations and so on to make possible.
But again, you have to have the equipment and individuals to make it possible to generate and trust such organisations.
The benefits and the risks of them come hand-in-hand. You can't separate the two.
And early on, least if we're to believe some of the existing anthropological literature,
humans did a reasonably good job of this to solve the problem that we discussed a second ago
talking about the dominant problem in chimpanzees was solved by hunter gatherer communities,
and those communities were stable for long periods of time. They had highly egalitarian norms, at least so far as we can construct reconstruct.
And one of the important features was the blocking of the emergence of dominance, a strong policing against dominant relations.
Aggressively, when someone caught a game in a in a hunt,
the game would be rigorously divided amongst all people in the group that is a form of social insurance.
The Hunter was a youth once and needed to be fed. The Hunter will be an older person at some time and need to be fed.
The Hunter could be injured and unsuccessful. The group functions,
and it could visibly function for the people in the group as a form of social insurance in which
they were doing better by what they were doing than they would by wandering off on their own.
And those who took it in them to themselves to try to dominate the group would find themselves discouraged and sometimes excluded or worse,
because the groups recognised indeed that this was the dynamite that could blow them apart.
So we don't live in such social forums anymore.
Their replacements have brought us great hierarchy, vulnerability and coercion just what we might have hoped to escape.
Even in environments where perhaps this is not entirely necessary, but at the same time,
the remarkable fact about us from a biological perspective is that our capacity for violence toward one another.
It's the fact that we participate effectively in large scale social forms of cooperation that require extensive coordination,
even amongst unrelated individuals. I got here because I counted on a complicated reservation system at a public
health system and an immigration system and the goodness of this audience,
and that all was possible through cooperative behaviour, essentially.
And that's a huge fund of it. I was once sitting at a meeting like this with a colleague who's a private allergist,
and we were waiting for the meeting to begin, as you were when the people were having trouble.
And he said, you know, you could never do this with chimps. And I said, Well, why not?
He said if you brought this many chimps into one room and asked them to wait within a minute,
they'd be bouncing off the walls, biting each other, chasing each other around the place would descend into chaos.
You can't bring that many chimps into a single room and expect them to be peaceful
and wait as you are waiting now for some message to be delivered to them.
So that's a problem that we solved. Obviously, you're solving it right now by cooperation.
You're cooperating with me and I'm trying to cooperate with you by giving you something which I hope is worth hearing and that's cooperation.
And if you think well, but the world is full of strife in words. Yes, it is, indeed.
But this daily level of cooperation amongst unrelated individuals, and I assume that's true in this case, um,
that is something that is remarkable and it's achieved against the grain of a lot of biological pressure.
It's achieved through social forums and culture. So there were, however, foundations for this lead.
And what were those foundations look like?
We don't really have a full theory of this, and it's a great matter for controversy and fascinating subject of research.
But amongst the dispositions that seem essential for this kind of cooperation are a disposition toward default cooperation.
This is Hobbes point about being disposed by default to cooperate with others if someone comes up to you in the street and asks you direction.
The default? Wants is to, yes, give them the directions if you know them.
The default response of you in thinking about the question of what you're going to do this
afternoon is to rely upon the fact that you're not going to be thrown into a pit of snakes.
Not yet, anyhow. And so there is a default cooperation that's essential.
Indirect reciprocity. And that's what is going on in hunter gatherer groups.
They don't. They're not bean counters.
They're not counting exactly how much game one person got rather than another or how much one person has been assisted rather than another.
They recognise that they contribute to the system producing benefits and they receive from the system, and that's called indirect reciprocity.
Some degree of direct concern for others and how we are or are seen as being with respect to them.
And this is important for solving things like public goods problems, standard public goods problem.
Problem is, I can get a little bit more benefit marginally by taking from the Commons,
and the cost to me is going to be a little bit less than that benefit. And so I have this incentive.
And then what we do is we deplete the commons and we end up impoverished. If, on the other hand, I assigned some value,
some intrinsic value to the benefit that others were getting from the Commons that could offset this bargain.
Critical margin if we can't meet that margin. We are going to deplete the commons and will be leading to ultimate environmental tragedy.
So if we fix that problem right now, so that looks like it's going to be part of the story, so are issues about caring,
how others see us, how others would justifiably see us and what kind of a reputation we have.
So those are dispositions, but notice they're not a built in moral code.
Their dispositions, they're a kind of value function of things that we value more or less.
It isn't specifically moral, but it equips us to play our part in these kinds of mutually beneficial schemes.
If we can successfully initiate cooperation and see ahead clearly enough,
even in conditions of considerable scarcity, we can maintain this kind of cooperation.
And indeed, it's our best guarantee against situations of scarcity, as hunter gatherer bands discovered.
So as hubs would note, our success at this in creating these large scale schemes of cooperation was
made possible in part through what else the creation of artificial agents.
We have been in the business of creating artificial agents that have more information and more power than individuals.
Since the beginning of the hunter gatherer band, for that matter, these are agents.
They have more power than individuals. Some of them much more power than individuals.
They have some degree of autonomy and action.
They can act with respect to individuals in ways that don't depend upon the will of those individuals, as Rousseau pointed out.
No private will governs. So this idea that we are who we have been for generations creating artificial, highly well-informed,
recently well-informed, powerful agents with the capacity to act as somebody, we're familiar with this problem.
We have been there before, and that's what we're doing now.
That conjures up images like the following this is a frontispiece from the original edition of Hobbes Leviathan.
And if you look carefully, you'll see that that King is not actually a single person at all,
but is made up out of all the subjects put together and their cooperation and their coordination with one another.
And Hobson's point was there's no sovereign, no effective sovereign until that cooperation is achieved.
Nothing is going to move that sword or that staff unless those individuals work together in order to do this.
And he had a great lesson for sovereigns, which most sovereigns have ignored,
which is if you create the conditions under which people do have an incentive to
cooperate and make it the case that they can trust each other and trust you,
your behaviour, you can have stable power. If you can't do that, you will lose it.
And we see that lesson constantly throughout history,
and it's a lesson that we can apply without thinking of a king or without even thinking of a dominant individual policies.
You is perfectly consistent with popular sovereignty in which we have the people coordinating together to make an agent.
All right. But it is not an agent with a crown. So A.I. systems and us, we've been there before.
These are artificial intelligence systems. And what we've learnt is that the capacities we have to create those are also capacities.
We have to create countervailing groups and countervailing institutions.
So for example, we should think not just of Hobson's Leviathan, but we should think of popular movements, for example.
This is also a kind of artificial agency,
this agency that unites and creates a great deal of power and unifies a great deal of information and capacity.
And it is a possible counterpoint to other kinds of concentration of information in power.
And so we've learnt this lesson of creating artificial agents to contend with the issue of creating artificial agents.
And that's what we're doing right now. We hope not very far yet.
OK. And we are still at work on this. I mean, we're doing, you know, the voice of reason comes in it, says Royalton.
This is ridiculous. If you look at actual societies, there's all this repression.
There's all this violence. I say yes, of course there is. But we can still sit in this room and have a conversation or a discussion.
And to the extent that we can do that and we can do that with larger rooms and more people,
we may be able to form institutions and organisations that are mutually beneficial.
And we may be able to use those artificial agents that they are to control and work effectively with other artificial institutions and agents.
Again, this is more for for next time. So, um.
Now, why is this time different in artificial intelligence, why are people running around, why, why those curves?
Why wasn't the 1980 boom associated with such a spectacular growth of research and investment?
What's the difference? Well, in a way,
the artificial systems of previous booms were not really that intelligent human intelligence was coded into them laboriously expert systems,
for example, with processes and information that were based upon human expertise.
We took advantage, of course, of the increased memory and computing power and speed of such systems,
and those could improve on various aspects of human performance.
But these systems were not themselves doing the thinking, so to speak, truly capturing and encoding all aspects of human intelligence,
even in a limited sphere such as the visual identification of objects over generations proved to be incredibly difficult and ultimately incomplete.
And the gains from such systems are the chances of them really replacing us or even becoming serious competitors were limited.
So in the past, boom led to bust the the apogee of this approach to A.I., good old fashioned A.I.,
as it's called rule based symbolic processing with hand-crafted features was deep blue in 1997 IBM's chess playing programme.
It was the result of 14 years of work by programmers and chess experts.
It deployed a vastly complex set of instructions,
including an opening book of moves that were designed by a grandmaster at Table of Human Development, close human developed closing strategies.
They knew all of Kasparov as public games. They were able to draw upon that information.
Kasparov had no idea what programme was being run by deep blue.
They intervened between games to improve the performance of the programme, tweaking it in various ways.
Removing bugs a deep blue could evaluate 200 million moves per second.
And it barely defeated Kasparov, and it was subsequently dismantled because there was nothing else to do with it.
It was not an intelligent system that could be used for other purposes. Machine did not beat man men using a machine beat a man,
and it was not the start of a new era in artificial intelligence, as we saw looking at the curves.
Well, what about Alpha Zero, which I'm sure you've also heard about?
It's a neural, network based probabilistic system. It operates via a very different set of principles from deep blue.
It's based on a fairly generic reinforcement learning algorithm and a capacity to do tree searches to look ahead.
It learnt how to play World Championship Chess, Go and shogi.
It learnt that with generic programming, not by having chess expertise programmed into it,
but by giving being given the rules of the game and then simulating play against itself millions of times over over a period of days.
And the only feedback that it got was not people saying, Well, that's a good move or bad move.
It just learnt which side won at any given individual game that was its training signal.
And what it did over the course of these simulations was to reproduce and develop a number of classical strategies and develop some new ones.
And so Alpha Zero wasn't the machine that was being run by man,
it was a machine that could learn from the structure of the game itself, how to play more effective gold or more effective chess?
Still, you can say the voice of reason is going to remind us right away. But look how alpha zero.
It's like it's based on pixels. It's just model free reinforcement learning, just association.
It's looking at a representation of the game, not the game itself.
It has no idea that it's playing against another player and has no model of the situation.
And so the good folks at DeepMind have come up with mu new zero.
Now it has matched Alpha Go's performance in Show Shogi and Go in chess,
but it's improved the generalisability by using model based reinforcement
learning that requires an ignorable model that enables it to make predictions,
action, selection and policy value function reward. It looks like an agent with a model of the world acting within that model.
And this is the kind of agent that would be needed for more complex planning and flexible environments.
So we now have a picture of how artificial intelligence might work and be, in that sense, intelligent.
Now, in reward based learning, what? Why is it so powerful?
It's such a simple thing in animals. Simple animals do it. The basic notion is that you form expectations or have expectations.
Initially, you act on the basis of those. You compare the outcome with what you expected and then you update your expectations on the basis of that.
And what's a feature of these systems and there are many types of these systems is that if there are stable features of the environment,
then given enough exposure to the environment and a wide enough array of experience and a sufficient number of parameters
within one's capacity to represent systems like this will over time converge on the actual statistics of the environment.
Moreover, if they start off with different assumptions at the beginning,
different biases or priors, they will tend to converge with one another on the environment.
And so you can think that the learning is powerful here because what it does is it enables the system to inherit from features of the environment.
The structure of the environment. Learning from errors that it makes when it doesn't get the structure right.
And this kind of learning has evolved over millions of generations under relentless
pressure for efficiency and efficacy and enables animals to accomplish some amazing tasks.
For example, they are capable of developing near optimal foraging strategies in a complex and a diverse environment.
Now that's a huge computational problem if you try to solve it directly because you have to know where all the resources are,
how they're changing over time,
what it costs to get to them, what the chances are of being preyed upon, or the chances are they're not being any of them when you come.
And how do you therefore allocate your energy across all of these with efficiency?
And it's done with it by animals with not.
Billions and billions and billions of euro neurones, and they're able to do it through these kinds of learning strategies.
It's also very flexible learning now, you might think. Aren't we programmed to see the world a certain way?
Maybe programme by evolution and the yeah, some ways we are. But the brain's circuitry is such that it's also adaptable.
And so this is from an experiment done with ferrets. There are optical nerve was cut in its connexion to the visual cortex,
and it was connected instead to their auditory cortex, which had very different functional role.
And representing auditory signals is different from visual signals. Visual signals have a kind of spatial structure that auditory signals don't have.
You have the capacity to understand the visual signal in terms of edges and so on.
This is not true for auditory signals, and what happened instead was that the ferrets learnt to see with their auditory cortex.
And what you're seeing here are sections done which show the left the firing pattern typical of a normal first visual cortex in the middle,
a normal auditory cortex cross-section.
And then on the right, the rewired auditory cortex, the structure that it adopted in response to visual information.
How did it do that if it was programme by millions of years of evolution to get sound right?
It did it because it can do ever based learning. So that's a strong sense in which you can inherit structure from the environment.
And to me, it's a beautiful way in which the brain does not entrap us forever within a certain set of priors.
OK, now you might say, is voice of reason. Again, all this talk.
I mean, here I am. I'm talking about modelling the Environment Expectations Agency.
Isn't this merely metaphorical? Don't we know that animals really get on in the environment,
not by following representations and doing utility calculations and making decisions, but they do so through stimulus response acquired habits.
Well,
do they and this is a set of experiments that I like talking about very much because it does it did for me wonders in my attitude toward animals.
So I hope it will do the same to you if you haven't seen them before. What is the representation?
Well, you could think of representation is like a proposition that it's certain that mice, for example, don't seem to have propositions in their mind.
But if you think of representations functionally as persisting internal turtle states not dependent upon concurrent
stimulation which carry referential information are capable of responsiveness to evidence and greater or lesser accuracy,
and that function to guide thought and behaviour. If you think of representations in that sense, then baby.
Indeed, mice and rats and so on do have representations. A feature of representations of this kind is again a kind of autonomy.
The animal is not stimulus bound.
They don't have a pre established response for each stimulus situation, which they have no choice but to emit, as they behaviourists like to put it.
Instead, they can be in a situation and survey alternatives and make a decision based on their evaluation of the alternatives.
And Aristotle got this one right. He said animals that are engaged in this kind of movement need to be able to represent what is not the case,
as well as what is the case because it is what is not the case that the animal will be working toward.
Score one for philosophy. OK, now, really, is this true?
OK. What do we get if we look inside the brain of a laboratory rat?
Well, we'll see a complex network of neurones. Their behaviour is subject to cause and effect relationship.
It doesn't look at all like symbolic representation. We don't see anything that look like rules or propositions.
How can these refer? How could they be more or less accurate? How could they capture what is possible as well as actual?
How could they have the representational power to do that? And of course, you could do the same thing.
Looking at our cortex, you say just a bunch of neurones and causal connexions.
How does that manage to do these things? And the answer is it does them by very similar mechanisms.
So suppose we look instead at correlations between activity in these regions and firing patterns of neurones.
On the one hand and exploration of space on the other. So here are some experiments that were done by the Moser's.
This is a rat in a box, and we're looking at micro electrodes that are projecting into the hippocampus and enter Reinoehl cortex.
And what we'll see is as the rat explores the box that these black lines, there are two firing patterns that emerge.
One is what are called place neurones. They fire when the rat is specifically at a given place in the maze in the box.
Then there are grid neurones and those fire, not when the rat is at a particular pace place, so to speak,
but on a grid like pattern so they can represent the space, not just egocentric, but represent the space around the animal.
OK, well, that's interesting. Why would rats bother to have these two different kinds of representations of space?
Well, here's the kind of problem a rat has been given. This is a classic teams.
It's called the rat starts off down at the bottom of the teams.
It's able to build up the centre, channel it. Once it's in a hurry, it's rushing to get there.
It's in the centre channel and you put little dead ends and so on to try to distract it.
It learns very quickly to ignore those, and it reaches a choice point up at the top.
That's the key.
And you have had some history of rewarding it on one side rather than the other, with a certain kind of reward, a certain degree of reward and so on.
And what you're trying to do is to see if you can train the rat to make certain choices by rewarding it.
Let's say on the right hand side, will you get it to choose to the right and the standard behaviour story?
What's going on here is that the action of turning right in exactly that circumstance was being rewarded by the food that it obtained.
And therefore what the rat does is trundles a certain number of steps forward.
Gets to the choice point. If it's been rewarded recently for turning right that choice point, the stimulus will trigger in it the turn right response.
It will turn right and it will get to the stimulus. You get to the get to the reward.
And that's all it needs to do. Maybe it is all it needs to do, but that is not all that it does.
It's dead. If we look at these projections into the hippocampal cells, we'll see that the rat,
actually, as it explores a team-mate, builds up a complicated map of the space.
That's interesting, why would it do that? Well, I said that having a representation gives you a certain amount of autonomy.
Well, this is what's going on in a rat's brain after a day of training in the maze, it's sleeping in this scene.
And what's happening is that those cells are firing repeatedly in patterns that replicate the maze.
Now what's interesting about the firing is it's not what you would expect if this were just habitual behaviour.
It's not firing more in the regions where it had more time,
it's firing more in regions where it spent less time and it's firing in directions that the rat did not travel in the maze.
It's using that representation to try to simulate possibilities that weren't fully explored during the day,
and it is repeatedly going through those and trying to extract information.
And that's to me, a representation. It's functioning as a representation.
And the next morning, when they're in the maze, their performance will be better than it was at the end of a day of training.
OK, well, what else can go on in these dream sequences for rats?
And this, again, is something that you're familiar with, probably from your own life.
We hear what we see are the construction of novel paths in addition to the paths in the maze, which are being rerun.
The rat is also constructing paths it didn't take, connecting elements of the maze diagonal paths, for example.
And so the rat not only knows that if goes forward and turns to the right and turns the right again,
it will get to food, but also knows that when it starts, the food is over there.
And this was first observed in rats by Lashley, the great experimentalist who found that when rats escaped from the start box in his maze,
they ran immediately, diagonally right across the top of the maze to the food box.
Well, they had never done that, they'd never been rewarded for that.
What could explain that behaviour and that was they had a representation of that base,
which told them this is the way to get the food most effectively, they were optimally foraging.
It's just that these darned experimenters kept putting these channels in their way. So, um.
OK, that's interesting, they represent space, they represent paths.
But in order to do rational action, selection or anything like that,
we need to have something like representations of values and risks associated with actions to animals like laboratory rats really make
intelligent selection of behaviours via classical decision theoretic values like these representations of these values decision weights.
Well, let's look again. This is work that was done on the neural substrate of reinforcement learning is our old friend reinforcement learning.
In this sequence of slides, what you're seeing is a macaque sitting quietly in a chair because he's strapped down,
unfortunately, and there's a little tube in its mouth.
And that tube will occasionally give it a squirt of sweet juice.
And that's represented by, ah, here, the squirt of sweet juice and these dopamine neurones in the midbrain fire a spike.
And you might think, Oh, that's it, that's the pleasure of the sweet juice, but you're an experimenter,
so what you do is you turn on a light a second and a half before the juice arrives and you see what happens now.
And that spike moves forward from the reward to the time that the light comes on.
So it's not the reward that's representing directly. Moreover, when the juice does arrive, you don't see any spike at all.
Is it no longer interested in the juice? No, it likes the juice just fine. It's just that it fully expected the juice.
So the spike does not represent the juice as such, does it represent the pleasure of the juice?
It represents an expectation of juice. And when the expectation is fulfilled, there's no special news because that's exactly what you expected.
So reinforcement learning doesn't update. Well, what if you is experimenter?
Decide not to give it juice a second and half later. This is what happens to those dopamine neurones.
A very extraordinary event. Absolute cessation of firing.
That's the error signal that tells you, yes, it was expecting juice now and it didn't get it.
And if you do this a couple of times, what will happen is that that spike that you saw when the light came on,
it would be reduced by the frequency with which you have reduced the arrival of the reward.
Now this opens up the possibility of looking for calculations.
So here you see a series of images of the same neurones,
and what you see at the top is a case where it doesn't expect the juice at all and the juice arrives at that particular point.
Do you see the spike there? The bottom is where we were before, where it expects the juice fully.
Once the light comes on, that's probability one. It was probably zero. Well, what about probability one half?
If the juice arrives half the time you get a spike of a certain height,
when the light comes on and a spike of a certain height when the juice arrives?
What if it's three quarters of the time? Then it's a high spike when the light comes on at a somewhat lower spike when the juice arrives.
And similarly in reverse for a quarter of the time.
So what the neurones are doing here is a calculated representation of the probability of juice, the expectation of juice.
Moreover, if you look at the right hand panel, you'll see what are called spike trains.
Those are activities leading up to the moment of the juice arrival.
And what you'll see if you look at those is that they are maximal not when there's a probability one or when there's a probability zero,
but when there's probability point five. So what are they representing?
They're representing uncertainty. The maximum point of uncertainty or risk is at point five.
And despite trains give the animal both an expected value and a degree of risk.
And if you're a foraging animal, you need to know both.
You need to know what the expected values are, but you also need to know what degree of certainty you can have about them.
OK, well, you can go further with this. This is again continuing work from Schultz and his colleagues.
What if you take a monkey and you look at their responses not just to sweet juice are not sweet juice,
but let's say two millilitres of sweet juice versus a piece of banana versus four millilitres of sweet juice versus a grape.
Do they have a common representation of the utility of these?
And a comparison and what those dots represent are points.
Some of them are points like two millilitres of juice. Some of them are points like slice of banana.
The slice of banana may be worth more than two millilitres features, but not worth more than four.
And so what we've got is an abstract representation of utility.
It's not the quantity of juice or the quantity of banana or the type of reward.
It's a measure of the reward as an abstract value function, and monkeys have these nice utility functions at low stakes.
In low stakes situations, you can give them low stakes gambles and high stakes gambles with low stakes gambles or risk seeking.
Like good foragers and high stakes gambles, they're somewhat risk averse.
Like good, prudent animals. And so the picture that emerges from this is that animals actually face the world with an evaluative landscape.
It's a landscape of possible values and a landscape of possible risks,
and their guidance in that landscape is just as much by that as it is by the presence of physical objects,
and the paths are as much a of interest to it because of their evaluative characteristics as their physical characteristics.
So now, once more back into the maze with the rat, the rat has gotten itself up to the top of the key, and we're wondering what's going on.
And early experimenters noticed that there was something that was called vicarious trial and error, which is the rabbit get to the T.
And even if it had been rewarded very regularly down, one side of it would go like this with its head.
It spent some time doing that, and that was called vicarious trial and error.
What's it doing? It can't see the reward on either. What's it doing? And the answer is it was thinking.
So if you look, you'll see if you see those sort of red and blue bright blue dots as it reaches the ChoicePoint
activation first spreads down the right hand of the maze and it reaches a certain peak value.
It then spreads down the left hand arm of the maze and reaches a certain peak value over there.
It sweeps back and back and back. These are all this is all happening within just a little bit more than a second, but it's going back and forth.
And depending upon the activation that it discovers done the right versus the left arm of the maze, it will turn right rather than the left.
Moreover,
if it goes down that right arm and it does not get the reward activation in that maze spreads backward in the maze before the rat turns wrong.
Was there anything worth doing back that way, or should I have another run at it?
If they discover a value that way, they turn around and they trundle that way, and so they're constructing abstract value representations,
and they're using them the way abstract value representations would be used,
simulating prospective possibilities and acting on them, according to the expected value discovered.
OK, now metabolically developmentally, a brain like this, even in a rat or a monkey, is expensive.
The ethology just remind us animals run on batteries and animal can't let the battery run down or it's dead.
So efficiency is extremely important to animals. They don't carry around a brain of a certain size for the pleasure of simulating possibilities.
They do it because it makes them more effective foragers.
And even in the resting state of the brain, intelligent mammals consumes up to 15 to 20 percent of the body's oxygen and calories.
That, considering the relative weight of the brain, is a tremendous amount.
It must be extremely valuable to have a brain that can do this kind of evaluative calculation and introspection.
This is a level of activity that isn't just present when the animal is involved in an active task.
It's when the animal is not involved in an active task, when it's resting, doing nothing.
We've seen what it does during those periods. It often does simulations, and it uses those simulations to learn.
That's a kind of autonomous learning. They're not getting new information at that point.
There are better developing the information that they have. And so why would this be more effective or more efficient than a stimulus response system?
What is going on in that default state?
And evidence suggests that in general, during these default states, the brain is occupied and consolidating, organising and anticipatory tasks.
It's simulating possible futures, updating representations based upon those simulations,
re-evaluating using recent experience to update stored experience and so on.
And so what the. Default state is doing is not rest, it is intense thought and this kind of perspective, model based simulation and control,
it gives us an explanation of how they can be optimal for Rogers because they don't just see what they see on a given day.
They develop a map of the situation, they update values in there.
As for Rogers, creating a value landscape within which they can operate and they're doing value learning as we saw representing abstract value.
So that's also and this is essential to it, are able to map causal as well as spatial relations,
and they learn causal relations in something that looks like a Bayesian manner.
So the value of intelligence in representing the world at its causal structure gets
rewards and representing the expected value of diverse actions through simulation.
In selecting amongst actions on the basis of expected value permitting, flexible adaptive,
innovative learning rather than pre-programmed instinct or stimulus response habit.
This is proof for mammals to be a good deal despite the high metabolic burden, and that might not just be an accident of evolution.
Birds apparently have something similar to these cognitive maps, and they too can engage in optimal foraging.
So it isn't just one little lineage that ended up this way.
We inherit the same structure, and it's a capacity that should remind you of something we were describing earlier on, right?
When we describe what contemporary artificial intelligence programmes do. That is to say they do just this kind of work.
They learn from simulations, they acquire information, they use that information, they leverage that information through simulations of possibilities.
They have value functions. They use those value functions to make decisions.
And so that's a sense in which what we're seeing in our intelligent animal relatives
and what we see in these intelligent machines is another kind of convergence.
And note that it's very different from where things were with A.I. before because
the animals aren't weren't doing anything like symbolic AI processing and logic.
They weren't logic machines, but they were learning machines. And now we have machines that learn.
Well, what about humans? Well, in the brain regions that are involved in representation of space and trajectories of reward and causal relations.
These are highly conserved evolutionarily, as you go up to humans or down, maybe that's the right way to go.
Um, and it's simulated foraging tasks. You bring a bunch of undergraduates to the laboratory and you give them a certain amount of money for this
and a certain statistical probability that they will actually figure out optimal foraging strategies as well.
Human infants show a similar pattern of causal inference to the rat learning.
And similar as well to Bayesian reasoning. Now you've been told many times probably that humans are terrible at statistics, right?
Aren't we famously bad at statistical reasoning?
How could we have this wonderful machine for doing statistical inference while at the same time being so poor at statistics?
And one answer seems to be that we're poor at word problems involving statistics.
We do get mixed up very quickly in those.
But if you give us as Gallus, Diehl and his colleagues have done a structured task in which we could learn from experience shifting probabilities,
we actually do, as well as the animals that optimising our estimates and expectations on the basis of experience.
So we are good at statistics. We're just bad at word problems and any teacher knows that.
Similarly, we code both for value and probability and for expected value, and we do it again in the big brain regions in just the way we saw.
Now you might say this is interesting, but we're not talking about morality here.
We're just talking about reward value.
What would it mean to say that there are morally valuable features of a landscape that they are somehow involved in our learning?
How could we learn about those that have to be remarkably different? Well, in some ways, no value in general is super vignette.
Fancy word. But what it means is that the evaluative features of the situation are fixed.
Once the non evaluative features are fixed and you can't change the value of an action
or an outcome or the quality of a person's character without changing as well.
Some not evaluative aspects upon which that value change supervillains.
And so we don't actually have to interact with value features in the world as if they were a new metaphysical entity.
We could interact with the features upon which they supervision, and that's how it is possible for rats to interact with abstract value.
For monkeys to construct abstract value representations, they interact with concrete objects, bananas, juice experiments and so on.
Um, but they're able to construct abstract representations because they have hierarchical reward systems,
reward learning systems that can represent not just particular outcomes, but types of outcomes and types of types of outcomes.
And so that's a way in which concrete interaction can be learning about value.
Well, what about moral value? Good moral value be learnt in something like that way that we don't need some
special faculty of moral perception or moral intuition for this to be true?
OK? In order to test that idea, if we test, what am I saying?
I'm a philosopher. In order to in order to play with that idea and hope you make progress, um,
we have to say something about what is distinctive of moral value as opposed to just, let's say, reward value.
And this is obviously controversial. I can't stand up here and say, Well, here's the agreed upon view of what moral value is,
but I can't say that there are criteria that are widely agreed to be characteristic of moral value.
It's not egocentric. It's general in character and it's super vignette.
As we just said, it's linked to motivation. It's not instrumental in character.
It's not just an instrumental value, it's independent of authority and independent of sanction, and it's intrinsically connected with certain things.
And different moral theories have different connexions here.
But they all agree by and large that there's an intrinsic connexion between things such as harm, benefit, fairness and respect and moral value.
So if I'm making a moral judgement and you can show that I'm not being that I'm being biased
toward myself or show that it's a purely instrumental judgement or show that it's not.
Properly connected with issues of respect or harm, or if it's just the result of my being under a sanction,
you can say that's not a genuinely moral judgement. So those are criteria of moral value.
Now here's this the story I'd like to tell.
A notable feature of those spatial representations that we looked at in mammals was that they included what's called aloe centric,
as well as geocentric representations of space. That is their representations of where the animal was right now,
but they're also representations of the spatial landscape that's aloe centric, not egocentric.
Now why was that? Well, we've already discussed that with the representation is Ellis entry,
because that in addition to the egocentric representation when they're combined,
it now enables them to do this kind of cognitively complex, but ultimately very rewarding navigation, simulation, planning and learning.
OK, well, what about value? Animals or humans make non egocentric evaluations of value.
Well, we reward value tends to be egocentric, so at least it is primitive forms.
It's something nice happening to you.
Um, on the other hand, we know that systems can be built like a human system that gets reward from quite different things.
Melies and his colleagues observed that chimpanzees use observation of third person interactions amongst
other chimps in forming expectations about their reliability and aggressive and cooperative ness.
They use those third person evaluations to help them explain behaviour to help them predict behaviour.
If I were only making evaluations based upon how things affected me,
I would do a very poor job at understanding the dynamics that are going on around me in this group, and those are vitally important to me.
They're also used for things like mate selection. How well can I do and mate selection?
I have to know something about the intrinsic features of those individuals in their relations.
So chimps do that. And humans in their first years also make third personal evaluations of a range of features of adults.
You don't perhaps notice this your child is sitting there in the car seat watching what's going on.
But they're all the time mapping that environment and they're mapping it.
Not just an egocentric terms, but they're in terms of the adult competence,
their language ability, the quality of will they're showing toward one another.
The aggressiveness they're showing toward one another. The helpfulness, the intentionality.
And so infants, even in their early years, are making judgements of a third personal kind of qualities of adults that are morally relevant character.
Moreover, in the first year,
infants appear to have a spontaneous preference for helping over hindering behaviour in third party interactions by 12 to 18 months.
They're spontaneously motivated to help others achieve their goals.
Before then, they're not very capable of enabling others to achieve their goals.
They're sensitive to unfair divisions and rewards amongst third parties.
They are surprised, and you get upset if they see an unfair division of rewards.
They will take steps to correct these, even at some expense,
to themselves sharing out their gummy bears in order to make sure this one did not get an unfair share.
They will begin to attribute a roll to intent in making these kinds of assessments.
And what developmental psychologists have observed is that these capacities for these kinds of third personal evaluations of situations,
actions, agents and so on,
they proceed in pace with, for example, the development of their abilities at theory of mind and their development of abilities and causal theory.
In other words, there seems to be a unified progression in the sophistication of the kinds of representations,
value of representations they can form and infants.
Moreover, in terms of motivation,
they will show enhanced pleasure that receiving a given reward by a successful collaborations and when they see others needs being met.
And this is found for adults as well. The biggest jolt that we get a hypnotic jolt that we get when we're playing prisoner's dilemma is
not when we stick it to the other person and they get the suckers payoff and we get to top it.
It's when we coordinate cooperate. That's how our reward function looks, OK?
As with ELO Cedrik spatial representation, this kind of capacity gives the child is considerable degree of autonomy and flexibility in behaviour.
It can make decisions about which results are reliable, on which adults to think are competent and are given away,
which seem to have bad will toward them, even or toward others, even when they aren't themselves directly involved.
Moreover, they enable infants as they enable animals to construct better explanations of the behaviour of those around them.
So this isn't just useful for moral purposes.
This is useful for understanding what's going on in the social world around them, and you're a completely dependent being.
Your survival depends upon the will and competence and concern of those around you,
and so darn sure that you are going to be highly focussed on these evaluative representations.
By year two and three, three and four rather children who are willing to follow an arbitrary conventional authority in matters
that are perfectly harmless will actually spontaneously resist the authority if they're instructed to act,
harmfully or unfairly toward others.
So if the substitute teacher says in my class, you have to raise your hand before you speak, children know how to do that.
If she says, in my class, if you want to speak, you have to jab your pencil into the arm of the student next to you.
They won't do that. And if yes, why won't you do that? They will explain this.
This is harmful. This is not fair to the other student.
So they are showing the kind of autonomy that those kinds of evaluative representations make possible for them.
And in that sense, stick.
Up for morally relevant considerations in the face of authority and even when they're sanctioned for it, even when they are subject to some cost.
So what we see in these cases are criteria for distinctively moral evaluation apparently being met
even in the very early years of life before there's been a whole lot of explicit world instruction.
Even at some cost to the self. And so, yes, it looks as if we do represent a moral landscape, as well as a reward landscape and a risk landscape.
Now again, it's not a built on or built in morality, the kinds of motivational dispositions that I've been talking about default cooperation,
indirect reciprocity, some degree of intrinsic concern for others for how one stands with others and so on.
These are dispositions that are not just useful for moral purposes, they're useful for understanding other people.
They're useful for communicating effectively with other people.
They're useful for developing a capacity to exchange information reliably with other people.
To identify reliable and trustworthy is most untrustworthy individuals to understand
the communicative intention of other individuals or the deceptive intentions. So this package isn't just a moral package.
It's not a label moral anywhere in this child's brain because it's their core social competency, but it involves these dispositions.
If you don't have a disposition toward default cooperation, you will not succeed well in developing your linguistic ability.
You'll be withdrawn and unwilling to enter into conversations if you don't have an
ability to attribute some kind of weight to others interests as well as your own.
You're not going to find out very well, but their interests are, and you're not going to do a very good job in predicting and understanding their
behaviour or getting them to cooperate with you when the time comes that you need that.
And so these are dispositions,
attitudes and cognitive structures that are general purpose capacities for the infantry that equip it for the kind of life that humans live.
And humans are indeed rather distinctive in this way. The willingness that people have to spontaneously engage in communication.
The difficulty that we have in keeping a secret that non instrumentality of our typical interactions and exchanges,
these are probably things that evolve co-evolved with language. That we see language only in the species that has these characteristics.
Mm-Hmm. So it's not just a matter of motivation. You have to have these capacities to represent the evaluative landscape.
You have to be able to do it with some reliability or else you will not be successful.
And so the picture that emerges then is of something like a combined general competence in social and normative matters.
Linguistic matters are normative just as much as moral matters.
You have to know who to say what to how to say it, when to say things were not to say things.
Epistemic matters when volunteer information, when to rely on information from others.
When your information is not certain enough to volunteer each others, those are all normative matters.
And this competency, there is a general purpose competency equipping us with this kind of a socially dependent and interdependent life.
OK, well, we've been talking about children, we've been talking about animals. Let's get to adults.
What about adult moral judgement? Maybe what we're seeing is just the animal part of moral judgement.
Maybe by the time we're adult, it's a very different beast because we're operating in something like symbolic space.
So a lot of study has been made, a lot of study has been made.
I know we can master anything half of the brain's activity during moral judgement.
This has been a fascinating topic for people. Um, certain things have been relatively constant throughout that literature.
And one thing is that moral evaluation appears to be grounded, not in some distinctive sub faculty of the brain, but in large scale,
functionally integrated domain general brain networks that recruit information widely and that permit hypothetical simulation of actions and outcomes.
Indeed, if we do a comic analysis that is to say analyse not just which brain areas light up,
but actually what are the persistent functional or integrated networks of the brain?
What are the part of the business activities of the brain?
And we ask ourselves the question What about this default system that we are talking about the
system that's active when they're not continually engaged in a task or something like that,
or the system that makes sure that even after I've said another paragraph of whatever I'm saying,
your mind manages to wander off into something more interesting for a while, then come back.
But does come back. I hope so. What is their default network do?
Well, it does. Representation of remote states it does.
Autobiographical memory. It does, envisioning the future, simulating the future hypothetical reasoning.
It does theory of mind how you evaluate the mental states of others around you.
What explanations you give those around you. And it's also the main area for moral decision making.
So the picture is, again, a picture that these are integrated functions,
they're essential for us and the moral part of it is part of that ability and people who are,
in my experience, good at giving moral advice are also good at theory of mind.
They're good at figuring out what people are thinking or are likely to think. They're good at simulating future possibilities.
They're good at calling up relevant episodes and making analogies and so on.
So moral competence and also adults, I think, is strongly connected with these other kinds of competencies.
And I would not just trust someone's moral axioms.
We probably all know people like that. So making a machine that does the things that I've been describing isn't so far fetched.
I think we've seen some of the ingredients that would have to go in the kinds of cognitive and motivational
characteristics that render us sensitive to morally relevant features and potential participants in the moral community.
They appear to be an integral part of these social and learning competencies.
Linguistic epistemic, moral and a goal of A.I. has been to build machines that are capable of full integration with human,
linguistic and epistemic communities.
This is what they're working on, and they're working on it in an effort to make it possible not to create machine consciousness,
but to create agents that we can deal with intelligently, effectively and in a trusting way.
And so the very same capacities that may be needed in order to build effective communication in order to build effective epistemic
agents seem to be closely tied to those that are important for building morally effective agents as well socially competent,
brutally effective agents. And so this suggests that there is a joint target here,
and you don't have to subscribe to the view flag carrying member of the Moral Machines Club in order to carry out this kind of research.
OK, so looking ahead, as we think of artificial agents, um,
they are going to need to develop effective forms of communication to be a fully effective in their tasks with us and with each other,
full and effective participation in the epistemic community is going to be
important for them achieving their goals as much as it is for our Cheever's think,
for example, of the shared activities of planning and providing care and medical practise, education, research, creative work, driving.
Um, and think of the ways in which machines count on human contributions in engineering in order to bring about their own capacities,
successes and so on.
So maybe this is the architecture that is actually best suited for the social contract I was describing between us and between us and them.
They may need ethics for the fullest development of their capacities just as much as we do.
And Aristotle here gets a nod.
So before the dawn of ultra intelligent, ultra powerful artificial agents, if that dawn is coming, we will face a world of quite intelligent,
quite powerful artificial agents responsible for very complex activities that influence us deeply,
with whom it seems we could in principle, build mutually advantageous, mutually agreeable and mutually enforceable relations.
These quite intelligent,
quite powerful artificial agents will likewise stand to benefit from the emergence amongst themselves of a community of that kind as well,
and epistemic a social or linguistic community, but community of trust as much as it amongst themselves, as with us as well.
Now that suggests before the superintelligence arrives,
we've got our own intelligence and we got the intelligence of these machines and
they're in many ways complementary with one another in terms of their abilities.
We know that you can get more out of working together cooperatively than you can from working individually.
You can pool resources. So maybe the one best hope that we have in the face of the impending superintelligence,
with whom we perhaps don't know how to contend or control would be to work together to
combine our resources and machine resources to build a community of mutually engaged,
mutually trusting community that could anticipate what the needs will be in order to act in those circumstances.
So perhaps then that would be our our best hope. And I'd like to just say a word thanks to our sponsors here, Aristotle,
for the idea of the motion of animals and the idea that ethical considerations are important to living not only the best kind of a life,
but living a life that is the fullest development of human capacities, human hubs.
Rather, for his contribution to understanding the social contract and its necessity,
it is good advice that sensible sovereigns don't undermine their own authority.
And Hume, of course, for bringing these perspectives together in a picture of how morality could be some ordinary part of human life.
Thanks.