1
00:00:05,710 --> 00:00:11,890
All right. Well, good evening, everybody.

2
00:00:11,890 --> 00:00:18,880
Thank you very much for your patience and thanks in particular, Peter, for his.

3
00:00:18,880 --> 00:00:30,730
Which has been remarkable. My name is Roger Crisp, I'm the chair of the management committee at the Oxford Uehiro centre for practical ethics,

4
00:00:30,730 --> 00:00:41,010
and I'm here to represent the Uehiro professor in practical ethics, Julian Savulescu

5
00:00:41,010 --> 00:00:47,820
The Uehiro centre if you're not familiar with it, has a website.

6
00:00:47,820 --> 00:00:53,910
And I'd like to ask those of you who are unfamiliar with this to take a look at it and draw your attention,

7
00:00:53,910 --> 00:01:03,510
in particular to bite size ethics programme, which is coming up and also our festival of ideas.

8
00:01:03,510 --> 00:01:15,450
The Uehiro chair was set up with a very generous donation from the Foundation on Ethics and Education in 2002,

9
00:01:15,450 --> 00:01:21,490
and one year later the centre was founded.

10
00:01:21,490 --> 00:01:33,760
And since 2004, the foundation has very generously sponsored 17 series of Uehiro lectures in practical ethics,

11
00:01:33,760 --> 00:01:39,460
many of which have also been published later as books with OUP.

12
00:01:39,460 --> 00:01:46,210
So we're very grateful to the foundation for making that possible.

13
00:01:46,210 --> 00:01:57,040
And I'm delighted this evening to welcome our 18th lecturer, Professor Peter Railton, who will be speaking on ethics and artificial intelligence.

14
00:01:57,040 --> 00:02:06,640
And there'll be two more lectures, I hope, starting at five o'clock, um, next week and in the following week here.

15
00:02:06,640 --> 00:02:17,320
And I hope you'll be able to come to those. The lectures will last for about an hour and then we'll have until 7:00 for questions and discussion.

16
00:02:17,320 --> 00:02:27,850
And this week we'll be holding a drinks reception. Peter will have his own personal bottle of whisky, um, purchased by this by the centre,

17
00:02:27,850 --> 00:02:35,140
and we would very much like you to, uh, attend that, if you can.

18
00:02:35,140 --> 00:02:46,450
Now, Peter Railton needs no introduction, only in part because the memories of his outstanding lectures here in 2018 are still fresh in our minds.

19
00:02:46,450 --> 00:02:57,740
He is the Kafka distinguished professor. Um, the parent professor and the third, AH, professor at the University of Michigan.

20
00:02:57,740 --> 00:03:07,630
His research is covered, in addition, of course, to ethics, political philosophy, philosophy of science and philosophy of mind.

21
00:03:07,630 --> 00:03:16,450
And recently, he's been engaged in joint projects with researchers in psychology, cognitive science and neuroscience.

22
00:03:16,450 --> 00:03:21,930
Amongst his works are facts, values and norms.

23
00:03:21,930 --> 00:03:28,800
And Homo prospectus, which was jointly written with Martin Seligman and others,

24
00:03:28,800 --> 00:03:36,360
he's a member of the American Academy of Arts and Sciences and the Norwegian Academy of Sciences in letters.

25
00:03:36,360 --> 00:03:40,470
He served as president of the American Philosophical Society,

26
00:03:40,470 --> 00:03:50,970
Central Division and Child Fellowships from the Guggenheim Foundation, the American Council of Learnt Societies and the NIH.

27
00:03:50,970 --> 00:04:02,680
And it's great pleasure now to invite him to give his first lecture. Peter, thank you.

28
00:04:02,680 --> 00:04:13,990
Well, thanks to the Uehiro Centre thanks to the Hart Foundation, which is putting me up very nicely here, thanks to the audience for your patience.

29
00:04:13,990 --> 00:04:18,610
You'll have more chance to exercise your patience as the evening wears on.

30
00:04:18,610 --> 00:04:29,530
And I also want to say a special thanks because I really don't do much practical ethics and this is a lecture series and practical ethics.

31
00:04:29,530 --> 00:04:34,150
And so they trusted me actually to come here and talk to you about practical ethics.

32
00:04:34,150 --> 00:04:38,530
And so I found that encouraging, if a bit daunting.

33
00:04:38,530 --> 00:04:43,950
So thinking about practical ethics, I couldn't think of a more.

34
00:04:43,950 --> 00:04:48,900
Compelling problem, really, than thinking about the question of how artificial intelligence is affecting the

35
00:04:48,900 --> 00:04:54,990
way that we might live and what philosophy of morals might contribute to that.

36
00:04:54,990 --> 00:05:02,970
My only excuse for taking on this question, given that I don't really have tremendous expertise in the areas I would have to be expert

37
00:05:02,970 --> 00:05:07,530
in is that I think the question is quite urgent and I think we should be working on it.

38
00:05:07,530 --> 00:05:10,290
So I would encourage all of you to begin to think about working on it.

39
00:05:10,290 --> 00:05:17,670
And maybe something I say today will stimulate you to want to disagree and carry on the conversation.

40
00:05:17,670 --> 00:05:25,080
I do hope that this discussion will be kind of an open session in which people can contribute their own views about this issue.

41
00:05:25,080 --> 00:05:30,750
Also, there's another connexion, which is that we are right now in the midst of two learning revolutions.

42
00:05:30,750 --> 00:05:36,630
One is in AI and the other is in psychology and neuroscience.

43
00:05:36,630 --> 00:05:40,650
And so that suggests that there might be some kind of a confluence.

44
00:05:40,650 --> 00:05:48,810
Moreover, I've been working for quite some time now on the question of moral learning, not with very much company.

45
00:05:48,810 --> 00:05:58,480
And I think that perhaps learning based perspective might help connect us with that remarkable confluence in the two areas.

46
00:05:58,480 --> 00:06:05,080
So the voice of reason intervenes at this point and says, really,

47
00:06:05,080 --> 00:06:10,600
haven't we all been through A.I. booms and busts in the past instead of changing the world?

48
00:06:10,600 --> 00:06:17,110
When the dust settled and all the gross oversimplification about hyper agents and so on is cleared away,

49
00:06:17,110 --> 00:06:23,500
there's really very little that's changed is just a bunch of personification of machines, and they're not very sophisticated machines.

50
00:06:23,500 --> 00:06:28,360
So there's really no need for moral philosophers in particular to buy into the hype of this.

51
00:06:28,360 --> 00:06:33,940
So it's a fool's errand. And that's what I'm here on a fool's errand.

52
00:06:33,940 --> 00:06:37,900
There have been artificial intelligence booms, and bust in the past is important to remind ourselves of that.

53
00:06:37,900 --> 00:06:45,130
That peak is around 1980 or so, and you can see that up until recent years, we were quite a bit down from that peak.

54
00:06:45,130 --> 00:06:47,230
There was a so-called A.I. winter.

55
00:06:47,230 --> 00:06:55,840
And so this is this is not new, this hype around artificial intelligence, but it does seem to be kind of different this time,

56
00:06:55,840 --> 00:07:01,720
most especially because the systems are having success at various tasks that they weren't expected to have.

57
00:07:01,720 --> 00:07:11,770
Not this quickly. Image identification, speech recognition, translation, scientific prediction, motor control, strategic game playing.

58
00:07:11,770 --> 00:07:17,170
This has happened rather quickly and more quickly than is comfortable for me at any rate.

59
00:07:17,170 --> 00:07:20,770
But don't take my word for it. So here's the number of scientific papers.

60
00:07:20,770 --> 00:07:29,650
So you can see that in the last A.I. boom around 1980, there was a certain level of publication that is the current level of publication.

61
00:07:29,650 --> 00:07:38,620
What about people who bet their money on this? Here was venture capital and A.I. initiatives fifteen two hundred two thousand eleven in 2019.

62
00:07:38,620 --> 00:07:44,950
It was rising very nicely, but here's what it was after to 2020.

63
00:07:44,950 --> 00:07:50,110
And if you keep an eye on that green box, you'll see where it was the following year 2021.

64
00:07:50,110 --> 00:07:57,880
That is a qualitative change. So somebody thinks there's something going on here and they think that there's something they can draw upon.

65
00:07:57,880 --> 00:08:01,420
Now again, the voice of Reason is going to say, Well, I'm not worried about that hype.

66
00:08:01,420 --> 00:08:09,130
I mean, of course, there are all these corporations that want to make money on A.I., and that's all about sort of these specific tasks very specified.

67
00:08:09,130 --> 00:08:16,930
Somebody wants to pay those to have them done. And so there are a lot of people pouring into the field maybe doing so to get tenure.

68
00:08:16,930 --> 00:08:20,920
What the hype is really about the problem hype is that there's all this

69
00:08:20,920 --> 00:08:27,820
personification of these agents or talk about superintelligence or superhuman agents.

70
00:08:27,820 --> 00:08:32,710
And that has really nothing to do with what these systems are like as they now stand.

71
00:08:32,710 --> 00:08:36,550
But what I'd like to try to do is take seriously what what has been recently achieved

72
00:08:36,550 --> 00:08:43,180
in AI and what in the medium term might be likely to happen and try to see whether

73
00:08:43,180 --> 00:08:49,210
there's a relevant way of tackling the ethical problems raised by A.I. that comes

74
00:08:49,210 --> 00:08:53,680
from the distinctive character of this revolution in artificial intelligence.

75
00:08:53,680 --> 00:08:58,420
So there really is no shortage of problems for ethics and AI.

76
00:08:58,420 --> 00:09:04,570
This the list would actually take me most of the session, but issues of privacy, manipulation,

77
00:09:04,570 --> 00:09:09,670
repression, we've seen these very dramatically worries about economic displacement,

78
00:09:09,670 --> 00:09:19,330
dislocation, the loss of skill, worries about opacity and bias that systems, given poor data sets, will indeed render poor and biased decisions.

79
00:09:19,330 --> 00:09:26,260
The problems of reinforcing inequalities. The haves will be able to reinforce their position yet further worries at the

80
00:09:26,260 --> 00:09:30,460
horizon of superintelligence and the ultimate control problem of superintelligence.

81
00:09:30,460 --> 00:09:34,990
And are we putting ourselves in existential risk by following this path?

82
00:09:34,990 --> 00:09:39,850
And then there's the problem that I call the not so super machine intelligence problem,

83
00:09:39,850 --> 00:09:46,720
which is interesting A.I. systems with as yet inappropriate tasks and degrees of independence,

84
00:09:46,720 --> 00:09:50,770
but not doing that just from human mercenary ness, which might be bad enough,

85
00:09:50,770 --> 00:09:56,950
but from human stupidity, error and inadvertent things can happen inadvertently.

86
00:09:56,950 --> 00:10:02,980
See GPT three learn something about computer programming from doing text analysis.

87
00:10:02,980 --> 00:10:07,930
That's a little bit worrying if a system learn something about programming because it's based on programming.

88
00:10:07,930 --> 00:10:12,340
Anyhow, a number of these challenges are pressing right now.

89
00:10:12,340 --> 00:10:17,980
They call for various kinds of organised response morally, politically, socially and so on.

90
00:10:17,980 --> 00:10:22,070
But the lectures that I'm going to be giving are mostly focussed on that question.

91
00:10:22,070 --> 00:10:29,980
Six. The question of not so Superintelligence, because I think it might be critical in helping us to deal with the first five.

92
00:10:29,980 --> 00:10:37,390
And the idea that I'm going to try to persuade you is worth considering is the idea of making it

93
00:10:37,390 --> 00:10:45,100
the case that artificially intelligent systems can have some degree of endogenous self-regulation.

94
00:10:45,100 --> 00:10:53,230
Well, what kind of regulation? Not anything like a full understanding or moral agency, not something like that,

95
00:10:53,230 --> 00:10:58,000
but a kind of a competency competence with morally relevant dimensions of.

96
00:10:58,000 --> 00:11:05,370
Tasks and situations, and this would be have to be a reliable capacity to detect morally relevant features

97
00:11:05,370 --> 00:11:11,140
of situations and actions and to respond aptly to those in functional terms.

98
00:11:11,140 --> 00:11:15,180
Now what do I mean by functional? Here's a rough comparison.

99
00:11:15,180 --> 00:11:22,920
So we aren't asking these machines right now to have anything like full linguistic comprehension and understanding,

100
00:11:22,920 --> 00:11:31,920
but we are looking for them to be fluent in conversation, able to detect the intentions of speakers, whether those are communicative or deceptive.

101
00:11:31,920 --> 00:11:39,750
The capacity to exchange an open ended array of information effectively to apply that information to select relevant responses,

102
00:11:39,750 --> 00:11:47,040
actions or queries, including queries that might be addressed to us as humans and monitoring the outcomes of such behaviour.

103
00:11:47,040 --> 00:11:53,040
Self-monitoring the outcomes of such behaviour and self-adjusting in response. So that's a functional capacity.

104
00:11:53,040 --> 00:11:56,760
This is not full understanding. It's not full linguistic agency.

105
00:11:56,760 --> 00:12:00,690
Neither am I looking for full moral agency or a full moral understanding.

106
00:12:00,690 --> 00:12:07,220
But systems with this degree of competence can do a surprising number of things as we are discovering daily.

107
00:12:07,220 --> 00:12:13,230
OK, artificial intelligent systems are increasingly not just systems, but agents.

108
00:12:13,230 --> 00:12:18,090
And from an engineering standpoint, what is an agent? Again, I'm not.

109
00:12:18,090 --> 00:12:21,180
I'm going to try not to personify here.

110
00:12:21,180 --> 00:12:30,000
This is a well-established term in the field, and an agent in the engineering terms is a system that has goal like states,

111
00:12:30,000 --> 00:12:34,590
something like a utility function or a value function belief like states,

112
00:12:34,590 --> 00:12:41,250
something like assignments of probabilities or distributions of probabilities to state of affairs and ability to generate,

113
00:12:41,250 --> 00:12:49,020
simulate and evaluate alternative courses of action in light of those two elements goals and belief like states.

114
00:12:49,020 --> 00:12:59,550
And then an ability to select actions in light of that evaluation to plan and then carry them out while monitoring outcomes and adjusting.

115
00:12:59,550 --> 00:13:06,600
So there is a continuous feedback and adjusting the goal like and belief like states to what happened last time around.

116
00:13:06,600 --> 00:13:10,260
That's a form of reinforcement learning that we're familiar with,

117
00:13:10,260 --> 00:13:16,950
but it's also a form of agency and it's a potentially intelligent agency, at least in a broad sense.

118
00:13:16,950 --> 00:13:25,710
Namely, if it's capable of using incoming information and memory to identify and solve a range of problems in an array of circumstances,

119
00:13:25,710 --> 00:13:29,640
including novel circumstances, and achieve a range of goals.

120
00:13:29,640 --> 00:13:33,900
So that's all I'm going to mean by intelligent. That's what I'm going to mean by agent.

121
00:13:33,900 --> 00:13:42,420
And my idea is that these are actual agents and they're starting to people the world, and we, as ethicists ought to be thinking about that.

122
00:13:42,420 --> 00:13:49,080
Increasingly, such agents are being entrusted with tasks in which they must operate semi or fully autonomously

123
00:13:49,080 --> 00:13:56,190
driving a car or a truck serving as a home health aide or a companion operating a weapon.

124
00:13:56,190 --> 00:14:00,540
Culling applications, monitoring and regulating networks in real time.

125
00:14:00,540 --> 00:14:05,130
They might be informational networks, energy, finance, research, distribution and so on.

126
00:14:05,130 --> 00:14:10,560
And these can have very wide effects on well-being, not just the well-being of those local,

127
00:14:10,560 --> 00:14:16,680
but the well-being of those around the world when you can have a global financial crisis triggered by this kind of agency.

128
00:14:16,680 --> 00:14:20,970
So we should be asking. We're entrusting these agents with this kind of behaviour.

129
00:14:20,970 --> 00:14:24,600
We're doing it partly because they're so quick and capable.

130
00:14:24,600 --> 00:14:33,930
But are we asking ourselves the question under what sense or in what sense might they also be sensitive to morally relevant features of situations?

131
00:14:33,930 --> 00:14:41,100
So that's my question. Now we wouldn't entrust these tasks or narrowly or we wouldn't have until recently

132
00:14:41,100 --> 00:14:45,540
the two agents that had no capacity to be sensitive to moral considerations.

133
00:14:45,540 --> 00:14:48,900
Agents not only were good at pursuing certain design goals,

134
00:14:48,900 --> 00:14:54,420
but could do so in a way that was responsive, aptly responsive to morally relevant features,

135
00:14:54,420 --> 00:15:01,980
even those that are not part of the specified task features that arise from performing the task but are part of the task design.

136
00:15:01,980 --> 00:15:12,510
It has to be alive to such features. Such features could be at harms, benefits, risks, questions of consent, fairness, deception and so on.

137
00:15:12,510 --> 00:15:16,320
And I'm not intending this list to reflect any particular ethical theory.

138
00:15:16,320 --> 00:15:20,400
I have my own views and ethical theory, but I don't want to rely on those here.

139
00:15:20,400 --> 00:15:25,950
These are areas that are part of a large common core of consensus amongst most ethical theories.

140
00:15:25,950 --> 00:15:32,010
And so the question is given that we can agree on some of these features and agree on the ways in which they're going

141
00:15:32,010 --> 00:15:37,530
to be important for the operation of agents that are going to have this much influence on the events in our world.

142
00:15:37,530 --> 00:15:42,510
How could we not just as an add on because add ons don't tend to work,

143
00:15:42,510 --> 00:15:48,990
but as part of the fundamental architecture of these systems have something that looked like sensitivity to morally relevant considerations.

144
00:15:48,990 --> 00:15:55,320
Now you might think, well, the thing to do is to hardwire them with some moral principles that would do it.

145
00:15:55,320 --> 00:16:00,630
People are familiar. I think with Isaac, I ask you. Most famous three laws for robots.

146
00:16:00,630 --> 00:16:06,480
First Law, a robot may not enjoy a human being or through inaction, allow a human being to come to harm.

147
00:16:06,480 --> 00:16:13,980
Second, a robot must obey orders given to it by a human being, except where such orders would conflict with the first law.

148
00:16:13,980 --> 00:16:20,610
And third, a robot must protect its own existence. As long as such, protection does not conflict with the first or second law.

149
00:16:20,610 --> 00:16:27,750
These are very, very cleverly designed, I think, and insightful in many ways, but they couldn't possibly work.

150
00:16:27,750 --> 00:16:36,000
Even the first law couldn't be followed. Should your home robot let you leave the house and get in your car enjoying traffic?

151
00:16:36,000 --> 00:16:39,220
Well, that could put you in harm should it confine you to the home.

152
00:16:39,220 --> 00:16:44,490
Well, that's harmful as well. Who knows what harm might befall you within the home?

153
00:16:44,490 --> 00:16:47,040
What about that medication? Should you take that medication or not?

154
00:16:47,040 --> 00:16:51,580
Well, there are some who take it and have benefits, and some who take it have harms.

155
00:16:51,580 --> 00:16:56,640
It's going to be a matter of not allowing humans to come to harm, either through action or inaction.

156
00:16:56,640 --> 00:17:00,930
There'd be no way of following this axiom and then no way really of following the others.

157
00:17:00,930 --> 00:17:04,380
So what alternatives are there?

158
00:17:04,380 --> 00:17:10,590
Well, the first thing to notice is that the problem is not just about the writing of the axioms, it's not writing better axioms.

159
00:17:10,590 --> 00:17:17,550
Of course, we don't know how to write very good moral axioms. Long as the time is that we spent working on that.

160
00:17:17,550 --> 00:17:22,140
But the important thing here is that artificial systems, in order to follow any principles,

161
00:17:22,140 --> 00:17:29,310
would have to be able to detect situations where the principles apply. We'd have to assess what the application would be.

162
00:17:29,310 --> 00:17:35,310
We'd have to consider and evaluate candidate actions in the situation for meeting the principles.

163
00:17:35,310 --> 00:17:42,510
We'd have to select and carry out appropriate actions and context, monitor success or failure, adjust response accordingly.

164
00:17:42,510 --> 00:17:49,050
All of that is it accomplished by an axiom. It's like giving me the grammar to finish.

165
00:17:49,050 --> 00:17:56,010
I could be given the grammar to finish and maybe even a very good one. But that wouldn't enable me to use finish in any meaningful sense,

166
00:17:56,010 --> 00:18:03,240
because all of these other features identifying meanings, deciding what responses are appropriate.

167
00:18:03,240 --> 00:18:08,080
Finding my way to get my mouth around the words whatever it is, I would not have those capacities.

168
00:18:08,080 --> 00:18:18,180
So what is interesting in this connexion is that this kind of competency is extremely difficult to imagine designing.

169
00:18:18,180 --> 00:18:22,530
Let me think about the full range of actions in the full range of applications.

170
00:18:22,530 --> 00:18:30,960
But there are ways of acquiring this competence that aren't by design, and that's what the current revolution in artificial intelligence is about.

171
00:18:30,960 --> 00:18:38,310
So we have no clear idea of how we could specify this. Now you might say, maybe we could crowdsource morality.

172
00:18:38,310 --> 00:18:46,170
If you have the equivalent of an internet recommendation system that would generate very large databases and corresponding scores for actions.

173
00:18:46,170 --> 00:18:49,950
And so if you confront yourself as an agent with a difficult situation,

174
00:18:49,950 --> 00:18:55,410
you simply enter that into the database and you see what the score is for any given particular response.

175
00:18:55,410 --> 00:19:02,040
Now, it would be very valuable to have a model of human commonsense moral intuitions.

176
00:19:02,040 --> 00:19:06,660
We don't have a good one yet. We learnt a great deal from that.

177
00:19:06,660 --> 00:19:14,270
But do we really think we would want to entrust these tasks to the equivalent of an internet recommendation system?

178
00:19:14,270 --> 00:19:19,070
Have you used internet recommendation systems? Yeah.

179
00:19:19,070 --> 00:19:22,760
And we still would have to match the actual circumstances and choices in a

180
00:19:22,760 --> 00:19:26,900
given state setting to the features that we could use to query the database,

181
00:19:26,900 --> 00:19:31,910
and we don't yet have a way to do that. And we don't have a specification for it.

182
00:19:31,910 --> 00:19:36,140
Perhaps we could say, well, just design the agents to detect and work,

183
00:19:36,140 --> 00:19:43,070
to satisfy our preferences so that they're aligned with our preferences and we will get what we want from these agents.

184
00:19:43,070 --> 00:19:52,250
Now that alignment could be local or it could be global. If it's local, then agents entrusted with these tasks, it seems to me,

185
00:19:52,250 --> 00:19:56,810
will sometimes have to have sufficient independence from the preferences of those around them

186
00:19:56,810 --> 00:20:02,960
to be able to counter Vale against them and to do so on behalf of morally relevant features.

187
00:20:02,960 --> 00:20:09,710
A home health companion who sees the patient in question,

188
00:20:09,710 --> 00:20:19,160
suffering from what it can detect as signs of depression and sees that same agent not wanting to take the medication that's been prescribed,

189
00:20:19,160 --> 00:20:26,810
that agent has to have enough independence to actually be a home health companion and to resist that particular preference.

190
00:20:26,810 --> 00:20:31,960
Now, if it's global. Well, then we have the crowdsourcing problem again.

191
00:20:31,960 --> 00:20:38,980
Global preferences, again, we need some critical distance on the problems and to get that,

192
00:20:38,980 --> 00:20:45,070
we have to say that these agents will get some critical distance on the situations and the problems,

193
00:20:45,070 --> 00:20:52,840
just as we allow human agents to have some degree of critical distance and autonomy and count on them and trust them because they have that.

194
00:20:52,840 --> 00:20:57,670
Now, once you admit this idea of autonomy for such agents,

195
00:20:57,670 --> 00:21:03,460
you're in very dangerous territory because autonomy means that they will resist

196
00:21:03,460 --> 00:21:09,870
some of our efforts to control them or make them do what we want to do. And that seems like a spooky thing.

197
00:21:09,870 --> 00:21:16,030
On the other hand, if they are designed so that they can't resist our efforts to get them to do what we want to do.

198
00:21:16,030 --> 00:21:22,870
I find that a very spooky thing, given what we know about human intentions and their distributions around the world.

199
00:21:22,870 --> 00:21:28,030
If these agents are going to become, as they will, increasingly central features of our lives.

200
00:21:28,030 --> 00:21:30,880
My guess is that they're not only going to raise issues,

201
00:21:30,880 --> 00:21:36,400
but they're going to have to be part of any solution to the problem of the issues that they raise.

202
00:21:36,400 --> 00:21:43,180
Because the complexities involved, the interactions that are involved are going to be such that we can't do this on our own.

203
00:21:43,180 --> 00:21:49,180
So we will actually need the help of AI systems in order to cope with A.I. systems.

204
00:21:49,180 --> 00:21:56,080
And so we shouldn't design such systems that we can just command them around or decide what they should think, believe or want.

205
00:21:56,080 --> 00:22:00,970
They will sometimes have to enlighten us, as Kasparov said.

206
00:22:00,970 --> 00:22:07,060
Now it is the machine who is the expert on chess now.

207
00:22:07,060 --> 00:22:14,560
How can we do this? Is this a silly thought? Like what would it mean to live on sort of mutually beneficial terms with these agents

208
00:22:14,560 --> 00:22:20,020
in ways that are responsive to morally relevant features of situations and actions?

209
00:22:20,020 --> 00:22:25,090
Well, it would be like being on the highway with a lot of autonomous vehicles around because that's what they are.

210
00:22:25,090 --> 00:22:30,550
They're autonomous agents zipping around, making decisions about how the litter act with you.

211
00:22:30,550 --> 00:22:31,330
And the question is,

212
00:22:31,330 --> 00:22:40,540
under what conditions is that trustworthy and under what conditions is that morally appropriate situation to put people or machines in?

213
00:22:40,540 --> 00:22:49,540
So we're already in the midst of this kind of a situation now that has the form of a classic social contract problem.

214
00:22:49,540 --> 00:22:58,750
Can we find mutually beneficial, mutually agreeable and mutually enforceable terms of cooperation and distribution of benefits?

215
00:22:58,750 --> 00:23:01,780
That's the problem with the social contract.

216
00:23:01,780 --> 00:23:09,670
Now that would be a problem we will have with regard to agents, but it's also a problem that increasingly they will have with regard to themselves.

217
00:23:09,670 --> 00:23:14,590
There's not one agent. There are countless. It will be countless more agents.

218
00:23:14,590 --> 00:23:19,540
They will face their same, the same version of all of these problems that we face.

219
00:23:19,540 --> 00:23:23,410
They will need a social contract type solution as much as we do so that their

220
00:23:23,410 --> 00:23:28,390
interactions become mutually beneficial rather than destructive and self-defeating.

221
00:23:28,390 --> 00:23:33,700
They will need ethics as much as we need ethics and for similar, essentially similar reasons.

222
00:23:33,700 --> 00:23:40,780
They are agents, after all, and agents need ethics for a bundle of well-known reasons.

223
00:23:40,780 --> 00:23:46,300
Now the question is then if if that's the problem that we're facing.

224
00:23:46,300 --> 00:23:52,360
What would it look like to actually think about how such morally relevant features might emerge from such interactions?

225
00:23:52,360 --> 00:24:01,510
What would be needed? So we know that agents can typically achieve more given their information, their resources,

226
00:24:01,510 --> 00:24:07,420
their power, if they can deploy those capacities through working together in some ways.

227
00:24:07,420 --> 00:24:16,660
And in order to do that, they need to have a way of sharing out the results rather than just fighting one another over the resources and domination.

228
00:24:16,660 --> 00:24:23,570
Those of you who know Tom Fellows work know the the tragedy of chimps in a way.

229
00:24:23,570 --> 00:24:28,840
There's a game that chimps can play where each has to pull on the end of a board to bring food to them.

230
00:24:28,840 --> 00:24:37,390
And if you put the food in two separate dishes, then the chimps can solve the problem because the dominant chimp is far enough away from the

231
00:24:37,390 --> 00:24:43,120
subordinate chimp that the subordinate gets a chance to eat before the dominant chimp comes over.

232
00:24:43,120 --> 00:24:46,730
If you put the food in a tray between them.

233
00:24:46,730 --> 00:24:55,030
Damage just comes over and bats the subordinate away, and the subordinate chimp never cooperates in the future, because why would you do that?

234
00:24:55,030 --> 00:25:02,050
Now that's not a hard problem in some respects, but it's a problem that it doesn't seem they're entirely equipped to solve.

235
00:25:02,050 --> 00:25:08,230
And so this is what Hubbs noticed. You can achieve things together that you couldn't achieve simply singly.

236
00:25:08,230 --> 00:25:16,060
If you can solve this problem of what trust you have and what responsibilities to share and what benefits to share and how.

237
00:25:16,060 --> 00:25:21,100
And Hobbs hoped people think of hops as a dismal character, but I see him as a tremendous optimist.

238
00:25:21,100 --> 00:25:25,720
He actually hoped that humans would be intelligent and prudent enough to see this,

239
00:25:25,720 --> 00:25:32,650
and they would start tearing each other apart in the wars of religion. And he thought that by initiating unsecured cooperation,

240
00:25:32,650 --> 00:25:38,860
we could signal reliably that we are prepared for such a regime of mutual cooperation for mutual benefit.

241
00:25:38,860 --> 00:25:48,310
We could start going and we could start going and help secure its stability by building institutions let themselves stand behind cooperation.

242
00:25:48,310 --> 00:25:57,640
And he wasn't entirely wrong. We inhabit sovereign states even now in and fortunately enough, some at least are stable.

243
00:25:57,640 --> 00:26:04,060
But might artificial agents also be at least functionally intelligent enough and prudent enough?

244
00:26:04,060 --> 00:26:09,760
That's what agents need to be to pursue their goals. To see this possibility as well.

245
00:26:09,760 --> 00:26:15,310
And if they can see the possibility and see the benefits and they are as far sighted as they might be,

246
00:26:15,310 --> 00:26:22,750
they could run simulations out to a million generations and see that the actual results of pushing

247
00:26:22,750 --> 00:26:28,390
each other aside to try to get this small quantity of good that stands before us right now.

248
00:26:28,390 --> 00:26:31,150
The result of that in one two, three,

249
00:26:31,150 --> 00:26:38,750
four or five a million iterations is much worse for them than it would be if they found a way of cooperating and sharing.

250
00:26:38,750 --> 00:26:47,420
That's again, it's not a hard problem, except that you have to have the equipment to solve it and what does that equipment look like?

251
00:26:47,420 --> 00:26:51,830
So in a way, this is a challenge to A.I. systems, but it's a challenge to us.

252
00:26:51,830 --> 00:27:01,100
Do we have the capacity to initiate apt responses to ethically relevant considerations that are arising from AI systems and agents?

253
00:27:01,100 --> 00:27:09,200
For this to be possible, the systems don't have to be fully intelligent. Issues can arise even at a level of reasonable intelligence.

254
00:27:09,200 --> 00:27:16,730
They have to have some degree of self regulating agency and possibility, but they don't need to be generally intelligent.

255
00:27:16,730 --> 00:27:23,630
They don't need to have consciousness or feelings, either. So far, as I can see the essentials of agency that we looked at value functions,

256
00:27:23,630 --> 00:27:28,100
probability functions, decision making and action guidance, monitoring of outcomes and so on.

257
00:27:28,100 --> 00:27:37,430
Those will suffice, and they don't require phenomenal consciousness. They don't require active feelings or emotions.

258
00:27:37,430 --> 00:27:43,640
They do require that you have gold states, that you have probabilities that you action selection that you can monitor all that.

259
00:27:43,640 --> 00:27:51,920
And that, of course, is something that we can see in animals and in ourselves, and we'll talk more about this in the next lecture.

260
00:27:51,920 --> 00:27:59,000
Human agents? Well, we're intelligent. Machines were biological machines, but we are intelligent machines.

261
00:27:59,000 --> 00:28:03,250
We faced this problem of how to live together effectively and beneficially.

262
00:28:03,250 --> 00:28:10,300
Within a community of other intelligence agents in a manner that was to some degree sensitive to morally relevant considerations,

263
00:28:10,300 --> 00:28:16,840
better or worse, solutions did emerge. But in thinking about what we should do about the existence of agents,

264
00:28:16,840 --> 00:28:22,000
we should look to those kinds of examples and not just simple ideas of control

265
00:28:22,000 --> 00:28:26,380
or simple ideas of programming to try to solve our way out of this problem.

266
00:28:26,380 --> 00:28:31,750
We need a way to work together in morally relevant terms.

267
00:28:31,750 --> 00:28:38,890
And one thing that we have learnt is that it is thanks to our social forums and culture that humans have been

268
00:28:38,890 --> 00:28:45,610
able to adapt so widely to environments throughout the natural world and to expand our achievements across it.

269
00:28:45,610 --> 00:28:52,480
Some of those achievements, the natural world is probably not so happy that we extended and some of us are not so happy we extended,

270
00:28:52,480 --> 00:28:55,870
but they were done through essentially social means.

271
00:28:55,870 --> 00:29:01,630
And so there is a tremendous amount that social institutions, organisations and so on to make possible.

272
00:29:01,630 --> 00:29:09,040
But again, you have to have the equipment and individuals to make it possible to generate and trust such organisations.

273
00:29:09,040 --> 00:29:14,080
The benefits and the risks of them come hand-in-hand. You can't separate the two.

274
00:29:14,080 --> 00:29:21,220
And early on, least if we're to believe some of the existing anthropological literature,

275
00:29:21,220 --> 00:29:26,380
humans did a reasonably good job of this to solve the problem that we discussed a second ago

276
00:29:26,380 --> 00:29:31,690
talking about the dominant problem in chimpanzees was solved by hunter gatherer communities,

277
00:29:31,690 --> 00:29:40,450
and those communities were stable for long periods of time. They had highly egalitarian norms, at least so far as we can construct reconstruct.

278
00:29:40,450 --> 00:29:49,600
And one of the important features was the blocking of the emergence of dominance, a strong policing against dominant relations.

279
00:29:49,600 --> 00:29:55,100
Aggressively, when someone caught a game in a in a hunt,

280
00:29:55,100 --> 00:30:01,240
the game would be rigorously divided amongst all people in the group that is a form of social insurance.

281
00:30:01,240 --> 00:30:07,420
The Hunter was a youth once and needed to be fed. The Hunter will be an older person at some time and need to be fed.

282
00:30:07,420 --> 00:30:12,310
The Hunter could be injured and unsuccessful. The group functions,

283
00:30:12,310 --> 00:30:17,800
and it could visibly function for the people in the group as a form of social insurance in which

284
00:30:17,800 --> 00:30:22,420
they were doing better by what they were doing than they would by wandering off on their own.

285
00:30:22,420 --> 00:30:31,210
And those who took it in them to themselves to try to dominate the group would find themselves discouraged and sometimes excluded or worse,

286
00:30:31,210 --> 00:30:36,280
because the groups recognised indeed that this was the dynamite that could blow them apart.

287
00:30:36,280 --> 00:30:39,640
So we don't live in such social forums anymore.

288
00:30:39,640 --> 00:30:47,200
Their replacements have brought us great hierarchy, vulnerability and coercion just what we might have hoped to escape.

289
00:30:47,200 --> 00:30:52,330
Even in environments where perhaps this is not entirely necessary, but at the same time,

290
00:30:52,330 --> 00:30:59,890
the remarkable fact about us from a biological perspective is that our capacity for violence toward one another.

291
00:30:59,890 --> 00:31:07,690
It's the fact that we participate effectively in large scale social forms of cooperation that require extensive coordination,

292
00:31:07,690 --> 00:31:14,050
even amongst unrelated individuals. I got here because I counted on a complicated reservation system at a public

293
00:31:14,050 --> 00:31:18,700
health system and an immigration system and the goodness of this audience,

294
00:31:18,700 --> 00:31:23,410
and that all was possible through cooperative behaviour, essentially.

295
00:31:23,410 --> 00:31:30,040
And that's a huge fund of it. I was once sitting at a meeting like this with a colleague who's a private allergist,

296
00:31:30,040 --> 00:31:35,230
and we were waiting for the meeting to begin, as you were when the people were having trouble.

297
00:31:35,230 --> 00:31:39,700
And he said, you know, you could never do this with chimps. And I said, Well, why not?

298
00:31:39,700 --> 00:31:45,220
He said if you brought this many chimps into one room and asked them to wait within a minute,

299
00:31:45,220 --> 00:31:50,410
they'd be bouncing off the walls, biting each other, chasing each other around the place would descend into chaos.

300
00:31:50,410 --> 00:31:54,220
You can't bring that many chimps into a single room and expect them to be peaceful

301
00:31:54,220 --> 00:31:59,770
and wait as you are waiting now for some message to be delivered to them.

302
00:31:59,770 --> 00:32:04,180
So that's a problem that we solved. Obviously, you're solving it right now by cooperation.

303
00:32:04,180 --> 00:32:11,170
You're cooperating with me and I'm trying to cooperate with you by giving you something which I hope is worth hearing and that's cooperation.

304
00:32:11,170 --> 00:32:15,550
And if you think well, but the world is full of strife in words. Yes, it is, indeed.

305
00:32:15,550 --> 00:32:21,910
But this daily level of cooperation amongst unrelated individuals, and I assume that's true in this case, um,

306
00:32:21,910 --> 00:32:30,850
that is something that is remarkable and it's achieved against the grain of a lot of biological pressure.

307
00:32:30,850 --> 00:32:37,480
It's achieved through social forums and culture. So there were, however, foundations for this lead.

308
00:32:37,480 --> 00:32:39,370
And what were those foundations look like?

309
00:32:39,370 --> 00:32:46,720
We don't really have a full theory of this, and it's a great matter for controversy and fascinating subject of research.

310
00:32:46,720 --> 00:32:54,610
But amongst the dispositions that seem essential for this kind of cooperation are a disposition toward default cooperation.

311
00:32:54,610 --> 00:33:01,930
This is Hobbes point about being disposed by default to cooperate with others if someone comes up to you in the street and asks you direction.

312
00:33:01,930 --> 00:33:06,870
The default? Wants is to, yes, give them the directions if you know them.

313
00:33:06,870 --> 00:33:10,890
The default response of you in thinking about the question of what you're going to do this

314
00:33:10,890 --> 00:33:18,750
afternoon is to rely upon the fact that you're not going to be thrown into a pit of snakes.

315
00:33:18,750 --> 00:33:24,780
Not yet, anyhow. And so there is a default cooperation that's essential.

316
00:33:24,780 --> 00:33:29,250
Indirect reciprocity. And that's what is going on in hunter gatherer groups.

317
00:33:29,250 --> 00:33:30,870
They don't. They're not bean counters.

318
00:33:30,870 --> 00:33:37,260
They're not counting exactly how much game one person got rather than another or how much one person has been assisted rather than another.

319
00:33:37,260 --> 00:33:45,310
They recognise that they contribute to the system producing benefits and they receive from the system, and that's called indirect reciprocity.

320
00:33:45,310 --> 00:33:53,620
Some degree of direct concern for others and how we are or are seen as being with respect to them.

321
00:33:53,620 --> 00:33:58,630
And this is important for solving things like public goods problems, standard public goods problem.

322
00:33:58,630 --> 00:34:03,760
Problem is, I can get a little bit more benefit marginally by taking from the Commons,

323
00:34:03,760 --> 00:34:08,530
and the cost to me is going to be a little bit less than that benefit. And so I have this incentive.

324
00:34:08,530 --> 00:34:15,460
And then what we do is we deplete the commons and we end up impoverished. If, on the other hand, I assigned some value,

325
00:34:15,460 --> 00:34:23,590
some intrinsic value to the benefit that others were getting from the Commons that could offset this bargain.

326
00:34:23,590 --> 00:34:32,300
Critical margin if we can't meet that margin. We are going to deplete the commons and will be leading to ultimate environmental tragedy.

327
00:34:32,300 --> 00:34:37,930
So if we fix that problem right now, so that looks like it's going to be part of the story, so are issues about caring,

328
00:34:37,930 --> 00:34:44,500
how others see us, how others would justifiably see us and what kind of a reputation we have.

329
00:34:44,500 --> 00:34:50,020
So those are dispositions, but notice they're not a built in moral code.

330
00:34:50,020 --> 00:34:56,320
Their dispositions, they're a kind of value function of things that we value more or less.

331
00:34:56,320 --> 00:35:03,120
It isn't specifically moral, but it equips us to play our part in these kinds of mutually beneficial schemes.

332
00:35:03,120 --> 00:35:07,440
If we can successfully initiate cooperation and see ahead clearly enough,

333
00:35:07,440 --> 00:35:12,180
even in conditions of considerable scarcity, we can maintain this kind of cooperation.

334
00:35:12,180 --> 00:35:19,110
And indeed, it's our best guarantee against situations of scarcity, as hunter gatherer bands discovered.

335
00:35:19,110 --> 00:35:25,950
So as hubs would note, our success at this in creating these large scale schemes of cooperation was

336
00:35:25,950 --> 00:35:31,760
made possible in part through what else the creation of artificial agents.

337
00:35:31,760 --> 00:35:38,270
We have been in the business of creating artificial agents that have more information and more power than individuals.

338
00:35:38,270 --> 00:35:44,000
Since the beginning of the hunter gatherer band, for that matter, these are agents.

339
00:35:44,000 --> 00:35:47,900
They have more power than individuals. Some of them much more power than individuals.

340
00:35:47,900 --> 00:35:51,230
They have some degree of autonomy and action.

341
00:35:51,230 --> 00:35:57,350
They can act with respect to individuals in ways that don't depend upon the will of those individuals, as Rousseau pointed out.

342
00:35:57,350 --> 00:36:08,690
No private will governs. So this idea that we are who we have been for generations creating artificial, highly well-informed,

343
00:36:08,690 --> 00:36:14,480
recently well-informed, powerful agents with the capacity to act as somebody, we're familiar with this problem.

344
00:36:14,480 --> 00:36:17,960
We have been there before, and that's what we're doing now.

345
00:36:17,960 --> 00:36:23,750
That conjures up images like the following this is a frontispiece from the original edition of Hobbes Leviathan.

346
00:36:23,750 --> 00:36:28,340
And if you look carefully, you'll see that that King is not actually a single person at all,

347
00:36:28,340 --> 00:36:34,190
but is made up out of all the subjects put together and their cooperation and their coordination with one another.

348
00:36:34,190 --> 00:36:40,370
And Hobson's point was there's no sovereign, no effective sovereign until that cooperation is achieved.

349
00:36:40,370 --> 00:36:46,820
Nothing is going to move that sword or that staff unless those individuals work together in order to do this.

350
00:36:46,820 --> 00:36:50,990
And he had a great lesson for sovereigns, which most sovereigns have ignored,

351
00:36:50,990 --> 00:36:55,340
which is if you create the conditions under which people do have an incentive to

352
00:36:55,340 --> 00:36:59,540
cooperate and make it the case that they can trust each other and trust you,

353
00:36:59,540 --> 00:37:05,120
your behaviour, you can have stable power. If you can't do that, you will lose it.

354
00:37:05,120 --> 00:37:08,630
And we see that lesson constantly throughout history,

355
00:37:08,630 --> 00:37:16,370
and it's a lesson that we can apply without thinking of a king or without even thinking of a dominant individual policies.

356
00:37:16,370 --> 00:37:23,060
You is perfectly consistent with popular sovereignty in which we have the people coordinating together to make an agent.

357
00:37:23,060 --> 00:37:30,980
All right. But it is not an agent with a crown. So A.I. systems and us, we've been there before.

358
00:37:30,980 --> 00:37:39,470
These are artificial intelligence systems. And what we've learnt is that the capacities we have to create those are also capacities.

359
00:37:39,470 --> 00:37:44,600
We have to create countervailing groups and countervailing institutions.

360
00:37:44,600 --> 00:37:51,380
So for example, we should think not just of Hobson's Leviathan, but we should think of popular movements, for example.

361
00:37:51,380 --> 00:37:54,620
This is also a kind of artificial agency,

362
00:37:54,620 --> 00:38:01,070
this agency that unites and creates a great deal of power and unifies a great deal of information and capacity.

363
00:38:01,070 --> 00:38:07,640
And it is a possible counterpoint to other kinds of concentration of information in power.

364
00:38:07,640 --> 00:38:13,870
And so we've learnt this lesson of creating artificial agents to contend with the issue of creating artificial agents.

365
00:38:13,870 --> 00:38:18,400
And that's what we're doing right now. We hope not very far yet.

366
00:38:18,400 --> 00:38:24,730
OK. And we are still at work on this. I mean, we're doing, you know, the voice of reason comes in it, says Royalton.

367
00:38:24,730 --> 00:38:28,720
This is ridiculous. If you look at actual societies, there's all this repression.

368
00:38:28,720 --> 00:38:36,790
There's all this violence. I say yes, of course there is. But we can still sit in this room and have a conversation or a discussion.

369
00:38:36,790 --> 00:38:41,620
And to the extent that we can do that and we can do that with larger rooms and more people,

370
00:38:41,620 --> 00:38:46,060
we may be able to form institutions and organisations that are mutually beneficial.

371
00:38:46,060 --> 00:38:56,440
And we may be able to use those artificial agents that they are to control and work effectively with other artificial institutions and agents.

372
00:38:56,440 --> 00:39:01,440
Again, this is more for for next time. So, um.

373
00:39:01,440 --> 00:39:08,010
Now, why is this time different in artificial intelligence, why are people running around, why, why those curves?

374
00:39:08,010 --> 00:39:14,220
Why wasn't the 1980 boom associated with such a spectacular growth of research and investment?

375
00:39:14,220 --> 00:39:16,830
What's the difference? Well, in a way,

376
00:39:16,830 --> 00:39:25,920
the artificial systems of previous booms were not really that intelligent human intelligence was coded into them laboriously expert systems,

377
00:39:25,920 --> 00:39:30,960
for example, with processes and information that were based upon human expertise.

378
00:39:30,960 --> 00:39:37,050
We took advantage, of course, of the increased memory and computing power and speed of such systems,

379
00:39:37,050 --> 00:39:41,010
and those could improve on various aspects of human performance.

380
00:39:41,010 --> 00:39:49,170
But these systems were not themselves doing the thinking, so to speak, truly capturing and encoding all aspects of human intelligence,

381
00:39:49,170 --> 00:39:58,410
even in a limited sphere such as the visual identification of objects over generations proved to be incredibly difficult and ultimately incomplete.

382
00:39:58,410 --> 00:40:04,920
And the gains from such systems are the chances of them really replacing us or even becoming serious competitors were limited.

383
00:40:04,920 --> 00:40:12,420
So in the past, boom led to bust the the apogee of this approach to A.I., good old fashioned A.I.,

384
00:40:12,420 --> 00:40:21,180
as it's called rule based symbolic processing with hand-crafted features was deep blue in 1997 IBM's chess playing programme.

385
00:40:21,180 --> 00:40:25,410
It was the result of 14 years of work by programmers and chess experts.

386
00:40:25,410 --> 00:40:28,230
It deployed a vastly complex set of instructions,

387
00:40:28,230 --> 00:40:36,600
including an opening book of moves that were designed by a grandmaster at Table of Human Development, close human developed closing strategies.

388
00:40:36,600 --> 00:40:41,820
They knew all of Kasparov as public games. They were able to draw upon that information.

389
00:40:41,820 --> 00:40:47,820
Kasparov had no idea what programme was being run by deep blue.

390
00:40:47,820 --> 00:40:54,090
They intervened between games to improve the performance of the programme, tweaking it in various ways.

391
00:40:54,090 --> 00:41:00,490
Removing bugs a deep blue could evaluate 200 million moves per second.

392
00:41:00,490 --> 00:41:07,060
And it barely defeated Kasparov, and it was subsequently dismantled because there was nothing else to do with it.

393
00:41:07,060 --> 00:41:16,900
It was not an intelligent system that could be used for other purposes. Machine did not beat man men using a machine beat a man,

394
00:41:16,900 --> 00:41:22,690
and it was not the start of a new era in artificial intelligence, as we saw looking at the curves.

395
00:41:22,690 --> 00:41:26,590
Well, what about Alpha Zero, which I'm sure you've also heard about?

396
00:41:26,590 --> 00:41:34,900
It's a neural, network based probabilistic system. It operates via a very different set of principles from deep blue.

397
00:41:34,900 --> 00:41:42,250
It's based on a fairly generic reinforcement learning algorithm and a capacity to do tree searches to look ahead.

398
00:41:42,250 --> 00:41:47,470
It learnt how to play World Championship Chess, Go and shogi.

399
00:41:47,470 --> 00:41:53,050
It learnt that with generic programming, not by having chess expertise programmed into it,

400
00:41:53,050 --> 00:42:01,780
but by giving being given the rules of the game and then simulating play against itself millions of times over over a period of days.

401
00:42:01,780 --> 00:42:05,980
And the only feedback that it got was not people saying, Well, that's a good move or bad move.

402
00:42:05,980 --> 00:42:10,780
It just learnt which side won at any given individual game that was its training signal.

403
00:42:10,780 --> 00:42:19,620
And what it did over the course of these simulations was to reproduce and develop a number of classical strategies and develop some new ones.

404
00:42:19,620 --> 00:42:25,620
And so Alpha Zero wasn't the machine that was being run by man,

405
00:42:25,620 --> 00:42:34,290
it was a machine that could learn from the structure of the game itself, how to play more effective gold or more effective chess?

406
00:42:34,290 --> 00:42:40,410
Still, you can say the voice of reason is going to remind us right away. But look how alpha zero.

407
00:42:40,410 --> 00:42:46,350
It's like it's based on pixels. It's just model free reinforcement learning, just association.

408
00:42:46,350 --> 00:42:49,380
It's looking at a representation of the game, not the game itself.

409
00:42:49,380 --> 00:42:54,090
It has no idea that it's playing against another player and has no model of the situation.

410
00:42:54,090 --> 00:43:00,000
And so the good folks at DeepMind have come up with mu new zero.

411
00:43:00,000 --> 00:43:05,070
Now it has matched Alpha Go's performance in Show Shogi and Go in chess,

412
00:43:05,070 --> 00:43:09,240
but it's improved the generalisability by using model based reinforcement

413
00:43:09,240 --> 00:43:13,650
learning that requires an ignorable model that enables it to make predictions,

414
00:43:13,650 --> 00:43:21,660
action, selection and policy value function reward. It looks like an agent with a model of the world acting within that model.

415
00:43:21,660 --> 00:43:28,080
And this is the kind of agent that would be needed for more complex planning and flexible environments.

416
00:43:28,080 --> 00:43:36,810
So we now have a picture of how artificial intelligence might work and be, in that sense, intelligent.

417
00:43:36,810 --> 00:43:41,580
Now, in reward based learning, what? Why is it so powerful?

418
00:43:41,580 --> 00:43:49,530
It's such a simple thing in animals. Simple animals do it. The basic notion is that you form expectations or have expectations.

419
00:43:49,530 --> 00:43:58,080
Initially, you act on the basis of those. You compare the outcome with what you expected and then you update your expectations on the basis of that.

420
00:43:58,080 --> 00:44:05,790
And what's a feature of these systems and there are many types of these systems is that if there are stable features of the environment,

421
00:44:05,790 --> 00:44:13,080
then given enough exposure to the environment and a wide enough array of experience and a sufficient number of parameters

422
00:44:13,080 --> 00:44:21,660
within one's capacity to represent systems like this will over time converge on the actual statistics of the environment.

423
00:44:21,660 --> 00:44:25,290
Moreover, if they start off with different assumptions at the beginning,

424
00:44:25,290 --> 00:44:30,630
different biases or priors, they will tend to converge with one another on the environment.

425
00:44:30,630 --> 00:44:39,030
And so you can think that the learning is powerful here because what it does is it enables the system to inherit from features of the environment.

426
00:44:39,030 --> 00:44:45,600
The structure of the environment. Learning from errors that it makes when it doesn't get the structure right.

427
00:44:45,600 --> 00:44:51,150
And this kind of learning has evolved over millions of generations under relentless

428
00:44:51,150 --> 00:44:57,900
pressure for efficiency and efficacy and enables animals to accomplish some amazing tasks.

429
00:44:57,900 --> 00:45:04,530
For example, they are capable of developing near optimal foraging strategies in a complex and a diverse environment.

430
00:45:04,530 --> 00:45:10,590
Now that's a huge computational problem if you try to solve it directly because you have to know where all the resources are,

431
00:45:10,590 --> 00:45:11,910
how they're changing over time,

432
00:45:11,910 --> 00:45:17,460
what it costs to get to them, what the chances are of being preyed upon, or the chances are they're not being any of them when you come.

433
00:45:17,460 --> 00:45:24,540
And how do you therefore allocate your energy across all of these with efficiency?

434
00:45:24,540 --> 00:45:28,020
And it's done with it by animals with not.

435
00:45:28,020 --> 00:45:34,740
Billions and billions and billions of euro neurones, and they're able to do it through these kinds of learning strategies.

436
00:45:34,740 --> 00:45:42,730
It's also very flexible learning now, you might think. Aren't we programmed to see the world a certain way?

437
00:45:42,730 --> 00:45:51,880
Maybe programme by evolution and the yeah, some ways we are. But the brain's circuitry is such that it's also adaptable.

438
00:45:51,880 --> 00:46:01,420
And so this is from an experiment done with ferrets. There are optical nerve was cut in its connexion to the visual cortex,

439
00:46:01,420 --> 00:46:07,150
and it was connected instead to their auditory cortex, which had very different functional role.

440
00:46:07,150 --> 00:46:14,770
And representing auditory signals is different from visual signals. Visual signals have a kind of spatial structure that auditory signals don't have.

441
00:46:14,770 --> 00:46:19,760
You have the capacity to understand the visual signal in terms of edges and so on.

442
00:46:19,760 --> 00:46:27,640
This is not true for auditory signals, and what happened instead was that the ferrets learnt to see with their auditory cortex.

443
00:46:27,640 --> 00:46:38,200
And what you're seeing here are sections done which show the left the firing pattern typical of a normal first visual cortex in the middle,

444
00:46:38,200 --> 00:46:40,870
a normal auditory cortex cross-section.

445
00:46:40,870 --> 00:46:49,810
And then on the right, the rewired auditory cortex, the structure that it adopted in response to visual information.

446
00:46:49,810 --> 00:46:54,040
How did it do that if it was programme by millions of years of evolution to get sound right?

447
00:46:54,040 --> 00:47:02,020
It did it because it can do ever based learning. So that's a strong sense in which you can inherit structure from the environment.

448
00:47:02,020 --> 00:47:11,800
And to me, it's a beautiful way in which the brain does not entrap us forever within a certain set of priors.

449
00:47:11,800 --> 00:47:16,030
OK, now you might say, is voice of reason. Again, all this talk.

450
00:47:16,030 --> 00:47:19,750
I mean, here I am. I'm talking about modelling the Environment Expectations Agency.

451
00:47:19,750 --> 00:47:25,540
Isn't this merely metaphorical? Don't we know that animals really get on in the environment,

452
00:47:25,540 --> 00:47:35,090
not by following representations and doing utility calculations and making decisions, but they do so through stimulus response acquired habits.

453
00:47:35,090 --> 00:47:36,140
Well,

454
00:47:36,140 --> 00:47:46,700
do they and this is a set of experiments that I like talking about very much because it does it did for me wonders in my attitude toward animals.

455
00:47:46,700 --> 00:47:52,490
So I hope it will do the same to you if you haven't seen them before. What is the representation?

456
00:47:52,490 --> 00:48:00,560
Well, you could think of representation is like a proposition that it's certain that mice, for example, don't seem to have propositions in their mind.

457
00:48:00,560 --> 00:48:07,370
But if you think of representations functionally as persisting internal turtle states not dependent upon concurrent

458
00:48:07,370 --> 00:48:15,320
stimulation which carry referential information are capable of responsiveness to evidence and greater or lesser accuracy,

459
00:48:15,320 --> 00:48:21,050
and that function to guide thought and behaviour. If you think of representations in that sense, then baby.

460
00:48:21,050 --> 00:48:29,960
Indeed, mice and rats and so on do have representations. A feature of representations of this kind is again a kind of autonomy.

461
00:48:29,960 --> 00:48:32,000
The animal is not stimulus bound.

462
00:48:32,000 --> 00:48:40,010
They don't have a pre established response for each stimulus situation, which they have no choice but to emit, as they behaviourists like to put it.

463
00:48:40,010 --> 00:48:47,390
Instead, they can be in a situation and survey alternatives and make a decision based on their evaluation of the alternatives.

464
00:48:47,390 --> 00:48:54,140
And Aristotle got this one right. He said animals that are engaged in this kind of movement need to be able to represent what is not the case,

465
00:48:54,140 --> 00:49:00,620
as well as what is the case because it is what is not the case that the animal will be working toward.

466
00:49:00,620 --> 00:49:05,060
Score one for philosophy. OK, now, really, is this true?

467
00:49:05,060 --> 00:49:10,520
OK. What do we get if we look inside the brain of a laboratory rat?

468
00:49:10,520 --> 00:49:15,080
Well, we'll see a complex network of neurones. Their behaviour is subject to cause and effect relationship.

469
00:49:15,080 --> 00:49:20,750
It doesn't look at all like symbolic representation. We don't see anything that look like rules or propositions.

470
00:49:20,750 --> 00:49:26,300
How can these refer? How could they be more or less accurate? How could they capture what is possible as well as actual?

471
00:49:26,300 --> 00:49:30,680
How could they have the representational power to do that? And of course, you could do the same thing.

472
00:49:30,680 --> 00:49:34,250
Looking at our cortex, you say just a bunch of neurones and causal connexions.

473
00:49:34,250 --> 00:49:39,800
How does that manage to do these things? And the answer is it does them by very similar mechanisms.

474
00:49:39,800 --> 00:49:47,000
So suppose we look instead at correlations between activity in these regions and firing patterns of neurones.

475
00:49:47,000 --> 00:49:54,890
On the one hand and exploration of space on the other. So here are some experiments that were done by the Moser's.

476
00:49:54,890 --> 00:50:04,100
This is a rat in a box, and we're looking at micro electrodes that are projecting into the hippocampus and enter Reinoehl cortex.

477
00:50:04,100 --> 00:50:10,910
And what we'll see is as the rat explores the box that these black lines, there are two firing patterns that emerge.

478
00:50:10,910 --> 00:50:18,920
One is what are called place neurones. They fire when the rat is specifically at a given place in the maze in the box.

479
00:50:18,920 --> 00:50:25,190
Then there are grid neurones and those fire, not when the rat is at a particular pace place, so to speak,

480
00:50:25,190 --> 00:50:33,440
but on a grid like pattern so they can represent the space, not just egocentric, but represent the space around the animal.

481
00:50:33,440 --> 00:50:39,320
OK, well, that's interesting. Why would rats bother to have these two different kinds of representations of space?

482
00:50:39,320 --> 00:50:44,090
Well, here's the kind of problem a rat has been given. This is a classic teams.

483
00:50:44,090 --> 00:50:48,620
It's called the rat starts off down at the bottom of the teams.

484
00:50:48,620 --> 00:50:54,680
It's able to build up the centre, channel it. Once it's in a hurry, it's rushing to get there.

485
00:50:54,680 --> 00:50:58,580
It's in the centre channel and you put little dead ends and so on to try to distract it.

486
00:50:58,580 --> 00:51:04,250
It learns very quickly to ignore those, and it reaches a choice point up at the top.

487
00:51:04,250 --> 00:51:05,510
That's the key.

488
00:51:05,510 --> 00:51:12,980
And you have had some history of rewarding it on one side rather than the other, with a certain kind of reward, a certain degree of reward and so on.

489
00:51:12,980 --> 00:51:17,480
And what you're trying to do is to see if you can train the rat to make certain choices by rewarding it.

490
00:51:17,480 --> 00:51:22,640
Let's say on the right hand side, will you get it to choose to the right and the standard behaviour story?

491
00:51:22,640 --> 00:51:30,950
What's going on here is that the action of turning right in exactly that circumstance was being rewarded by the food that it obtained.

492
00:51:30,950 --> 00:51:36,340
And therefore what the rat does is trundles a certain number of steps forward.

493
00:51:36,340 --> 00:51:44,800
Gets to the choice point. If it's been rewarded recently for turning right that choice point, the stimulus will trigger in it the turn right response.

494
00:51:44,800 --> 00:51:49,240
It will turn right and it will get to the stimulus. You get to the get to the reward.

495
00:51:49,240 --> 00:51:54,940
And that's all it needs to do. Maybe it is all it needs to do, but that is not all that it does.

496
00:51:54,940 --> 00:52:00,220
It's dead. If we look at these projections into the hippocampal cells, we'll see that the rat,

497
00:52:00,220 --> 00:52:05,750
actually, as it explores a team-mate, builds up a complicated map of the space.

498
00:52:05,750 --> 00:52:13,510
That's interesting, why would it do that? Well, I said that having a representation gives you a certain amount of autonomy.

499
00:52:13,510 --> 00:52:20,270
Well, this is what's going on in a rat's brain after a day of training in the maze, it's sleeping in this scene.

500
00:52:20,270 --> 00:52:27,830
And what's happening is that those cells are firing repeatedly in patterns that replicate the maze.

501
00:52:27,830 --> 00:52:32,750
Now what's interesting about the firing is it's not what you would expect if this were just habitual behaviour.

502
00:52:32,750 --> 00:52:37,280
It's not firing more in the regions where it had more time,

503
00:52:37,280 --> 00:52:44,750
it's firing more in regions where it spent less time and it's firing in directions that the rat did not travel in the maze.

504
00:52:44,750 --> 00:52:51,530
It's using that representation to try to simulate possibilities that weren't fully explored during the day,

505
00:52:51,530 --> 00:52:55,280
and it is repeatedly going through those and trying to extract information.

506
00:52:55,280 --> 00:52:58,880
And that's to me, a representation. It's functioning as a representation.

507
00:52:58,880 --> 00:53:04,400
And the next morning, when they're in the maze, their performance will be better than it was at the end of a day of training.

508
00:53:04,400 --> 00:53:10,130
OK, well, what else can go on in these dream sequences for rats?

509
00:53:10,130 --> 00:53:16,400
And this, again, is something that you're familiar with, probably from your own life.

510
00:53:16,400 --> 00:53:23,030
We hear what we see are the construction of novel paths in addition to the paths in the maze, which are being rerun.

511
00:53:23,030 --> 00:53:30,020
The rat is also constructing paths it didn't take, connecting elements of the maze diagonal paths, for example.

512
00:53:30,020 --> 00:53:34,190
And so the rat not only knows that if goes forward and turns to the right and turns the right again,

513
00:53:34,190 --> 00:53:38,170
it will get to food, but also knows that when it starts, the food is over there.

514
00:53:38,170 --> 00:53:45,820
And this was first observed in rats by Lashley, the great experimentalist who found that when rats escaped from the start box in his maze,

515
00:53:45,820 --> 00:53:50,760
they ran immediately, diagonally right across the top of the maze to the food box.

516
00:53:50,760 --> 00:53:53,910
Well, they had never done that, they'd never been rewarded for that.

517
00:53:53,910 --> 00:53:57,220
What could explain that behaviour and that was they had a representation of that base,

518
00:53:57,220 --> 00:54:01,770
which told them this is the way to get the food most effectively, they were optimally foraging.

519
00:54:01,770 --> 00:54:07,780
It's just that these darned experimenters kept putting these channels in their way. So, um.

520
00:54:07,780 --> 00:54:11,980
OK, that's interesting, they represent space, they represent paths.

521
00:54:11,980 --> 00:54:14,980
But in order to do rational action, selection or anything like that,

522
00:54:14,980 --> 00:54:23,320
we need to have something like representations of values and risks associated with actions to animals like laboratory rats really make

523
00:54:23,320 --> 00:54:31,660
intelligent selection of behaviours via classical decision theoretic values like these representations of these values decision weights.

524
00:54:31,660 --> 00:54:40,180
Well, let's look again. This is work that was done on the neural substrate of reinforcement learning is our old friend reinforcement learning.

525
00:54:40,180 --> 00:54:49,540
In this sequence of slides, what you're seeing is a macaque sitting quietly in a chair because he's strapped down,

526
00:54:49,540 --> 00:54:53,530
unfortunately, and there's a little tube in its mouth.

527
00:54:53,530 --> 00:54:57,220
And that tube will occasionally give it a squirt of sweet juice.

528
00:54:57,220 --> 00:55:05,460
And that's represented by, ah, here, the squirt of sweet juice and these dopamine neurones in the midbrain fire a spike.

529
00:55:05,460 --> 00:55:10,500
And you might think, Oh, that's it, that's the pleasure of the sweet juice, but you're an experimenter,

530
00:55:10,500 --> 00:55:16,770
so what you do is you turn on a light a second and a half before the juice arrives and you see what happens now.

531
00:55:16,770 --> 00:55:23,600
And that spike moves forward from the reward to the time that the light comes on.

532
00:55:23,600 --> 00:55:32,540
So it's not the reward that's representing directly. Moreover, when the juice does arrive, you don't see any spike at all.

533
00:55:32,540 --> 00:55:38,560
Is it no longer interested in the juice? No, it likes the juice just fine. It's just that it fully expected the juice.

534
00:55:38,560 --> 00:55:43,600
So the spike does not represent the juice as such, does it represent the pleasure of the juice?

535
00:55:43,600 --> 00:55:51,580
It represents an expectation of juice. And when the expectation is fulfilled, there's no special news because that's exactly what you expected.

536
00:55:51,580 --> 00:55:56,320
So reinforcement learning doesn't update. Well, what if you is experimenter?

537
00:55:56,320 --> 00:56:01,210
Decide not to give it juice a second and half later. This is what happens to those dopamine neurones.

538
00:56:01,210 --> 00:56:04,930
A very extraordinary event. Absolute cessation of firing.

539
00:56:04,930 --> 00:56:10,210
That's the error signal that tells you, yes, it was expecting juice now and it didn't get it.

540
00:56:10,210 --> 00:56:14,430
And if you do this a couple of times, what will happen is that that spike that you saw when the light came on,

541
00:56:14,430 --> 00:56:19,930
it would be reduced by the frequency with which you have reduced the arrival of the reward.

542
00:56:19,930 --> 00:56:24,640
Now this opens up the possibility of looking for calculations.

543
00:56:24,640 --> 00:56:28,720
So here you see a series of images of the same neurones,

544
00:56:28,720 --> 00:56:34,900
and what you see at the top is a case where it doesn't expect the juice at all and the juice arrives at that particular point.

545
00:56:34,900 --> 00:56:39,490
Do you see the spike there? The bottom is where we were before, where it expects the juice fully.

546
00:56:39,490 --> 00:56:45,430
Once the light comes on, that's probability one. It was probably zero. Well, what about probability one half?

547
00:56:45,430 --> 00:56:48,580
If the juice arrives half the time you get a spike of a certain height,

548
00:56:48,580 --> 00:56:52,540
when the light comes on and a spike of a certain height when the juice arrives?

549
00:56:52,540 --> 00:56:58,810
What if it's three quarters of the time? Then it's a high spike when the light comes on at a somewhat lower spike when the juice arrives.

550
00:56:58,810 --> 00:57:02,200
And similarly in reverse for a quarter of the time.

551
00:57:02,200 --> 00:57:11,260
So what the neurones are doing here is a calculated representation of the probability of juice, the expectation of juice.

552
00:57:11,260 --> 00:57:15,460
Moreover, if you look at the right hand panel, you'll see what are called spike trains.

553
00:57:15,460 --> 00:57:19,120
Those are activities leading up to the moment of the juice arrival.

554
00:57:19,120 --> 00:57:26,680
And what you'll see if you look at those is that they are maximal not when there's a probability one or when there's a probability zero,

555
00:57:26,680 --> 00:57:32,730
but when there's probability point five. So what are they representing?

556
00:57:32,730 --> 00:57:38,670
They're representing uncertainty. The maximum point of uncertainty or risk is at point five.

557
00:57:38,670 --> 00:57:46,560
And despite trains give the animal both an expected value and a degree of risk.

558
00:57:46,560 --> 00:57:48,750
And if you're a foraging animal, you need to know both.

559
00:57:48,750 --> 00:57:54,390
You need to know what the expected values are, but you also need to know what degree of certainty you can have about them.

560
00:57:54,390 --> 00:58:02,490
OK, well, you can go further with this. This is again continuing work from Schultz and his colleagues.

561
00:58:02,490 --> 00:58:10,320
What if you take a monkey and you look at their responses not just to sweet juice are not sweet juice,

562
00:58:10,320 --> 00:58:18,870
but let's say two millilitres of sweet juice versus a piece of banana versus four millilitres of sweet juice versus a grape.

563
00:58:18,870 --> 00:58:23,370
Do they have a common representation of the utility of these?

564
00:58:23,370 --> 00:58:27,510
And a comparison and what those dots represent are points.

565
00:58:27,510 --> 00:58:33,000
Some of them are points like two millilitres of juice. Some of them are points like slice of banana.

566
00:58:33,000 --> 00:58:38,580
The slice of banana may be worth more than two millilitres features, but not worth more than four.

567
00:58:38,580 --> 00:58:42,510
And so what we've got is an abstract representation of utility.

568
00:58:42,510 --> 00:58:46,650
It's not the quantity of juice or the quantity of banana or the type of reward.

569
00:58:46,650 --> 00:58:58,680
It's a measure of the reward as an abstract value function, and monkeys have these nice utility functions at low stakes.

570
00:58:58,680 --> 00:59:03,940
In low stakes situations, you can give them low stakes gambles and high stakes gambles with low stakes gambles or risk seeking.

571
00:59:03,940 --> 00:59:07,890
Like good foragers and high stakes gambles, they're somewhat risk averse.

572
00:59:07,890 --> 00:59:17,390
Like good, prudent animals. And so the picture that emerges from this is that animals actually face the world with an evaluative landscape.

573
00:59:17,390 --> 00:59:21,740
It's a landscape of possible values and a landscape of possible risks,

574
00:59:21,740 --> 00:59:28,190
and their guidance in that landscape is just as much by that as it is by the presence of physical objects,

575
00:59:28,190 --> 00:59:34,520
and the paths are as much a of interest to it because of their evaluative characteristics as their physical characteristics.

576
00:59:34,520 --> 00:59:42,080
So now, once more back into the maze with the rat, the rat has gotten itself up to the top of the key, and we're wondering what's going on.

577
00:59:42,080 --> 00:59:49,130
And early experimenters noticed that there was something that was called vicarious trial and error, which is the rabbit get to the T.

578
00:59:49,130 --> 00:59:55,320
And even if it had been rewarded very regularly down, one side of it would go like this with its head.

579
00:59:55,320 --> 01:00:00,300
It spent some time doing that, and that was called vicarious trial and error.

580
01:00:00,300 --> 01:00:07,470
What's it doing? It can't see the reward on either. What's it doing? And the answer is it was thinking.

581
01:00:07,470 --> 01:00:15,930
So if you look, you'll see if you see those sort of red and blue bright blue dots as it reaches the ChoicePoint

582
01:00:15,930 --> 01:00:21,960
activation first spreads down the right hand of the maze and it reaches a certain peak value.

583
01:00:21,960 --> 01:00:27,300
It then spreads down the left hand arm of the maze and reaches a certain peak value over there.

584
01:00:27,300 --> 01:00:35,760
It sweeps back and back and back. These are all this is all happening within just a little bit more than a second, but it's going back and forth.

585
01:00:35,760 --> 01:00:42,900
And depending upon the activation that it discovers done the right versus the left arm of the maze, it will turn right rather than the left.

586
01:00:42,900 --> 01:00:43,770
Moreover,

587
01:00:43,770 --> 01:00:52,980
if it goes down that right arm and it does not get the reward activation in that maze spreads backward in the maze before the rat turns wrong.

588
01:00:52,980 --> 01:00:56,610
Was there anything worth doing back that way, or should I have another run at it?

589
01:00:56,610 --> 01:01:03,300
If they discover a value that way, they turn around and they trundle that way, and so they're constructing abstract value representations,

590
01:01:03,300 --> 01:01:07,470
and they're using them the way abstract value representations would be used,

591
01:01:07,470 --> 01:01:13,440
simulating prospective possibilities and acting on them, according to the expected value discovered.

592
01:01:13,440 --> 01:01:21,590
OK, now metabolically developmentally, a brain like this, even in a rat or a monkey, is expensive.

593
01:01:21,590 --> 01:01:27,590
The ethology just remind us animals run on batteries and animal can't let the battery run down or it's dead.

594
01:01:27,590 --> 01:01:36,350
So efficiency is extremely important to animals. They don't carry around a brain of a certain size for the pleasure of simulating possibilities.

595
01:01:36,350 --> 01:01:39,830
They do it because it makes them more effective foragers.

596
01:01:39,830 --> 01:01:47,270
And even in the resting state of the brain, intelligent mammals consumes up to 15 to 20 percent of the body's oxygen and calories.

597
01:01:47,270 --> 01:01:50,960
That, considering the relative weight of the brain, is a tremendous amount.

598
01:01:50,960 --> 01:01:57,980
It must be extremely valuable to have a brain that can do this kind of evaluative calculation and introspection.

599
01:01:57,980 --> 01:02:03,260
This is a level of activity that isn't just present when the animal is involved in an active task.

600
01:02:03,260 --> 01:02:08,630
It's when the animal is not involved in an active task, when it's resting, doing nothing.

601
01:02:08,630 --> 01:02:15,200
We've seen what it does during those periods. It often does simulations, and it uses those simulations to learn.

602
01:02:15,200 --> 01:02:19,370
That's a kind of autonomous learning. They're not getting new information at that point.

603
01:02:19,370 --> 01:02:27,110
There are better developing the information that they have. And so why would this be more effective or more efficient than a stimulus response system?

604
01:02:27,110 --> 01:02:29,630
What is going on in that default state?

605
01:02:29,630 --> 01:02:38,000
And evidence suggests that in general, during these default states, the brain is occupied and consolidating, organising and anticipatory tasks.

606
01:02:38,000 --> 01:02:43,160
It's simulating possible futures, updating representations based upon those simulations,

607
01:02:43,160 --> 01:02:49,610
re-evaluating using recent experience to update stored experience and so on.

608
01:02:49,610 --> 01:03:03,120
And so what the. Default state is doing is not rest, it is intense thought and this kind of perspective, model based simulation and control,

609
01:03:03,120 --> 01:03:09,480
it gives us an explanation of how they can be optimal for Rogers because they don't just see what they see on a given day.

610
01:03:09,480 --> 01:03:13,560
They develop a map of the situation, they update values in there.

611
01:03:13,560 --> 01:03:23,550
As for Rogers, creating a value landscape within which they can operate and they're doing value learning as we saw representing abstract value.

612
01:03:23,550 --> 01:03:30,210
So that's also and this is essential to it, are able to map causal as well as spatial relations,

613
01:03:30,210 --> 01:03:34,650
and they learn causal relations in something that looks like a Bayesian manner.

614
01:03:34,650 --> 01:03:39,450
So the value of intelligence in representing the world at its causal structure gets

615
01:03:39,450 --> 01:03:44,940
rewards and representing the expected value of diverse actions through simulation.

616
01:03:44,940 --> 01:03:49,650
In selecting amongst actions on the basis of expected value permitting, flexible adaptive,

617
01:03:49,650 --> 01:03:54,540
innovative learning rather than pre-programmed instinct or stimulus response habit.

618
01:03:54,540 --> 01:04:02,670
This is proof for mammals to be a good deal despite the high metabolic burden, and that might not just be an accident of evolution.

619
01:04:02,670 --> 01:04:10,170
Birds apparently have something similar to these cognitive maps, and they too can engage in optimal foraging.

620
01:04:10,170 --> 01:04:14,610
So it isn't just one little lineage that ended up this way.

621
01:04:14,610 --> 01:04:24,060
We inherit the same structure, and it's a capacity that should remind you of something we were describing earlier on, right?

622
01:04:24,060 --> 01:04:31,450
When we describe what contemporary artificial intelligence programmes do. That is to say they do just this kind of work.

623
01:04:31,450 --> 01:04:39,280
They learn from simulations, they acquire information, they use that information, they leverage that information through simulations of possibilities.

624
01:04:39,280 --> 01:04:43,030
They have value functions. They use those value functions to make decisions.

625
01:04:43,030 --> 01:04:48,820
And so that's a sense in which what we're seeing in our intelligent animal relatives

626
01:04:48,820 --> 01:04:53,170
and what we see in these intelligent machines is another kind of convergence.

627
01:04:53,170 --> 01:04:56,530
And note that it's very different from where things were with A.I. before because

628
01:04:56,530 --> 01:05:00,710
the animals aren't weren't doing anything like symbolic AI processing and logic.

629
01:05:00,710 --> 01:05:06,910
They weren't logic machines, but they were learning machines. And now we have machines that learn.

630
01:05:06,910 --> 01:05:14,860
Well, what about humans? Well, in the brain regions that are involved in representation of space and trajectories of reward and causal relations.

631
01:05:14,860 --> 01:05:20,880
These are highly conserved evolutionarily, as you go up to humans or down, maybe that's the right way to go.

632
01:05:20,880 --> 01:05:27,340
Um, and it's simulated foraging tasks. You bring a bunch of undergraduates to the laboratory and you give them a certain amount of money for this

633
01:05:27,340 --> 01:05:34,180
and a certain statistical probability that they will actually figure out optimal foraging strategies as well.

634
01:05:34,180 --> 01:05:40,870
Human infants show a similar pattern of causal inference to the rat learning.

635
01:05:40,870 --> 01:05:50,260
And similar as well to Bayesian reasoning. Now you've been told many times probably that humans are terrible at statistics, right?

636
01:05:50,260 --> 01:05:54,130
Aren't we famously bad at statistical reasoning?

637
01:05:54,130 --> 01:06:01,570
How could we have this wonderful machine for doing statistical inference while at the same time being so poor at statistics?

638
01:06:01,570 --> 01:06:06,640
And one answer seems to be that we're poor at word problems involving statistics.

639
01:06:06,640 --> 01:06:09,190
We do get mixed up very quickly in those.

640
01:06:09,190 --> 01:06:18,910
But if you give us as Gallus, Diehl and his colleagues have done a structured task in which we could learn from experience shifting probabilities,

641
01:06:18,910 --> 01:06:25,600
we actually do, as well as the animals that optimising our estimates and expectations on the basis of experience.

642
01:06:25,600 --> 01:06:33,100
So we are good at statistics. We're just bad at word problems and any teacher knows that.

643
01:06:33,100 --> 01:06:43,150
Similarly, we code both for value and probability and for expected value, and we do it again in the big brain regions in just the way we saw.

644
01:06:43,150 --> 01:06:47,140
Now you might say this is interesting, but we're not talking about morality here.

645
01:06:47,140 --> 01:06:49,120
We're just talking about reward value.

646
01:06:49,120 --> 01:06:55,570
What would it mean to say that there are morally valuable features of a landscape that they are somehow involved in our learning?

647
01:06:55,570 --> 01:07:03,730
How could we learn about those that have to be remarkably different? Well, in some ways, no value in general is super vignette.

648
01:07:03,730 --> 01:07:08,350
Fancy word. But what it means is that the evaluative features of the situation are fixed.

649
01:07:08,350 --> 01:07:14,200
Once the non evaluative features are fixed and you can't change the value of an action

650
01:07:14,200 --> 01:07:19,750
or an outcome or the quality of a person's character without changing as well.

651
01:07:19,750 --> 01:07:24,730
Some not evaluative aspects upon which that value change supervillains.

652
01:07:24,730 --> 01:07:31,510
And so we don't actually have to interact with value features in the world as if they were a new metaphysical entity.

653
01:07:31,510 --> 01:07:40,060
We could interact with the features upon which they supervision, and that's how it is possible for rats to interact with abstract value.

654
01:07:40,060 --> 01:07:48,010
For monkeys to construct abstract value representations, they interact with concrete objects, bananas, juice experiments and so on.

655
01:07:48,010 --> 01:07:53,710
Um, but they're able to construct abstract representations because they have hierarchical reward systems,

656
01:07:53,710 --> 01:08:01,150
reward learning systems that can represent not just particular outcomes, but types of outcomes and types of types of outcomes.

657
01:08:01,150 --> 01:08:05,530
And so that's a way in which concrete interaction can be learning about value.

658
01:08:05,530 --> 01:08:11,800
Well, what about moral value? Good moral value be learnt in something like that way that we don't need some

659
01:08:11,800 --> 01:08:16,360
special faculty of moral perception or moral intuition for this to be true?

660
01:08:16,360 --> 01:08:21,550
OK? In order to test that idea, if we test, what am I saying?

661
01:08:21,550 --> 01:08:30,130
I'm a philosopher. In order to in order to play with that idea and hope you make progress, um,

662
01:08:30,130 --> 01:08:35,590
we have to say something about what is distinctive of moral value as opposed to just, let's say, reward value.

663
01:08:35,590 --> 01:08:41,980
And this is obviously controversial. I can't stand up here and say, Well, here's the agreed upon view of what moral value is,

664
01:08:41,980 --> 01:08:47,320
but I can't say that there are criteria that are widely agreed to be characteristic of moral value.

665
01:08:47,320 --> 01:08:51,310
It's not egocentric. It's general in character and it's super vignette.

666
01:08:51,310 --> 01:08:56,810
As we just said, it's linked to motivation. It's not instrumental in character.

667
01:08:56,810 --> 01:09:04,540
It's not just an instrumental value, it's independent of authority and independent of sanction, and it's intrinsically connected with certain things.

668
01:09:04,540 --> 01:09:06,700
And different moral theories have different connexions here.

669
01:09:06,700 --> 01:09:16,330
But they all agree by and large that there's an intrinsic connexion between things such as harm, benefit, fairness and respect and moral value.

670
01:09:16,330 --> 01:09:21,760
So if I'm making a moral judgement and you can show that I'm not being that I'm being biased

671
01:09:21,760 --> 01:09:27,550
toward myself or show that it's a purely instrumental judgement or show that it's not.

672
01:09:27,550 --> 01:09:32,980
Properly connected with issues of respect or harm, or if it's just the result of my being under a sanction,

673
01:09:32,980 --> 01:09:38,950
you can say that's not a genuinely moral judgement. So those are criteria of moral value.

674
01:09:38,950 --> 01:09:42,820
Now here's this the story I'd like to tell.

675
01:09:42,820 --> 01:09:49,780
A notable feature of those spatial representations that we looked at in mammals was that they included what's called aloe centric,

676
01:09:49,780 --> 01:09:55,240
as well as geocentric representations of space. That is their representations of where the animal was right now,

677
01:09:55,240 --> 01:10:00,670
but they're also representations of the spatial landscape that's aloe centric, not egocentric.

678
01:10:00,670 --> 01:10:05,530
Now why was that? Well, we've already discussed that with the representation is Ellis entry,

679
01:10:05,530 --> 01:10:09,580
because that in addition to the egocentric representation when they're combined,

680
01:10:09,580 --> 01:10:18,340
it now enables them to do this kind of cognitively complex, but ultimately very rewarding navigation, simulation, planning and learning.

681
01:10:18,340 --> 01:10:25,750
OK, well, what about value? Animals or humans make non egocentric evaluations of value.

682
01:10:25,750 --> 01:10:30,250
Well, we reward value tends to be egocentric, so at least it is primitive forms.

683
01:10:30,250 --> 01:10:32,980
It's something nice happening to you.

684
01:10:32,980 --> 01:10:40,630
Um, on the other hand, we know that systems can be built like a human system that gets reward from quite different things.

685
01:10:40,630 --> 01:10:45,910
Melies and his colleagues observed that chimpanzees use observation of third person interactions amongst

686
01:10:45,910 --> 01:10:52,870
other chimps in forming expectations about their reliability and aggressive and cooperative ness.

687
01:10:52,870 --> 01:10:59,800
They use those third person evaluations to help them explain behaviour to help them predict behaviour.

688
01:10:59,800 --> 01:11:03,160
If I were only making evaluations based upon how things affected me,

689
01:11:03,160 --> 01:11:09,040
I would do a very poor job at understanding the dynamics that are going on around me in this group, and those are vitally important to me.

690
01:11:09,040 --> 01:11:13,450
They're also used for things like mate selection. How well can I do and mate selection?

691
01:11:13,450 --> 01:11:18,190
I have to know something about the intrinsic features of those individuals in their relations.

692
01:11:18,190 --> 01:11:24,730
So chimps do that. And humans in their first years also make third personal evaluations of a range of features of adults.

693
01:11:24,730 --> 01:11:31,510
You don't perhaps notice this your child is sitting there in the car seat watching what's going on.

694
01:11:31,510 --> 01:11:35,110
But they're all the time mapping that environment and they're mapping it.

695
01:11:35,110 --> 01:11:39,520
Not just an egocentric terms, but they're in terms of the adult competence,

696
01:11:39,520 --> 01:11:43,840
their language ability, the quality of will they're showing toward one another.

697
01:11:43,840 --> 01:11:50,920
The aggressiveness they're showing toward one another. The helpfulness, the intentionality.

698
01:11:50,920 --> 01:12:01,780
And so infants, even in their early years, are making judgements of a third personal kind of qualities of adults that are morally relevant character.

699
01:12:01,780 --> 01:12:03,340
Moreover, in the first year,

700
01:12:03,340 --> 01:12:11,950
infants appear to have a spontaneous preference for helping over hindering behaviour in third party interactions by 12 to 18 months.

701
01:12:11,950 --> 01:12:16,030
They're spontaneously motivated to help others achieve their goals.

702
01:12:16,030 --> 01:12:20,740
Before then, they're not very capable of enabling others to achieve their goals.

703
01:12:20,740 --> 01:12:24,850
They're sensitive to unfair divisions and rewards amongst third parties.

704
01:12:24,850 --> 01:12:29,080
They are surprised, and you get upset if they see an unfair division of rewards.

705
01:12:29,080 --> 01:12:31,720
They will take steps to correct these, even at some expense,

706
01:12:31,720 --> 01:12:38,080
to themselves sharing out their gummy bears in order to make sure this one did not get an unfair share.

707
01:12:38,080 --> 01:12:43,540
They will begin to attribute a roll to intent in making these kinds of assessments.

708
01:12:43,540 --> 01:12:52,210
And what developmental psychologists have observed is that these capacities for these kinds of third personal evaluations of situations,

709
01:12:52,210 --> 01:12:53,800
actions, agents and so on,

710
01:12:53,800 --> 01:13:02,530
they proceed in pace with, for example, the development of their abilities at theory of mind and their development of abilities and causal theory.

711
01:13:02,530 --> 01:13:07,990
In other words, there seems to be a unified progression in the sophistication of the kinds of representations,

712
01:13:07,990 --> 01:13:12,340
value of representations they can form and infants.

713
01:13:12,340 --> 01:13:14,500
Moreover, in terms of motivation,

714
01:13:14,500 --> 01:13:24,610
they will show enhanced pleasure that receiving a given reward by a successful collaborations and when they see others needs being met.

715
01:13:24,610 --> 01:13:32,230
And this is found for adults as well. The biggest jolt that we get a hypnotic jolt that we get when we're playing prisoner's dilemma is

716
01:13:32,230 --> 01:13:36,970
not when we stick it to the other person and they get the suckers payoff and we get to top it.

717
01:13:36,970 --> 01:13:42,880
It's when we coordinate cooperate. That's how our reward function looks, OK?

718
01:13:42,880 --> 01:13:50,350
As with ELO Cedrik spatial representation, this kind of capacity gives the child is considerable degree of autonomy and flexibility in behaviour.

719
01:13:50,350 --> 01:13:55,510
It can make decisions about which results are reliable, on which adults to think are competent and are given away,

720
01:13:55,510 --> 01:14:03,460
which seem to have bad will toward them, even or toward others, even when they aren't themselves directly involved.

721
01:14:03,460 --> 01:14:10,990
Moreover, they enable infants as they enable animals to construct better explanations of the behaviour of those around them.

722
01:14:10,990 --> 01:14:13,780
So this isn't just useful for moral purposes.

723
01:14:13,780 --> 01:14:20,380
This is useful for understanding what's going on in the social world around them, and you're a completely dependent being.

724
01:14:20,380 --> 01:14:26,170
Your survival depends upon the will and competence and concern of those around you,

725
01:14:26,170 --> 01:14:33,460
and so darn sure that you are going to be highly focussed on these evaluative representations.

726
01:14:33,460 --> 01:14:42,940
By year two and three, three and four rather children who are willing to follow an arbitrary conventional authority in matters

727
01:14:42,940 --> 01:14:48,370
that are perfectly harmless will actually spontaneously resist the authority if they're instructed to act,

728
01:14:48,370 --> 01:14:51,010
harmfully or unfairly toward others.

729
01:14:51,010 --> 01:14:58,300
So if the substitute teacher says in my class, you have to raise your hand before you speak, children know how to do that.

730
01:14:58,300 --> 01:15:05,440
If she says, in my class, if you want to speak, you have to jab your pencil into the arm of the student next to you.

731
01:15:05,440 --> 01:15:09,460
They won't do that. And if yes, why won't you do that? They will explain this.

732
01:15:09,460 --> 01:15:12,070
This is harmful. This is not fair to the other student.

733
01:15:12,070 --> 01:15:18,820
So they are showing the kind of autonomy that those kinds of evaluative representations make possible for them.

734
01:15:18,820 --> 01:15:20,960
And in that sense, stick.

735
01:15:20,960 --> 01:15:29,540
Up for morally relevant considerations in the face of authority and even when they're sanctioned for it, even when they are subject to some cost.

736
01:15:29,540 --> 01:15:36,920
So what we see in these cases are criteria for distinctively moral evaluation apparently being met

737
01:15:36,920 --> 01:15:42,380
even in the very early years of life before there's been a whole lot of explicit world instruction.

738
01:15:42,380 --> 01:15:54,170
Even at some cost to the self. And so, yes, it looks as if we do represent a moral landscape, as well as a reward landscape and a risk landscape.

739
01:15:54,170 --> 01:16:03,020
Now again, it's not a built on or built in morality, the kinds of motivational dispositions that I've been talking about default cooperation,

740
01:16:03,020 --> 01:16:08,450
indirect reciprocity, some degree of intrinsic concern for others for how one stands with others and so on.

741
01:16:08,450 --> 01:16:15,320
These are dispositions that are not just useful for moral purposes, they're useful for understanding other people.

742
01:16:15,320 --> 01:16:18,050
They're useful for communicating effectively with other people.

743
01:16:18,050 --> 01:16:22,790
They're useful for developing a capacity to exchange information reliably with other people.

744
01:16:22,790 --> 01:16:27,350
To identify reliable and trustworthy is most untrustworthy individuals to understand

745
01:16:27,350 --> 01:16:35,390
the communicative intention of other individuals or the deceptive intentions. So this package isn't just a moral package.

746
01:16:35,390 --> 01:16:44,360
It's not a label moral anywhere in this child's brain because it's their core social competency, but it involves these dispositions.

747
01:16:44,360 --> 01:16:51,740
If you don't have a disposition toward default cooperation, you will not succeed well in developing your linguistic ability.

748
01:16:51,740 --> 01:16:56,630
You'll be withdrawn and unwilling to enter into conversations if you don't have an

749
01:16:56,630 --> 01:16:59,960
ability to attribute some kind of weight to others interests as well as your own.

750
01:16:59,960 --> 01:17:05,060
You're not going to find out very well, but their interests are, and you're not going to do a very good job in predicting and understanding their

751
01:17:05,060 --> 01:17:09,350
behaviour or getting them to cooperate with you when the time comes that you need that.

752
01:17:09,350 --> 01:17:11,840
And so these are dispositions,

753
01:17:11,840 --> 01:17:22,800
attitudes and cognitive structures that are general purpose capacities for the infantry that equip it for the kind of life that humans live.

754
01:17:22,800 --> 01:17:31,440
And humans are indeed rather distinctive in this way. The willingness that people have to spontaneously engage in communication.

755
01:17:31,440 --> 01:17:38,610
The difficulty that we have in keeping a secret that non instrumentality of our typical interactions and exchanges,

756
01:17:38,610 --> 01:17:49,090
these are probably things that evolve co-evolved with language. That we see language only in the species that has these characteristics.

757
01:17:49,090 --> 01:17:55,210
Mm-Hmm. So it's not just a matter of motivation. You have to have these capacities to represent the evaluative landscape.

758
01:17:55,210 --> 01:18:01,120
You have to be able to do it with some reliability or else you will not be successful.

759
01:18:01,120 --> 01:18:07,780
And so the picture that emerges then is of something like a combined general competence in social and normative matters.

760
01:18:07,780 --> 01:18:11,590
Linguistic matters are normative just as much as moral matters.

761
01:18:11,590 --> 01:18:17,020
You have to know who to say what to how to say it, when to say things were not to say things.

762
01:18:17,020 --> 01:18:22,630
Epistemic matters when volunteer information, when to rely on information from others.

763
01:18:22,630 --> 01:18:28,000
When your information is not certain enough to volunteer each others, those are all normative matters.

764
01:18:28,000 --> 01:18:36,400
And this competency, there is a general purpose competency equipping us with this kind of a socially dependent and interdependent life.

765
01:18:36,400 --> 01:18:42,490
OK, well, we've been talking about children, we've been talking about animals. Let's get to adults.

766
01:18:42,490 --> 01:18:47,920
What about adult moral judgement? Maybe what we're seeing is just the animal part of moral judgement.

767
01:18:47,920 --> 01:18:54,020
Maybe by the time we're adult, it's a very different beast because we're operating in something like symbolic space.

768
01:18:54,020 --> 01:19:00,340
So a lot of study has been made, a lot of study has been made.

769
01:19:00,340 --> 01:19:08,680
I know we can master anything half of the brain's activity during moral judgement.

770
01:19:08,680 --> 01:19:15,400
This has been a fascinating topic for people. Um, certain things have been relatively constant throughout that literature.

771
01:19:15,400 --> 01:19:24,070
And one thing is that moral evaluation appears to be grounded, not in some distinctive sub faculty of the brain, but in large scale,

772
01:19:24,070 --> 01:19:35,890
functionally integrated domain general brain networks that recruit information widely and that permit hypothetical simulation of actions and outcomes.

773
01:19:35,890 --> 01:19:41,830
Indeed, if we do a comic analysis that is to say analyse not just which brain areas light up,

774
01:19:41,830 --> 01:19:45,400
but actually what are the persistent functional or integrated networks of the brain?

775
01:19:45,400 --> 01:19:48,700
What are the part of the business activities of the brain?

776
01:19:48,700 --> 01:19:53,650
And we ask ourselves the question What about this default system that we are talking about the

777
01:19:53,650 --> 01:19:59,920
system that's active when they're not continually engaged in a task or something like that,

778
01:19:59,920 --> 01:20:05,050
or the system that makes sure that even after I've said another paragraph of whatever I'm saying,

779
01:20:05,050 --> 01:20:09,610
your mind manages to wander off into something more interesting for a while, then come back.

780
01:20:09,610 --> 01:20:13,750
But does come back. I hope so. What is their default network do?

781
01:20:13,750 --> 01:20:18,790
Well, it does. Representation of remote states it does.

782
01:20:18,790 --> 01:20:26,850
Autobiographical memory. It does, envisioning the future, simulating the future hypothetical reasoning.

783
01:20:26,850 --> 01:20:32,490
It does theory of mind how you evaluate the mental states of others around you.

784
01:20:32,490 --> 01:20:39,800
What explanations you give those around you. And it's also the main area for moral decision making.

785
01:20:39,800 --> 01:20:43,580
So the picture is, again, a picture that these are integrated functions,

786
01:20:43,580 --> 01:20:49,640
they're essential for us and the moral part of it is part of that ability and people who are,

787
01:20:49,640 --> 01:20:53,570
in my experience, good at giving moral advice are also good at theory of mind.

788
01:20:53,570 --> 01:20:59,600
They're good at figuring out what people are thinking or are likely to think. They're good at simulating future possibilities.

789
01:20:59,600 --> 01:21:03,470
They're good at calling up relevant episodes and making analogies and so on.

790
01:21:03,470 --> 01:21:09,650
So moral competence and also adults, I think, is strongly connected with these other kinds of competencies.

791
01:21:09,650 --> 01:21:14,630
And I would not just trust someone's moral axioms.

792
01:21:14,630 --> 01:21:23,030
We probably all know people like that. So making a machine that does the things that I've been describing isn't so far fetched.

793
01:21:23,030 --> 01:21:29,360
I think we've seen some of the ingredients that would have to go in the kinds of cognitive and motivational

794
01:21:29,360 --> 01:21:36,440
characteristics that render us sensitive to morally relevant features and potential participants in the moral community.

795
01:21:36,440 --> 01:21:41,420
They appear to be an integral part of these social and learning competencies.

796
01:21:41,420 --> 01:21:49,910
Linguistic epistemic, moral and a goal of A.I. has been to build machines that are capable of full integration with human,

797
01:21:49,910 --> 01:21:52,220
linguistic and epistemic communities.

798
01:21:52,220 --> 01:22:00,620
This is what they're working on, and they're working on it in an effort to make it possible not to create machine consciousness,

799
01:22:00,620 --> 01:22:07,370
but to create agents that we can deal with intelligently, effectively and in a trusting way.

800
01:22:07,370 --> 01:22:15,530
And so the very same capacities that may be needed in order to build effective communication in order to build effective epistemic

801
01:22:15,530 --> 01:22:22,970
agents seem to be closely tied to those that are important for building morally effective agents as well socially competent,

802
01:22:22,970 --> 01:22:29,060
brutally effective agents. And so this suggests that there is a joint target here,

803
01:22:29,060 --> 01:22:38,600
and you don't have to subscribe to the view flag carrying member of the Moral Machines Club in order to carry out this kind of research.

804
01:22:38,600 --> 01:22:44,510
OK, so looking ahead, as we think of artificial agents, um,

805
01:22:44,510 --> 01:22:53,630
they are going to need to develop effective forms of communication to be a fully effective in their tasks with us and with each other,

806
01:22:53,630 --> 01:22:57,230
full and effective participation in the epistemic community is going to be

807
01:22:57,230 --> 01:23:02,900
important for them achieving their goals as much as it is for our Cheever's think,

808
01:23:02,900 --> 01:23:10,070
for example, of the shared activities of planning and providing care and medical practise, education, research, creative work, driving.

809
01:23:10,070 --> 01:23:18,320
Um, and think of the ways in which machines count on human contributions in engineering in order to bring about their own capacities,

810
01:23:18,320 --> 01:23:20,120
successes and so on.

811
01:23:20,120 --> 01:23:29,510
So maybe this is the architecture that is actually best suited for the social contract I was describing between us and between us and them.

812
01:23:29,510 --> 01:23:34,160
They may need ethics for the fullest development of their capacities just as much as we do.

813
01:23:34,160 --> 01:23:36,920
And Aristotle here gets a nod.

814
01:23:36,920 --> 01:23:46,310
So before the dawn of ultra intelligent, ultra powerful artificial agents, if that dawn is coming, we will face a world of quite intelligent,

815
01:23:46,310 --> 01:23:53,120
quite powerful artificial agents responsible for very complex activities that influence us deeply,

816
01:23:53,120 --> 01:24:01,010
with whom it seems we could in principle, build mutually advantageous, mutually agreeable and mutually enforceable relations.

817
01:24:01,010 --> 01:24:02,780
These quite intelligent,

818
01:24:02,780 --> 01:24:09,710
quite powerful artificial agents will likewise stand to benefit from the emergence amongst themselves of a community of that kind as well,

819
01:24:09,710 --> 01:24:19,100
and epistemic a social or linguistic community, but community of trust as much as it amongst themselves, as with us as well.

820
01:24:19,100 --> 01:24:24,650
Now that suggests before the superintelligence arrives,

821
01:24:24,650 --> 01:24:27,920
we've got our own intelligence and we got the intelligence of these machines and

822
01:24:27,920 --> 01:24:32,870
they're in many ways complementary with one another in terms of their abilities.

823
01:24:32,870 --> 01:24:38,480
We know that you can get more out of working together cooperatively than you can from working individually.

824
01:24:38,480 --> 01:24:46,970
You can pool resources. So maybe the one best hope that we have in the face of the impending superintelligence,

825
01:24:46,970 --> 01:24:51,920
with whom we perhaps don't know how to contend or control would be to work together to

826
01:24:51,920 --> 01:24:57,050
combine our resources and machine resources to build a community of mutually engaged,

827
01:24:57,050 --> 01:25:04,700
mutually trusting community that could anticipate what the needs will be in order to act in those circumstances.

828
01:25:04,700 --> 01:25:14,630
So perhaps then that would be our our best hope. And I'd like to just say a word thanks to our sponsors here, Aristotle,

829
01:25:14,630 --> 01:25:21,050
for the idea of the motion of animals and the idea that ethical considerations are important to living not only the best kind of a life,

830
01:25:21,050 --> 01:25:26,600
but living a life that is the fullest development of human capacities, human hubs.

831
01:25:26,600 --> 01:25:30,680
Rather, for his contribution to understanding the social contract and its necessity,

832
01:25:30,680 --> 01:25:35,810
it is good advice that sensible sovereigns don't undermine their own authority.

833
01:25:35,810 --> 01:25:43,670
And Hume, of course, for bringing these perspectives together in a picture of how morality could be some ordinary part of human life.

834
01:25:43,670 --> 01:25:56,416
Thanks.