1 00:00:02,620 --> 00:00:10,260 Well, thanks very much. So this slide is like a happy medium between the first one and most slides at all. 2 00:00:10,260 --> 00:00:16,000 And so we just have a few bullet points for you. 3 00:00:16,000 --> 00:00:19,210 So I'm at the public school. 4 00:00:19,210 --> 00:00:26,820 I'm not authorised to speak on behalf of the Pivarnick School, but I will sample some of the things that we are doing there. 5 00:00:26,820 --> 00:00:35,020 So Peter asked me to talk about the work that's going on inside are in school and I haven't gone round and ask everyone. 6 00:00:35,020 --> 00:00:40,660 We don't have a single group of people. We've got lots of people following their own research interests. 7 00:00:40,660 --> 00:00:49,810 And many of these research interests do have connexions with AI, machine learning and ethical issues of a number of different sorts. 8 00:00:49,810 --> 00:00:56,410 So this first slide is just a sample of some of the things I know that is going on in the school. 9 00:00:56,410 --> 00:01:05,070 And many of them have already been mentioned, although not all first, social media and democracy. 10 00:01:05,070 --> 00:01:11,430 So as a school of government, we're very concerned about the quality of the democracy we're in. 11 00:01:11,430 --> 00:01:20,370 We're very concerned about the way in which Facebook and other companies have perhaps been affecting 12 00:01:20,370 --> 00:01:25,600 the way in which elections are conducted and the way in which information is getting out. 13 00:01:25,600 --> 00:01:29,100 And there are a couple of things that have been discussed in the school. 14 00:01:29,100 --> 00:01:33,570 I'm not sure anyone's produced any research on it yet, but these are developing areas. 15 00:01:33,570 --> 00:01:36,330 One is the obvious area we've already heard about. 16 00:01:36,330 --> 00:01:47,700 I think a type of deliberate manipulation by which Cambridge Analytica has been accused of manipulating marginal voters towards the extremes. 17 00:01:47,700 --> 00:01:53,490 We don't know how serious that is as an issue. Views differ about it. 18 00:01:53,490 --> 00:01:57,150 But there's another issue perhaps we're more concerned about, 19 00:01:57,150 --> 00:02:03,540 which is how this can happen in an accidental way, and that the business model of Facebook and others, 20 00:02:03,540 --> 00:02:13,530 the click driven module model where you got money for diverting traffic to particular sites, more outrageous site, the more money you'll get. 21 00:02:13,530 --> 00:02:18,030 And so there is no connexion between truth and money. In fact, may be the reverse. 22 00:02:18,030 --> 00:02:24,040 And so the quality of the debates is being corrupted by a business model. 23 00:02:24,040 --> 00:02:29,450 And we need to think about the ways in which that can be done with. 24 00:02:29,450 --> 00:02:38,270 And the second area may be related in a way, but all around the world there is a move to have digital governments one way or another. 25 00:02:38,270 --> 00:02:43,880 We see this with Gulf War UK. Are we seeing all around the world? 26 00:02:43,880 --> 00:02:52,430 Is it a good thing or is it a bad thing? A while ago, people would have discussed in primarily the digital divide and the way in which poorer, 27 00:02:52,430 --> 00:02:57,530 unrepresented groups just don't have access to the internet and therefore not services. 28 00:02:57,530 --> 00:02:59,480 And it's still a concern. 29 00:02:59,480 --> 00:03:07,760 But the introduction of machine learning, deep learning into the provision of government services could be a very serious problem. 30 00:03:07,760 --> 00:03:15,110 One area where it's already being discussed, it was mentioned in the last session is recruitment in government services. 31 00:03:15,110 --> 00:03:25,610 But there are probably many other areas where bias can create then and reinforce existing disadvantage. 32 00:03:25,610 --> 00:03:30,530 This, of course, takes third. The private sector as well. 33 00:03:30,530 --> 00:03:42,290 Notion of automating inequality again machine learning picks up vulnerabilities groups that already disadvantaged, perhaps regarded as high risk. 34 00:03:42,290 --> 00:03:50,620 Well. Become even higher risk through the way they're labelled by machines and something I've only just begun to understand. 35 00:03:50,620 --> 00:03:57,040 Well, I don't understand. I've been introduced to is the way in which. 36 00:03:57,040 --> 00:04:01,420 Companies are typically not building their own machine learning systems from 37 00:04:01,420 --> 00:04:06,220 scratch because the buying in modules or using modules that they don't understand. 38 00:04:06,220 --> 00:04:12,370 Now if you have different companies buying in the same modules and learning the same way. 39 00:04:12,370 --> 00:04:16,180 Is it not likely the same biases will appear? Time and time again? 40 00:04:16,180 --> 00:04:22,660 So if we have if you simply have a bias St. John manager, then OK, that's going to be unfair there. 41 00:04:22,660 --> 00:04:27,580 But somewhere else that's going to be a different bias. And so to some degree, the biases will cancel out. 42 00:04:27,580 --> 00:04:34,810 But if we got the same systems being used over and over again, we see it with credit check agencies or only a couple of agencies. 43 00:04:34,810 --> 00:04:39,470 Now, if you're forced to be blacklisted on one agency, you're going to be screwed everywhere. 44 00:04:39,470 --> 00:04:43,960 The same thing is quite likely to happen in recruitment and in provision of other services. 45 00:04:43,960 --> 00:04:48,880 So it's not only the bias, but it's the same bias everywhere. It's going to be a big problem. 46 00:04:48,880 --> 00:04:53,380 So this is something obviously we want to look into as well. Finally, 47 00:04:53,380 --> 00:04:56,890 something I don't think has come up in this seminar before we got a couple of 48 00:04:56,890 --> 00:05:02,280 colleagues working on autonomous weapons and machine learning and the army. 49 00:05:02,280 --> 00:05:12,070 And one of my colleagues, Tom Simpson, was formerly a soldier. Dapo Akanbi works on the ethics of war. 50 00:05:12,070 --> 00:05:16,960 We already know about drones changing war. What about drones and instruct themselves? 51 00:05:16,960 --> 00:05:21,910 What about some weapons that would pick their own targets? Are we ready for that? 52 00:05:21,910 --> 00:05:28,600 We regulate it. Should we make sure it doesn't happen? So these are some of the issues of going around in the school at the moment. 53 00:05:28,600 --> 00:05:35,680 And I, my own work does relate to the first three, although in terms of outputs, 54 00:05:35,680 --> 00:05:42,520 I have very little to show, which is why I say nothing to show for that. 55 00:05:42,520 --> 00:05:50,290 So the work I've been doing, some of my interest in this area comes from work I've been doing for maybe two decades on the ethics of risk, 56 00:05:50,290 --> 00:05:58,490 and I'm particularly interested in the question of risk and regulation. Now, if you think about why do we regulate anything, 57 00:05:58,490 --> 00:06:07,010 I positive regulation is typically about harmony of standards or making information available to industry, 58 00:06:07,010 --> 00:06:10,880 we have first responders European standards so that people know what they're getting 59 00:06:10,880 --> 00:06:15,440 and you can have a type of interaction between items of different manufacturers, 60 00:06:15,440 --> 00:06:17,310 for example, if they're all using the same standard. 61 00:06:17,310 --> 00:06:23,840 So now you can replace your shower unit quite easily because they're all the same size or the same size because they're regulated. 62 00:06:23,840 --> 00:06:33,890 So positive regulation is about harmony and ease. But negative regulation or negative regulation I regard as a reaction to risk. 63 00:06:33,890 --> 00:06:42,160 So we regulate because there are risks out there. I hear I'm very influenced by something I read by the. 64 00:06:42,160 --> 00:06:46,570 What was once upon a time called the Better Regulation Task Force. 65 00:06:46,570 --> 00:06:51,310 And it was just one line from the better regulation task force about universities. 66 00:06:51,310 --> 00:06:57,100 And we were complaining the universities about how heavily we were being regulated and the Better Regulation Task Force said 67 00:06:57,100 --> 00:07:06,400 it's a surprise that the universities be being regulated to such a degree because there has never been a university failure. 68 00:07:06,400 --> 00:07:12,580 And there has never been a VC sacked for fraud or embezzlement. So the risk in this one of the risks in the sector. 69 00:07:12,580 --> 00:07:18,310 Well, the main risks in the sector, it will be someone who deserves a two one gets a two two, roughly speaking. 70 00:07:18,310 --> 00:07:21,580 That's our main risk other than financial collapse. 71 00:07:21,580 --> 00:07:28,620 And so the level of regulation around the universities, given those minor risks, appears to be completely disproportionate. 72 00:07:28,620 --> 00:07:31,950 So I found this fascinating because it opens up a world of thinking about risk and 73 00:07:31,950 --> 00:07:36,810 regulation because what it suggests is that negative regulation before you regulate, 74 00:07:36,810 --> 00:07:41,070 first of all, I can find your risk. So what is your risk? 75 00:07:41,070 --> 00:07:44,960 What is it you're regulating again? So we tend to have the opposite view. 76 00:07:44,960 --> 00:07:51,690 We need some regulation. Let's get some regulation the other way around. What's the risk and what can we do? 77 00:07:51,690 --> 00:07:58,150 Well, first of all, what is the risk? That's the first question. If you haven't identify the risk you're regulating in the dark. 78 00:07:58,150 --> 00:08:02,960 Second thing is that you should make sure your regulation addresses the risk. 79 00:08:02,960 --> 00:08:09,040 Sounds obvious, but it rarely happens, actually, that quite often we regulate because something needs to be done. 80 00:08:09,040 --> 00:08:12,880 We have to be seen to be doing something. Someone has identified this problem. 81 00:08:12,880 --> 00:08:17,560 We've got to run around and we come up with the six principles of regulation for this area. 82 00:08:17,560 --> 00:08:26,110 And now we've got our principles regulation. What do we do? Nothing. We've spent a lot of time and money coming up with our rules for regulation. 83 00:08:26,110 --> 00:08:29,860 Then next thing is, if you've got regulation, 84 00:08:29,860 --> 00:08:36,490 you've got to make sure that your regulation actually reduces the risk because there are times when regulation increases the risk. 85 00:08:36,490 --> 00:08:43,050 That's also very common. How does that happen? Well, sometimes it's just really bad regulation. 86 00:08:43,050 --> 00:08:51,780 More often it's ineffective regulation and roughly speaking, a belief that you're safe is very dangerous. 87 00:08:51,780 --> 00:08:56,700 That is, if you believe the area is well regulated, you'll drop your guard. 88 00:08:56,700 --> 00:09:03,150 And that's what we've seen in the aviation industry lately. That's what we've seen in the building around building regulations. 89 00:09:03,150 --> 00:09:08,190 Who would have thought we could have had unregulated cladding pretty much on buildings, but that's what we had. 90 00:09:08,190 --> 00:09:12,150 We all assumed this was properly regulated. We looked at it and it wasn't. 91 00:09:12,150 --> 00:09:18,270 So if you think an area is regulated, then you relax and you don't take your own personal god. 92 00:09:18,270 --> 00:09:25,680 So it's actually better to have no regulation than bad regulation, Mary, because then we can all take care for ourselves. 93 00:09:25,680 --> 00:09:30,660 So these are the background, some of the background assumptions I'm making. We have to think about second round effects. 94 00:09:30,660 --> 00:09:34,440 We also have to think about the fact that generally speaking, 95 00:09:34,440 --> 00:09:39,660 the agencies we're regulating have a financial interest in avoiding that regulation if we can. 96 00:09:39,660 --> 00:09:47,220 Regulation is typically ineffective. If you're sceptical about this, consider most of us work in universities. 97 00:09:47,220 --> 00:09:51,570 We're getting regulation all the time coming from a level above. 98 00:09:51,570 --> 00:09:57,750 What do you do when you get some regulation that stops you that would stop you doing something you want to do? 99 00:09:57,750 --> 00:10:03,870 One of your first instincts and mine is to work out how I can carry on doing what I want to do. 100 00:10:03,870 --> 00:10:05,630 Compliant with the regulation. 101 00:10:05,630 --> 00:10:14,820 OK, so that is our instinct when we are regulated and we don't identify with the goals of the regulation, we all do it all the time. 102 00:10:14,820 --> 00:10:25,060 So most regulation is useless and is counterproductive. Doesn't mean we shouldn't have it means we should be very, very careful about how we regulate. 103 00:10:25,060 --> 00:10:32,320 But a second point. Regulatory drift, negligence, gifting countries and markets, what I mean here. 104 00:10:32,320 --> 00:10:41,640 Well, so I'm also very interested in the regulator once we've got a regulation and how the regulator behaves, particularly over time. 105 00:10:41,640 --> 00:10:51,430 Because if we have a new area and this is not just any area of new technology, quite often when it comes in as a lot of fuss, 106 00:10:51,430 --> 00:10:57,730 a lot of action, a lot of money goes, a lot of people declare themselves the experts and we and some of them are. 107 00:10:57,730 --> 00:11:02,640 And we can get some OK regulation to begin with. 108 00:11:02,640 --> 00:11:05,140 What happens over time? 109 00:11:05,140 --> 00:11:11,500 Well, if nothing bad happens, we have a regulation, nothing bad happens, we begin to lose interest, something else comes along. 110 00:11:11,500 --> 00:11:20,330 We take our eye off the ball. So there's been a negligence that will come in that we will think this area is OK and we will begin to neglect its. 111 00:11:20,330 --> 00:11:29,550 Later on, we'll have a government that says there's too much red tape, we've got to cut red tape, two regulations for every everyone that comes in. 112 00:11:29,550 --> 00:11:35,270 Doesn't matter what they are. OK, this is an area that hasn't been to be affected, so maybe we will too. 113 00:11:35,270 --> 00:11:45,290 Well, my friend Judy Brown in Cambridge calls regulatory gifting where the regulations giving given back to the industry as a form of self-regulation. 114 00:11:45,290 --> 00:11:50,180 Normally, the industry promises something in return, but it never gets it right. 115 00:11:50,180 --> 00:11:57,320 So regulation is reduced and the benefit is entirely for the industry itself rather than others. 116 00:11:57,320 --> 00:12:06,560 Third, regulatory capture. Well, so this is a way in which corporations cosy up to the regulator will entertain the regulators. 117 00:12:06,560 --> 00:12:12,590 They will put pressure on the government and regulation becomes more and more favourable to the industry. 118 00:12:12,590 --> 00:12:20,150 Finally, one thin market so this is from our colleague Karthick Ramana, who has written I'm not sure if he said this or I said it, 119 00:12:20,150 --> 00:12:28,780 but I think the sense in which what he works on is the most boring thing you could possibly work on, which is the. 120 00:12:28,780 --> 00:12:38,290 Regulatory standards in the accountants accountancy industry and what he says is this this is incredibly boring, 121 00:12:38,290 --> 00:12:42,730 he wants to talk about regulating accountants. But in fact, his business school, no, 122 00:12:42,730 --> 00:12:49,430 this is one of the most important thing that goes on at the moment because so many auditors have failed in the work. 123 00:12:49,430 --> 00:12:54,800 So what is happening? Why is auditing accountancy found so difficult to get right? 124 00:12:54,800 --> 00:13:00,620 Well, it's because hardly anyone can do it, and they're all employed by accountancy firms. 125 00:13:00,620 --> 00:13:05,310 And so you have thin markets in these very specialised areas. 126 00:13:05,310 --> 00:13:13,340 So areas of new high technology, that's almost no one in the world with the knowledge to regulate the area. 127 00:13:13,340 --> 00:13:19,280 So if you think about financial markets and derivatives or some of these markets where almost no one understands that if you want to regulate, 128 00:13:19,280 --> 00:13:25,640 you've got to bring the. Same thing is happening in solar radiation management, I'm sure the same thing is happening. 129 00:13:25,640 --> 00:13:32,330 I, as a small number of people, the usual suspects turn up every conference they give a keynotes set on the panels they probably 130 00:13:32,330 --> 00:13:36,920 once worked for organisations or they're going to work for those organisations in the future, 131 00:13:36,920 --> 00:13:46,190 even if they believe they're completely unbiased. They will see the world the particular way they will likely to. 132 00:13:46,190 --> 00:13:53,180 Be much more sympathetic to the corporate interests and less sympathetic to consumer interest, for example, what do we do about that? 133 00:13:53,180 --> 00:13:58,550 Do we have massive regulation and even high regulation of the regulation? And what about corporate capture for that? 134 00:13:58,550 --> 00:14:03,350 I think we have to go in the other direction and in some areas around about environmental policy. 135 00:14:03,350 --> 00:14:08,480 For example, civil society groups have been assigned to check on the regulator. 136 00:14:08,480 --> 00:14:17,480 I saw this years ago with gene watch when genetic engineering came in and they always made what you might think the most extreme annoying arguments. 137 00:14:17,480 --> 00:14:23,480 But we need people in the world to irritates the regulator, to get them to take things, to look for the middle ground. 138 00:14:23,480 --> 00:14:28,520 So we need civil society groups to stir things up and never give anyone a moment's rest. 139 00:14:28,520 --> 00:14:32,570 Nothing. It the remains best. And that's probably what we need in this area, too. 140 00:14:32,570 --> 00:14:33,800 I'm not sure it's happening yet. 141 00:14:33,800 --> 00:14:40,730 We've got people writing very good books, but I'm not sure I'm seeing a lot of civil society follow up from that, so I don't know where we can. 142 00:14:40,730 --> 00:14:44,230 All the money's on the other side, of course, in this case. 143 00:14:44,230 --> 00:14:54,400 The question of values of what values should or are embedded in technology is, of course, a perennial question. 144 00:14:54,400 --> 00:14:57,310 Technologists and society at large, however, 145 00:14:57,310 --> 00:15:06,250 have often taken this relationship for granted and sometimes falsely assume that technology is value neutral. 146 00:15:06,250 --> 00:15:13,390 A normative conception of value, of course, has deep philosophical roots and outside philosophy. 147 00:15:13,390 --> 00:15:20,020 There is a considerable body of work on the relationship between technology and values. 148 00:15:20,020 --> 00:15:25,570 For instance, the interdisciplinary field of science and technology studies, or stress, 149 00:15:25,570 --> 00:15:32,560 builds on the insight that values are embedded in technological choices and specific technologies. 150 00:15:32,560 --> 00:15:37,210 Far from being independent of human desire and intention, 151 00:15:37,210 --> 00:15:44,950 Sheila just enough explains technologies are, quote, subservient to social forces all the way through. 152 00:15:44,950 --> 00:15:49,510 In his seminal 1980 article, Do Artefacts Have Politics? 153 00:15:49,510 --> 00:15:54,160 Langdon winner argued that technologies can have political properties, 154 00:15:54,160 --> 00:16:00,220 either by influencing power relations or corresponding to certain structural arrangements. 155 00:16:00,220 --> 00:16:08,740 Technologies can reinforce unjust and unequal relations, and technical choices can lock in particular values. 156 00:16:08,740 --> 00:16:16,900 He cited the example of Robert Moses. His 20th century designs of New York City reportedly contributed to deepening 157 00:16:16,900 --> 00:16:25,670 racial hierarchy through infrastructure that restricted public transport flows. 158 00:16:25,670 --> 00:16:34,320 There's also been constrict, constructive, considerable constructive responses to these kinds of worries since the early 1990s, 159 00:16:34,320 --> 00:16:42,230 for example, an approach known as value sensitive design or vesti drawing on fields such as anthropology, 160 00:16:42,230 --> 00:16:50,210 philosophy and engineering, has sought to bring values into technological processes via a range of theoretical, 161 00:16:50,210 --> 00:16:53,120 methodological and practical proposals. 162 00:16:53,120 --> 00:17:02,160 The study attempts to find intersections between technical innovation and human flourishing through methods such as stakeholder analysis. 163 00:17:02,160 --> 00:17:06,840 I'd like to structure the rest of my remarks in three parts, so that's two to four. 164 00:17:06,840 --> 00:17:14,220 First, I'll introduce the problem of value alignment in AI ethics and given time restraint constraints, 165 00:17:14,220 --> 00:17:25,770 I'll just stick to some high level normative points rather than try to cover a lot of the innovative technical research that's going on in that area. 166 00:17:25,770 --> 00:17:31,980 I then want to highlight what I will call a political turn in value alignment research. 167 00:17:31,980 --> 00:17:42,390 And finally, I will situate the question of value alignment in what I call a social ecology of AI, an ethical AI development. 168 00:17:42,390 --> 00:17:47,830 I should acknowledge that my remarks draw heavily from work I've done with my collaborator, 169 00:17:47,830 --> 00:17:52,710 Yasmin Gabriel, a philosopher and a senior research scientist at DeepMind. 170 00:17:52,710 --> 00:17:57,510 But the views expressed should not be attributed to DeepMind. 171 00:17:57,510 --> 00:18:04,920 I will return to your son's own work in a moment. OK, so the second point on value alignment and AI. 172 00:18:04,920 --> 00:18:15,150 There has long been a view expressed in science fiction since the field emerged in the 1950s that AI poses a distinctive moral challenge. 173 00:18:15,150 --> 00:18:22,290 The value alignment challenge refers to the one of aligning powerful technologies with human values has risen 174 00:18:22,290 --> 00:18:28,590 to the fore in light of the potential for something that has been called Artificial General Intelligence, 175 00:18:28,590 --> 00:18:36,720 or AGI. This is A.I. which would match or exceed human level intelligence across different domains. 176 00:18:36,720 --> 00:18:40,770 A prominent A.I. researcher referred to earlier, Stuart Russell, 177 00:18:40,770 --> 00:18:47,520 describes a failure of value alignment as arising when quote We perhaps inadvertently 178 00:18:47,520 --> 00:18:53,880 imbue machines with objective objectives that are imperfectly aligned with our own. 179 00:18:53,880 --> 00:19:00,810 Since the 1990s, A.I. researchers have recognised this as a specific area in need of attention. 180 00:19:00,810 --> 00:19:01,800 More recently, 181 00:19:01,800 --> 00:19:10,290 it has become a theme in specialised discourses such as philosophy and computer science and increasingly discussed in the public sphere. 182 00:19:10,290 --> 00:19:16,650 A major concern is that an AGI incentivised around a particular reward might pursue that reward 183 00:19:16,650 --> 00:19:23,130 without respecting the wider values of its designers users or affect other affected persons, 184 00:19:23,130 --> 00:19:27,780 generating systematic pressure towards value misalignment. 185 00:19:27,780 --> 00:19:34,410 Relatedly, Nick Bostrom has pointed to the existential risks of a superintelligence that is one that 186 00:19:34,410 --> 00:19:41,110 greatly exceeds the cognitive performance of humans in virtually all domains of interest. 187 00:19:41,110 --> 00:19:47,020 But existing A.I. systems already exhibit a degree of value misalignment by amplifying 188 00:19:47,020 --> 00:19:52,120 social disadvantage in ways that diverge from the purported values of equality or justice, 189 00:19:52,120 --> 00:19:56,650 for instance, held by their designers or wider stakeholders. 190 00:19:56,650 --> 00:20:07,480 For example, algorithmic bias and unfairness have been documented widely now in criminal justice systems, social welfare and insurance markets. 191 00:20:07,480 --> 00:20:13,480 Social media algorithms have come under intense scrutiny for their counterproductive consequences, 192 00:20:13,480 --> 00:20:25,480 and these misalignment risks are all exasperated, exacerbated by a lack of accountability in what Frank Pasquale has called the Black Box Society. 193 00:20:25,480 --> 00:20:35,620 Such challenges have led to the emergence of a vibrant fairness, accountability and transparency or fat star research community. 194 00:20:35,620 --> 00:20:40,030 So value alignment, we can say, has two dimensions. 195 00:20:40,030 --> 00:20:46,030 It has has technical dimensions, and that is how to align AI with human values, 196 00:20:46,030 --> 00:20:51,700 as well as normative dimensions, deciding what values to align artificial agents with. 197 00:20:51,700 --> 00:20:54,940 And I'll focus here just on the latter. 198 00:20:54,940 --> 00:21:03,730 There are features of AI's their potential speed, scale and complexity, which makes value alignment in this area distinctive. 199 00:21:03,730 --> 00:21:08,380 I think from other technologies, as artificial intelligence become more advanced, 200 00:21:08,380 --> 00:21:19,270 they have a wider range of decisions open to them and as they make such decisions in areas formerly reserved for human judgement and control. 201 00:21:19,270 --> 00:21:25,000 This increased. You might call it autonomy becomes morally significant. 202 00:21:25,000 --> 00:21:37,840 And but normatively, there has been a range of responses to this to to this in the literature on safety and control of advanced A.I. 203 00:21:37,840 --> 00:21:43,600 The standard approach uses instructions to ensure value aligned outcomes. 204 00:21:43,600 --> 00:21:48,340 Here, the idea is to include as much safety or other value. 205 00:21:48,340 --> 00:21:54,710 Preserving criteria as possible in the design is instructions. 206 00:21:54,710 --> 00:22:05,210 But unsatisfied with the risks of the standard model, some are focussed on creating agents that behave in accordance with the user's intentions. 207 00:22:05,210 --> 00:22:11,450 A third approach advanced by Stuart Russell in his book, a recent book called Human Compatible Advocates, 208 00:22:11,450 --> 00:22:17,180 shifting the onus to artificial agents to infer human preferences. 209 00:22:17,180 --> 00:22:25,970 In this approach, a utility function would capture preferences in an algorithmic form that a machine can process. 210 00:22:25,970 --> 00:22:30,830 Importantly, machines would be designed so that they defer to humans. 211 00:22:30,830 --> 00:22:40,520 He calls it a kind of humility and ask them they would ask permissions and act cautiously when guidance is unclear 212 00:22:40,520 --> 00:22:50,820 and allow themselves to be switched off the switched off being important for that AGI or superintelligence worry. 213 00:22:50,820 --> 00:22:53,940 But each of these approaches has limitations. 214 00:22:53,940 --> 00:23:03,600 For one thing, they struggle to generate a legitimate basis to discuss or decide what value should be embedded in our systems. 215 00:23:03,600 --> 00:23:09,180 So now I'd like to turn to my third point on the political turn. 216 00:23:09,180 --> 00:23:16,830 Johannes Zimmer has recently made the case for political philosophy as distinct from moral 217 00:23:16,830 --> 00:23:22,560 philosophy or social science to be given greater consideration in technology ethics. 218 00:23:22,560 --> 00:23:28,590 And he uses the specific example of self-driving cars to illustrate his point. 219 00:23:28,590 --> 00:23:34,230 In his view, relying on moral theories is problematic, since it does not always. 220 00:23:34,230 --> 00:23:45,320 It is not always clear which theory is correct and society's, he thinks should generally aim to preserve human agency and autonomy. 221 00:23:45,320 --> 00:23:51,470 Tabulating preferences, on the other hand, which would be a more social scientific approach, 222 00:23:51,470 --> 00:23:56,720 perhaps can include discriminatory and unfair preference preferences. 223 00:23:56,720 --> 00:24:06,500 A political philosophy approach has three advantages, according to him the British respecting value pluralism. 224 00:24:06,500 --> 00:24:08,870 The fact that people have different values. 225 00:24:08,870 --> 00:24:18,890 Respecting human agency in a time of me and being sensitive to issues of legitimate authority or why we should abide by certain decisions. 226 00:24:18,890 --> 00:24:26,270 And this thesis fits with what I would like to characterise as an early political turn in value alignment research. 227 00:24:26,270 --> 00:24:33,380 And the pathbreaking contribution along these lines was recently made by Yasmin Gabriel in an article published last month. 228 00:24:33,380 --> 00:24:44,140 Gabriel disentangled A.I. that aligns with instructions, intentions revealed preferences, ideal preferences, interests and values. 229 00:24:44,140 --> 00:24:53,350 And drawing on with this analytical clarity enhancer, he advocates for a principle based approach to alignment, 230 00:24:53,350 --> 00:25:01,180 drawing on the great liberal philosopher John rules this insight on the fact of reasonable pluralism. 231 00:25:01,180 --> 00:25:06,280 He suggests that the key challenge is not to identify true moral principles, 232 00:25:06,280 --> 00:25:16,570 but rather to identify fair principles for alignment that receive reflective endorsement despite widespread variation in people's moral moral beliefs. 233 00:25:16,570 --> 00:25:25,060 Gabriel suggests three possibilities for deriving such fair principles based on what Rawls called an overlapping consensus. 234 00:25:25,060 --> 00:25:33,730 These are global human rights law and discourse, hypothetical agreement from behind the veil of ignorance and social choice theory, 235 00:25:33,730 --> 00:25:37,240 given time constraints, I can't elaborate on the specifics. 236 00:25:37,240 --> 00:25:45,370 Instead, I just want to point to five reasons why I think Gabriel's approach is immensely generative for future thinking about A.I. ethics. 237 00:25:45,370 --> 00:25:50,470 First, as calls for democratic oversight of tech A.I. technology growth, 238 00:25:50,470 --> 00:25:54,850 this approach offers the basis to engage constructively with the diverse values 239 00:25:54,850 --> 00:26:01,360 people hold and to channel these into the design and development of AI technologies. 240 00:26:01,360 --> 00:26:07,090 Second, the proposals offer technologists a new conceptual mapping for their work, 241 00:26:07,090 --> 00:26:14,920 grounded in the perspective of ordinary citizens with proper regard for vulnerable people and groups in particular. 242 00:26:14,920 --> 00:26:22,450 Third, as numerous public and private bodies begin to specify principles for the governance and use of A.I., 243 00:26:22,450 --> 00:26:26,650 his approach offers new ways to evaluate these initiatives. 244 00:26:26,650 --> 00:26:33,040 Fourth, crucially, Gabriel's approach recognises the transnational nature of the value alignment challenge, 245 00:26:33,040 --> 00:26:40,000 given that the potential global reach of AI technology, and he builds this into his proposals from the outset. 246 00:26:40,000 --> 00:26:46,870 Finally, it demonstrates how the tools in a methodological sense of political philosophy and political theory, 247 00:26:46,870 --> 00:26:54,730 can inform public policy and deliberation on transformative technologies like A.I., as well as technical choices involved. 248 00:26:54,730 --> 00:26:58,180 So finally, I want to turn to this point. 249 00:26:58,180 --> 00:27:02,860 My last point, which is inspired by the political turn. 250 00:27:02,860 --> 00:27:10,150 I want to say that I want to suggest that future A.I. ethics research should pay more attention 251 00:27:10,150 --> 00:27:15,490 to the evolution of what I call the social ecology that's geared towards solving problems, 252 00:27:15,490 --> 00:27:20,050 addressing public problems and learning from the failures of existing technologies. 253 00:27:20,050 --> 00:27:22,870 And this reflects some of my own research interests. 254 00:27:22,870 --> 00:27:29,980 Prominent AI researchers have suggested innovating in how we conceive and analyse the ethical impact of A.I. 255 00:27:29,980 --> 00:27:35,620 For instance, Kate Crawford and Ryan Keller have promoted the use of what they call social systems 256 00:27:35,620 --> 00:27:41,470 analysis to account for all the possible effects of AI systems on all parties in design, 257 00:27:41,470 --> 00:27:45,550 deployment and regulation to meet, Gebru says. 258 00:27:45,550 --> 00:27:51,820 Rightly, I think that AI researchers should learn about the ways in which their technology is being used, 259 00:27:51,820 --> 00:27:58,540 question the direction institutions are moving in and engage with other disciplines to learn from their approaches. 260 00:27:58,540 --> 00:28:05,890 She says those studying fairness, accountability, transparency and ethics in AI should forge collaborations across disciplinary, 261 00:28:05,890 --> 00:28:16,110 geographic, demographic, institutional and social and socioeconomic boundaries, and help lift the voices of those who are marginalised. 262 00:28:16,110 --> 00:28:18,750 So apart from deriving a ranking of values, 263 00:28:18,750 --> 00:28:29,400 then we must additionally ask how to engender what a philosopher Alex Springer calls critical responsiveness amongst agents, 264 00:28:29,400 --> 00:28:32,880 given their situated ness within a social ecology. 265 00:28:32,880 --> 00:28:41,640 In her book Communicating Moral Concern, Springer argues critical engagement is a kind of moral work, and we bring different perceptions, 266 00:28:41,640 --> 00:28:51,090 talents and social relations to it, rather than trying to build an account of generic agents through philosophical moral theory to promote this. 267 00:28:51,090 --> 00:29:00,660 She suggests at best, we can illuminate the process by which we each give varying weights to the concerns that emerge around and within us. 268 00:29:00,660 --> 00:29:09,570 The social ecology. So, so is this terrain that, she says, quote, We struggle to make a distinctive and viable space place. 269 00:29:09,570 --> 00:29:21,290 Sorry for our evolving projects. Typically, we use normative theory to narrow the relevant agents who have specific duties or responsibilities. 270 00:29:21,290 --> 00:29:28,140 I'm interested in how it can also assist us to develop complementary ways of thinking about pluralism. 271 00:29:28,140 --> 00:29:34,470 The scope of Mali engaged agents not only as victims of injustice or inequality, 272 00:29:34,470 --> 00:29:39,150 but as constructive protagonists in collective projects of human flourishing. 273 00:29:39,150 --> 00:29:45,210 The fact that A.I. research is largely concentrated in particular parts of rich Western 274 00:29:45,210 --> 00:29:50,790 countries means certain background assumptions are likely to define the parameters of value, 275 00:29:50,790 --> 00:29:59,280 line research and practical mechanisms. There is a worry that high profile and privileged voices may stifle the possibility of 276 00:29:59,280 --> 00:30:05,160 alternative viewpoints that mask underlying contentiousness or conflicting interests. 277 00:30:05,160 --> 00:30:07,290 Indeed, there is also growing recognition. 278 00:30:07,290 --> 00:30:15,990 The value alignment efforts must guard against distorting effects of Western biased ideological blinkers and neglect of feminist perspectives. 279 00:30:15,990 --> 00:30:24,600 As one researcher, Shakir Muhammad notes, A.I. is currently localised and quote within restricted geographies and people. 280 00:30:24,600 --> 00:30:33,280 He continues, We are talking about A.I. Researchers like himself rely on inherited thinking and sets of unquestioned values. 281 00:30:33,280 --> 00:30:41,070 We reinforce selective histories. We fail to consider our technology's impacts and the possibilities of alternative paths. 282 00:30:41,070 --> 00:30:46,260 We consider our work to be universally beneficial, needed and welcome. 283 00:30:46,260 --> 00:30:51,150 So an account of social ecology can help evaluate the interactions the social and 284 00:30:51,150 --> 00:30:57,090 moral learning processes that grapple with these epistemic and legitimacy challenges. 285 00:30:57,090 --> 00:31:03,840 Through this lens, many agents have standing to contribute to the generation of relevant practical knowledge. 286 00:31:03,840 --> 00:31:07,740 I can think of at least seven sets of agents. There are more. 287 00:31:07,740 --> 00:31:13,380 These are technologists, technology companies, the media, universities, policymakers and regulators, 288 00:31:13,380 --> 00:31:21,630 social movements and community organisations and all individuals, I think conceived as citizens in the Republican sense. 289 00:31:21,630 --> 00:31:26,190 So Social Ecology Lens considers how to motivate these agents participation in 290 00:31:26,190 --> 00:31:31,500 creating and sustaining apt institutional structures and social processes. 291 00:31:31,500 --> 00:31:38,970 After all, any mechanism for determining legitimate values, I think, depends to some extent, 292 00:31:38,970 --> 00:31:43,950 at least on the emergence of ethically robust and trustworthy institutions. 293 00:31:43,950 --> 00:31:49,620 The development of social practises that facilitate the effective communication and moral concern, 294 00:31:49,620 --> 00:31:59,850 and individuals with sufficient allegiance to realising ideals like justice in the face of competing pressures, such as self seeking economic gain. 295 00:31:59,850 --> 00:32:06,360 And I'll just give a couple of very brief comments and to sort of flesh it out a little bit before I finish. 296 00:32:06,360 --> 00:32:11,340 So consider how technology firms can become subject to greater democratic oversight. 297 00:32:11,340 --> 00:32:19,410 They must find openings for genuine, creative and constructive engagement with all communities and people affected by their work. 298 00:32:19,410 --> 00:32:23,220 They can also innovate with corporate governance arrangements, 299 00:32:23,220 --> 00:32:31,110 recognising that such regimes are more malleable than commonly assumed in Orthodox models of capitalism. 300 00:32:31,110 --> 00:32:38,790 A social ecology lens can encompass such questions of political normative political economy on things like the Constitution, 301 00:32:38,790 --> 00:32:47,640 incentives of firms, the social function of finance and the dynamics of market competition and cooperation and through an ecological lens. 302 00:32:47,640 --> 00:32:55,740 Universities, for example, can have a crucial role in terms of educating future technologists and policymakers, 303 00:32:55,740 --> 00:33:00,600 as well as promoting interdisciplinary approaches to value alignment. 304 00:33:00,600 --> 00:33:08,010 As a relatively new area of enquiry, there is an opportunity to harness different branches of knowledge as part of a unified research agenda. 305 00:33:08,010 --> 00:33:15,680 But interdisciplinary alignment research requires enhanced trust and collaboration across intellectual bound. 306 00:33:15,680 --> 00:33:20,960 Including the development of shared vocabularies. So finally, 307 00:33:20,960 --> 00:33:30,230 citizens social movements and ultimately the state should ensure that both the public sphere and in policymaking 308 00:33:30,230 --> 00:33:36,350 that value alignment efforts are subject to continued scrutiny from the perspective of the common good. 309 00:33:36,350 --> 00:33:40,280 So when we begin to desegregate agency in these kinds of ways, 310 00:33:40,280 --> 00:33:47,780 I think it's possible to envision not only how to improve value alignment in the design of specific technologies, 311 00:33:47,780 --> 00:33:51,722 but also how society at large can support that goal.