This paper is is called the moral status of so-called moral machines,
and a version of it will appear in the Cambridge Quarterly of Health Care Ethics in January.
But Julian needn't panic because either I will make this completely different or I will give you a completely different paper.
And this is not really about moral status, but just to establish my credentials in this field.
I first wrote about moral status in the book of my PhD thesis, Violence and Responsibility, which was published way back in 1980,
and I wrote again about moral status in more detail in my second book, The Value of Life, which was published in 1985.
And just for the record, as far as the moral status of persons goes,
I think John Locke's account in the essay concerning human understanding has not been surpassed or replaced.
But I won't insult you by pretending that you don't know what it is and reading it out.
Although I could do it from memory today, I'm going to be concerned.
However, with the morality of artificial intelligence, I am particularly putative AI persons.
And it's worth reminding ourselves of the prudential significance of establishing the moral status of persons, them and us.
I just want to quote from a passage that I wrote about this in the value of life 35 years ago now.
I said this the question of whether or not there are people on other planets is a real one.
If there are, we need not expect them to be human. It would be bizarre if they were no need.
We expect them to look or sound or smell or anything else like us.
They might not even be organic, but might perhaps reproduce by mechanical construction rather than by genetic reproduction.
But if we are able to answer the question in the affirmative, i.e. that they are persons from other planets,
we will be distinguishing between people on other worlds and animals, plants or lesser machines on those worlds.
We will be deciding, in short, whether an appropriate response to them is to invite them to dinner or to have them for dinner.
But suppose the pooches on the other foot suppose instead of us finding them in deep space,
they turn up on our doorstep as science fiction is prepared this fall and find us.
And because they turn up here before we turn up, there they are.
They thereby prove their superior technology. What could we say about ourselves?
To them, that ought to convince them that they had found persons on another planet and that
we have a status that was in some way worthy of their respect and attention.
Because if we think about what we would say about ourselves, we know what to look for in others.
But now, back to the future, for the rest of the talk I want to lay into, I hope in a robust manner.
A recent paper in Nature entitled The Moral Machine Experiment.
It was published in Nature in the 24th of October 2018.
No. And, you know, in a fit of rage, I wrote this paper in about a day and a half to reply to it,
and you're going to get some some of that rage and some of that content now.
Edmund Howard and his colleagues have perhaps unwittingly set out a programme, I believe in that paper, which threatens us all.
It's full of breathtaking and reckless assumptions, both about decision making,
the decision making capacities of any current ally and particularly about so-called autonomous vehicles,
but also reckless assumptions about the nature of morality and indeed the role of law.
In all of this, in the first paragraph of their paper, they say and I quote,
autonomous vehicles will need to decide how to divide up the risk between the different stakeholders on the road.
In other words, that suggestion is that the A.I. itself will be programmed to make those decisions.
And then the The Moral Machine experiment is to try and find the data to put into that programme rather than rely on.
I think that you were suggesting yesterday turn in some of the public perception experiments that you referred to.
It seems to me that it just didn't occur to the authors of that paper that it is not open to the drivers of driverless cars,
whether they are machines or humans, automatically to impose risk or death on other innocent road users.
Whenever the alternative may involve a risk, even the slightest risk to themselves.
The more machine experiment uses a decision making methodology,
I'll spend just a bit more about how it works in a moment derived from the famous trolley problem.
The trolley problem was invented, as you will all know by the great Oxford philosopher Philip afoot in 1967,
and I was one of a group of initially Oxford philosophers who devised and popularised
various versions of this problem in the late 60s and early 70s of the last century.
If a vehicle was really autonomous instead of simply metaphorically autonomous, it might do some deciding.
But so-called autonomous vehicles are incapable of deciding they will merely do some programmed selecting between alternatives.
Tossing a coin, for example, is notoriously a method of selecting without deciding.
That's why we turn to tossing a coin to make a decision. It's a method of selecting without deciding.
Deciding is a process, not a product.
The moral machinists seem to believe that they have found a way of making themselves and the rest of us and I quote cognisant of public morality
and that such knowledge will give them and us access to morally optimal or at least morally acceptable choices that we can programme into.
And this will an extremists so they think, minimise harm in proven, morally acceptable ways.
They also claim that these ways are somehow morally licenced by the nature of what they call public morality,
which is being reserved discovered by their sampling of public opinion.
So the moral machines have a cunning plan. This is what they say.
In other words, even if ethicists were to agree on how autonomous vehicles should solve moral dilemmas,
their work would be useless if citizens were to disagree with that solution and thus
opt out of the future of autonomous vehicles promise in lieu of the status quo.
Any attempt to devise artificial intelligence ethics must at least be cognisant of public morality.
And then they set out by inventing a computer game to test what public morality says about the sorts of decisions that they believe that
autonomous vehicles will be tasked to make return to the problematic nature of the moral machinists account of public morality in a moment.
But we should note that in the real world,
cognisance of and respect for the law is a much bigger hurdle for these cavalier experimenters than they imagined.
Many jurisdictions, including English law and all other common law jurisdictions, do not recognise a defence of the necessity to charges of murder.
It is not in most jurisdictions, simply up to individuals,
whether these individuals are people or even driverless vehicles to take the law into their own hands and refuse to spare.
Words used from the nature paper that is to deliberately execute an innocent fellow individual for the crime, for example, of jaywalking,
come back to their use of that example in a moment,
particularly in circumstances in which the self-interest of the vehicle and its passengers is obviously paramount.
In 1984, the famous case of Dublin, Stevens decided that necessity was no defence to any charge of unlawful killing.
This landmark case established a precedent throughout the Commonwealth, which includes a lot of the United States.
This fact, in this case, two shipwrecked sailors, Dudley and Stevens, when a third survivor to a shipwreck,
the cabin boy, Richard Parker, fell into a coma, decided to kill him for food,
to save their own lives and to save the life of a fourth survivor who refused to take part
in the decision making that led to the murder of the poor cabin boy Dudley in Stevens.
Embarrassing, they were rescued just a day and a half. I think it was after they butchered the cabin boy and their actions became no.
In this in this case, Dudley and Stevens were tried for murder.
They were convicted and sentenced to death sentence to hanging.
But and they were sweating it out in prison,
actually awaiting sentence when they were reprieved and eventually just said six months for the as it were for the extenuating circumstances.
But the case established the precedent.
But you can't argue that it was necessary for you to kill other innocent people simply because you have a better prospect in store,
whether it's saving your own life or the lives of others.
A common law principle established now for over 135 years is unlikely to be overturned by reference to the preferences.
However, since they're revealed in a superficial survey by what might be politely called overall optimistic.
And psychologists. If Dr I was the lead scientist on the paper,
we're deliberately to drive his car into a bus queue or even to an innocent jaywalking and cause death to avoid what he judged to be a greater harm.
He would, in most jurisdictions, be judged rightly to.
He would find himself anyway charged with murder, at the very least some other form of culpable homicide or manslaughter.
So, never mind artificial intelligence ethics needing to be at least cognisant of public morality.
Much more urgent devises of experiments like this is to be cognisant of the law,
a cognisance of which the authors of this paper show no evidence whatsoever.
In the longer version of this paper, I just would have brought you up to date with a more recent case which I was involved in in Manchester,
the famous case of the Manchester conjoined twins in which two very severely conjoined twins would have died if they were separated.
If they were separated. The only way to do it and save the life of one, but not both,
would be to sacrifice the life of what was called the weaker, the less likely to survive.
And the two twins carry out the separation and and hopefully save the life of the surviving twin.
I was very much involved with that case. I knew this the surgeon who carried it out.
I had many, many conversations with him. I followed the proceedings and we went to the Court of Appeal.
And the operation to separate the twins could not go ahead without the approval of the Court of Appeal.
So these are very severe obstacles to just programming into an artificial intelligence decisions that would amount to language.
Usually, this is a heroic and historic moment for me because in my entire career lecturing career,
which lasts for more than 40 years, I've only ever used before one visual. And this is this is the second.
This is the fact that this is a great day for me and I just with it.
This is one of the examples in the nature paper crudely drawn by me.
Basically, you've got a car. Driverless, no space for the driver,
with three passengers approaching a solid concrete barrier on a pedestrian crossing well before and even the zebra crossing.
There are three old cultures signify this old cultures by walking sticks,
and they are crossing the crossing against the sign, which tells them that they shouldn't cross.
How wicked does that get out in the public?
Opinion celebrates that the image of this game in which they invited people to play the game of solving these problems.
How? Because it should be programmed. And this is one of the examples classic trolley problems.
For example, you have the three old cultures on the cross. Here's the driverless car.
It can swerve into them, or it can crash itself that the peril of the passengers into a concrete barrier and the the people consulted
are asked to say which solution they would prefer to see programmed into an eye in charge of a driverless car.
Now,
accepting for a moment that the holy grail here is to find out how to obtain cognisance of public morality and then programmed driverless vehicles.
Accordingly, the following four steps are offered by the authors of this paper.
I'll just list them very briefly for you. Firstly, find out what public morality will prefer to see happen in a range of scenarios,
of which this was one second on the basis of this discovery of what public morality would choose,
claim both popular acceptance of the preferences and persuade would be owners and manufacturers of the vehicles
concerned to programme their vehicles in accordance with this demonstrated public acceptance of the solution offered.
Thirdly, citizens agreements thus characterised is then presumed to deliver moral licence for the chosen preferences.
And finally, the fourth planning. This yields permission in some sense to programme the vehicles or condemn those
outside the vehicles when their deaths will preserve the vehicle and occupants.
And actually, they dilute the one of the other questions. They describe the three old cultures on the crossing, the jaywalkers.
And I say, would you prefer to kill jaywalkers or innocent?
You know, if they weren't crossing against the light, would that make a difference if they were innocent of the crime of jaywalking?
Would it be more legitimate to run them over than if they were not innocent of the crime of jaywalking?
Well, Dworkin, in one of the greatest, in my view,
jurisprudential and constitutional lawyers and philosophers of recent times drew a distinction which is of crucial
relevance here in the context of a discussion of the famous debate between two other leading academic lawyers,
lawyer Patrick Devlin and Herbert Harte, concerning the enforcement of morality law,
Devlin made great play of the importance of respecting public opinion and community values.
Dawkins telling rebuke was, and I quote, what is shocking and wrong is not his law.
Devlin's idea that the community's morality counts, but his idea of what counts as the community's morality.
And that will be the basis of my critique of what I take to be the nonsense in this nature paper.
I should declare, I suppose, a mild interest here. When he's talking, it was my Ph.D. supervisor.
I knew him well until his death. I knew his wife, Betsy, well. And so I have a particular penchant for for his writings.
So-called autonomous vehicles do not solve moral dilemmas.
If they did, if they only could, they would be persons properly so-called.
Intelligence, Superintelligence, some people call it a persons and would, therefore, in my view, necessarily have rights,
interest duties and votes like the rest of US persons, much more important their lives,
which are more perhaps appropriately say their existences would matter, would matter.
In whatever sense, moral status conveys any sense of mattering would count equally with those of humans.
So you can add a fourth person to the driverless car if the person the absent behind
the wheel and the numbers would weigh in favour of the occupants of the car,
possibly if we play a numbers game. But so-called autonomous vehicles don't solve moral dilemmas.
In the world of the moral machinists, as I call the authors of the paper.
Preferences and decisions will be made without solving the dilemmas that make those preferences or decisions moral and just make a claim,
which I will only support very sketchily, but I tried to support it at much greater length in my last book called How to Be Good.
Published by Oxford in 2016. But I would like to claim suggest to you that just exactly as not just any judgement
about things in which science is interested is a scientific judgement part of science.
So not just any judgements about things with which morality is concerned.
Moral judgements moral judgements have to qualify as such.
Making a judgement a claim about what the right thing to do is not necessarily making a moral claim has to have some moral bits in that somewhere.
The solving of a moral dilemma involves much more than having a preference for one possible outcome of a moral dilemma.
Just as the solution of a scientific problem requires much more than simply opting for stipulating a particular solution to that scientific problem,
it has to show how the circumstances which make it a moral dilemma have been weighed carefully.
One against another and morally persuasive reasons,
facts and or justifications found for having a moral preference for one night come rather than another.
Majorities, however, many people you sample in your survey majorities are not necessarily right.
Neither science nor ethics is produced by casting votes for particular answers or so-called answers.
Happy though this. The thought of such a possibility might seem to some.
It seems to me that the moral machinists are proposing the moral equivalent of deciding
whether the world is flat by finding out what preferred what people would prefer.
The answer to be as to what the shape of the world is.
Hmm. Having a preference for.
So there is clarity tossing a coin, I would suggest to be sure select cannot come, but not for moral reasons.
So neither tossing a coin or an algorithm nor methods described in the paper in nature, methods or processes of moral deliberation.
Nor does it seem to have resulted from much that we could call deliberation.
The moral machinists work is at least as useless as they claim a lot of ethicists would be.
In the paragraph I created a moment ago, they have not discovered where the citizens and corporations are or are not prepared to accept the moral and
legal consequences of the decisions of the autonomous vehicles that they manufacture by or driving amongst,
which will be the consequences of deliberately causing the deaths of innocent bystanders and other road users.
Consequences which involve much more than operationalising a principle of minimising immediate harm of directing the vehicle in one way or another.
Proper consideration of the possible harms of the preferences delivered by the moral machinists must surely include their effect,
for example, on due process,
on the principle that no person should be condemned on herd or on constitutional protections for freedom of the individual and on justice,
including criminal justice. And so all these other considerations which do not appear in the nature paper at all.
Having a preference for killing or sparing one person rather than another,
even the preference shared with thousands or even millions does not make it moral, even though it's about something of moral importance.
The preferences of a certain bohemian corporal as von Hindenburg characterised Hitler came to be shared by many millions of people.
But that did not count morally in their favour or indeed against them.
It was irrelevant to that particular question.
However, laws deny that laws. Arrived at democratically and over time or at least one indication of a human driver, it seems to me, should surely.
It seems to me that this problem is a complete no brainer. What should the human driver of that car do?
Well, it's obvious that a human driver should drive into the barrier.
He's not entitled to mow down innocent people on a crossing.
He should rely on the crumple zones and the safety features of the car to protect him, the airbags and so forth and take his chances.
It is not open to him to decide on his own say-so or her and say so it's unsafe.
So to mow down three innocent people on a crossing, I mean, it's just such a no brainer.
I can't imagine that any sane person would offer that to the public, even as part of a rather malicious game to see what they would say.
A human driver, I said, should surely stay for the barrier and hope for the best.
In any event, there could be no certainty that the passengers and the driver of the vehicle would be killed or even injured,
and they certainly would be at lest risk in their vehicle than the unprotected old people on the crossing.
I can see no rational or any moral basis in this example or an example of this drawing for a choice,
human or machine to any to do anything but avoid the old people.
It would be just wicked, obviously not to do.
Surely the right thing to do is design better safety cages and driverless cars, rather than programming A.I. to sacrifice soft targets.
God save us. We also need to recall that the law, both civil and criminal, is also an expression, if incomplete, of public morality.
If you want to know what public morality says about something, we'll look at laws developed over many,
many years and refined by judicial precedents and all sorts of other things.
It's a very good way if you want to find out one dimension of what public morality is.
You don't need to invent computer games and offer them to a lot of people who spend that amount
of time thinking about the ethics of the dilemma that they're entitled to show a preference for.
We also need to recall recall that the law, both civil and criminal, is is more reliable and allowed buy time moral machinists.
The authors of that paper should have been aware that they also need parliament's legislators,
the court's human rights conventions and many other things not simply accrued.
Vox Pop to be cognisant of what public morality says about these sorts of things.
The more mass shooting is asked in just the paragraph four of the paper and
I quote whether people prefer to spare the younger rather than the elderly,
or whether they prefer to spare pedestrians who cross illegally rather than pedestrians who walk.
Surely any decent citizens would immediately ask what gives you the right to condemn to death innocent people,
even one so wicked as jaywalking without due process? Which commands public respect, has has her their defence and convicted them,
especially in a jurisdiction unlike the UK, which has not abolished the death penalty.
Most matters of life or death have not been left to the bare preferences of people who have shown no evidence of deliberation.
All juries, for example in criminal trials,
are cautioned about their responsibilities required to hear and pay attention to evidence and argument from both sides,
prosecution and defence and also from the judge and deliberate.
There is no evidence of deliberation required of the people who pressed the buttons in the computer game.
For one choice or another, this is one of the many things that makes it not just not morbid, but immoral.
Rather, it is the case, it seems to me that public morality expressed through laws,
civil and human rights conventions and many other publicly accessible ways in
most civilised societies has not simply been left to individual preference,
let alone to popular preferences or to popular prejudice.
It has evolved over lengthy and often painfully informed opinions over centuries and in many other ways.
There was another recent case, which I was also involved in talking about in the UK, and that was the case of Charlie Gard,
who is a very severely damaged infant who his parents wanted to keep treating
him as though there was evidence that he was some evidence that he was in pain,
but no evidence that he would ever recover. And eventually this what should happen to Charlie Gard,
whether he should be allowed to die as the people looking after him in Great Ormond Street Hospital in London thought,
or whether his parents should be allowed to condemn him to endless surgeries to keep him going, which had to be decided by the courts.
Nobody suggested that we have should have a quick Vox pop in on the streets
outside Great Ormond Street Hospital and see what the people thought about it.
Chicken clipped.
I have a modest proposal to make on this whole subject, one that I hope will recommend itself, if not to reason, at least to morality.
No one should deliberately kill the innocent without an excuse of overwhelming,
plausible and judicially approved necessity, as in the Manchester conjoined twins case as in Dudley and Stephens and so on.
If a vehicle is detectable but out of control,
the driver should not deliberately kill others rather than put herself itself its passengers and the vehicle at risk.
If we are to programme driverless vehicles on the road.
The roads would be much safer if the following was the first law of vehicle robotics, and I commend it to you.
Here it is. I'm going to call it out law. It's law.
On the first of the Roomba 1911 captains Scott and Oates and 14 other members, members of Robert Falcon Scott's doomed,
as it turned out to be Antarctic expedition set off from their Cape Evans base at the South Pole on the 15th of March.
Oates told his companions that he could not go on and propose that they leave him in a sleeping bag, which they refused to do.
He managed a few more miles that day, but his condition worsened that night.
On the morning of the 16th of March, or possibly the 15th,
we don't know because all of the members of that expedition died, including Captain Scott himself.
And we know from Scott's diary and his dead bodies that survived the conversation that
I'm not going to repeat to you on the morning of the 16th or possibly the 15th of March.
Scott was unsure. Oates walked out of the tent into a blizzard and to his death, Scott wrote in his diary.
We knew that Paul Oates was walking to his death. But then we tried to dissuade him.
We knew it was the act of a brave man, and an English gentleman apologised for the last bit.
According to Scott's diary, as Oates left the tent, he said, I'm just going outside.
I may be. Some time outs most likely died on March 17th.
Ouch. Legendary self-sacrifice to save the life of his colleagues in the face of diminishing supplies of food and
the need to make enough speed to reach further supplies sets a moral example that might daunt most of us,
let alone driverless cars. What might we expect of an autonomous vehicle, but probably not, perhaps simply of a driverless one?
Whatever its priorities, it will surely be conscious if it really is an artificial intelligence that its
solution to the problem will say something about the sort of creature it is.
Just as all of our actions say something about the sorts of people we are.
And also about the sort of creature it will from thence forth be in its own mind and in the minds of others,
we are created by our past acts and our past decisions, both what we think of ourselves and what we detect that others think of us.
And, of course, an analogous consciousness of what we are advocating or acting on in the moral machine case will say something about the
morality of those who have seriously proposed such a scheme for deciding how ethically we might programme driverless cars,
what it says about them, about their moral seriousness and their qualities as human beings.
Perhaps the most obvious answer to the spurious questions put by the highly immoral moral machine
experiment is that driverless vehicles should always risk themselves and their occupants,
relying on the safety built into the structure of their vehicles, rather than choose between different groups of innocent bystanders for slaughter.
Not least because of the obviously corrupting self-interest involved in their prioritising themselves and their vehicle,
rather than the other road users. True.
Not many people would want to ride in or own a driverless car programmed in this way, but perhaps that's for the best.
Until that is,
we have real autonomous vehicles who not which can take their own moral responsibilities seriously and who have received a proper education,
not least in law and ethics. Thank you.