Thank you very much. Thank you for inviting me. And thank you all for attending this fourth in-person and online.
I am very excited and honoured to be here today.
And of course, as you can see from the title and the reasons why you're here is that for the next half hour or so,
I will be speaking to you about autonomous weapons and of your responsibility.
This presentation is based on an article that I am finalising,
and I will try to explain things as clearly as possible for an audience that is not ready for the last of order,
which is also why I prepared this short PowerPoint with some keywords that hopefully will help you follow along.
My reasoning, which perhaps at times can be a bit complex, but I welcome all comments and questions and clarifications at the end.
I should also add as a disclaimer that I'm recovering from a cold, so that sounds great.
So if at some point my voice drops, please let me know.
So before getting to my main argument, perhaps I give you a bit of background about how I got to this topic in the first place.
Like many lawyers, I am not particularly familiar with the tech aspects of autonomous weapons.
But what I do find interesting about the advent of these technologies is that they prompt
dozens of international lawyers to dissect some old rules that we took for granted in a way.
And they make us question different elements, and they make expression of these old rules could potentially apply in light of these challenges.
For instance, if you take international law,
it's now the advent of one of those weapons is making us question whether in order to violate the principle of distinction and the other
rules and the standard is targeting decisions needs to be made by a human or that can be delegated to an artificial intelligence.
And in the real world of international criminal law, I think the doctrine of secure responsibility offers also the opportunity to think about some
elements of this doctrine that we didn't necessarily have the chance to to question before.
Super responsibility is, as you are probably all aware,
a criminal law doctrine that entails the responsibility by omission of a military commander
or a civilian superior that failed to prevent or repress the crimes of their subordinates.
We find it in the first initial part of law to the Geneva Conventions before removing the
statutes of the International Criminal Court of the Tribunals of Primary Courts and so forth.
And superior responsibility is a bit unique in international criminal law.
And because of these, it has attracted some criticism, because it is a form of sui generis liability.
It has been described as a fallback option.
So when everything else fails for the charge, it wouldn't stick under any other motive liability towards people in leadership position.
Then we go for a superior responsibility and in this sense, superior responsibility of courts with our moral preferences.
Those that are in a leadership position that could have done something to prevent
or punish the crimes of their subordinates and didn't do so be held to account.
But on the other hand, the criticism is that there is a gap between the culpability of the superior.
We simply omitted to act and the actual responsibility, which is for the crimes, the war crimes ordinance.
Now in all of this, enter autonomous weapons you might know about.
Different stakeholders have been engaged in a series of debates about this weapons.
What characteristics should they have? How to regulate them?
How can they be used in compliance with international law?
From the perspective of criminal law,
the debates are focussed on the difficulty or even impossibility prescribed criminal responsibility to an individual.
When autonomous weapons are employed and it is called the so-called responsibility that is created by the use of autonomous weapons.
Now various proposals have been put forward as solutions to this problem, real or perceived on their responsibility gap.
And one of them is superior responsibility.
And the way that superior responsibility has been designed as a solution to the problem of the responsibility is in one way to consider autonomous
weapons themselves as subordinates whose crimes need to be prevented or repress or otherwise to rethink and change the doctrine so as to accommodate,
in a way, machine criminality. Opportunities, violence.
I am not particularly convinced by either of these solutions.
And I will tell you why. In the course of my presentation, which is mostly a response to the first kind of proposals,
those that to those that believe that we can interpret severe responsibility as entailing the responsibility
of superiors for crimes committed by sort of the advanced weapons themselves or the subordinates.
And I will focus on those that are, I believe, are the challenges or some of the challenges to the applicability of the doctrine in the circumstances.
And this is the type of console that humans exercise over a part of most weapons is different from a qualitative perspective,
from the type of control that is needed for a Super Hornet relationship to exist, which is needed for super responsibility to supply.
Second, I argue that another challenge is represented by underlying findings that we need to be prevented because it
means that have been committed by a punishable subordinate and needs to be a crime committed in settlements.
And finally, I will explain.
However, instead, if we focus on the superiors supervisory duties,
we can still find some use for us in the doctrine of secure responsibility within the debate concerning one the.
So let's start with the first element, the super organic relationship.
This is the main challenge, I think, to the applicability of the doctrine.
And I will come back to this even if we look at the codification of the doctrine for various purposes.
For instance, in the first part of the universe, various statutes,
we see that the people that the doctrine refers to, the superiors in this audience are identified in various ways.
There are two subordinate forces, the subordinates, people or persons.
There are members of the armed forces under the superiors,
command or otherwise persons under the effective command and control of a superior and so forth.
Based on this wording, we never really had a thought that all of these words identified clearly individuals and therefore
the superior subordinate relationship would be an interpersonal relationship between individuals.
And I think that this interpretation still stands and is.
In a way, it is undermined by going back to the origins of the doctrine of secure responsibility.
Because today we know that the doctrine applies to military superiors for commanders and non-military superiors.
But the origins of this doctrine are in the military sphere.
The doctrine of secure responsibility is the criminal law.
Corollary of the principle of responsible command.
And responsible command means that members must foster a command culture that is in line with that.
A child must train their troops, must discipline them, must threaten them,
sometimes a punishment so as to make sure that they will use force only in ways
there are a lot of impermissible in line with international humanitarian law.
Responsible also implies a certain level of organisation and position of discipline.
Responsible command and responsible command structures are also invited.
Of course, authority can be delegated, legal obligations can be delegated, but they can only be delegated to another responsible moral agent.
And in case obligations are not complied with, someone will be responsible and obligations will be still overseen by someone.
And so we have the responsibility of the superior as a consequence.
I think there is really no question about the fact it was venomous.
Weapons are not responsible for agents, so we should not be giving them any obligations under the law.
We cannot not threaten them with punishment.
Of course, they can be programmed in compliance with your show and should be programmed to be using not your show.
But I don't think that this is the same thing as educating troops and fostering respect for the command culture,
which is based on the principle of responsible mind, because certainly autonomous weapons do not understand the legal character prohibitions.
It is also important to know that in order for a secure subordinate relationship to exist.
We need what is called effective control of the superiors over the subordinates.
We know this from the case law of international criminal courts and tribunals.
Of course, you know that the term effective control in international law means many different things.
In the context of super responsibility, it means something very specific.
It means the material ability of the superior to prevent or repress the commission of the crimes or to submit the matter to the room before it is.
And the case law, like international criminal courts in particular,
lets us come up with a list of indicative factors that might show the existence of effective control.
You can see them on the PowerPoint. For instance, they include the power to issue orders engaged in hostilities,
the commander's capacity to ensure compliance with orders, supporting units to remove, replace this being investigations and so on.
Now, according to some offers, this information can be translated to the type of relationship that a commander would have with this weapon.
I think instead that this factors are quite clearly anthropocentric in nature.
Of course they were in the first place developed having in mind the relationship between individuals.
But I think that, for instance,
adjusting an algorithm that guides an autonomous weapon or in deciding to withdraw it from one place and deployed elsewhere,
are not initial effective control. It is not the same as redeploying troops.
It is not the same as issuing orders for troops to act in compliance with a challenge.
These decisions are decisions are similar to decisions made about any other type of weapon in the arsenal,
and yet it is only in relation to autonomous weapons that it is suggested that these decisions instead
show that the weapons themselves are subordinates for the purposes of security responsibility,
and that someone most effective control over them, as it has over there, are supporting the troops.
And I think this is a fundamental misconception of the nature of effective
control as opposed to the relationship between humans and any type of weapon,
including autonomous weapons. And the law, as it stands today, where autonomous weapons are used for warfare.
They're not combatants. They're not somewhere between combatants and means of warfare.
They are weapons tools. They are objects of the law.
And they they're not subjects of the law. So they're not addressed by equal obligations or prohibitions.
And the type of control that is exercised over these weapons is not different from the type of exercise or any other weapon.
Here I refer you to the work of Tim MacFarland, who is a lawyer who also understands and knows about the tech side of autonomous weapons.
So someone who has certain authority, I'd like me to speak about these things.
And I think he's convincingly demonstrated that autonomous weapons do not differ in any legally
relevant way on the basis of their physical properties or functional aspects from other
types of weapons and the type of control that is exercised over how autonomous weapons are
operated and how they're used is planned or executed is the same as any other weapon.
For instance, in operations, decisions will be made about what type of weapon to use,
what type of explosives to employ for a certain attack, but the range of the weapon at the blast radius and so forth.
And in a way, these are all decisions are dissimilar from deciding what level of autonomy to use in a certain operation.
When autonomous weapons are employed. In the realm of autonomous weapons.
Specifically, we speak about autonomy in the sense that a weapon can be a human supervise or will be autonomous or semi-autonomous.
And autonomy is basically a function of human control over the weapon.
But it is still a function of control that is a technical type of control over a tool.
It is not the type of control that our commander over there, troops.
So we also speak about, for instance,
meaningful human control or sufficient levels of human control that our states are still trying to decide what exactly they mean by this.
But all of these notions are employed to mean the level of human control that is needed.
Some of these weapons, the main sort of tools that do what you want them to do.
And so it is on the basis of all of this these arguments and this research that I end up
concluding that a weapon is not a subordinate for the purposes of severe responsibility.
I do think that the doctrine remains applicable with respect to human subordinates that operate or supervise a weapon and their superiors.
But still, then the responsibility of the human operator needs to be established in order
for the derivative responsibility of their superior to also be established.
And this brings me to my second point, which is about the underlying problem.
And I think in relation to the underlying crime that needs to be prevented or repressed by the superior.
There are two issues that arise. The first is that the subordinate needs to be susceptible of punishment.
And the second is that the crime has been committed in all its other means.
So as I mentioned before, autonomous weapons are not considered to enjoy a legal personality.
They're not addressed by crime rules or international humanitarian law.
They cannot be punished based on criminal law.
It is true that in some jurisdictions we have entities that are not individuals that have been granted legal personally do for instance,
corporations in some states. But if we take, for instance, the doctrine of respondent superior in the United States,
according to which a corporation itself can be punished under criminal law for
the acts that its agents have undertook undertaken on behalf of the corporation.
I think this is presents basically the opposite problem that we have with the weapons, because in the case of corporations,
the responsibility of the corporation still extends from the actions of a not for a person, the agent.
But in the case of autonomous weapons, still, we have to send a firm responsibility to an individual for the acts of autonomous weapons,
including on the basis of secure responsibility. We should also keep in mind, having said in many other jurisdictions, for instance, in Germany.
Criminal law is instead rooted on the principle of blind worthiness.
And this means that an actor must be able to decide between doing right and doing wrong, and can only be punished if it is blameworthy.
Hasn't decided to do something wrong. And this principle excludes the criminal responsibility of legal entities and
would also exclude the responsibility of machines no matter how much they may be,
because they will never be able to be blameworthy in this sense.
The principle of worthiness would also affect, however,
the possibility to hold a human or greater celebrity responsible if either was not possible for them to store the weapon,
for instance, because they were out of the movie or if they do not understand how the technology.
Besides to to to ask because it's a wonder isn't this machine learning or observing was workings are inexplicable to the operator in these
circumstances the operator would not be worthy of the weapon on and as a consequence would be held that operators superior responsible.
We would be further exacerbating even further the gap between the culpability of the superior and their responsibility,
which I mentioned at the beginning. Consider also that severe responsibility arises for failure to repress,
which includes punishment and estimating the crimes of the competent authorities.
And because punishment is.
Sort of the most visceral quality of legal personalities and autonomous weapons cannot be punished because their punishment could be,
for instance, reprogramming or deactivating.
And this wouldn't pursue any of the of the goals that we normally associate with punishment in criminal law.
Then it would mean that the superior cannot simply exercise this duty.
If you cannot punish the weapons with criminal exercise weapons.
It can also not to make of the matter and the authorities, because the weapon cannot be one of doubt and so forth.
And this means that if the material ability of the superior to punish is not there,
there is also no effective control in the sense of superior responsibility.
Which brings me back to the initial point. That, however, is not as important as.
Relatedly, when we think about the underlying crime for which the superiors responsibility would arise.
We should ask ourselves whether this has to be a crime committed know incitements or the objective elements of our core arms race.
The mental or subjective element. Or real. And the actual element usually.
And generally the answer to this question is, yes, we do need all of the elements of a crime to be committed.
Otherwise, it's a crime. Not every roadblock is a crime after all.
According to Simon, said, when it comes to autonomous weapons,
it would be sufficient for only the off the rails to be committed without the requirements.
So, for instance, they can take the problem of infecting civilians.
It's intentionally attacking civilians. If an autonomous weapon does it.
So this argument goes it is sufficient that civilians have been attacked.
It's not sufficient. It's not necessary to prove that the machine did so with intent.
I don't think that there is any reason to change the interpretation of what we mean by a commission when it comes to autonomous weapons.
The commission of a crime, I think, implies that it needs to be committed properly in all its elements.
The only partial exception is represented when the subordinate can avail itself of an excuse.
But in the interests of science and ecosystem, it's even more complex than what I've been saying so far.
It will only result. Under criminal law as extensive.
However, I think that is pretty clear that autonomous weapons cannot form the required rea for a crime.
They cannot act with intent. Because again, the answer is linked to the notions of blame, worthiness and culpability.
And I think that these are here into human notions.
And if autonomous weapons or then unable to form demonstrate are maybe the correct analogy is with dangerous animals.
But then if you look at countries that do have laws on dangerous animals, for instance,
that you take the responsibility of the owner of dangerous animals is some form of strict liability.
It is not a horrible of severe responsibility. The fact also that an underlying crime must have been committed in all its elements, of course,
affects the possibility of the for the operator to be held responsible as well because they might also not informed the required real.
And as a consequence, this person's superior would not be responsible for the wrongdoings of the weapons.
Now, so far, I've told you about all the reasons why I think it's your responsibility is not the way to go.
I still think that it can be useful in the discussions concerning autonomous weapons.
If we think about the supervisory duties of the superior, so we should think about those obligations that stem from the union to prevent.
Again from the case of international criminal courts and tribunals,
we have some examples of measures that can be taken to prevent the crimes of the subordinates.
And I think that at least some of these measures can be interpreted, again, of course,
with respect to human subordinates so as to keep weapons within human control and the human chain of command and control.
For instance, if you take the obligation to give adequate training child to the subordinates.
I think, of course, this means making sure that the subordinates.
Again, I'm talking about humans are aware of the rules of international humanitarian law,
but it also means making sure that they're trained in how to use the means and methods of warfare, their disposal in compliance with ISIL.
And if this means of warfare and autonomous weapons, it means we need to understand how the system works.
They need to understand how to operate it, then supervise it.
They need to understand when it is not functioning and what to do when it is not functioning.
And in turn, the commander must be able to understand sufficiently how this technology works so that it can be employed in accordance with show.
And then, if anything goes wrong, it should be preventing the use of the weapon.
Tate also issuing orders aimed at bringing the relevant practice in accordance with international humanitarian law.
This can be done in different ways when it comes to homeless veterans, for instance.
Commanders can order their subordinates to use a specific level of motivation for asserting that they can
circumscribe the geographical or temporal use of the weapon so that it is used in compliance with bilateral.
Spirits also have a duty to prevent further or recurring crimes.
And in this sense, securing reports about the use of the weapons in past attacks is important because the first sign that a weapon was functions,
that this might not be a war crime.
But then at that point, the operator is on notice and their commander is on notice as well that something is potentially going wrong.
And at that point, the commander might even have to order that the weapon not be used until it can be reprogrammed or adjusted.
Now to conclude. I understand the appeal of superior responsibility as a solution to the responsibility gap.
But I do not think that in its current formulation, the doctrine allows us to do so.
I told you why. Of course, the doctrine could change as a matter of a treaty or customary law.
I don't see this happening, but even if it did,
I think we would run the risk of turning severe responsibility in the form of strict liability for criminal negligence,
so that anyone who is in putative control of the weapon would be responsible as a superior for whatever malfunctioning happens.
And I don't really think that this is a development that we should wish for.
So instead of closing the responsibility gap, I think this would create further problems to the culpability of the superior.
And I think that instead of creative interpretations of criminal law,
we should focus on supervisory duties and making sure that the weapons remain under adequate disposal of humans,
under a human chain of command and control. So that one person is responsible and that person superior can also be responsible.
So at the end of the day, the responsible thing is supposed.
When we mix the technical specifications of the weapon, we agree on a sufficient level of human control and not by interpreting criminal law.
In other words. Thank you so much for your attention.