In this episode, The Changing Character of War program welcomes Dr. Kenneth Payne,
a senior lecturer in the School for Security Studies at King's College London.
His talk explores the evolutionary basis of strategic behaviour from chimpanzees to computers,
teasing out the connections between liberal warfare and human evolution.
So the colour coding here is three parts of the book comes in three parts.
The first part is all about evolution or fixes on evolution, and that's my thesis there.
The second part is a bit about culture, which I'm going to gloss over in just three or four slides.
And then the last part is probably the bit that some that might be of most interest to you is the bit
about artificial intelligence and the ways in which artificial intelligence will shape strategy.
And the overall thesis of the book in a nutshell is that there have been and everybody likes to be kind of reductive about revolutions in warfare.
I think if the Toffler's, for example, that posited three revolutions in human history, the agricultural revolution,
the information age, the story, the Industrial Revolution, and the Information Age Revolution I wouldn't bet it.
The Toffler's beginning for two revolutions, a cognitive revolution early in human history that some, I argue,
set up the basis for some underlying and enduring principles of strategy in the sense that most of us would understand strategy.
And then an artificial intelligence revolution that challenges the evolved
cognitive psychology that had been a longstanding part of our human history.
So two revolutions, the evolution of a cognitive package to do with language and and theory of mind,
and then latterly a revolution in artificial intelligence, which is just getting underway.
So part one and the bits about evolved psychology, my thesis is that there is a dynamic relationship between strategy and evolution.
That is to say that the need to think strategically was a strong factor in shaping our human
evolution and also the way in which we evolved in pace with constraints on how we did strategy.
Hopefully I wouldn't say too much about what strategy is, but in a nutshell, for those of you that might not be familiar with the term,
I'm talking here primarily about military strategy, the use of organised social violence in pursuit of some political goal.
My thesis is that some another Tom Jones who is not my thesis is not even Tom's, but he's done some tremendous work on it.
The general thesis is that Greek surmise has an important effect on winning or losing in combat,
and the need to cooperate together in progressively larger human groups has been one of the fundamental driving forces shaping human evolution.
I got started on this project reading a book that I commend you all in the late seventies and early 80 seconds devours chimpanzee politics.
If you haven't seen that. Grab a copy of that highly entertaining account of life and death in a chimpanzee colony colony in Arnhem,
and the game's inverted commas that the chimpanzees played in order to get ahead and assume a dominant position within the hierarchy,
colony and devil at the end of the book asks the question.
Interesting question Do chimpanzees think strategically?
And he answers that, yes, they do think strategically, but I think it is a slight oxymoron.
They do so instinctively. There's not he doesn't necessarily suppose,
a particularly sophisticated degree of self-reflection and reflection about the minds of
others on the part of chimpanzees is much of their strategic behaviour is innate to them,
is instinctively acquired.
And I think not that I want to disagree on this eminent primatologists in the world, but I think it is something of a disservice to chimpanzees.
And I think the view on chimpanzee cognition and theory of mind has moved on since the late seventies.
So on the one hand, he's doing a disservice to chimpanzees because they are capable of thinking about the intentions of others,
not as not to the same degree that we are, but they are to a certain degree.
And on the other hand, he does limit service,
because I think there's is the supposition that we as humans think in rational and calculating terms about strategy is unfair.
We do a lot of unconscious, instinctive, horrific type thinking about strategy, just as the chimpanzees do.
So Deval set me going and the question was, do animals fight wars?
And if they do, do they fight wars in the strategic sense? Do they make use of violence in an instrumental and passive way?
And my answer was this was answer fight pretty totalitarian wars of the entire destruction of other and colonies in other species,
that they do so on a fairly automatic situational basis.
They're responding in an evolved fashion to instinctive clues from the environment about how they ought to behave.
Chimpanzees, on the other hand, certainly fight wars come on tonight in the second.
And then he did so with a little bit more of what we might call strategic thought, a little bit more conscious reflection on what they're doing.
And so that was my starting point. Way too much about the middle part of the book.
Middle part of the book is a dilemma coming to anybody looking for general patterns in strategic behaviour.
It's you've got the whole gamut of human history.
What can you say about strategic behaviour that unites strategy across all these different varieties and cultural circumstances?
That's book classes wrestle with, of course, is the tension between historicism something that's particular to the moment in theory,
the ability to say something general across all particular areas or multiple areas.
The second tension that the tension between historicism, the local, the particular and history with a capital H,
the idea that history is driving forward towards some particular purpose.
That that's the that's the tension I wrestle with in the middle part of the book,
the bit about culture that I'm not going to spend too much time thinking thinking about here today.
And then lastly, the last part of the book, can artificial intelligence strategy save that for the end?
Here to kick it off, we come to chimpanzees and their ability to think strategically.
This is a paper in 2014 in which the chimpanzees demonstrated an ability that might have
appealed to social scientists from the middle of the 20th century to win a game of strategy.
This is the game they're playing here is a little bit like pieces of rock.
You have to guess what your adversary actually can't see.
The chimpanzee is play. It's the chimpanzee here. Is that back to back?
So they can't see what the adversary is choosing. One of them is tasked with matching the shapes.
And and if he guesses, if he matches the shapes that the other chimp throws, he wins.
The other is tasked with mismatching the shapes that the chimp race.
So he knows he's got to think about what the other chimp is going to throw and then produce a mismatch.
So you've got a strategic game to play,
a strategic game in which they've got to try and guess what the other person going to do and behave accordingly.
Chimpanzees outperform humans at this game, which if you were a scholar of the standard social science model of the rational activity,
expected utility payoffs, that would be a puzzle. Why are the chimpanzees so good at it and why the humans are less good at it?
And if you change the payoffs or if you change whether one's a match or a mismatch and they tell them very quickly,
the chimpanzees change tack and start behaving according to how the change suggests that they should on a mathematical basis, they come.
Very close to the optimal mathematics solution that humans are hopeless at it comparatively.
The question is why and what does that mean? And the answer for me is not that chimpanzees are better strategists.
It's that they're pursuing a different strategic logic. The chimpanzees are responding more instinctively to the payoffs.
The humans are trying to second guess what the adversary is doing.
Humans are making that strategy using theory of mind.
They're trying to play the idea that, you know, the innate game and the chimpanzees aren't playing that game.
They just want to sweep that they get for winning the game. And it's the over complication that we put into it,
trying to read what the intention of others are the producers in this narrowly to play game a suboptimal outcome.
So chimpanzees more rational actors than humans are in that game.
Chimpanzees, we know because they're evolutionarily close to us.
We parted company for them some 6 million years ago. Very useful comparator in terms of thinking about strategy.
And in answer to Duval's question, you know, do chimpanzees fight wars?
When he wrote that in 79, I think it was the goomba extreme war had just ended.
I'm not sure whether where they could all have actually published about it,
but the dominant view at the time in anthropology and actually in climatology as well was still,
you know, chimpanzees were fairly peaceful, fun loving kind of characters.
Since when that view has changed almost 180 degrees and the survey that shouldn't say we're going to say random, of course, Richard Wrangham,
famous primatologist at Harvard,
it turns out that some chimpanzee life is extremely violent at best interest group when you're struggling for dominance as it develops.
Chimpanzees were on them and also between groups in terms of these chimpanzee wars.
And the the paper by Wrangham that compiles all the available evidence around in 2006
about how many wars were fought and what the levels of casualties were in these wars.
And it turns out the chimpanzee life inside the group is much, much more violent than Detroit in the 1970s,
and that some the levels of war were between chimpanzee groups allowing for small
sample size was quite likely to be on a par with hunter gatherer communities.
And we'll see what that was in the moment.
Here's some chimpanzees behaving you might think strategically, perhaps in response to innate subconscious inclinations.
They're conducting a chimpanzee patrol here and they patrol into the territory of neighbouring chimpanzees and they're doing the tactics
when it comes to waging a war like the going to extreme war is the raid and ambush that typically attack the 10 to 1 force ratios.
And so if they can't not heavily outnumber their adversary, they'll stage a silent retreat back away until such time as they do.
WRANGHAM Theory is that this behaviour, which contrasts markedly with that of bonobos,
they tend to fight wars, spend a lot of time having a good time.
Range theory is that the distribution of food which forced chimpanzees to forage in an isolated fashion,
may be particularly vulnerable to predation in this way by their fellow chimpanzees,
because they had to fissure into small groups to go foraging for food, they could very quickly be outnumbered and defeated by other chimpanzees.
The question is, is that strategic behaviour or not?
On the one hand, the chimpanzees are capable of waging a sustained war against another group of chimpanzees.
In that extreme war they annihilated the other group of chimpanzees.
That survey every last chimpanzee.
I think there were a couple of females left over that were absorbed back into the original population and they waited over a period of years,
you might say, well, that's not necessarily strategic behaviour and you know,
they're following an evolved inclination to conduct these patrols and to stage ambushes and I think you'd probably be right about that.
For me, the big difference that makes our behaviour more strategic in the sense that we're familiar with is our capacity for this theory of mind,
for intuiting what other people want and communicating on the basis of that in order to derive an effective strategy.
So I start with chimpanzee wars and I say yes.
On the one hand they can do instinctive strategy, but there's something different about humans.
And what's different about humans is our ability to communicate and to plan for an imagined future.
And the basic theory that elaborate in part one is that there is enough violence in
the hunter gatherer world in which we evolved for almost all of our human history.
There is a sufficiently high level of violence like in that chimpanzee world.
To force us to become effective co-operators, we have to cooperate.
Otherwise, we go out of business. That's essentially the theory.
So, as you might know, Robin Dunbar's work here may have heard Dunbar's number.
Dunbar's number? Yeah. And under the back? I hope not.
Robin Dunbar. I've never met Robin Dunbar. So. So Dunbar's number is the size.
Your neocortex correlates very effectively with the size of social group in which you live in the larger neocortex, the theory goes.
The greater the ability. You have to track the key essentials of life in a social group.
And these key centres are essentially who's up, who's down and who's shagging who.
These are the things that you won't be able t t to understand.
And if you if you correlate a chimpanzee neocortex sized chimpanzee cortex size relationship,
you can extend that line outwards to humans and come up with what's called Dunbar's number.
Dunbar's number for humans is 150. That's the size of hunter gatherer community in which we spent a lot of our early human evolution.
It's a very rough rule of thumb because actually the site you hang out with on a day to day basis would be near 50 people.
That would be your kinship band, people you primarily be related to.
And you come together in this gathering of 150 for an overnight camp, perhaps for reasons of security.
Dunbar's number is one aspect of the theory of human evolution called the social theory of cognitive evolution,
which says a lot of our functions are derived to help us navigate this complex, increasingly co-operative,
interdependent world 50 to 150 people with whom we would spend most of our time.
That's the group that we hung out here now.
The second big theory to think about is Frederick Manchester's mathematical laws of armed conflict dating to 1916.
Anybody familiar with those yet to be familiar with? They say without without stretching the point.
Point one, this is an abstraction. It's a mathematical abstraction.
It's not meant to correlate precisely to real world behaviour.
But Manchester's argument was that the benefits that would accrue from being in a larger group would be disproportionate to the size of that group.
That's what he call square. If fighting is structured so that it's a melee, basically a wall against all, then the fighting power of the group,
all of the things being equal will increase geometric t relative to the size of the group.
The advantages from being in a larger group will increase geometrically if fighting is structured.
So it's one on one. So imagine a series of duels at dawn.
Or imagine, as I do later in the book, Hoplite Warfare, where the formations force you to fight 1 to 1 rather than in an overall melee.
Then it still matters to be big, but it matters in a linear fashion.
Say provided fighting is disorganised or there is a capacity to concentrate fire, not renown,
strategic principle, concentrating force where you can then the fighting power increases geometrically.
So the question is, is, is there enough fighting in hunter gatherer communities early in human evolution to
force the agglomeration of humans together into larger groups and in larger groups?
Perforce, they have to fight.
You have to be in a group with people to whom you're not directly related, and you have to find a way of cooperating with strangers.
And the answer I make, the claim I make in the book and I'm not alone in making plenty of other people make it as well,
is that, yes, there was enough fighting in hunter gatherer times to force people to cooperate together.
So the result I suggest is that there was sufficient violence early in human history to put
pressure on humans to start cooperating with people with whom they might not know each other.
Manchester's law provides a mathematical formulation of that.
That is to say the prevalence of war solves what's known as the free riding problem.
Free riding problem is essentially a problem of loafing. If you get enough people together, there'll always be some people who slack off.
And if it's a matter of fighting somebody to the death, I personally would be on the side of the slacker office.
So what is it that's going to make me cooperate?
That's going to stop me from slacking off?
One answer is to this the prevalence of war forces,
cooperation and scrutiny of how people are behaving within a particular book say the problem to be solved,
you'll be aware, is the classic problem of the prisoner's dilemma. And the payoffs is just in the prisoner's dilemma,
such that non-cooperation tends to be the favoured option, even if cooperating would be better for you.
The solution to that problem, as Samuel Bowles argues, as Peter Turchin argues,
is that there is sufficient violence to force the read out that on co-operators and to force cooperation.
And actually you can see echoes of that in a couple of days.
Research is there. So the basic theory of cooperation in an evolutionary sense is from Robert TRIVERS
Trivers argues that we cooperate with non kin on the basis of reciprocal altruism.
That is, if I scratch your back, I do so with the supposition that in years to come you might scratch my back.
So I will behave altruistically to you on the assumption that I'll get some payback later on.
That's the basic theory of reciprocal altruism,
and that requires you to have a make a pretty good bet about it that the probability of being paid back in that future date,
of course, the incentive then is to cheat, to get away with having your back scratched without returning the favour, so to speak.
And Dan Ariely and Travis himself have demonstrated that very ably trivers in the in the world of animals.
Ariely, in some fascinating experiments on his students, proved that our inclination is to cheat and then lie about it.
But what actually proved this is quite interesting is that we lie about it not just to the people we're cheating, we lie about it to ourselves.
So we have the capacity to persuade ourselves that we are good, honest co-operators, even while we're cheating.
So we can see that the natural inclination is not to be a free,
cooperative member of society, even though the pressure of war suggests that we should be.
But there is, because of that pressure of war, I argue, still some inclination to cooperate.
And you can see that.
When you do real world experiments with the prisoner's dilemma, in theory, in game theoretical terms, the incentive is to cheat and not to cooperate.
In practice, if you give the prisoner's dilemma to normal people, they tend to cooperate, even where it's a one shot game.
So there's no possibility of playing the same person again. So you can't build a reputation for trust that might shape your behaviour.
Was a one shot game with a stranger.
We have a tendency to wait to cooperate and the argument is that tendency to cooperate has evolved within us as a result,
as I say, these pressure of warfare. And in fact, even if you give the prisoner's dilemma to genuine real life prisoners who ought to know better,
they tend to cooperate more than is suggested by the prisoner's dilemma.
So this evolutionary picture then that I take out of the first part combines this notion of reciprocal altruism.
Lanchester His idea,
the victory goes to the big battalions in a disproportionate sense and comes up with what I think is the the dominant approach to strategy,
which is to be strong everywhere, if you're strong everywhere. Of course, strategy is very easy.
And the strategy that the strong evolved with involves scale, mass and concentration of force.
And and that couldn't I gave you Voltaire a moment ago.
Victory doesn't belong to the big chains. It belongs to the best shooters.
Lanchester would disagree with that, and I would suggest the evidence from evolutionary history is against that as well,
which we generally get used to the big battalions, as Stone said of the pope.
How many divisions? SIEGEL And say there is still that in-built supposition that mass pays off in the end.
But you will say, you know, even if you're strong, you don't want to be a muffett.
You also want to combine your strategy, the strong with being strong everywhere with some sensible tactics.
And that's what the chimpanzees do.
They make they in conducting a raid and an ambush rather than pitched battle against the same number of adversary chimpanzees.
What they're doing effectively is building a force ratio that Lanchester would say is dominant to create a localised force ratios,
even a strong use rate and an ambush. And if you look at this book wall before civilisation,
that pattern is very strongly evident across ethnographic studies and archaeological studies
that the dominant strategy in warfare is rate an ambush rather than pitched battle.
And if you're on the on the weak side, that's when strategy gets really interesting,
because you have to find a way of offsetting these advances to decrease the others.
So you adopt all these sort of approaches and surprise, deception and fortification, manoeuvre.
And if you are the borrowers, as the Americans call the British army, you come up with the British way of warfare,
which is to try and be more flexible and you get throwing about it and reverse.
Three. Max Fraser if you don't know, Max raises work and you should.
He's a statistician at Oxford and he's produced extensive database on all sorts of fascinating stuff.
The world, the world and figures and the general picture is that if he has a general motif is that the world is getting a better place,
which is not always how it feels. A day's out for an American election,
but he has compiled a whole range of statistical information about it's the most comprehensive list that I've seen of archaeological
evidence for prevalence of warfare and hunter gatherer communities and ethnographic studies of warfare in hunter gatherer communities.
And on the right here, you can't make out any of the detail here, but on the right this is violent deaths per 100,000 in state societies.
Down at the bottom here is the world in 2007.
So that's that's that's the the [INAUDIBLE] pit that we live in at the moment.
Battle deaths 0.33 deaths per 100,000 people for the world in 2007.
Whereas if you look at this hunter gatherer community study in the 1840s in California.
1450 deaths per hundred thousand, saying hunter gatherer communities tend to have the most violent men,
but they tend to have more violence, more violent deaths than do more modern state communities.
So was there enough warfare in the course of human history to drive consumption behaviour and get rid of free riders?
The answer for me is yes and that's the claim I take out of the first part of the book.
I didn't spend a bit of time thinking through the implications of what that means for human cognition and
for the heretics that we've developed to help us think strategically when we go into any of that here.
That's part one of the book and an evolutionary history of warfare. It pays to be big.
There's enough fights around to force us to agglomerate in large groups and to force it to learn patterns of cooperative
behaviour and shortcuts for thinking about who we trust and who we shouldn't trust to allow us to think strategically.
In part to the book. I take that forward and see if it holds in the cultural era.
And the answer is that it kind of does.
But stuff complicates it for reasons of time. I'm not going to get into.
I use three fairly well-trodden examples.
It's all about the Greeks and hoplites.
And I talk about the invention of writing and particularly the invention of strategic studies as a separate way of thinking reflectively about war.
And the take out from that study. The Greeks is that writing provided a first example of artificial intelligence,
if you like, a means of offshoring, part of our cognition outside of our brain.
I won't say any more about that example to Clausewitz and Napoleonic warfare.
Again, I'm not going to say too much more about that. I will say a little bit about nuclear weapons.
And the take out from the middle part of the book is that Manchester's law breaks down in the cultural era because
fighting power is not just about the number of men that you have at your disposal and their ability to concentrate force.
It's also about technology in the cultural areas technologies of warfare, weapons technologies, but not just weapons technologies.
Technology screws Manchester's abstract thinking about the importance of scale when it comes to strategy.
And the apotheosis of that, of course, is nuclear weapons, where the analogy can inflict as much damage as a thousand plane fighter bombing raid.
On take, you say scale, which you have nuclear weapons becomes less of an issue.
Of course, you need scale in order to develop nuclear weapons in the first place.
I said at the beginning, I posited two revolutions in military affairs, the one the cognitive revolution,
the flexible package to construct grips flexibly in order to deal with violence in the world.
And second, artificial intelligence. And one of the other contenders for a revolution in military affairs that changed everything is nuclear weapons.
Robert Jervis, many of you will know, said that nuclear weapons were a revolutionary development in warfare.
They were the revolutionary development in warfare.
Bernard Brody, the American strategic thinker on hearing about the destination of the first nuclear bomb,
said that he thought it made everything else he had ever written redundant and obsolete.
He thought that the entirety of strategic history was redundant and obsolete because nuclear weapons are transforming so much.
I think he was wrong. He was wrong. But I also think that this is wrong in the sense that nuclear weapons are revolutionary.
They're an important development in the history of warfare, for sure.
But I didn't think the revolution and the reason is at the bottom of their strategy.
Even with nuclear weapons remains a matter of our evolved human psychology.
We still making decisions about the employment of nuclear weapons,
using precisely the same cognitive architecture that we evolved with 4050, 200,000 years of our human evolution.
So all these factors in nuclear strategic theory, escalation, the need to demonstrate credibility, miscalculation,
the scope for misperception, they are all subject to the same evolved cognitive, psychological foibles that we brought with us before.
Jervis said that nuclear weapons were revolutionary on the basis that they would transform the balance between denial and punishment.
And of course, you have a nuclear standoff through mutually assured destruction, and the threat of retaliation is how you use the weapon.
So is the threat of using the weapon rather than the actual use of the weapon that delivers you more strategic result.
And what he said is there's no point trying to deny nuclear weapons.
You can't defend against them. And instead, your strategic use of nuclear weapons relies entirely on the threat of punishment.
I'm not entirely sure that is revolutionary. Firstly, because denial and punishment both featured before the onset of nuclear weapons.
You could certainly deny through the construction of fortification or the use of terrain, use of things.
And you could also threaten punishments through retaliatory attack.
You didn't need nuclear weapons. Individual nuclear weapons gave you an assured ability to punish.
But punishment was certainly a feature of strategy even before the development of nuclear weapons.
So I'm not sure in Genesis account that nuclear weapons are particularly revolutionary.
That's what I'll say about them here. Reasons of time.
I wanted to get ahead to the artificial intelligence because I'm aware that that's
probably why most of you here not to hear me rambling on about chimpanzees warfare.
So in very quick order, my headline for artificial intelligence is that it is a revolution in military affairs,
is a revolution in strategic affairs precisely because it challenges the underlying evolved psychological
abilities that we've brought to bear to tackle strategic problems since we emerged on the planet and even before,
you know, antecedent species. Here are some examples of artificial intelligence from recent months.
Let's go with the easy ones. Do you believe what's going on there? Yeah.
Google's AlphaGo. Yeah. The part of the world.
The world leading go. Yeah. Did you release it all for one? The World Championship.
And before that? Very quickly. We very quickly adopt hindsight bias with achievements like that.
And assuming, of course, it beat him before that pretty well. Every A.I. expert at DeepMind was saying it couldn't be done,
couldn't be done for 20 years because they'd all been trying to do it goes right for those you don't know who wins or version where the ball game.
It's a very simple constrained what now developer would call a toy universe.
It's not like the complex, messy reality of winning in Vietnam.
There are only a certain number of moves that can take place at two different varieties moves that can take place on the keyboard,
but they can do so with such dazzling combinatorial complexity that there are more possible moves mathematically than there are atoms in the universe.
So you can't solve a game of going through brute force calculation,
which is why A.I. developers have said it would be 20 odd years before anybody came
close to performing at the level of the world champion still is beating him anyway.
Within DeepMind, they were pretty confident they trash him and they truly did.
So that's not one this one you. But you know, that one is a wreckage or helicopter flying upside down.
But more significantly, the Knights and rings, remote control helicopter flying upside down from 2006.
And this is one of the first deep learning nets that you don't know how deep learning net and rough terms.
It's analogous to the connections of neurones in your brain.
It's a very loose analogy, but that was the inspiration that started AI researchers off down this sort of
what's called constructivist constructionist approach to artificial intelligence.
So Hayes was the first one to figure out how to fly a helicopter from first principles.
It encountered the universe of the helicopter and it flew and crashed the flew and crashed.
It flew and crashed on a simulator. But he could afford the models to actually and even over again.
And then eventually it reached that takeoff point and it mastered helicopter flight to the point at which it could roll a helicopter,
which is no mean feat for any helicopter pilots, because normally they crash at that point.
So not only could you grow the form to pretty complex manoeuvres as well, this one down here is in the last couple of weeks.
Anybody seen this one? You know, this guy is he's an American Air Force colonel.
He retired. He is a a superiority combat instructor.
Who is his job? Dogfighting. Top gun stalwarts we've spent his career doing.
Now he works as a consultant for air companies.
And this is him, as they say, having his ass handed to him by an artificial intelligence system that's running off blackberry pie,
running on the BlackBerry Pi mobile phone. And he was unable to beat it in repeated air to air combat activities.
And that's only now in the last couple of weeks. And it's really combat against human vessels is something that obviously the Pentagon
has engineers been working on since the 1980s with no particular degree of success.
It's just a complicated control problem to be able to process all the relevant
information in the required amount of time to to defeat a sophisticated human adversary.
But it's now been done. Pentagon, by the way, still planning to put a man in the cockpit of the next generation long range bomber,
which is something that they might think about again.
And you know what this is? That's that's an intelligent design.
He's a visionary design are where they brand the computers.
How did you know that? And it was ever known that if you actually geek out yet that's exactly what it is
it's very old now is a nicer antenna to go on a satellite communication antenna.
And it was designed using A.I., which is why it looks like that wacky shape.
It's not the sort of thing that I would sit down with it, design me an antenna I'd come up with,
used a straight forward thing like that one bend and it came out looking like that.
And that is the optimal solution that is designed better than it than a human could do it.
And I think together these things show a range of the sort of military relevant activities that I is involved with already.
This is this is weapons design right now.
Already you can read this in stations for them on on the US Army website solicitations to use air to design bunker penetrating munitions.
So air is already involved in designing weapon systems.
As I'm walking over here with Rob and we were talking about the carriers, the Navy carriers was the favourite project of the Royal Navy.
We might also have been talking about the US Air Force man and long range bomber.
These are weapon systems that are designed partly with the efficiency imperative in mind.
You know, war has an efficiency imperative. If you don't do it well, you go out of business.
But partly also of course to reflect the institutional proclivities of the organisations that design them.
Bureaucratic politics in action that's only sustainable.
Well, have been designed by human and I probably wouldn't design you an aircraft carrier.
You might not like what it designed you, but it would design you something a bit different from from what you would come up with.
So alien weapons design, this is not just to be in strategic situations or operational situations,
situations with less structure to them when it comes to decision making and this is off to a tactical level,
activities across a range of military savings activities.
And my argument is to my good friends who work for DeepMind, who show me on a regular basis that they are not involved in defence related research,
and indeed it is one of their criteria for being bought by Google that they
would not engage in defence related activities a little analogous to to nuclear
weapons development and the idea of dual use technologies where you have civil nuclear capability capable of being switched over to military ones,
but even more so in this case, because air is entirely about autonomous decision making in imperfectly understood environments,
and that is an inherently military strategic capability, regardless of their desire to keep them separated.
So let me let me close this section for reasons of time. Some questions.
The way I've tackled air here in the book is through some dualism isms that I think a
particularly relevant considering how I will affect strategic level decision making.
First, the distinction between narrow and generally I.
So when Stephen Hawking tells you that we've unleashed the demon we must unleash, the demon air is coming to take us over.
He's talking about generally that is not just unbounded flexibility, capable of reaching decisions across a range of activities.
It's not just a chess computer. It might also decide what steel garden, but it's its air that is capable,
that is motivated to some extent intrinsically by itself, to do these things.
That's what he's frightened about. It's a flexibly I is motivated to do it.
So in fact, I'm not so much concerned about that for all sorts of reasons.
But I think there's plenty of you because.
Concerned about women still talking about narrow domains, specifically in the tactical impact of narrow domain specific.
So that's one big distinction when we're talking about narrowly.
I was talking about, Lisa, of being beaten there. I mean, to my generally, I have a look at the top, a robotics challenge from this year.
The robots are still incapable of doing fairly basic stuff like opening doors, climbing over fences without toppling over.
And I think, you know, the complexity,
the combinatorial explosion in doing some of the things that we take for granted is still beyond what our current level of I can do.
Second, you ask, is between tactical and strategic decision making.
So flying that helicopter up upside down or defeating us, colonel,
in that's where combat is a tactical level activity tactical level military activity still require some generalist knowledge,
flexible skills across a range of activities. And they're still hard.
It's not a chess computer. It's altogether more complex to learn to fly a helicopter.
The last couple of of Judaism's before I. I stopped talking with you.
One is this question of autonomy and self-directed A.I., and this opens a real can of worms.
To what extent is artificial intelligence capable of making decisions for itself?
And to what extent is it responding to inputs that you put in ahead of time ex-ante?
And in an ideal world, you would like to have the idea of your strategic goal within your tactical goal,
and you would tell the machine to achieve that, and it would go out and achieve it.
And that's what the Army calls mission command. It does it today. You tell them what you want to achieve.
You tell them your commander's intent, and you leave them to figure out how it is to achieve them.
So in an ideal world, you would quite like that to be your relationship with autonomous performance.
In terms of chief, they are autonomous to the extent that they are carrying out activities that you have told me to do upfront,
and that's fine, well and good,
except that you don't always know what it is that you want them to do upfront because you can't anticipate situations that may arise down the street.
It's fine, well and good because you may be Satisficing is the term is a dishonest term
is between different goals that you want to achieve that might be intention.
Can you specify which goal you want machine to achieve?
Which goal is the most important or otherwise?
Boston's Beijing paid ticket counter is going to be read Boston superintelligence if you if you want an intro to the to the to the
scary Elon Musk Stephen Hawking site of artificial intelligence Bostrom book on superintelligence is a pretty good place to start.
And his nightmare scenario is the paperclip counter the paper accounts it has the
best intentions in the world in fact has no intention is not is not out to get you.
It's just been tasked with counting to 100. You want it to count 100 paperclips and tell you when it's done, that there are 100 paperclips there.
And then they end. In Bostrom scenario, it conquers the known universe in order to count these paper because it can't count
them to the degree of certainty that it is happy 100% that they're 100 paperclips.
So in 98, 99, 100. Job done.
Did I get it right? Just have one more check. 93, four, five, six.
And it gets 90 to 100 and it's reasonably short. It has done it twice.
Now you can have one more check because it's a vanishingly small probability that it counted wrong twice and it will carry on doing this.
And as a genius artificial intelligence system that has been tasked with this thing and nothing else,
it will secure all the resources, Bostrom argues. We don't know that it's obsidian.
It will secure all the resources in the universe in order to keep counting the paperclips to make sure it's got 100 paperclips.
That is an example of the agent principal problem gone disastrously wrong.
All you wanted to know is is the roughly 100 paperclips that unless you specify it up front,
your very clever artificial intelligence will keep counting and securing the resources to enable it to keep counting.
So the principle problem is a key thing.
How do you put in enough safeguards to ensure that the machine is going to do exactly what you want and not what it thinks you want?
And if you can see the military dimension flashing autonomous weapons systems,
I just asked you to kill this one guy, and now you're doing all sorts of other stuff that I didn't ask me to do.
I talk in the book about consciousness and the possibility of machines becoming conscious, and I won't go into that here.
The last the last few before I finish,
what is the distinction between that I make between evolved human cognition and artificially intelligent machines?
Doing strategy is probably a false dichotomy.
It's not a dichotomy. It is not probably it is a false dichotomy.
Artificial intelligence and human intelligence are probably not going to be seen as as separate entities for all that long.
And HCI is human computer interface.
And the extent to which you are using artificial intelligence systems as part of
your normal cognition and pokémon's augmented reality is an example of that.
Or the use of a chip in your brain that's reading your prosthetic arm.
To tell you what your prosthetic arm is feeling as an example is where mind computer merge if you like another dollop of project.
The quadriplegic woman at some at MIT has flown in F-35 through the power of her thoughts.
She's flying distant, not real. I 35 about ten of these in existence but she's flown in a simulator using the power of her mind.
So I think the breakdown between what is evolved human cognition and what is artificially intelligent cognition will probably blend a little bit.
So that's another another duality. They talk about defence and civilian research.
I'm pretty hard over on in the book. It's all defence research. It's all about decision making in complicated environments.
And to me, this is real universes. A lot of A.I. recession.
Longstanding A.I. researchers are pessimistic about A.I., and I think they are say, by inclination because they've been at it a long time.
And it's a frustrating game.
And they are by inclination because they want to counter unjustified hyperbole, the sort of, you know, massacres with is talking about.
But but I think that that pessimism is to a certain extent, unwarranted.
You know, toy universes can be pretty complicated and they are doing pretty well in these twin universes.
Let me finish for for reasons of time, if nothing else with I then offer by way of conclusion the book Ways in which artificial
intelligence is going to change strategy and perhaps take a lot more in questions,
the sort of the sorts of things in regards to, oh,
that A.I. changes the balance between offence and defence because it allows like a concentration of force.
Hey, I shift your tolerance for risk. Firstly, because you're not losing any of your own people.
Secondly, because your actions are not being driven by your own emotional cognition, A.I. makes the man in the loop unsustainable.
Apartment issued defence position on artificial intelligence weapons.
There will always be a man in the loop, a lawyer making a decision on what machines can and can't kill.
That is inherently unsustainable. And it's fine when you're attacking the Taliban in Helmand province and you can lawyer everything back up to London.
It's not fine if you're tackling an autonomous system operated by a high tech adversary in, say, China,
because they might not care to have their lawyer in the loop deliberating or reporting their machine in court.
To which case you eventually say the man in the loop will be forced out of the loop.
By dint of the speed of artificial intelligence, A.I. will change radically.
The structure of armed forces is another point that I make in the Strip.
A lot of people out of the armed forces to kick off with no change, the sort of people that are employed.
So that's my last slide. I started from the point of considering the way in which our evolved human
psychology has shaped our strategic behaviour and for our entire human history.
Up until fairly recently and really fairly recently,
the Pentagon is now using artificial intelligence systems in rapid change to identify and prosecute targets.
So fairly, fairly recently, our entire strategic decision making about the use of violence in the collective good
has been taken on of these evolved cognitions that I say started with Manchester
Law one and continues to cooperate that is now gradually being replaced with what and
that I'll leave it and take any questions that you may have about the statements.