Okay. Good morning, everybody, and welcome to the last lecture.
And thank you for your loyalty and staying through two terms of this.
As you know, today is a little bit different in that I'm going to give a talk that I've been giving many times over the last 15 years,
really, but it's developed over that time. And I enjoy about once a year telling people the current state.
So the title is Who Invented the Great Numerical Algorithms?
And maybe the first thing to say is that although that's a picture of Oxford, regrettably Cambridge would be more appropriate.
This all began at a discussion in the Numerical Analysis Group over coffee some years ago, where in fact it was my Giles,
I think, who was arguing persuasively that people like us academics actually didn't invent the important stuff.
The important stuff got invented by real people out there in industry or wherever, solving real problems.
And what academics did was tidy it up. So it occurred to us at that point, well, let's look at the data and get a sense who did invent,
who did have the big ideas, who created the big algorithms in numerical analysis?
And so the way this talk works is I'm going to tell you, I'm going to show you 29 pieces of paper, 29 slides, about 29 major algorithms.
And in each case, we'll try to figure out who was the main inventor, usually two or three people.
And what were they? Were they in academics, in industry? What was their life?
Very hastily, of course, you can't do 29 things at leisure and then at the end come back to this question and try
to make some generalisations about what sort of people invented the great algorithms.
For amusement, moral, more or less.
It's good to bear in mind the so-called Ziegler's Law of the Pandémie, which is that no scientific law is named after its original discover.
So as we go along, you can see how valid or invalid that may be.
So this is the list of 29 algorithms, and I've grouped them into three sections before the war, roughly speaking, after the war,
and then sort of in the time of my career, roughly speaking, I stop at 2000 and at the end I'll have a word to say about things more recent than 2000.
Okay. So let's begin with 1940. It's chronological.
And the very first thing we'll talk about is Newton's method for nonlinear equations, which we use all the time in this course.
We've used it a lot. Very major idea in numerical computing.
Well, there's Newton, of course. We all know him. And there's no doubt he was one of the key people involved with this.
Not much to say that you don't already know. He was one of the greatest thinkers of all time.
He was an eccentric, annoying academic at Trinity College who spent many years at Trinity, thoroughly an academic for most of his career.
Certainly at the time that he did his legendary work, he was an academic at Trinity.
Eventually he became a celebrity and master of the Mint and moved to London and did all sorts of governmental things.
But that was after his research period. It is somebody we don't know as much about.
Of course, we've all heard the expression Newton Roughton. But who was this guy?
Well, it turns out he was a mathematician, more or less of Newton's time, who worked with Newton and was one of the people in the calculus wars.
You know, there were limits and Newton who should get the credit for calculus.
This became a very ugly chapter in intellectual history, with the British being particularly ugly in their fanatical defence of Newton.
Apparently Rapson was one of the key guys doing that.
The history of flux fluctuations. You can imagine which side of the argument that was on.
Simpson, I think is a great hair figure.
He was really a very major numerical person with a fine mind, very creative, who did a number of important things.
You'll notice in the comments here that Newton didn't use Newton's method, but really only for very low order scalar polynomials.
It was Simpson who moved to non polynomial problems and also who treated systems of equations.
So Simpson somehow saw the bigger vision of what Newton had done.
Who was Simpson. He was at the Royal Military Academy in Woolwich, which is London, more or less.
And I guess that makes him an academic with an interest in military applications.
Now least square fitting is the second on the list. Another huge topic.
It's, among other things, the foundation of the whole field of statistics, you might say.
This has in common with the first topic that it was founded in great acrimony, this time by Gauss and Legend.
So just like Newton and Leyden to co-invented the calculus,
gauss and the genre independently invented least squares, fitting the genre was the first to publish.
But Gauss may have had the idea first. Who knows? They got in a big fight.
Another ugly episode. Gauss, like Newton, is, of course, one of the great figures of all time.
He did everything in physics and mathematics. He's a mathematician and also a physicist of the highest order.
His career, as you know, was quite spectacular as a teenager, sort of in a sixth form college, as you might say.
He did unbelievable things. You know, the binomial theorem, the quadratic, reciprocity, arithmetic, geometric means an incredible guy.
And in 1795, he would have been 18 years old when he invented least squares fitting.
He was very interested in orbits of asteroids and so on, and that was one of his interests.
So he was an academic, though very much involved in the practical scientific problems of his day.
Alexandra, another well known mathematician. Of course, he was active during the difficult times in French history, the revolution.
And after that, you can see. Well, at 1805 in France, that's the Year of Trafalgar, right?
You can just imagine how complicated things were. He invented leaf squares also in connection with commentary orbits.
For many years this was the picture of Alexandra. And then four or five years ago, thanks to the miracle of the web,
somebody noticed that there were two different people called Alexandra, and they both had the same picture.
There's a politician and a mathematician. And if you looked at either one, you would be pointed to this picture.
So one of them must be wrong. And it turned out the mathematician was wrong.
So far as I'm aware, the only known image of Alexandra is this caricature from some collection of caricatures of mathematicians.
Now Gaussian elimination for linear systems of equations, another absolutely fundamental algorithm that one is certainly an example of Ziegler's law.
It's ancient. It was described by the Chinese in the third century A.D. but even then it was old news to the Chinese, I think.
So the idea of eliminating variables, I guess, is a pretty natural one, and it goes way back.
As soon as people start having coupled systems of equations, of course, they're going to try to eliminate variables.
So the history there is very murky. Gauss himself was one of the players, of course, though not particularly the pivotal one.
He, as far as I know, only did symmetric systems.
The non symmetric case was seemingly written down first by Jacobi, though that doesn't mean he was the first to have thoughts in those directions.
Jacoby was, as I've mentioned, one of my great heroes.
I think he's an extraordinary person who only lived a few years.
Look at that. An academic at Berlin and Königsberg died in his mid-forties,
but in that time wrote a dozen papers in both pure and applied mathematics that changed all sorts of things.
Now go quadrature.
That's a beautiful, powerful idea for integrating functions, the benchmark against which all other methods of integration are measured.
This is the cleanest example I know of. A counterexample I know to Stigler is law.
It really was invented by Gauss. I don't think anyone before Gauss did this at all because it's a tricky idea.
It's not something you would invent so casually. So Gauss did it out of the blue in 1814, when he was 37 years old.
He published that paper.
He did it, by the way, via continued fractions, which is a technique that was big in the 19th century and nowadays we don't know much about.
It's considered now a kind of an odd way to approach things.
And in fact, it was Jacoby who a few years later turned to Gauss Quadrature into an application of orthogonal polynomials,
which is how everyone perceives it now. So notice these ideas we're talking about are absolutely fun fundamental ideas.
In Numerics, we've had Newton's method and Gauss Quadrature and linear systems of equations by Gaussian elimination least squares.
These are crucial ideas for mathematics and science.
What about formula for odds, which we've certainly talked about in this course?
Adams Formulas? Well, certainly one of the key people there was Adams.
But before him, for the first order formula at least, is Euler.
And Euler, we know, was another one of these astonishing figures like Newton and Gauss, to whom we owe much of modern mathematics.
Euler lived at also a complicated time.
He was Swiss from Basel, but he spent his career in Prussia and in St Petersburg.
His he had some rather good mentors.
So one of the protectors, whenever we you call them, one of them was Catherine the Great and another was Frederick the Great.
So that's pretty good to have on your resume. Not in the letters of recommendation.
Actually, the letters might not have been so good. He annoyed them a little bit.
But anyway, Euler had this astonishing academic career, one of the most prolific mathematicians of all time, a great master of mathematics.
Adams was a Cambridge man. He spent his life at Cambridge.
You can see he was born in 1819. And then when he was 24 or so, he was the top student in his year at Cambridge.
That's fourth I think I mentioned this in one of my lectures was in the same year at
Cambridge and where Adams was the senior wrangler back fourth was the second wrangler.
But that's fourth scores, and the exams were about half as high as Adams.
So Adams was a true genius, one of these extraordinary people that people are still talking about years later.
So he had this fantastic career at Cambridge, very much involved in astronomical questions.
And a mathematician fundamentally, when he was quite young,
he studied the perturbations of the orbits of Uranus and predicted there must be another planet out there.
But as the story goes, the Astronomer Royal refused to point the telescopes in the right direction.
So in the end, the French discovered Neptune instead of the Brits.
He has a good reputation for having declined honours. He was offered a knighthood and said no.
He was offered the Astronomer Royal Post and said No. Sounds good to me.
Now back fourth is a much less important figure overall.
But we've all heard of Adams best fourth methods. Now, who was him?
He. Well, he was at the Royal Military Academy in Woolwich, also.
And it seems he worked on these methods 30 years later.
But those were maybe the first publications. It's not clear Adams published anything in the 1850s.
And although he was a ballistics expert and you think ADAMS mapping odds ballistics,
it seems that it was actually shapes of drops that were his initial application.
I'm a little unclear on that. Now, the other great classic formula for Ode to Ease is running a cut of formulas.
And these date to about 50 years later three Germans basically it seems wrong
and Hein and Qatar were very much involved at the turn of the 20th century.
Like Adam's formulas. As I say at the top, these are generalisations of Euler.
Euler, the first order Euler formula is both in atoms and the wrong kind of formula, if you like.
It's when you go to higher orders that the two diverge.
So I wrote this paper in 1895 and I, as I've mentioned to you in the class, he's another one of my heroes.
He did so many fundamental things. We'll see his name again in a moment.
Holy. I don't know much about he guess. I guess he was in theoretical mechanics, which is what they call the applied mathematics and move days.
And then Qatar is well known for the current condition in aerodynamics.
So he was a fluid person and a mathematician. And then after that,
a lot of people were interested von Mises the this is a great machination also wrote a major paper on wrong a kind of method in the modern era.
It's John Butcher who's still alive in New Zealand. He put them on a sound mathematical footing.
So throughout the last 40 years, really, John Butcher has been the man in the cut the area.
But no way that he invent them.
And I think this is the last of my pre 1940 topic finite differences for PDAs, which we've seen a lot of in this course.
It's pretty hard to say who invented that. But there are some remarkable pre-computer people and I mentioned.
Three groups there. Richardson and Southwell were both British mathematicians.
Engineers who? Use these so-called relaxation methods.
Before there were machines. It's just amazing. We've talked a bit about Richardson.
You did a problem related to him on the last assignment. He was a remarkable, very interesting figure in all sorts of ways.
Southwell is the only Oxford contribution to this talk.
He was a legendary engineer at Oxford, apparently a fantastic lecturer.
People found him extraordinary. So this is Richardson there and Southwell there.
Friedrichs and LaVey wrote this legendary paper in the 1920s,
which for theoretical purposes more or less founded the Leapfrog Method and moreover, the idea of stability analysis.
They didn't interpret that in terms of things blowing up on a computer, of course.
Nevertheless, they had the key idea. And then when computers came along, many people got involved.
Of course, too. Two of the key ones were Von Neumann and LAX.
Von Neumann, of course, much older than LAX, a very sadly died.
Young von Neumann, as you know, was one of the great mathematician scientists of all time, certainly of the 20th century.
You can see that he, like Jacoby, died in his forties.
Peter LAX is still with us. He's won the Abel Prize and so on.
So von Neumann had this young kid, Peter Laks, working with him at Los Alamos during the war.
Basically, Laks went on to have an academic career at the Crown Institute.
He won all the prizes. Von Neumann was very much involved in government work.
He was, for example, on the Atomic Energy Commission for a couple of years.
Okay. So then you have the war and post-war era.
So the next one I'd like to mention is floating point arithmetic.
Now, most people don't know who invented that.
If you ask the typical numerical analyst who invented that, they don't know.
I didn't know when I started to work on this topic. But it turns out it was invented very clearly at sea, not by Babbage,
as you might have guessed, but by Conrad Sousa, who was engineer, mathematician in Berlin.
Very interesting, unusual person. So during the Nazi era, he had this company in Berlin called Sousa Operator Bow,
and they built a succession of computers which were incredibly modern in concept, obviously not in hardware.
But you can see, for example, I mentioned there is 22 bit floating point binary arithmetic, 14 bits for the fraction, eight bits for the exponent.
It's just the way computers work now. Although now we can afford more bits.
Of course. The clock speed was one hertz. It was programmable.
Although you didn't store the program, the only stored the data.
This particular machine of his that I mentioned there was destroyed in the air raid in 1945.
I don't know that much about that. It seems that incredibly he wasn't so much wrapped up in the Nazi stuff.
The line, they say, is that the Nazis did not appreciate how important computers were.
How true that is, I don't know. He then lived for a long time.
No, this until 1995. In the late forties, early fifties, he was a name that people paid attention to when they were recreating computers.
But then he lived on and on. And he was a fantastic artist.
So here's an example of one of his watercolours. Very remarkable man, I think.
Lyons, one of the key ideas for representing functions and surfaces.
So this is one that clearly did have some roots in industry.
Strangely in the car industry. So you see two of the key early names are the custodial and busier.
And these were both French people involved with cars at C Tire and at Renault.
The Casio was basically a mathematician and a physicist.
Bazzi seems to have been an engineer so far as I know DeCastro is still alive, but I don't know if that's true.
One of the sad things over the years with me giving this talk is that every time I give it, a few more people have died.
Now most of them are gone. But Carl, the book is very much with us.
Schonberg was this remarkable man from Romania, from the intellectual elite in Romania, if you like, who then came to the US, as many people did.
He was at Chicago and Harvard and Princeton and twice more Pennsylvania, eventually Wisconsin.
And he wrote some key early papers on aeroplanes and a book on Sloan.
Carl de Boer was then a very major mathematician who.
What's the first mathematician, I guess you'd say, to really bring blinds into the modern era somehow?
Interestingly, he although he's very much an academic for most of his life, at a key stage, he was working at General Motors.
So he was born in what became East Germany.
And he came over here at age 22, over here to the west, and made his life in America and certainly made it big.
He's now actually living in retirement in the San Juan Islands near Seattle.
Monte Carlo simulation. Well, if you're a good test taker, one thing you know about Monte Carlo is whether it was invented in Europe or America.
Well, obviously Europe, because if it were America, it would be Las Vegas method.
Right. So it's a European who's going to have invented this?
And these are the three key names. So you have Stanislaw Ulam, who is a Pole, who came to the US and became a very patriotic American.
Very much a mathematician, a pure mathematician even, and yet was heavily involved with Los Alamos.
And he was one of the key people who figured out how to make a hydrogen bomb work.
So there's really an exceptional case of a mathematician who would have thought of himself as very pure doing very major applied work.
John Barnes. Meanwhile him again, a very major guy in mathematics and physics and everything in between.
He spans the pure applied spectrum better than almost anybody.
His base was Princeton, I guess you'd say.
He was born in Hungary, but came to America when he was in his twenties and then was a key person in the Manhattan Project.
And Nicholas Metropoulos was a physicist, a Greek American.
And he has this complicated career where he was at Chicago and Los Alamos and kept going back and forth.
Obviously, there were two places important to him, very much wrapped up in the military industrial complex, I guess you'd say the official invention.
A lot of these things were officially invented after the war. You never quite know what happened during the war.
But anyway, the first papers are in 1947, very much in the realm of nuclear technology.
And then this major paper by Ulam and Metropolis and 49 other physicists very much involved Fermi and Whitmire.
Monte Carlo. Things just keep growing in importance. Stochastic issues.
The Simplex algorithm for linear programming.
This is the problem where you have a lot of variables and a lot of linear function of them and a lot of constraints.
And you want to find the optimum. Well, the two.
People who invented linear programming, at least if maybe not quite the simplex algorithm are Kontorovich in the Soviet Union and Danzig in the USA.
Very much war stories.
Kontorovich was in Leningrad St Petersburg and he was an academic mathematician but much involved with the planning of production.
Patrick Farrell isn't here, I think, but he was. He pointed me to something on the web about the problem of getting plywood to the right
place at the right time in order to fight off the Finns during the early stages of the war.
Kontorovich had these remarkable ideas about linear programming, and if you've read this novel, read plenty.
He's one of the characters in Red Plenty about the Soviet vision.
You know, we're so steeped in the sort of post-Soviet post-Soviet era that we we kind of forget the beautiful,
noble thought that many people had, which was that it makes sense to do things optimally.
Right. These guys had the idea that. Capitalism is a mess because nothing is optimal.
Whereas if you really do your mathematics right, you can have all the right inputs in all the right places at the right time,
and the outputs will come out optimal at some level.
That's right. It leaves out enough that at some level it's also deeply wrong.
But there was something truly beautiful there. And Kontorovich was part of that tradition.
George Danzig was very much in the Western tradition.
Now, here's a guy who was a major academic, a brilliant mathematician, truly brilliant.
He was a very, very special man. During the war when he's very young, you know, 1941.
So what is he, 27? He was head of the combat analysis branch of the U.S. Air Force.
What does that mean? One of the amazing things looking through the history of the war is how young the scientists were who had this incredible impact.
For example, the. The Manhattan Project.
The head of the theoretical division was Hans Baker, and he was the senior guy who bossed around the younger figures.
He was like 37 years old.
It's just astonishing that such key things were done by people in their twenties and thirties on the scientific side of the war effort.
Now you can see if you get the War Department Exceptional Civilian Service Medal, you're probably doing some good things.
A couple of years later, he got his Ph.D. and then in 47 outcomes, the simplex algorithm.
Kontorovich didn't quite write down the simplex algorithm, but it's all in the same territory.
Dan did the expression linear programming came along a year later,
and then the original think tank was the Rand Corporation in Santa monica, California.
The phrase think tank sort of came from that one. And Danzig was a key person in that original think tank.
So here's a man who's very much in the military industrial complex, if you like.
But his foundation was always academic.
He was at Berkeley for a few years and then Stanford for 40 years.
So very major figure making Stanford so great in areas of operations research.
Now conjugate gradients and land growth iterations. This is all about iterative linear algebra.
We've talked about that and you know, it begins in 1952.
That's the magical year. And these three people were clearly the key.
People have finished. And Stifel wrote the paper on conjugate gradients.
But at the same time, Lunchbox was writing his papers on very closely related techniques.
Now, one chose was a Hungarian Jew. He was born Lavi and changed his name to Lantos.
He worked with Thayer and Einstein, eventually moved to the West.
He went to America and was, of course, in the war era, too.
I believe he was a communist. And so things got a little bit hot for him in the fifties.
So he left the U.S. during the McCarthy era and ended up in Dublin.
And there's a very nice video of him from the 1970s talking about mathematics.
So if you Google, you can find him on YouTube and check. I was an American at the University of Chicago.
He spent some time at the legendary Institute of Numerical Analysis in Los Angeles, part of the National Bureau of Standard.
That was an exciting place for a few years where numerical things were happening.
Stiefel was an eminent professor at the EPA in Zurich,
and he was he's very well known as a pure mathematician and geometry, as well as for conjugate gradients.
Also a physicist. He looked at that intense look. He reminds me of like The Third Man by Orson Welles or something.
Fortran. Anyone know who invented Fortran? It's another one man thing.
John Backus. He invented Fortran, and it's an amazing story.
So most of the people I've been telling you about are among the great academic geniuses of all time.
In many cases, these are truly extraordinary people who take all the boxes of intellectual achievement.
Now, Backus was a little bit more human than that. It seems that his early years were a bit of a mess.
Somehow, at around late twenties, I guess he found him or mid-twenties.
He found himself at IBM, which was an old company.
But of course its great era was just beginning. And then he had this idea that maybe it would make sense to have a programming language.
And he persuaded the managers at IBM to fund this. So they built a little team.
And nearly three years later, released Fortran.
It's an incredible story.
We all think of Fortran as having been there forever and come, you know, it must have come from a team of thousands or something.
But I think this was a little group working for a few years at IBM and out it came.
And of course it changed the world with not quite literally the first programming language, but it was the one that changed the world.
He got the National Medal of Science, the Turing Award, all of these big gongs later on.
But anyway, he spent his career, I guess, at IBM.
Stiff Odds solvers. We've talked about that a bit. What do you do when you have different timescales in a time dependent problem?
That's a problem, an issue that keeps getting bigger and bigger because in multi physics, if you like,
multi scale physics these days, you so often need to couple together different time scales.
So this is at the heart of many, many things. And. The two people who put this problem on the map were Curtis and Hirschfeld there.
And they were a pair of chemists, and this was in their great paper in 1952, integration of this equation.
So these were pretty remarkable people. Curtis was an eminent chemist who did government work during the war.
And then Hirschfeld, who was truly a major figure in American chemistry.
So you can see, like so many of these people, he was an academic with war involvement.
So he was at Los Alamos leading a group. And like so many people, he wasn't very old.
You see, he's a group leader at age 32. And then he was the chief phenomenon of the bikini bottom test.
That's pretty amazing for a guy of a 35.
So obviously he and Curtis, he will have been involved with very applied problems then this great paper of theirs in 1952.
And later he got into the National Academy of Science, the National Medal of Science, which is the top thing in the US, basically.
Then the academic involvement came a little bit later.
German Dahlquist, who we've mentioned, wrote his great paper in 1963, and that sounded the subject of stiffness as a mathematical discipline.
In fact, Curtis and Hirschfeld had made a key mathematical mistake in their paper.
And then Geer was the person who made the subject famous at a practical level, beginning the software stiff solver.
Gear, by the way, was an academic for most of his career, but at this time of the work,
he was very much involved in the Argonne National Lab in the US.
And then after some decades he moved to NBC, which is a company, and so far as I know, he's now retired.
He's British, by the way. I didn't say that here, but a Brit who ended up in the US.
Now what about finite elements? This is one where that famously has an industrial connection, although a little murkier than you might imagine.
In some sense. The first paper on the subject is by Richard Courant, but it really didn't have much impact.
It's now recognised as a foundational paper. Courant was a remarkable man, so he was a German who was the head of the institute in Göttingen,
where Hilbert had been an extraordinary situation but then had to leave during the
Nazi era and came to the US and founded the current institute at New York University,
which became really the great applied mathematics place in the U.S.
But there were many aeronautical engineer who were involved, and some names are Martin and Turner and Irons and Kelsey and Poppy.
So a lot of these ideas were bubbling around. Now, who actually invented the term and sort of pulled it together?
These two people in 1960 seem to have been key.
So John, our guy, Chris, was a Greek who spent his career in Germany and he wrote this key book in 1960.
And Ray Plus, an American at Berkeley also was involved in 1960.
So 1960 would often be regarded as the sort of the date it all came together in finite elements.
But with these earlier roots classes, the remarkable man, another winner of the National Medal of Science,
you get the impression from this talk that everybody gets the National Medal of Science.
But that's not true. It's hard to get that.
He's actually best known not for having invented finite elements, but for his work on earthquake related engineering.
Other key old figures are babushka ends and Kasich.
Babushka is still alive. This great man from Czechoslovakia who moved to the US in 1968.
I think Kasich died a few years ago. He I guess was Polish, but his career in Wales.
Now in numerical linear algebra, most of the great algorithms have an orthogonal aspect,
and that all happened in the sixties, basically at the late fifties and sixties.
And I think the three key names are givens and householder.
And so Givens in 1958 introduced really the first numerical technique that was orthogonal, given the rotations.
Householder then made it much bigger with his givens reflectors.
So somehow householder was the person who put this stuff on the map.
And then Jane Garland saw the much wider picture of all of numerical linear algebra and its pervasive
importance throughout science and the pervasive importance of orthogonal methods throughout linear algebra.
So Ghalib was the big figure in this era, though maybe Householder was the one who first put together the functionality.
An early paper of globes that was major was linearly squared.
It's now so obvious to us that you can do linearly squares with orthogonal matrix algorithms, but that wasn't obvious back then.
People would do the normal equations and factor those somehow non orthogonal.
So Ghalib was born on February 29th. He would be 84 years old on Monday.
What about the QR algorithm? One of the most beautiful success stories in all of numerical analysis.
This is the one that has most perfectly become a black box that everybody knows and loves and uses without caring what's inside it.
So I think these are the four key names route is has her invented a thing called the ELA algorithm,
which was the non orthogonal version of the QR algorithm?
So in some sense that was the basic idea of factoring one way and then recombining the factors in the
other direction and then I guess independently a.k.a a woman in Soviet Union and John Francis in England.
Worked on the orthogonal version of that, which became the QR algorithm,
and Francis in particular introduced the bells and whistles we now know so well the the crucial idea of shifting.
You need to take your matrix and first try diagonals it or make it in Heisenberg form, do some shift chase bulges.
All that stuff is very much rooted with Francis and he was close to Wilkinson,
who then became the great figure in the area, wrote his magnum opus in 1965.
So certainly Wilkinson is the one we can credit for making eigenvalue solution such a well advanced subject.
Wilkinson is a very interesting man. He was a wunderkind at Cambridge and very young when the war started.
But so at age 2021, he was working in the war effort.
Then he became Turing's right hand man on the pilot ace computer at the National Physical Laboratory.
But Turing didn't get along well with the National Physical Laboratory, so he went off to Manchester, where he eventually died.
Wilkinson had a great career at the National Physical Laboratory and, as I say, wrote this book.
He got all the prizes.
The Turing Award, in particular for computer science, is only rarely given to numerical people, but he and Cahun and Backus are three at.
The Fast Fourier transform. This has one of the most complicated histories.
There's no doubt where that name comes from. It comes from Cooley in Tukey in 1965.
So Julian Tuki. So here's maybe the one case where the names I put in big letters were certainly not the first inventors.
The first inventor seems to have been Gauss in 1805. No impact at all.
But then it was invented over and over again. It seems wrong.
And my hero invented it. Thomas Mann shows good.
Another hero. Wheeler Gentlemen. Many people seem to have noticed that you can do a clever recursion and improve and square to enlarge in.
But somehow it was in 1965 that this all got named and changed the world.
Turkey was one of the major statisticians of the 20th century, a larger than life figure physically as well as mentally.
Extraordinary man, a prolific, amazing man who was an academic but heavily involved with non-academic things.
So he founded the Statistics Department at Princeton, but was always at Bell Labs and other consulting places.
Richard Garwin is a physicist with a major involvement with the H-bomb still around, I think.
And for example, he was interested in detection of nuclear tests.
I guess that would now be North Korea's nuclear test. James Cooley, so far as I know, is still around.
And he was at IBM. I not I don't know much about him.
I don't think he's such a major figure or certainly the heavy hitter here is Tukey and Garwin is also very well.
Kwasi Newton methods. As I guess I briefly mentioned last term,
are a way for doing Newton's method without doing all the linear algebra involved
in solving setting up and solving a big system of equations at every step.
So this was a beautiful idea, came along in the sixties and seventies.
The first paper in 1959 by William Davidson.
So he was a physicist at Chicago, an academic who spent his career at Haverford.
He was a Quaker. That paper didn't get accepted and he got mad.
So he he he put it in a drawer.
And then 30 years later, when the Journal of Optimisation was founded, they pulled it out of the door and published evidence paper at last.
Michael Power was for years the dominant numerical analyst in this country.
He just died last year. And he was a key people or a key person all along in these early development droid in another key person.
And Fletcher, who's still with us, another key person. So Royden had a complicated career.
I guess he was in English Electric during the crucial years, but then later was an academic and spent a number of years in Italy.
Roger Fletcher has basically been at the University of Dundee,
but in the early days was at Harwell, the nuclear lab where he and Powell worked together.
So that brings us to the post, the modern era, shall we say, or the early modern era, whatever you want to call preconditioning.
Key idea in solving system of equations. Many people claim the credit, but I think the one who is the really important figure is Vandervoort.
He wasn't the first to have the idea at all, maybe, but he's the one who put it on the map and thought it's pervasive, important.
Moreover, he invented incomplete factorisation, which remains one of the key preconditions.
So he's a remarkable man, still very much with us, relatively young.
Also like Conrad Sousa, an excellent artist, an extraordinary artist.
And there is one of many of his work. So he was very much an academic.
Spectral methods for solving differential equations we've talked about the phrase comes from
Steve Moore that he invented the phrase spectral methods and his post-doc David Gottlieb.
In the early years, the two of them sort of initiated this subject that was at MIT.
They were academics interested, especially in fluid mechanics.
Orszag then went on to a number of places and then died a couple of years ago.
Gottlieb was an Israeli who spent his career in Israel and the U.S., and that was at Brown for many, many years, but also at Tel Aviv.
I think you probably know who invented MATLAB since I mentioned it fairly often that Cleave Mueller.
Cleave Mueller is very much with us. Born in 39. So he's from Utah.
And grew up as a very smart kid, went to Caltech, did very well and an academic, but with involvement in other things.
Argonne National Laboratory, again, was a key thing for him.
So notice after Caltech, he did his Ph.D. at Stanford and then went to Michigan and New Mexico.
And it was when he was a professor in New Mexico that he did the MATLAB stuff.
So MATLAB came from his academic years, but then later he joined a company that Jack Little had found.
So notice at the bottom matlab is two thirds as old is FORTRAN.
Maybe you think of them both as infinitely old, so you don't distinguish them.
But for people of my generation we think of Fortran as having always been there and MATLAB is sort of the new upstart.
Well, that's not true. It. Multi grid methods.
One of the major technologies for solving the discrete equations associated with differential equations.
They were invented by Federico in the Soviet Union.
At least a two grid method at first. And then, voila, with another key name.
I don't know so much about Federico. Aki Grant is the person who made them big in the West.
Well, indeed, anywhere. I don't think Federico stuff led to so much.
But Brandt learned about it from Federico, so that was not an independent discovery.
But Brandt enlarged the field extraordinarily. And then Hotchpotch was an independent discovery,
a German academic who independently invented air guitar for fire and then later realised it was the same as multiple.
So another academic. This is no longer up to date.
He's now retired from Leipzig. I Tripoli arithmetic which we use all the time.
You you've never used anything else? Probably. This is the standard way we represent numbers on a computer.
Well, that comes from a committee, but there's no doubt that the driving force behind the committee was William Cohan.
So he's an extraordinary man at the University of California at Berkeley,
Canadian originally, but he spent his career at Berkeley, been there for many, many years.
He's another winner of the Turing Award.
He saw the importance of standardisation, of floating point arithmetic, and he banged heads together until it happened.
A remarkable success story. Not a symmetric cry laugh iterations.
Well, in 1952 we had conjugate gradients and lunchbox, and in the early seventies those took over a lot of computational science.
The non symmetric story is more complicated. The roots are equally old, but it didn't become important as quickly.
So for example, our A.D. is also from back in the fifties.
The first sort of practical use seems to have been at the Shell Petroleum Company.
But I don't know anything about Vincent Yusef.
Saad, an academic at Yale at that time, was a major, major person, and Hank Vandervoort another major person.
So Guimaraes comes from new facade. Basically, SAB and Jacoby Davidson come from Vandervoort, and a third very major person is Dan Sorensen,
who at that time was at the Argonne National Lab and then later Rice.
He with his collaborators, invented, implicitly restarted Arnold which is the standard way we now try to find eigenvalues of big matrices.
So Air Pack and the MATLAB I command coming from third.
Interior point methods for optimisation. So the simplex method works on the outside of a region of feasibility.
Interior point methods follow a curve on the inside, and there's no doubt that that was.
That made the headlines because of Karmakar Narendra karmakar in 1984.
Now, he wasn't the first to have ideas along these lines.
There were important precursors. Some of them, like Hutcheon, were much more theoretical.
Others like Margaret Wright. We're focusing on other aspects.
But it was Karmakar who realised that this could really have huge practical importance.
And in 1984, he at Bell Labs annoyed a lot of people by making outrageous claims and refusing to release the software.
It was a very touchy time in optimisation, but an extraordinary contribution.
So he did his degrees in electrical engineering and then was at Bell Labs when this happened.
And he's now in India and I'm not sure what he's doing. The fast multiple method is a fast way for solving systems.
I could accelerate a little bit and this comes from Green Card and Rocklin, as everybody knows, a remarkable pair of people.
Rocklin is a Russian who came to the U.S. in mid-career, now at Yale, so an academic.
But he was at Exxon at a crucial time. Leslie Green Garden is very much an academic at Courant Institute.
Basically, though, he's now running the Simon Centre for Data Analysis.
I say they both have eminent fathers as well. Rockman father also called Vladimir, was one of the major Soviet mathematicians.
Green Garden father is a Nobel Prize winner in physiology.
I think. No, I guess I have to, ma'am. Wavelets.
Very important technology in signal processing to tell the truth.
I'm not sure they're that important in numerical computation.
I'm not sure it's a huge, crucial fact in computation more broadly.
But whether it's major in, you know, sort of my kind of numerical computation, I'm not so sure on that.
But it's a beautiful, powerful thing in signal. The roots tend to be francophone.
John Marlay is one of the key ones. Other names you may recognise.
And then Ingrid DAVIES, she is the person who put them on the map in 1988.
She's interesting, very much an academic, but with links at AT&T.
A physicist by training but also a mathematician in her early years for publications are in physics.
But then she moved into mathematics at Princeton. And though that's out of date, she's now at Duke.
Finally, automatic differentiation, a way to compute derivatives not by finite differences, but by other computational methods.
That idea has been invented in some sense many times, but I think it came together and became important because of Andrea's remark.
I guess I haven't really put dates there, but this would have been in the 1980s, I guess, when he was at Argonne National Laboratory.
I should get the dates there. So he was at Argonne and he saw how to connect, spark, linear algebra to aid.
And from then it just took off.
He was in Germany for most of his career at the Humboldt University, but he's now retired from there and moved to Ecuador.
Okay. So now we come to the end where I try to make some generalisations about what we've seen.
So first of all, about the people who invented these great algorithms.
Let's ask. Well, there they are. Those are the names that I put in boldface.
So of course, I'm omitting a lot of precursors here. But these are the ones that I've selected.
So let's ask Firth, who is an engineer. Well, I think those people were engineers.
So they're are an important subset, but it's a reasonably small subset.
One, two, three, four, five, six, seven. Who is a physicist.
This is another important subset, slightly larger. I've given three people half a half billing.
So obviously Gauss and Newton, I've called half physics,
half mathematics because I think none of them would be pleased to be called just one and not the other.
Who was a chemist. Well, Curtis and her father.
I'm not aware of others on this list. Of course, the list has all sorts of biases.
So you can tell me of some I may have admitted. Who was a mathematician.
Well, most of them were mathematicians. Now, the way I like this up is I call computer scientists and statisticians mathematicians,
because most of them would be proud to be called mathematicians.
I certainly have not called physicists mathematicians because most of them would be mad to be called mathematicians.
But this is very striking, isn't it? Who was a professor.
Almost everybody. There are a few weird cases in England where people that would be called professors in other countries didn't have that title here.
And I really mean an academic, of course. So it's very striking how almost everybody was a professor.
But that may not be all they were. On the other side who had major involvement with government or industry.
And there, again, it's a huge number. This is very striking to me.
Something like 3/5 of these people. I don't just mean that they picked up a little money on the side doing consulting.
I mean, major, major involvements with serious applications.
Who was Griffith? Well, pretty good for the population.
With gear. Oh, yeah. Gear. Yeah. Who was German?
Swiss. Austrian. Well, it's always impressive.
The. That's history. A lot of them. Who was born in the only a few.
Who ended up in the U.S. in more. How old were they?
So, of course, everybody with white hair is very interested in the question of how well, what's the upper limit of doing interesting things.
So if you look at the numbers in each case, I've tried to estimate how old they were when they did their legendary work.
And a few cases, you see several numbers for people who have appeared more than once.
I find looking at those numbers that what's striking is the variety.
You have everything from 20 to 60, basically many people, very young, many people maybe not so young.
Now in numerical analysis, there's a prize given every couple of years called the Fox Prize, and the last two winners were from Oxford.
You have to be 31 or less. So all of these people are eligible.
Most of them, of course. Well, many of them died before there was a Fox Prize.
But I don't think any of them actually won the Fox Prize, although Green Guard was of a generation where he might have.
But I guess he did mentor something and.
Newton was pretty good at age 27. He could have won. Actually, no.
You have to give a good talk. I'm not sure Newton could give it. So back to the opening question.
You had these two models of where great algorithms come from.
One is academics doing their thing, and the other is real people confronting real problems.
And I believe that the answer is kind of a remarkable combination, that in most cases, these people are academic mathematicians.
They really are academic and they really are mathematicians.
But at the same time, they have a level of involvement with applications that transcends the usual transcends.
For example, anything I've ever done, I've never had anything approaching that kind of involvement with reality that many of these people have had.
Now, of course, partly it's hard to disentangle it from the war because that was such a transformative experience at a key time.
So then the question is, what's the first great numerical algorithm of the 21st century?
Now, as I say, I first gave this talk at the turn of the 21st century.
So the way I ended it then was with this picture.
But, you know, now we're 16 years into the 21st century, so I don't have any more slides.
But let me just tell you what I think the area would be.
Whether there's one or two or three algorithms to talk about, I don't know.
But I think the huge new thing in the last 15 years has to do with big data and randomness.
The use of randomness, I think it was essentially nowhere.
Well, Monte Carlo, we mentioned, but it has now become so much more important things to do with compressing data,
huge matrices, low rank compression, sampling them in random dimensions.
That is everywhere. It's very, very important.
So with without a doubt, there's at least one more entry in this list potentially from the last 15 years, maybe two or three.
And I would like to hear your views on that, if you have you. Okay.
Thank you very much.