Okay. So today we're going to talk about a lot of things rather hastily.
And the first is. The question of what is the definition of numerical analysis?
Now one definition is that it's Professor Profession's field or more broadly,
it's the field of the numerical analysis group upstairs, whatever that is.
But of course, it's a serious question, and one reason this has interested me for years is that there's a rather invalid
definition that people consciously or unconsciously adhere to sometimes,
which is that it's the study of rounding error is on a computer.
Now, that's an incredibly boring subject. Well, actually, it's quite interesting, but it's certainly not enough to justify my existence.
So I think the right definition is the study.
Of algorithms. Now if we stopped there.
That would be more or less computer science, I suppose you might say.
But in particular, it's the study of algorithms for the problems of continuous mathematics.
And I want to spell out really what continuous means. But the point is it's real and complex numbers, not just integers.
So for whatever reason, justified or not,
it's a fact that discrete algorithms and continuous algorithms tend to be carried out rather differently by rather by different people.
It's a shame that that division is so extreme, but it's a fact that it is pretty extreme.
And in computer science departments, broadly speaking,
you find people working on discrete algorithms and numerical analysts are interested in continuous problems.
So solving a system of equations, solving a differential equation.
I believe that's what the field is all about. Now, I have been interested in these, if you like, philosophical issues for a long time.
In the appendix of my book on Linear Algebra, you'll find an essay on the definition of numerical analysis.
But more recently, one of the handouts today is my essay from the Princeton Companion to Mathematics, published seven or eight years ago.
And this is a compact description of the whole field.
And if you have the time, I think it would be very interesting for you to read this.
This course is, in some sense, samples from this essay, if you like, expanded, of course, mathematically.
Let me draw your attention to a couple of things in this essay. One is the table at the end.
Table one on some algorithmic developments in the history of numerical analysis, where I've listed about 30 greatest hit algorithms.
And I've even gone so far as to list some names of the people who were associated with originating those algorithms.
We're all human. We like names. To have a name attached to something makes it more full for us.
But you can imagine it was a bit controversial to make a list like this.
And of course, I would never claim that the list is exactly right or the names are exactly right.
But it is an interesting, pretty good approximation to some of the things that have happened in this field over the years.
More philosophically, back to the definition of numerical analysis.
The final section of this essay, Section seven, called The Future, is very much about this definition and its implications in particular.
One of the themes that has fascinated me is that even problems that can be solved with
a finite number of operations in principle are often not solved that way in practice.
If the number of operations is n cubed and n is a million, nobody's going to do it that way.
You use iterative methods like conjugate gradients, even for finite problems.
That intrigues me. And you'll see that's a theme in this essay.
The final paragraph of Section seven. Is predicting the future.
And I think in the seven or eight years since then, some of this has already come to pass.
I think numerical algorithms more and more are embedded inside artificial intelligence, if you like.
And it's fascinating for us to watch how things evolve in the 20th century.
When you computed something that was sort of deterministic, you could run it again and be sure of getting the same answer.
That's less and less true in the 21st century as more and more things get handled in adaptive, intelligent ways.
Okay. So that's the end of section 1.8 of the course.
Now returning to our particular problem of matrix algorithms.
I want to give you an overview really in the form of a table.
Of matrix iterations.
Now the historical overview, as I've mentioned from various angles, is that the mathematical underpinnings are really from the 1950s.
But this stuff began to be used in the 1970s, and since then it has just grown more and more important.
Interpretive methods are used all throughout computational science.
Pre conditioners in particular are just amazingly important.
Now for my table, I'd like to emphasise the two biggest problems of iterative linear algebra, and those are eigenvalues and systems of equations.
So it's X equals B, B is given and we're trying to find X or x equals lambda x, which we haven't talked about much, but that's an eigenvalue problem.
You're given a matrix A and you want to find vectors x and scalar is lambda such that that equation holds.
And then there are two big classes of matrices and those are simply symmetric and non symmetric.
So either A equals a transpose or we don't assume that A equals a transpose.
Of course, if you pick a matrix at random, whatever that means, it will be non symmetric.
But if you look at applications maybe half the time they are symmetric, they might also be positive, definite, which is a further condition.
So let me now draw a table, which is merely an attempt to tell you some of the names of the main algorithms.
A number of which we've talked about, and we're about to talk about one more.
So let's see. It's a two by two block table.
And the first column is A, X equals B. The second column is X equals lambda x.
The first row of the block matrix is symmetric.
And the second row is not symmetric. So in each of these areas there are several different ideas around which four different problems may be best.
Conjugate gradients is where we started.
It's really the mother of all iterative algorithms, and let's remember that that requires the matrix to be symmetric.
Positive, definite. Actually, it's kind of interesting.
If you run conjugate gradients on a symmetric, indefinite problem, often it'll do pretty well, at least for a while.
And then it may fail. If you run it on a non symmetric matrix, it'll blow up properly immediately.
Then there are other methods, other iterative methods that don't require positive, definite ness.
And the two famous names are Min Revs and SIM l q.
So if you have a symmetric indefinite problem, you should learn more about those and related methods.
Now in the eigenvalue problem, absolutely analogous to C.G. is the method called the Lantos method.
Of course, land shows, when I say it that way, is not properly pronounced.
It's a Hungarian name. But the convention in most most people in English, say Lanchester.
But of course, there are other related things, and a crucial variation on that theme,
which in practice makes all the difference, is what's called implicitly restarted land shows.
And it's interesting that mathematically this came along in the fifties computationally in the seventies, but this was more like the nineties.
That's when a huge advance was taken. And then the other big game in town is JD, if you like, which stands for Jacoby Davidson.
Now, we're not going to talk about most of these except for the simplest version of land shows.
But these are big, well-developed technologies.
And let me just emphasise that it's these two from which everything starts.
Now non symmetric problems we haven't talked about.
But just to give you some names, there are really three different mathematical ideas in the non symmetric linear systems business.
One of them goes by names like CG and LZ Q are.
But I can tell you immediately what these methods are.
These methods. Start from the observation that if you're trying to solve X equals B,
well then you might consider a transpose a x equals A transpose B if you can solve that, of course, it's the same solution.
Now, this in some respects has a worse problem. The condition number of A is worse.
On the other hand, it's a symmetric, positive, definite problem.
So this first class of methods is really all about converting this to that and then using conjugate gradients.
But then other methods don't do that conversion and the two big classes.
Well, one of them is called GM Rez, which stands for generalised minimal residual min read stands for minimal residual.
So I haven't told you what that is, but it's an iterative algorithm invented in the symmetric case,
which was then generalised to the non symmetric case. The other class of methods is has a lot of names.
It's really alphabet soup in this business. Kumar and by c g I conjugate a gradient that's called and CGS,
which stands for conjugate gradient squared and then by c g stab, which stands for by conjugate gradient stabilised and so on.
When you see all these names, it looks a little untidy intellectually,
but the fact is these methods are spectacularly important in practical problems.
Big stab is a remarkable one. I have I got this right in the 1990s.
The mathematical paper that got the most citations of all was the paper that introduced
by CGG staff much more important than Fermat's Last Theorem and stuff like that.
Then of course there are analogues for non symmetric eigenvalue problems, which is a subject I've been very interested in over the years.
And the basic 1950s mathematics was introduced by somebody called Arno Lee.
And then in the nineties there's the implicitly restarted version, implicitly restarted only, which made it more and more practical.
And then there's also a Jakobi Davidson set of algorithms in the non symmetric case.
Now of course, I'm not really telling you very much about these things, but I want you to be aware of them.
Let me give you a few pointers to where you could learn more.
So if you wanted to learn more about the X equals B problem, one of the books on our reading list is by Yousef Saad.
His first book, if you like, and another one of the books on our reading list is the book called Templates.
And then similarly for the eigenvalue problem, Saad's other book.
And the other templates book. So those are some remarkable places to look.
Also references by Vandervoort and many other places.
Now in MATLAB, there are lots of codes that do lots of these things, and I'll list a few that I happen to have jotted down.
You know about PC G, right? So we've used that precondition conjugate gradients is the name of the MATLAB code.
There's also a code called Minions. There's also a code called Simple Q.
What else have we got in this business? There's a code called LS QR.
There's a code called G m rez. There's a code called Kumar.
There's a code called B, C, D. There's a code sigs and basically stab.
They all have codes with exactly those names.
If you want to do. I know the Orlan shows both of them.
Are done by the code called I was. And just to make sure you're with me on I eggs.
Traditionally in MATLAB, there's a command I which would find all the eigenvalues of a matrix.
So and by and matrix and eigenvalues that's an n cubed calculation.
It's grade event is less than 1000. It's impossible and there's 100,000.
I is an iterative code which aims to get some of the eigenvalues of a matrix, and we're going to do that in a moment.
By default, it will get you the six biggest eigenvalues, but of course you can specify you want some other method.
And I as I circled land chosen are nobody but it's really implicitly restarted so I exist you based on this
implicitly restarted technology and in particular it all came out of a Fortran package which is called a R pack.
So when the implicitly restarted stuff was being done at Rice University 20 years ago by Danny Sorenson and Richard Luke and others,
they made a Fortran package air pack and eventually MATLAB incorporated that.
It's just amazing. And on your assignment problem for you can use args and just it'll it'll work.
I think that's all I'm going to say in this overview. Are there any questions?
Okay. So what I'm going to do now is look at one of these.
The big one that we haven't touched yet are chose, which is mathematically very close to conjugate gradients,
but for eigenvalues instead of systems of equations. So let's talk about land chose.
Land chose was a remarkable man. He was, I think, a Hungarian Jew who was born Levi and changed his name.
And if I remember right, he went to the U.S. but he had been a communist at some point.
So this was in the early fifties. It wasn't a good time to be a former communist in the US.
So I guess he ended up in Ireland after he left the US. So lunch with her Asian.
1950. I'm a little unclear about how much Land chose and Destination Stiefel talked in those early years.
They're doing such close things now, in particular remain.
Let me remind you that in the case of conjugate gradients. We had the so-called rediscovery.
In 1970 by John Reed or was it 71?
I forget where he noticed that this 20 year old algorithm was more useful in practice than people realised if you regarded it as iterative.
Now Ranchos, which is the eigenvalue analogue, had just the same history with a rediscovery by somebody else at the same time.
And this was Chris Page. Very strange history.
I think maybe even the same year, plus or minus one, happening within a few miles of each other.
This was in Abingdon. This was in London. So Heston is in the Stiefel and land in the early fifties probably didn't communicate enough.
And then these guys 20 years later, sort of rediscover the algorithms and practice again independently.
Eventually, of course, people saw all sorts of connections.
So I'm going to give you an idea of what nachos is all about and write down the formulas and then we're going to play with it.
So we're given a symmetric matrix. Real symmetric matrix.
Now for the eigenvalue problem.
It doesn't matter whether it's positive definite when you're inverting a matrix, the location of zero matters enormously.
But if you just want the eigenvalues, it's all translation and variance. So we don't care whether it's positive.
Definite. So what Mann shows does.
In principle, the Land Show's algorithm constructs a tri diagonal ization.
Of a. And let me spell out what I mean by that.
It finds a tri diagonal matrix that A is similar to.
So if T equals an orthogonal matrix times a times the inverse of an orthogonal matrix.
So that's Q inverse. Then these two matrices, A and T are similar, so they have the same eigenvalues.
So this is what we call a similarity transformation.
So A and T have the same eigenvalues.
And T is tri diagonal. So the idea behind land shows is to take your matrix A and turn it into a much simpler matrix, which we can write like this.
Alpha one one, beta one, alpha two, theta two, Baidu two and so on.
Everything else is zero. And of course, as we very well know, when a matrix is mostly zero, that opens up all sorts of other possibilities.
If you want to find the eigenvalues of a tri diagonal matrix, there are lots of ways to do that with relatively little work.
In fact, you can you can do that pretty much in oh of end work as opposed to end cubed.
So if you can change a dense matrix into a tri diagonal one, that's a spectacular improvement.
Now when matlab computes I it try DAG analyses the matrix and then finds the eigenvalues of the tri diagonal.
When we do AIG's this iterative stuff, it's a different method of tri diagonal lasing.
And here's the amazing thing. You don't go all the way.
It's a million by million matrix. Let's say in principle, we would turn it into a million by million tri diagonal matrix.
But that would be n cubed work. You don't go that far. You just make it maybe 100 by 100 or 200 by 200.
By 200. You go partway along. Then you look at the eigenvalues of this far from complete tri diagonal matrix,
and very often they approximate the extreme eigenvalues of a very accurately.
It's amazing. So that's the key idea. So the big idea, which if you like, became the standard idea after Paige in 1971, don't go all the way.
Which would be of m cubed operations.
You go some much lesser distance, which in a typical application might be on the order of hundreds.
And then you have a small tri diagonal problem.
You compute its eigenvalues and you see what you've got.
So let's say to be very loose, take maybe hundreds of steps.
Of course, with 100 by 100 try diagonal matrix, you'll never get all the eigenvalues of a, but you can get some of them.
As you might imagine, the theory of this is a little involved, but it's also very beautiful.
It's again about approximation, as we saw with conjugate gradients.
With conjugate gradients,
we had a polynomial whose job in life was to be small on the spectrum while being normalised to take the value one at the origin.
Well, essentially the same polynomial comes up here,
and if I were to draw a picture to try to give an idea of how Len chose finds eigenvalues, let me draw that picture.
Here's the real axis. Now, the location of the origin doesn't really matter.
Nevertheless, I'll put it in because it helps us see this as a plot of a function.
Imagine that your matrix has a million eigenvalues, which are, let's say, located somewhere or other.
And maybe I won't make it definite. There may be located all over the place.
So that's a million eigenvalues. Okay. Now, what the Lantos iteration does, it turns out, is much like conjugate gradients.
It constructs a polynomial, which is small on that set.
The normalisation is a little bit different, but it's the same business of a smaller polynomial.
So it magically, in some sense, finds a polynomial that's very small on that set.
And the reason it finds eigenvalues is that, well, let's see if I want to test a parabola.
I couldn't do very well. It would be sort of, you know, some big parabola.
It's normalised by a monarch as it happens.
But as you get higher and higher degree, you can get smaller, better and better agreement to the eigenvalues.
And the reason land shows finds eigenvalues is it turns out that the optimal polynomials tend to go through approximately the extreme ones,
and then they do things like this. So I'm just giving you the intuition.
The monarch polynomial that is smallest at these red dots has the remarkable
property that typically it goes right through almost exactly the extreme ones.
So in this picture. Lantos is doing a good job of finding those three eigenvalues.
It's an amazing connection with polynomials and approximation and linear algebra.
Land shows probably didn't view it that way, but eventually that became clear.
So continual gradients and land shows are both about approximation theory.
They both converge at amazing rates for certain problems.
And let's show you that. I need to give you the formulas before we play with it.
It's a loop, just like conjugate gradients.
So here is the lunch house iteration.
You start with a scalar better not which is zero and a vector q not which is zero and a vector b which is arbitrary.
I'm trying to write it in a format that comes close to what we wrote for CG, and then we normalise B and call it Q one.
So we take B and divide by the two norm of B, which means that root mean square, the square root of the sum of the squares, of the absolute values.
That gives us a normalised vector, and we call that Q one.
And then the loop looks like this four equals one, two, three, and so on until we're happy we set the equal to eight times Q And we set Alpha N.
Equal to the inner product of Q, n and V.
We update V by saying V equals v minus.
Beta and minus one. Q and minus one.
Minus Alpha and Q in. So you can see here the flavour of a three term recurrence relation, which we also had with conjugate gradients.
So these are very closely related to orthogonal polynomials.
Three term recurrence relations. We compute the norm of the.
Call it better end and then we normalise to get the next Q.
So Q and plus one equals V divided by its norm.
So you can see it's a iteration very much like conjugate gradients.
And as with conjugate gradients, the work at each step is basically one matrix vector multiplication.
So if A is dense, that's n squared work at each step.
If A is sparse, it's less than n squared work. And now comes the remarkable step.
Compute the eigenvalues. Of the NBN tri diagonal matrix.
Of exactly the form that I wrote down.
So let's call it T sub n and I'll just call it tri diag.
But I've already written it down so you'll know what it means involving the parameters alpha one through alpha n and theta one through bad n.
I guess maybe only beta and minus one comes up. No way that I forget.
This is a small tri diagonal matrix. You can do that by calling matlab iig if you like, which is using other techniques.
But it's easy because this little n is a small number compared with the dimension of your matrix.
And that's the end. You stop when you're happy. And now let's see what happens if we do that.
So I'm going to run the code M6 under bar lunches.
If you look at the handout, you'll see, just as with conjugate gradients,
what I've written is a very pure version of Alonzo's code, which does nothing but this.
Essentially, of course, real software does much more, and in particular the whole business of implicit restarting I haven't told you about at all.
But it's very important, very powerful. Okay.
So let's play with the launch code.
I'll say a equals diag of sparse of 12999.
So that constructs a 999 dimension matrix whose entries are just one up to 999.
It's represented as a sparse matrix in MATLAB. And let's run Lantos on that.
20 steps and look at the result in its last four entries.
So it has gone through this loop. It's taken 20 steps and outcome these numbers.
And of course, we don't really know anything about those numbers. Now, you and I know the exact eigenvalues of either 1 to 999.
So you and I know that those are not very good approximations to the top four eigenvalues.
So we haven't had anything like Convergence yet. So let's try 40 steps.
Well, there's a bit of a hint of convergence at the top. Eigenvalue should be 999.
So you see we're beginning to get there, but it certainly isn't machine precision yet.
So let's take 60 steps. So now let's see.
We've got 999 and 997 and so on.
It's still not there. So let's take 80 steps.
And now you see it's beginning to converge. So the top one 999, we've got to about six digits.
We could keep going. But I'm going to make the problem more complicated.
So there we've got more than six digits. You can see some kind of convergence is happening.
And we've taken far fewer steps than the dimension of the matrix.
Even though that dimension isn't that great. Now let's do a hard problem.
Let's say A equals a random, sparse, symmetric matrix.
Using MATLAB is built in codes. Spar around a sim.
So it's a 10,000 by 10,000 matrix with 0.1% of its of its entries non-zero.
And just for fun, let's mess it up a little bit.
I'm going to make the 500 500 entry equal to ten and I'm going to make the 700 700 entry equal to eight.
Now we don't know the eigenvalues of this matrix,
but we know it's a big symmetric matrix and probably those ten and eight are going to appear somewhere approximately.
But there's no way of knowing exactly what will happen.
So let's try the same old thing.
First of all, let's take a look at it. You can see it's a.
Dense matrix by appearance, but not really so dense.
If we zoom, we'll discover as usual that it's not so dense.
Yeah. So it's a sparse matrix. So let's do our nunchucks thing.
Let's say. E equals land chosen.
I'll do the same thing. I'll. I'll take 20 steps. Sorry, that was 200 steps.
Well, let's skip that and we'll come back to that. Let's take 20 steps.
Let's take 40 steps. So let's ask, do we seem to be converging?
Well, we do. I'm sorry. You can see the 11.717 is looking pretty good.
9.8 to 6 is looking pretty good. So to the eye, it does seem that we're converging.
Let's take more steps. Let's take. 60 steps.
And now you see the strange effect that land shows. Subject to that.
Actually, there's only one eigenvalue of 11.717, but it tends to give you multiple copies.
They're called ghosts for reasons, interestingly, related to the picture I drew.
So you have to be cautious if you're running a pure chose iteration to exclude multiple eigenvalues that aren't genuine.
Luckily, the implicit restarting does all that, so real software will not have this problem,
but a pure lunchbox will have this problem if we keep going, if we make it 80 or 200, as we did a moment ago.
So there you see we've got two copies of both of the top eigenvalues.
Let's look at and minus ten. What have we got there?
We've got two copies of the top eigenvalue, two copies of the next one.
And then who knows, let's take 200 steps.
So there we have six copies of the top eigenvalue and four copies of the next eigenvalue.
So you see, a pure Lantos iteration is great for finding one eigenvalue, but not more than that.
But once you do the restarting, it's just spectacular. And in particular, if I say I have a let's put a tick tock around it.
So just to remind you, this is a 10,000 by 10,000 matrix.
That's not all that it doesn't have a structured sparsity.
It's got a mess of a sparsity pattern. So in 2.6 seconds on my laptop, it has now found the two eigenvalues we keep seeing,
11 and nine, also some extreme negative ones, and then the next two.
So these are the six extreme eigenvalues. And by the way, we could, if we just want the positive ones, of course there are ways to do that.
I could say eyes of a six.
And then I believe you say L.A., which stands for largest algebraic.
So this will now find the six largest algebraic. Or I suppose I want to add more of them.
I could say 12. So this implicitly restarted land shows.
Technology is quite amazing. That's all I'm going to say about it.
Any questions? It's a little surprising how late some of this developed historically.
In principle, the seventies could have done this, but it took people a long time as problems kept getting bigger and bigger.
I guess the incentive sort of steadily became stronger over the years.
So what I want to do in the last bit of this lecture is to show you the other handout,
the numerical software tools handout, and we're going to play with that online.
So this is officially section one, 1140.
The. So of course, you all know how to find things on the web.
Better than I do, I'm sure. But.
Maybe you're not so aware of the richness of some of the stuff that's out there related to numerical computation.
And the goal here is to make you aware of this list of things.
That is an interesting starting point for seeing many things that are out there.
So we have about 10 minutes and I'm going to click on 15 or 20 of the things on this sheet of paper to give you a sense of some of what there is.
So we'll go to. There.
As you can see, I've organised in various categories.
Now when I was your age, software libraries were very important and the things were collected together and you use the library.
They're much less important than they used to be. I think things are done more ala carte, I guess you'd say.
For example, net lib was a wonderfully important thing that came along even before the web for collecting numerical software.
It's still important, but not not as important as it was. If you go there, you'll see an old fashioned feel to it.
Perhaps it's but it's it's important. It's hosted at Oak Ridge National Laboratory and the University of Tennessee.
I won't mention all of these things. Traditionally, in the early days, a lot of this software would have been in Fortran and then C became important.
Those have certainly been the dominant languages of numerical computing over the years.
Of course, a lot of things are done in MATLAB and Python and other languages too.
Fortran 90 was a big change to a sort of more matrix oriented way of thinking.
And then, of course, you all know probably better than I do about GitHub,
which is the absolutely huge system for storing open source stuff on the web.
And for example, our software project TAB Fund is all hosted on GitHub.
I'm curious how many people here have used GitHub? That's impressive.
And I'll bet even last year it would have been half as many. It's just exploding.
I haven't checked on the web lately, but the number of projects hosted there is spectacular and you know, it's only a few years old.
It's just amazing. Okay.
But of course, GitHub is not numerical in general, although there is some numerical stuff there.
If you go to information portals, the next category here.
For people like me in the profession of numerical analysis, there's a weekly digest which means a lot to us.
These other things are also important in their way. Let's click on the one called S.W. Math, which is relatively new.
This comes out of the people in Germany who run Central Blot, and it's a software search engine, which seems to be pretty interesting.
I've only played with it a bit, but this is relatively recent and I think if you're interested in finding things, it you may find them there.
Let's just try something at random that I haven't tried. If I try, well, let's say fast for a transform.
Let's just see what happens. I don't know what will happen. So it gives you a lot of information about various things out there.
I don't know which hedge fund is there that was not intentional, but that's mine.
Interestingly the most famous of their 12 pages. I see.
But on each of these projects you could then find much more information.
And it's pretty impressive. By the way, I have some sort of virus in my computer, so occasionally odd things happen, and that's more than most.
I hope we're all right here. Yeah.
See, I get ads for things that I'm not supposed to get, right.
Let's hope we can recover here. So far, all of the ads have been perfectly clean.
But, you know, I expect any morning that the virus will have mutated if you go to numerical analysis journals.
I've already had a hand out on them. But of course, these days everything is available online.
So that's, again, a list of some of the ones that mean a lot to us. General purpose libraries.
I mentioned these were much more important years ago than they are now, but still there's some impressive stuff there.
Wilkinson Prizes. This is a very interesting one. 20 years or so ago, a prize in numerical software was founded.
And it's I think it's been awarded seven times and they're all remarkable.
So these seven projects are really very special things and differential algebraic equations.
Automatic differentiation, the fastest 48 transform in the West Triangle mess generation deal to finite elements
IP opt for interior point methods and most recently one by a guy here in our building,
Patrick Farrell is Dolphin Adjoint, which is an automatic system for generating add joints of finite element models.
They're all interesting and worth looking at if you're in those areas.
Now, of course, sometimes we do extended precision or symbolic things.
Maple and Mathematica are the most famous in that category.
Some people love Pari. MATLAB has its symbolic math toolbox, which is okay, but I think you can do much more in Maple and Mathematica.
Well, lots of things there. Linear algebra has always been the most developed part of computational science scientific computing.
It's extraordinary what is known there.
In the seventies you had these packages Ice Pack and Linpack, which really created the modern software business, and then they mutated into L.A. Pack,
which is has ever since been the standard scaler pack for parallel computing air pack, which I mentioned for eigenvalues.
Here are the templates books. In ordinary difference differential equations.
I don't have much to mention, although cab fun is very nice in pdes.
That's a huge field of course. And different packages tend to be good at different things.
So one that's very popular for conservation laws is claw pack or for geo physical conservation or geo claw and so on.
So that's a nice example of a system on GitHub. Open Source APL Htmg is an ancient but still evolving educational tool for finite elements.
What else comes on? Multi physics is commercial and though in principle we would all be using it.
I don't know many people who seem to use it who uses comsol. That's interesting.
It's a big company that does a lot of good stuff, but not in our world, I guess.
The MATLAB PD toolbox hasn't been touched in 50 years, so it's a piece of ancient code that still some people like because it's available.
If they've got MATLAB going on to optimisation, we're going to be talking about that more.
So I won't linger on this, but that's a very important, well-developed field with a wonderful online interface.
So nice. For example, the server in the Midwest in the U.S. is a very important resource for optimisation.
Other numerical topics?
Well, I've certainly mentioned the top 500 supercomputer site, which is interesting, the digital library of mathematical functions.
So every few years the US government shuts down or nearly shuts down.
It's about to happen again. And a couple of years ago it did shut down.
And you might think if the US government shuts down, how does that affect us here in the Numerical Analysis Group?
Well, the answer was two things. One is that one of us had planned a holiday in the Grand Canyon and had to go to Utah instead.
The other is and this was the killer,
the online digital library of mathematical functions was no longer accessible for eight or ten days, and that really was annoying.
So the US government does do something useful and there it is.
In CFD. Traditionally, a lot of the best stuff is commercial and you have to pay big money for it.
I guess fluent, for example, is in that category, but there is some very interesting open source stuff.
Phoenix is a very well developed one, and then lately one hears more and more about open phone.
Who's used Phoenix? Who's used open phone.
Interesting. Okay. The Dolphin Joint Project is in the Phoenix setting.
Statistics is, of course, a huge, huge, important subject, which tends,
for whatever reason, to be a little separate from the rest of numerical computing.
But Oxford is very much involved in this through R in particular.
Oxford is one of the big are developing places are as an open source statistics package.
Most of the ones listed there are expensive, not open source.
So who's use are? Oh, that's impressive. And even though who's in statistics?
Yeah, I thought so. So many of you have used our. Even though you're not in the statistics.
That's good. Certain things can be done on the web, I think.
Predicting the future is always difficult. We now live in an era where everything is stored on the web, the cloud.
You know, nobody saw that coming. But suddenly everything we do is stored on the web.
Whereas 15 years ago, people predicted a lot of computing over the web.
And of course, that is also there, obviously. But somehow it's the storage that has more changed our lives than the computing.
Especially at the high end numerical level. But there are some interesting things.
Of course, Wolfram Alpha is one that we all know. Just to mention some others, more mathematical.
If you have a sequence and you want to find what is that sequence, go to the online encyclopaedia of integer sequences.
If you have a number and you want to know what is that number, you know, 3.141592653.
What is that number? Then try the inverse symbolic calculator.
I'm going to talk about Chevy Fun at some other point if you're interested in Schwartz, because tarsal mapping, there's a beautiful package for that.
High performance computing is, of course, rapidly evolving.
And I see here I don't even have anything mentioning GPUs, but of course GPUs are a very important part of that these days.
Traditionally, they're the two paradigms of high performance, which is shared memory and distributed memory,
and there's open MP and MP for those graphics and visualisation I don't know much about.
There are a few commercial things, some of which are then gone. Open source.
Many people end up using MATLAB in practice, but of course for high end stuff, there are other packages out there,
so please take a few moments and play around and click on some of these links.
I guess that's it for today.