So, yes, so machine learning has ventured into its many parts of physics,
string theory is one, and that's happened quite recently, so I'm going to talk about that.
And the plan is, well, I'll give you a little bit of a motivation and the history which is going to be very short.
I'll go over some machine learning basics, which is going to do something very similar to what artists done earlier,
but perhaps phrasing it slightly differently.
So maybe that's helpful because hearing it twice presented in a slightly different way might be might be useful.
I have to tell you a little bit about string theory, because otherwise I can't explain what the applications are going to be.
And then eventually I'm going to put these two together and and tell you what you might be able to do with machine learning and string theory.
So let me give you a short history, and it's really a very short history,
because this subject machine learning and string theory started about three years ago,
and I think it's fair to say that it started in Oxford Theoretical Physics with these three gentlemen.
I mean, really, it's been a post-doc here, is happily smiling, standing in front of the CMS detector at CERN, where he is now a fellow.
And this is when Krypton do. If it was a postdoc at the time, he's now a long term fellow in Munich and young.
We hear who is at City University London, but has a long affiliation with Oxford.
And four years ago, we were discussing precisely how we would get this topic started,
how we would go about applying machine learning in string theory.
And this led to two papers one by volume, one by Yonghui, who were kind of the first papers doing this or exploring this sort of thing.
So this was two years ago, and it's been a burst of activity since.
But of course, it's a very new subject,
so people are still exploring as there's no there are no final conclusions and there won't be any final conclusions today.
So all I can tell you about is basically what this beginning looks like and what we're hoping for.
OK, so what are the motivations, why do you want to do this in the first place?
No string theory leads to very large data sets, and I'll explain a bit later why that is right and that.
But these are data sets which are very different from the usual data sets that you use, such as pictures and videos.
And these numbers keep changing, but the current world records for the number of solutions in string theory is this somewhat ridiculous number here.
Right? And so these are really, really large data sets, but quite different.
And, of course, machining provides techniques to deal with large sets of data,
so it's an obvious thought that you might be able to put those two together and be able to make some progress.
So can we can we uncover features of string data using techniques for machine learning?
So that's that's the obvious question.
It's perhaps slightly less obvious question, which has to do with this example that I talked about earlier about the the deep mines go enterprise.
So let me come back to that.
So as you can see, the number of so sensitive go games is quite large, but still a lot less than the number of string solutions.
But it is also a lot larger than the number of sets of chess games, which is why GO was a challenge for a very long time, computer wise.
And so this these curves to illustrate what DeepMind was able to do in this context,
so so the the dashed line here that corresponds to the supervised learning system that they initially devised, which beat the the world champion Lee.
So this was this was a system that was trained using human go games, basically with many human go games.
And the the the this blue line up here that that goes up,
that is the reinforcement learning curve, which was for a system that did not use human input.
It just knew the rules of go and learn to play the game by playing against itself.
As you can see for about two days and after two days, it's strength.
So this this vertical axis is the strength with which it plays after after about two days.
Its strength exceeded the strength of the supervised system and of course, then also of the of the strength of the world champion.
So that's quite impressive. And this curfew makes it even more impressive because this curve or these two curves,
they show the number of human moves that the system made in the context humanlike
moves that see professional player moves and the fact that this blue curve,
which corresponds to the reinforcement, is below the the other curve says that the reinforcement learning which had hadn't used
any human input is in fact making moves that are not humanlike like it invents new moves.
Yet it is stronger than the best human player. And to illustrate this in a paper by presenting some of these moves that the system has come up with.
So that is quite impressive, and that makes you think, well, perhaps if if such a system can reveal new structures of the board game,
maybe it can reveal new structures in physics or mathematics, and we might be able to learn something about the new network in the context as well.
So just to summarise, the two basic questions are then where exactly this one, right?
Can we can we somehow use machine learning to reveal structures, mathematical structures within string theory?
And this is sort of related to one of the questions we had earlier about. Can we understand better what the system is actually learning?
Can we look at it as more more than a black box?
And this is all very new. So. So this will illustrate this with a paper that is just from last year.
And then the second question is, is the one that post earlier, which is,
can we somehow use machine learning to help sort through this enormous amount of data?
OK, so let me go through some of the basics, so this is very similar to what I did, but perhaps presented slightly differently.
I think one of the very useful language, if you if you know the language is just the basic language of mathematics.
So if you just remember a few bits of very basic mathematics, new netbooks are actually very easy to understand.
And one thing to say is that you should just for the time, just think of it as a box,
which corresponds to a function f feature which takes as an input,
an end, dimensional vectors and real numbers, and produces an M dimensional outputs and real numbers.
And it's actually more complicated than that. It's not just a single real function.
It's a whole family of of functions which depend on some set of parameters feature that we call collectively theatre, right?
And so the the art of training a neural network is to somehow pick these parameters to in some sort of suitable fashion.
And how do we do that? Well, we have typically in Alice, in supervised learning, which is what I'm discussing here.
We have a training set out, which corresponds to instances of input externally and intended to target outputs.
Why? So these are the way I sort of the real, the real results that we would like to get, right?
So if if I were pictures of cats and dogs, then I would be one zero one is one is one is for Dawkins, who is for cats, for example.
And how do we train this while we form a function like this, which is called the lost function, right?
So we we look at the difference between the output of the neural net.
For a certain input and given adjustable parameters to we subtract from it the correct intended output,
why we scrap the whole thing, and we do this over some sort of batch from this training sample, right?
And then we we think of this whole expression as a function in those parameters theatre.
And we try to minimise it by a certain method that is normally taken to be what is called stochastic gradient descent,
which basically means you go down the steepest gradient in the feature direction and
you keep repeating this as you as you pick matches here from this training set.
And you hope that that way you get to a well-trained network.
And then, of course, us unseen data of the same kind to to test this network, see if it if it generalise as well.
In other words, if the loss on this data computed for the value of FT are not of the parameters that you've arrived at is actually sufficiently small.
And if that works out,
you declare success and you would use the Soul Train network with the parameters now set to those specific values and make predictions.
All right. So that's that's the basic process. Now I have to say a little bit about this black box here.
So what's the black box? So this is the the simplest version of what could be in this box, which is quite the perception.
And the perception is it's just this performs a sequence of two steps.
The first one is just enough time up, right? So it takes the image an input vector that forms the dot product with a Vector W, and that's a B.
And then the second step is it applies to the output of that some function.
So if you combine this thing in mathematical language, this is what the function looks like.
And W the vector is called the weights.
B is got the bias and it's the activation function.
And these two sets of probabilities together is what I call feature previously.
Right, so this this is these are the things that you want to trade in this particular
context and the activation function is what what are drew on the board earlier,
for example, this one choice, which is this function, which sort of interpolate between zero and one.
But the other choices? OK. One thing that hopefully you remember is that this this kind of equation here that is performed
in the first step is very closely related to the equation of a plane or a hyper plane.
It's a I have a hyper plane, which is defined by this equation, right in two dimensions.
It would just be a line if you if you think X is a two d vector, it would just be the equation of a line.
All right. Now. It is with this with this sort of geometry in the back of your mind, it's very easy to understand what the system does.
Suppose that your Vector X is such that it is above the line, right?
Then the output of the first element here will be positive.
All right. That's what it means to be above the line. Because there is exactly on the line greater than there is above the line, right?
If it's if it's if this outward is positive, you are somewhere here on the branch of this activation function where it is plus one.
So the output will be roughly plus one. If it's below the line, then this this Afghan transformation is negative.
You'll be on the negative branch of this sigmoid and the output will be zero.
So you can see what this is doing is a very basic system which decides whether a given point is above or below a line or a plane.
All right. So it's it's a very simple example if you want of of a pattern recognition.
OK, so this is actually a hands on subject.
So I want to do something hands on. I want to actually show you how this works in real time.
So let's this will take a little while, but let's let's see. So these systems are set up, this is, for example, within Mathematica,
so here is a set of points random points generated in the box and there are two kinds the blue and the yellow,
and you see they're roughly separated by a line. All right.
But let's suppose that we don't actually know that just yet.
We want we want to train a system to actually recognise that line so it can distinguish between the two kinds of points.
And so. So this is the training set plotted. This is what it looks like in practise, right?
It has the two coordinates, the X and the Y coordinate.
And then the the target is either one if it's a blue point or two, if it's yellow or yellow point.
OK, so then we can define ourselves a perception.
So he has the he has the perception that's how mathematical this place is, this is this is basically the A5 transformation.
This second bit is the logistic sigmoid activation function.
And then we can train this in real time.
And this is what you see here is precisely the lost function that I defined earlier.
And so if this goes down, that's a good thing.
And the the orange one is the loss on the training set and the blue version of the curve is lost on the validation set.
So you want both of them, you expect the blue one to be higher than the orange one, but you definitely want both of them to go down.
And now it's finished. And I can go and extract the values of the weights and the bias from my network,
and I can plot from those from the state to the line that they define in two dimensions.
And if I do that, that's what I get, right? Not surprisingly, right.
But it's so. So I mean, someone was asking earlier, Can you understand what a new network does better?
This is a little bit of an understanding of what it does, but of course, in a very, very simple case.
So this clearly distinguishes the the blue and the red and the blue and yellow points.
And so if you know, had an arbitrary point, picked an arbitrary set of X and Y coordinates and fed it into this network,
you would get a zero output or one output. And depending on what you get, you would be able to decide where the point is above or below that line.
OK, so. So this was the the simplest building block. But of course, it gets more complicated.
You can look at several of these in parallel. So each each one of these is now one of those perceptions from the previous slide.
You can look at any of them in parallel, but they have independent weights and have independent biases.
And now the output, of course, is not a single real number. It's a it's a vector with components.
And of course, that's a very inefficient way of riding the stone made a much better way.
If you sort of remember basic, the vectors and matrices is to combine all these weights into a matrix,
which I call a W and combine all these biases into Vector, which are called B.
And of course, then these are all the parameters that are called feature previously.
And you get to what you can sort of symbolise this whole operation like.
So. Now this becomes just a multiplication of a of a of a vector with a matrix plus an extra bit.
And then there's an activation function as before. So,
so one way of of saying what this does is when the previous system learnt about the
existence of a single hyper plane and this this such systems combine in parallel.
This learns about the existence of M hyper planes.
And then, of course, you can go further, you can take one of those building blocks, one of these perceptions in parallel.
And you can construct from it several layers that you can just do this sequentially, one after the other.
And of course, in each step in general,
you change the dimension of your input just depending on what the size of this matrix of this weight matrix is.
So this indicated that with those various numbers and one and two in and so forth.
And of course, what these dimensions are depends on the details.
And there's there's a much longer story there, but that is that is the basic structure.
And I want to show you another example where this is done in a slightly more complicated way.
So sort of same principle, but now with the set of two points is a lot less a lot as simple, right?
So is a sort of a pattern. Right.
And as before, we would like the new network to distinguish between the blue and the yellow points, but they're not linearly separated anymore.
So if we try the same method as before, so we just try a simple, simple perceptron.
Right. So the one that represents a line in which we try to change this?
Well, that doesn't work, right? As you can see, the loss is not really getting a very small.
I mean, you can see the the numbers here, it sort of zero point five, right?
That's that's not very impressive.
And that, of course, expected you would not expect a single line to be able to tell the difference between those two shapes.
All right. So we can we can stop this, but we can sort of still look at what it's actually done right and what it's done something very silly.
Right? OK, so but of course, that was a silly way of going about it anyway.
So we do something more complicated, right? So we we look at a neural network which has in its first layer for perceptions in parallel.
All right. So this is like to be pretty a for Buffy says that that that these are for four of these perceptions are arranged
in parallel and then we have a sort of final layer to put them all back together and make the outward real.
So we do we train again.
And while initially you get a little bit worried, but then then clearly there is a difference that it starts going down quite dramatically.
OK, we can probably stop it now. And then we look at the same same picture as before we read about the way it's read.
And so this is what it's done. If I run this again, it would probably do something else,
but the point is that it's somehow arranged the four lines that those four perceptions correspond to in a
way that allows you to distinguish the blue and the yellow points based on whether they're above or below.
Each one of those four lines. And you can see how this how you could generalise this to get to a proper pattern recognition.
And in fact, this is this is something we can we can actually do so.
Uh, I was talking about this and this said which is, which is this set of hand-written numbers.
It's sort of a standard test set that has been used.
Here is a small sample of this set. So it contains these handwritten numbers then.
And of course, the target is what they actually represent, which numbers actually represent.
And I can use a network which is in practise very similar to the previous one,
just a bit more complicated and uses the twenty eight times twenty eight inputs,
which is the size of the the pixel size of these handwritten numbers and then just goes to an output.
And of course, I should say this,
these are all these are all the handwritten numbers from the tonight and then I've just picked the one and the nine here as an illustration.
So we have a lot of binary classifier. And we can train this thing.
As before. Right, and it's it works quite well.
OK, stop it now. And.
So then this is just giving you an impression,
so this is sort of used a test sets so something that has not been used at a fraction of the original set, which has not been used for training.
I've just fed the picture into the train network and the right hand side after the error is the output that that the network actually provides.
So as you can see, in practically all cases that are plotted, it actually correctly identifies whether it's a one or nine.
And and in fact, the the number down here gives you tells you that it's a ninety nine point seven percent accuracy of predicting the correct outcome.
OK. Right, so so that was. The multi-layered perception,
there's one more thing I want to one more type of network I want to discuss which is in the within the context of unsupervised learning.
So that's that's learning where you're not providing the target you you're kind of hoping that the network
will discover the pattern within the data by itself without telling you anything about the answer.
And this particular network is called an auto encoder.
So let me explain how that works.
So the first part of the auto encoder is pretty much the same structure of a multilayer perception that we had on the previous slide.
And so just a sequence of such layers.
And this is combined with another such multilayer perception where now the dimensions go in the opposite direction.
So this goes from an end dimensional vector in sequences down to a vector with dimension and L.
This starts with the same dimension and L and gets it back to an end dimensional one.
And it's done in such a way that in the top one, the dimension is decreasing as I go from left to right.
And of course, and then in the bottom one, the dimension is increasing as they go from left to right.
OK. And. The idea is.
But somewhere here in the middle.
Well, first of all, I will then combine these two networks right, and I will fight what is called the encoders of this first network.
I will fit the output of that into the inputs of the second one.
And that way, I will ensure that my input goes through this bottleneck,
keeping in mind that the dimension here in the middle is a lot smaller typically than the one you started with.
Right? And what you are trained on? Well, I don't provide the targets.
I just have a set of possible inputs ixi that I put into the auto encoding on the left.
And what I will try to minimise answer the loss function in this case is just making sure that the input X and the output X are the same.
All right. So I'm trying to reproduce whenever fed into here at the end.
But what I'm doing, so I have to feed it through this bottleneck.
Right, so. So this must mean that somehow in the middle, the this audio to encode must learn some successful compression of the data.
OK. So again, let's do an example of that.
So this is for the same data set, the data set. And this is it starts looking a bit more complicated,
nobody sees it takes the twenty eight times twenty eight inputs, which corresponds to one of those pictures.
It increases damage first, but then it starts decreasing.
It goes all the way to two dimensions in the middle and then it goes back up to the same size again.
So let's train this thing.
So this is now the last, but it's remember that the loss meant the the difference between the input and the output being the same.
So this this looks reasonably good. And it's finished now.
And so. So now this is actually this actually isn't very good, so maybe I should I should run this again.
So this this actually happens, right?
So this is this is something that that can happen, that depending on your on the initialisation of your neural network,
sometimes you might not not go into the right direction. Let's hope this time it's better.
No, it's not OK. Anyway, that's not what it was supposed to be happening.
Of course, what was supposed to be happening is that the the blue and yellow points there were there
were split apart and we'll show you an example of where it actually did work later on.
Right. But so the blue points correspond to the lines, the yellow points correspond to the ones, right?
So that's that's the idea that the this hour to call it would be able to tell
them apart without actually telling the machine that there are these two types.
Yeah, I mean, maybe there's a so with a bit of good will, you could anyway.
So again, yeah, the axis. So remember that the opening quarter would compress it to something two dimensional in the middle.
So this these two, these two dimensions are the two axes. So it's kind of a latent space in the middle.
OK. Right. OK, so complete switch of topics.
Let's go to string theory, so let me remind you about some basics of string theory, because I'll have to put this in this context.
So string theory is a theory of strings meant to be the fundamental constituents of of nature and come in two times the open and the close strings.
So the open strings? Well, exactly what you think they are.
And when they propagate, they sweep out this sheet and the close strings they sweep out instead of cylinders as they move along.
And it's a theory that starts out with one free one, one free, undetermined dimension for constant, which is which is the string tension.
And which is, it turns out, only consistently defined in 10 or 11 dimensions,
and this is where the root cause of the trouble is, as we will see very shortly.
Now, the spectrum of of this view is very schematically. It's pretty much like you would expect for a string.
So the the map, the master's exit, the massive exploitation of the massive and massive massive excitations with mass m.
They are measured in units of this string tension, which is too high for prime.
And in those units, they're basically integers. And amongst the modes here, where N equals zero, so amongst the matchless modes,
we find always a graviton and we always find the kind of force carriers that we need to mediate the strong and the electric forces.
So in other words, string theory generically always has the typical types of forces, the competition and the forces that we know exist in nature.
So. So from that point of view, it looks like a reasonable starting point.
And it turns out that the string tension because because the gravity is always in there and we somehow have to reproduce Newton's constant,
which is in the appropriate units, a very large energy 10 to the 19 GeV.
Because we have to reproduce that. It turns out that in most cases, the string tension gets coupled to the Newton constant.
Otherwords is very large, and it's basically of the order of the Planck scale.
And for that reason, all these modes were any bigger than zero, which you might have worried about.
They become very, very heavy. All right.
So, so the idea then, is that the the physics that we currently observe that that will only be tied to these modes with an equal zero.
OK, so what about these dimensions? So that's that's really very embarrassing.
And the way to get out of that is that somehow we need to think of a six or seven, depending on whether it's starting 10 or 11 dimensions.
We need to think of these as being curled up in a very small scale something strings of his colleagues compact ification.
So the schematically it's you start in 10 11 dimensions,
you call up six or seven of these dimensions on a space that I keep will be will be calling X and somehow effectively then at least at scales,
which are much longer than this case in which you've built these things up.
You will be ending up with a four dimensional theory as you as you would like to.
And the spaces on which you do this coming up there can be very complicated.
That is a picture of one of them. It's it's called clube.
How many folds. But this is a very particular one called the bi cubic, which are come back to later.
And. But that's that's the basic process.
And then the question is, how does this for the effective theory that you obtains, we're actually dependent on the way that you curl this up.
And so here is the schematic way that works.
One feature of the curling up is, of course, the topology of the curling up, so I can only draw 2-D 2D curling up manifolds here.
So I am drawing a choice and the sphere at these are different apologies.
And so the before this theory would depend on which one of those you picked.
And so more specifically, what the topology determines is the actual forces that you get in your 40s and the metal content that you get.
And mathematically, this whole process is caught in sort of tied up with the field of mathematics that's called algebraic geometry.
And then this, of course, also the shapes of the toys that could have a very fat or very thin toys.
So the shape also matters, and the shape will somehow determine the coupling constants in here for this theory.
So that's roughly how the correspondence works. And for the purpose of this talk, I will only focus on this first aspect here.
So I will be worried about the kinds of forces and the kinds of particles that you obtain in four dimensions from this kind of construction.
So what are what are these two apologies of for cutting up? So again, I can only draw the 2D pictures very well.
So in two, do you have this fear you have the choice,
but you could have something with two handlers, you could have something with three handles, etc.
So this is called the genius of the curve. All right. So in two dimensions, it's a very simple.
You can basically classify the topology by the number of holes by this by this single integer tree.
And that's it, right?
And in fact, if you were to come back to find just two dimensions from this infinite sequence, only one of them, the tools would actually be allowed.
But we are not. We're not just creating a two dimensions. We're calling up six dimensions.
So, so a single integer is not enough. In fact, there is usually a multitude of integers.
So. So the bottom line here is that the topology on which you do this is is classified by a bunch of integers, by integer data.
OK, that's that's that's the main message. And because we in six dimensions will typically be many choices for that data.
And this is this is precisely related to this enormous number of solutions to string theory that I mentioned earlier.
It counts these counts, the different apologies which you can use to go from 10 to four.
And some of these choices remember the topology is tied with the particle content, some of these choices,
they will lead to particle content, which looks realistic, like the one that we actually see in nature and many others will not.
So how do we actually find this? More specifically? And so I have to add a little bit more information, I haven't told you quite the truth.
So typically this Space X carries extra structures, not just the space.
It's also things on this space. And one one thing that could be on the space is what is called line bundles.
So what's the line bundle? Well, I can.
I have to sort of lower dimensions.
And now the Space X, which is the black circle, is just drawing one dimensional because I need one other dimension to draw.
And so a line bundle would be a structure where you attach to each point on this circle just align, right?
I've done it here as an arrow to indicate an orientation, but in principle it should.
It should go from minus to plus infinity like a proper B line.
So that's that's one way of introducing a line bundle on this space x.
This number is usually called o o x by the mathematicians, right?
But you can see that you can do this in other ways. This is sort of not a very good picture, but I hope you can see what I mean.
I start with a line oriented this way, and then as I go round the circle, it changes its orientation.
And when it comes back, it's sort of pointing downwards. All right.
So you can twist. In other words, you can twist the line as you go around this, uh, the circle.
And this is called a one. And then, of course, the next one I couldn't draw, right,
so but you can see that why you go around the circle, you can twist it twice and three times,
etc. And so and you could change the orientation of the twisting, which means these these numbers could also be negative in principle.
So you see that line balance in a simple case. They're classified by integers.
Of course, this is the the manifold in which we consider this one, the this is very simple.
In reality, we want to do this on a six dimensional manifold and a six dimensional.
My foot might have many different loops and around each loop, the line and my twist as you go round it.
So in other words, you need many integers in this case, typically to describe nine models.
So. So the message is here we are again stuck with integer data.
So alignment is classified in six dimensions, typically by a bunch of integers.
So. What does this have to do with anything in terms of in terms of the physics that we get in four dimensions?
Well, line have what is called section. So what's a section? So let's draw the line, Mundell.
Access to manifold, again, a lot of these red areas, one of those that I discussed previously and the section is just,
uh, basically a kind of a function which picks out a value on each of those lines.
And the physical importance of these sections is that they're kind of the internal
wave function of the particles that we're trying to obtain in four dimensions.
So the upshot is that the number of independent sections that such a line bundle has.
Well, first of all, it's counted by a mathematical concept, which is called cosmology.
But. And commodity is denoted by this age symbol, and it comes in sort of different flavours.
Age Knowledge 100, which feeds for never mind the details.
But the point is that these numbers, which are just dimensions, numbers of different sections, independent sections that you get.
They in fact count the number of particles that you get in four dimensions.
And so, so it's a very nice way in which mathematics ties into physics.
That's what happens very frequently, that you have some sort of mathematical theory,
and it precisely plugs into this problem of wanting to compute the spectrum of the fortieth theory.
So mathematically, what you have to do is compute those numbers. H h one.
And it's two. For a given night Monday. And the point is that it's an absolutely horrendous calculation, right?
I mean, it looks simple, but it is trying to if a bunch of integer numbers in trying to get the number of a bunch of integer numbers out,
but to actually do this in practise is totally horrendous.
And so, so for this reason, it seems like a good problem for machine learning, and it's something that is absolutely not obvious.
Takes a very, very long computation. And actually, how am I doing on time? I'm thinking, should I show this calculation or not?
I think probably not. OK, good. Right? So take my word for it is horrendous and therefore be a good problem for machine learning.
So I skipped the example. And so get to the final part, machine learning string theory.
So. So this line, Malcolm, that I need to compute in order to know what my patio content is.
Can I can I somehow teach the machine to learn about that? In other words, can I teach a machine to learn?
This particular map out has explained to us that these neural networks, they can basically represent any function.
They're very expressive, right? They represent these function spaces. And can I teach a neural network to learn this function,
which takes an integer vector which represents one of those alignments twisting in a particular way?
And give out the commodity or the number of particles if you want.
And then perhaps the next question, which is more ambitious, can somehow not do this in a black box way.
But can I learn something about the mathematical structure of this map?
As I go about it.
So the training data is of this form, right, so we have an integer input vector and we have one of those commodity dimensions as the output vector,
and we get this training data from our horrendous calculation that we have to do right.
It's supervised learning, so we need the data. OK, so there is a space which is called deep two, never, never mind what it is.
The unimportant is that line buttons on this space are classified by three integers cannot K1 and K2.
And we can compute the commodity for, say, about a thousand of these integer triplets.
We get the answer and we do this, of course, for a certain range for these integers, which is in a box of size 10 or 20.
And so this is the same picture that you saw previously, so this is how the the lost function evolve.
So you see it trains reasonably well. And you can see that you can then check that within this box of 10 that we've trained in.
This will predict these these dimensions correctly with a 98 percent accuracy.
But so that looks that looks very good. But if you then go and increase the size of the box to 15 and you ask your new network,
which was trained on the smaller box to predict values in that range, the the success rate drops very dramatically.
So this is what I'd alluded to earlier generalisations beyond the domain that you've used for training.
They typically don't work very well because why should the network know how it would continue so that it was only trained on this particular box?
So the upshot here is.
Well, this works to some degree, with reasonably high accuracy, it's of course, much faster than the horrendous method once it's trained.
But 90 percent accuracy or something like that might, in fact, for some applications, not be good enough.
After all, you computing a dimension of a space, but you don't really want any uncertainty in knowing that answer, as I just explained.
If you go outside the box that you use for training, it would become very bad.
And of course, at this point, we have absolutely no inside of what it even means.
It doesn't tell us anything about the mathematics.
So this is where the second question comes back, so can we actually do this in a more sophisticated way and learn something about the mathematics?
And of course, we need some kind of intuition of what the mathematics is, and fortunately, we do have that.
And what we know is what we suspect from experience is that the function that we're trying to learn is actually not such a complicated function,
even though the calculation is horrendous and performed in this way, it's a function which is piece wise polynomial.
So for in this space of key vectors, there are regions which we don't know arbitrary,
but there are regions and in each region this function, the commodity function is described by a polynomial of a certain degree.
But we don't know what that is, either.
So we can devise a somewhat more sophisticated neural network, and maybe I don't want to go too much into the details,
but the idea of this new network is that we have two branches. The Apple one is somehow supposed to recognise what these regions are.
And you can think of it working in much the same way as the sort of pattern recogniser that are presented in these earlier runs.
All right. So it recognises these regions and the local branch.
This one here sort of recognises the polynomial, and it puts the two together, and I can train this network on data.
And then read out certain bits of information from these weights here.
Which which of these input factors come with the same polynomial formula?
So if I use this information, I can basically identify the regions. So let me let me just show you an example.
So this example is actually for this by cubic that I showed the picture before.
Its linemen are characterised by two into just K1 and K2.
And if I run the neural network over my training data, I get this kind of plot.
So the different colours here indicate the different regions. So in each of those regions, I know that the formula must be described by a polynomial.
And so I can now just go knowing what the regions are and just fit the right polynomial to it,
I need very few points because I know it's most a cubic polynomial, right?
A cubic polynomial in two variables doesn't have very many coefficients. I only need a certain number of points to do that.
I can do that for the Blue Ridge. I find it's just zero.
And I could do it for the yellow and green region, which in turn in fact turn out to be two parts of the same region.
And I find this formula. I can use the formula to clean up the regions, which were a little bit fuzzy at the edges.
And find the boundaries and end up with a final formula, which looks so simple that it's almost embarrassing.
Right? And and in fact, this formula is a formula of this kind were not known in mathematics,
and it's not completely clear to this date why the standard when this calculation that it's normally been performed,
which can take hours on the computer ends up with such a simple result.
But this particular formula, so you can you can think of this as some kind of conjecture generator.
So this is a conjecture that has been generated. It's not been proven because it's only been conjectured from a finite amount of data.
But you can now go and try to mathematically prove this. And that's indeed been done for this, for me and for others of of a similar kind.
So that's an example where you might have learnt something more than sort of the typical black box thing from in your network.
OK, so this is another example,
which is just here because it's pretty so this is again for this space deep to which where nine bills are characterised by three integers,
so it's three dimensional plots, you see, it's a lot more complicated. There are six regions there, but they all identified.
They've been cleaned up. I can read of the the polynomials. So a bit more complicated and there are six regions.
This is basically just to see that it works, and you can also go and mathematically prove this conjecture.
OK, so the summary here is at least instead of a modest way,
machining can be used to generate mathematical conjectures within string theory, but probably also more generally.
OK, so the second part has to do with can we somehow use machine learning to sift through this enormous amount of string data?
And one simple question that you might ask is, Well, I have these constructions of all these four dimensional models that come from compact.
You find from 10 to four and the spectrum of of these 40 theories depends on exactly how curled is up.
Some of them will be good models from a physics point of view, others not with a neural network be able to tell the difference.
But would a new a network be able to tell me which one of these choices corresponds to a standard model of particle physics and which one does not?
So that's that's that's one basic question you might ask.
And so one thing I need to say before is is that in order to get a model with the right forces,
first of all, because I need to cut up six dimensions, but also need five of these line ones.
Right. So but that just means picking five of these integer vectors, which we could summarise into some sort of matrix that are called OK.
All right. So so just think about we pick up space, and the models on this space will then be described, but just picking an integer matrix.
And we can create ourselves training data, which is a set of these interpretive matrices,
and they go to zero or one, depending on whether that leads to a standard model or not.
OK, so that's that's the kind of training data binary choice.
And the question again is, can we somehow teach the machine to distinguish between those two?
So we have to pick a space. I mean, never mind what the space is, but the point here is that it is also described by some set of integers,
so that's a particular 60 space for cooling up.
The nice thing about this space is that there are 17000 stand up models on there, which we have found by brute force.
And we add to the 17000 the same number of non-standard models, which we just get by randomly generating some matrices with integers.
And so he had two examples, like so that's one matrix, which in fact corresponds to the standard model.
That's the one matrix, which does not correspond to a standard model. So the question is, can you tell the difference?
I can't. Right. But the question is, can the machine?
So it's a relatively simple network, two or three layers, so this is another way of representing the data.
So this gives you an indication of how big the entries in this matrix are.
And this is the distribution of this typical size, and we take training and validation data from this lower end of the spectrum,
which corresponds to the matrices with small entries. And we take a test set from this upper end.
And we run we train the new network, and we find it's extremely successful on this low end trains very well validates very well,
but surprisingly it's also very successful on the test set.
So this is an example which I don't completely understand yet of where the new
network actually generalise as well beyond the domain that it has trained in.
So the bottom line here is we can, in fact, distinguish between those two types.
And again, there is a very complicated calculation actually to do this in the standard way.
And so this this will be once it is trained, it will be a lot faster than doing this computation.
OK, so now comes the outgoing caller, which went so spectacularly wrong before, but of course not.
I'm not doing this in real time because it would take too long.
So I I used the same dataset for an audio encoder,
and the latent space in the middle is again two dimensional so I can produce a nice two dimensional plot.
And this is what the plot looks like. Right. So the red points are the standard ones.
The new points are the non-standard models and they are neatly separated, right?
And this is from actually the set with small entries and one which it was strange.
And if I use a test set of unseen data from the matrices with bigger entries, the split persists, right?
Which which again seems to be saying it's whether generalising beyond the domain of training.
OK, so the auto encoder works very well in this case as well.
OK, so that's the end of it. So machine learning and strength has really just begun, and we don't really know what the good problems are,
what the good techniques are and how we combine the two. So this is all developing still.
But at least we can see that perhaps there is a there's an avenue there where machining can be used to generate conjecture, mathematical conjectures.
And I think there's some hope that machine learning can help us sifting through this enormous landscape of string solutions.
I think the question as to whether machining can really lead to substantial progress in string theory, that's still open.
But that's why we have to wait and see. And somehow you might have a hope that because the data sets that we have in string
theory and the kind of questions that we're asking are so different from the
usual kinds of questions which often have to do with pictures and videos and
speech recognition because those things are so different in the science context.
There might eventually also teach us something about machine learning that we didn't know before, but we'll have to see.
And thanks very much.