So hello, everyone, so welcome to the last seminary in Hillary term.
So today it's our pleasure to have Benjamin Guarch as the speaker.
So Ben is a principal research fellow in Machine Learning at UCL and a tiny research scientist area,
and he is also a visiting researcher at the Alan Turing Institute.
So he's a member of our society and a fellow of the Royal Statistical Society.
So he's a leading researcher in pack based. So personally, I have read many of his papers on Taipei's during my Ph.D. study.
So today he will talk. Tell us more about Taipei's and the new research in his group.
Thank you very much.
Thank you very much for that very nice introduction and for the invitation, which I'm afraid has had to be postponed over the past few months,
certainly wish were not meant to meet with all of you, but things are like they are.
I'm very pleased to present a short overview, really, of what Backpage is, as I was joking a few minutes ago,
saying that it certainly is much more popular than used to be 10 years ago when I was on the topic.
But still, I appreciate that.
I think it's good to have maybe a kind of a refresher on the Watpac base theory is and can bring in particular commission learning and statistics.
And then I'll cover also a few recent results from my group.
So I should say also disclaimer if you do have any questions, please.
And Mitchell said, I'm afraid I'm not quite sure I can see if you type in the chat, but please feel free to interrupt me if you do have any question.
Right. So you should see my slides moving forward.
So, yeah, as I was saying,
so I'm a researcher affiliated with in in France of the National Research Institute for Science and Mathematics and UCL since twenty eighteen,
where some of the members of the centre of the eye, which is recently inaugurated, we based in Ireland, been in London,
and that also serve as the scientific director of this joint venture between France and the UK called the General and that programme.
So if you if you are even remotely interested in hearing more about that, please feel free to get in that.
I'd be very happy to.
So my research sits at the crossroads of statistics and statistics, but also more recently, optimisation and application to machine learning.
So the few keywords in my highlighting here are stats for learning to impact base.
Obviously, as I would say, this is the core of my research contributions over the past year.
I also have a keen interest in competition statistics and much more recently I've taken interest in theoretical analysis of deep learning,
as you will see. So we were able to to obtain generalisation bounds for some architectures of the network.
So as you will very much gather from my talk,
I hope my personal obsession is absolutely generalisation and what it entails for a series of problems in machine learning and statistics,
how it can translate also to obtaining better theoretical understanding of some algorithms and also drive new algorithms.
Sorry, I'm trying to do, I think, identification's to let people in, but I don't know if you're taking care of this.
I'm trying to do. Yeah, ok. OK, I guess.
All right. Let me start with a very brief introduction and maybe kind of a thought provoking into Matari to the topic of judaisation.
So I'd like to start with a very bold claim or phrase, which is learning history to be able to generalise.
And you all know this kind of classical pictures, you know, this kind of classical pictures.
This one's taken from Wikipedia, for example.
And the crux of translation is trying to understand from example what can an intelligent system learn about what is underlying?
And you pretty much all agree that if you do memorise what you've already seen,
it's usually pretty bad, this of fitting in statistics and generalisations.
Precisely the opposite of that is your ability to escape of a fitting and trying to be able to understand
what is underlying and to understand really what generalising beyond the datasets truly means.
So let me start also with a very small appetiser, really food for thought.
There's been some evidence in the past few years that deep learning might be conflicting or even
actually breaking statistical learning theory as we know it on that very simple observation.
So most neural network is trained on that. Is that actually achieve a fantastic training error?
Very low training error, basically zero. And as statisticians, I mean, this should be a red flag, right?
In the sense that might not bode well for the performance this strong.
This suggests actually very strongly of fitting.
This is not what happens. Surprisingly, they're able to generalise from from massive data sets to achieve actually excellent test errors.
You might also know, of course, this kind of picture. This is very statistics one to one.
This one's taken from from Balcones Paper and also from coughers.
You all know this kind of trade off between actually increasing the complexity of space of hypothesis versus.
Optimising the test risks, so the goal of statistics and statistical learning in particular is really to aim for that sweet spot right
over there where you achieve the ideal Trade-Off between a good training loss and the patronisation ability.
That's conjectures actually from a series of others.
But starting really with this very influential paper from Michelle and co-authors that this might actually just be half of the picture.
So this is what the other half might look like.
And the idea that they put forward is really that rather than that sweet spot, we should aim for some kind of interpellation threshold.
So an explanation, a possible explanation for how massive architectures of networks actually journalise is very well indeed,
is that there might be this second phase, which they called the interpolating regime in which you've indicated it's very handwaving.
But in a nutshell,
you've learnt the data so well that you start to be able to generalise slightly around the few data points that you already collected.
So that's that's. I mean, we can't we can't hear you now.
No. It was working like 10 seconds ago.
Can anyone hear me? Yeah, yeah, we can hear it now. OK.
No. Oh, boy.
OK. Now we can hear you.
It should be back. Can anyone hear me? Yes, we can.
Yeah, fantastic. I'm very sorry about that's my Bluetooth headphones just decided to go off.
All right. The joys of going for landline, I guess.
OK, so let me just resume on that picture. So this is a kind of conjectures that are being addressed at the very moment by Magpie's.
Another thought provoking example, I think,
is I'd like to to focus on the semantic content of learning and the way that semantic representation can be used and leveraged,
really to accelerate learning and try to better understand. Actually, Putrajaya is from a few examples.
This is very much addressed by one of the big projects that I was very fortunate to join when I joined the UK two years ago,
which goes semantic information pursuit for multimodal data analysis.
So there's a lot of partners I know in particular, and the department is one of the pieces of that project.
And this is very much what we try to address. The last third and last example is what I like to to narrate as a tale of two learners.
So for one specific task, let's assume that you have a definition that works for a classical architecture.
I'm not going to comment on that. This is all very classical to all of you, surely.
But it's it's widely Invicta now that four very simple tasks, such as identifying an item in an image.
So you want to identify horses in images? Well, you would typically achieve accuracy's, which are excellence, basically a hundred percent.
And the way it works is usually through training samples, hundreds of thousands, maybe millions of annotated images.
And it is important here because you need some kind of of way to ensure that data is curated,
which see what this actually means in theory, because it translates in many ways in theory.
And then you train this on your favourite GPU and then you get you get a deployable dignon network.
That's the first contender. So that's considered the number two.
We are also extremely good identifying horses on images, actually, they do achieve the same accuracy,
they have 100 percent accuracy and surprisingly, they are also able to do other things.
So, for example, there are very good transferring to other items which resemble horses, for example, zebras or unicorns.
And my three year old daughter is all about unicorns at the moment.
So this ability is obviously very far from what you would expect from the network, especially with the same training samples.
So train samples in that particular example is an equal five or six basically.
So what I'm saying with this little tail is that there is a striking mismatch between theory
and the way we actually use algorithms and the way we understand intelligent systems approach.
And this is very much where my research is sitting for for the past few years and for the upcoming years for sure,
trying to address that gap, trying to better understand how semantic representation can be used to leverage,
learning how we can actually hope to aim for more intelligent algorithms in the sense that it would be much more frugal,
require much less resources and try to leverage more internal representation.
So back to that original claim. Learning is indeed to be able to generalise, but not from scratch.
So if you do consider each learning tasks as a very fresh road, being blind to context, then you are unlikely to be efficient.
So this is a call for incorporating structure and semantic information and what I like to call implicit representations of the world that we humans,
for example, are using constantly without even realising it.
So the hope is that it very much drives a new series of algorithms,
which probably will be on the long run, but hopefully much more frugal in terms of resources.
So that's very much where my efforts on bank base will sit in the sense that we would like to drive this with
my group would like to drive this actually from the theory and try to translate this into working algorithms.
So that brings me to the bottom of my talk.
So, again, a very short primer on what bank base is very much inspired by the tutorial that I gave with John Shortener on the slide.
And this was this was that said two years ago. So let me start with a very simple settings.
And again, if you do have any questions or remarks, please feel free to amuse yourself.
I'm saying this for there's a joint there a bit late. Right.
So that's very simple setting again, bear in mind that pretty much all of what I will be saying also extends to much more complicated, sad things.
For example, I would restrict myself to a supervised case,
but there are a few times for backspace in particular, for unsupervised learning or contrastive learning.
I'll touch briefly on this in the last part. But all right.
So this is implicit saying I have a learning algorithm say let's assume that I have pairs of inputs and outputs.
So, again, superclass connotations. I'm not going to comment on that too much.
The underlying distribution P this is the underlying data generating distribution.
We just assume that we collect a sample of the examples and decide the assumptions.
Of course, extremely central and key in statistics.
But you will say in the second part that we actually have a few attempts and successful contributions to to relax that assumption.
And this is also in the sense of translating this into much more working algorithms and more realistic assumptions.
So let me start briefly with again a very broad overview of statistical learning theory.
So let me take just this example. If you fix an algorithm, A if you fix the function plus H, if you fix also the sample size,
which you could play with is changing random samples and then just looking at the errors.
So you look at the test errors from those samples. In CESCO, learning theory is all about trying to understand the behaviour of that sample.
So rather than focussing on the mean, what makes a lot more sense is to focus on the tail of the distribution of those samples.
So what you want to do ultimately in of in theory is finding bands which hold as high probability statements of a random samples.
If you compare this, just as for test, it's basically the same kind of interpretation.
So if you do assume that you have 99 percent confidence level, say,
the chance of the conclusion of what is classical learning to rebound might tell you not being true, those are less than one percent.
So this is where you should interpret those probabilistic statements.
And this is the celebrated back framework's of Pakistan's for probably approximately correct, can be traced back to the 80s with aliens.
And basically the way it goes is you would typically define a convenience
parameter delta so that you can control the probability of making a huge mistake,
that you can control this by Delta. Then you understand the phrasing, high incidents as if you take the the the the complementary sets of that event,
then the probability of being not in the case of a large mistake.
So approximately correct. That comment on this in the slides, then this probability is actually arbitrarily close to one.
So that's why I like to claim as well that still any theories about controlling what happens to the tester with high confidence?
What can you achieve from that particular assembly so it can do basically two things, you can first learn a predictor.
You can train a machine learning algorithm which delivers a predictor on these two major and then you can certify this performance.
So by learning, I mean that you could train them using any kind of learning principles.
So this covers pretty much everything that you could think of.
You could have regularised methods like Bayesian techniques, networks and so on and so on.
This can be informed or not by applying knowledge. But the crux is you can also certify the performance of that predictor.
And this is typically what is addressed by transition bands, as you will see now in in the minutes.
So those two goals obviously interact with each other.
And this research is pretty much very much involved in the interplay between those two goals and how you can use one to leverage the other.
So no crossing into the actual definition of legislation is.
So, again, this is all very classical. Please feel free to ask you to ask questions or make remarks or maybe you'll go slightly faster on this.
But so, again, very classically, we use a lost function to measure the discrepancy between the actual outputs.
Why and what you do predict for. So with this very simple notation that's evaluated in X, and obviously those are random variables.
So if you want to be able to make comparisons between both, you take the expectation that's what I'm going to call the out of simple risk or are out.
So the expectation of respect to the underlying distribution of data and obviously you can't compute.
That's the results of the empirical counterparts. The in risk transition is answering that question in the middle of a slide.
If you have a predictor, which does well on the path that you've collected so far,
would you still do well on new pairs if you do collect new data points, which is to form reasonably good.
And for that, we use the transition gap, which is simply the difference between the assessment of risk and the simple risk.
Probabilistic statement, as was mentioning before, might be in the form of at the bands like Bets.
So there's also research on lower bands. I'm not going to touch on that in this thought.
But basically, those upper bounds come with a series of flavours of this excellent term is actually the one which is which is of the utmost
interest overseeing those in those those results and the different flavours I'm going to be touching upon are either distribution,
free of distribution, dependence in the sense that it might depend on the underlying distribution of the data,
meaning that you could kick in actually additional assumptions on the underlying distribution dates.
For example, if you assume mundaneness, if you assume the existence of Kaif moments of the data,
if you assume a linear dependence between X and Y with the cushion noise and so on and so on,
those are old model specifications and the band can be made tighter and tighter with those assumptions.
But hopefully the band would also be valid with a minimal set of assumptions, if any.
Really. The other kind of flavour that it comes in are the algorithm for your independent flavours in the sense that
this might be true only for a specific age or it could be true for a very wide range of age functions.
And if this is the case, I think you already see this coming.
That is, you can optimise the band for the specific age that you would be more inclined to use.
So getting back to the pack framework, so in a nutshell, this is a kind of prototypical form of pack of pack pounds, not jet pack based.
So with that probability, with probability of it very close to one, really the transition of an act of this age, which is these blue parts of it here.
So the transition gap, as we had before, is that most disturb this excellent term that we can control.
And hopefully in good cases, we can also compute, not just control.
So you could think of this as a measure of complexity of the problem that you're looking at.
And this typically scales. I see I see someone typing the chat, actually do see magicians.
Could you provide an example of a description free of the independent approaches? Yes, absolutely.
But that's going to come in a few slides. That's the right. All right.
And. Feel free to enrich yourselves if you can.
All right, so those those statements are really eye candy statements on the tail of the distribution of these are out random variable.
All right. Again, keep in mind that this is very similar to what you would achieve.
The statement from a full test at level one man is Delta, so pack based.
So here comes Taipei's is really about patronisation bounce, not just for hypothesis, but rather for Distribution's hypothesis.
So hence the connexion to base statistics that we will see Backpage is actually fairly different from Ferentz.
So let me also try to convince you, if you're not already at this stage, that you should,
as a statistician and as you should care about translation in a sense that transition towns really are a safety check.
The way I see it, it would give you a guarantee for data that you have not yet collected or you might never collect.
But to give you a theoretical guarantee of the performance of one specific algorithm on any unseen data.
And as you will see, the most interesting cases of judaisation bounce are when they provide this control, which is computable.
So if you can actually compute numerically the right hand side of a transition bounce,
then you're in business basically because you would have the tightest kind of control that you could hope for, for that particular algorithm.
They also explained why some algorithm actually work, if you want to, to explain why some of them don't work, you're more interested in lower bounds.
But as far as the bounds they do explain what some of them actually work and were saying.
If you are able to optimise the right side of the bound with respect to the hypothesis or something even more complex,
really, then it can lead to designing new algorithms. I'm going to be commenting on this in a minute.
So at this stage, this is a text message. I very much hope that will convince you that Backpage is a framework which is extremely generic in the
sense that you would say that it's been applied to a lot of frameworks in statistics and machine learning.
The only thing you need to do to apply really based reasoning is to be able to define distributions of hypotheses
and what you've done that you can basically use the Backpage machinery and then try to to derive patterns.
So it really much is a technique to think about what transition means for those good times.
And as I was briefly saying it, it has a Bayesian flavour.
So it leverages the flexibility of Bayesian trends, as you will see, and so you can derive new results from it.
So this is, as I was saying, very much inspired by that material that the shows to say that we we ran the workshop two years before at Napes,
where we in particular, we highlighted and discussed the difference and the differences really between Bayesian inference and fact based learning.
You was he was a panellist, by the way, I see that you're here, are you all right?
And so, as I was mentioning as well, if this is something of interest to you. So we do.
We do. We do have quite a few openings in my group here in London.
But I'd be very happy to elaborate on this. Uttaranchal, the questions maybe so before jumping into the formal definition, a fact based.
Let me just briefly remind you what transition bounce without fact based look like,
hence answering the questions for distribution, free distribution, dependence.
So this is the very simple building block of transition where you have basically just one eye,
but it's obviously not very realistic, but you have to start somewhere.
So this is right. Plus the results.
And so what it says is that the transition gap with high probability will be no more than the log of one of the confidence
with which you states that the inequality divided by the square root of how many points that you have in your training set.
So this tells us one of those square root of M, and this is very simple example.
If you do have a finite function plus and which is again, in a sense the very worst case approach, those bounds are extremely conservative.
I remember discussing this with a colleague recently that those bounds are actually a worst case approach.
They consider that the worst situation that could arise is these rates in practise,
although the right conditions might be even better, performance may be actually even better.
So again, illustrating that mismatch between between theory and algorithms, this is something which has guided me for for as long as I remember.
Really. So so, you know, you you all know those classical results and transition building during the 60s on, for example,
that picture of and then just to mention the complexity and all those techniques actually targeting specific algorithms.
So those are very much algorithm dependent approaches.
And rather than algorithm free, the algorithm free comes more with a pack based approach, as you will see.
So those techniques take some accounts, other correlations between the different hypotheses.
If you have a finite number of hypotheses, obviously you have an infinite function plus or even comfortably.
And they do take some regression, but quite imperfectly, to be honest.
And so this is where Backpage also brings in something new. As I was saying, Backpage considers distributions of hypothesis rather than hypothesis.
And so you have a perfect control on the on the correlations between the different hypothesis.
All right, question, please. So did you just say now that the victim actually ran away and complexity were algorithm dependent?
Though he says, no, no, no, I was saying that those bounds of how this lights are out are.
No, sorry, I was I was sorry I had to choose between distribution and, ah, with independents and those bounds are valid, whatever the algorithm.
Yes, OK, yeah. Sorry, sorry. I'm not quite sure what I said.
I maybe I misspoke. Thanks. Thanks for catching this.
All right, so now coming to what is probably the most thought-Provoking slide of my talk,
at least this over with a slide that ends up being discussed in particular by patients.
That's my attempt at summarising the key differences between Bayesian statistics and fact.
Based on the top of the slides, you get a very summarised interpretation of of what base insurance is.
So by defining a prior distribution and specifying a statistical model.
So typically the likelihood function you get in an unique way up to in the monetisation constant, obviously, but you get in the unique way,
the posterior distribution, which is the object of interest and invasion statistics, and by by unique on this slide.
What I mean is that if you do pick a different player,
then the whole insurance machinery is very likely to be fairly different if you pick a different likelihoods as well.
This is very likely to lead to to much different results.
And as a matter of fact, the likelihood and the prior are chosen very carefully invasion invasion statistics.
And that choice is not independent from each other.
There's a very rich literature on eliciting proper Prior's targeting different statistical models leading to efficient insurance supersaver in base.
There's a considerable shift. So I should just say as a disclaimer that we do use prior into Syria in bank base,
but they have nothing to do really with proper Bayesian priors and proper Pistorius.
So backspaces or some other free approach in the sense that it does not require to have a statistical model.
There is no likelihood function. The only thing that you're using is lost function.
So you could argue that the loss induces a likelihood.
But even if you don't need to to define it analytically, you you can actually never consider this as as proper likelihoods.
So the prior is just a way, just a measure to explore the state of hypotheses.
So this is what I mean by explanation mechanism. It's just a way to navigate within the hypothesis.
It doesn't need to to actually be built with a particular inference scheme in mind is they need to be that
the Pusser is really the twisted prior after confronting with data just as much as invasion's statistics.
But as you will say, there's also a line of work in backspace which considered data dependent Pryors, for example.
So what I'm saying here is that the vocabulary is the same, really, but the object is fairly different.
And also lighting the differences between Beijing fans and back bays.
So, as you will see, the back base bounce will hold whatever the prior, whatever the area and whatever the death attributed to distribution.
And this is a massive difference.
As I was saying earlier, the choice of the prior and the likelihood function, for example, has a massive impact in Beijing status.
So you cannot just pick anything and then and then use the same machinery.
So that's a very salient difference between base and back base, so I'm going to consider it from now on, Pioppi and the ACU,
which only needs to be absolutely continuous with respect to peace as this title here and the rest as before was defined for hypothesising,
you know,
one to consider the risk of a whole distribution where we just basically integrate those previous divisions of the rest with respect to this area.
And if you want to measure the discrepancy, not between just two hypotheses,
but rather between two distributions of business, then you need to resort to divergences.
So most of the literature is holding for the back labeller divergence, as you will see in the minutes.
I should also say that there's a few attempts in the literature to extend this and go beyond AKL,
and I've actually much more general classes of divergences. Here we are.
This is now finally, after half an hour, the first classical palpation band,
so the prototypical bound from from David McAllister, from the series of papers at the end of the 90s,
and the prototypical form of a back based band is holding, as was saying for any prior any confidence level,
the probability for the musician gap to be at most these terms.
They had this big square roots right there where this probability is pretty close to one.
And again, this holds whatever the posterior.
So this for any prior and for any poster is actually extremely important in Backpage, because as you could imagine,
this opens the way to optimising the band for the best posterior distribution in some sense.
Are there any questions so far? And your remarks are criminals excuse me.
This is a bond for the bond that lost Functioning 011.
Yes, absolutely. Absolutely. Thank you.
All right, so it's touching on the whole base actually used to drive new learning algorithm, most bank based bands, we have this shape.
So basically on the left hand side, the arrow and the unscented.
So the arrows quantity and on the right hand side, this empirical error plus complex to determine if you can compute that complex to term,
if it either has an analytical form or you can approximate it reasonably.
Yes, I see a question from the. I mean, what's the definition of a risk of cute?
Oh, sorry, I went a bit fast and that's the risk of Q the rest of A distribution or any decision, for that matter.
It's just the integrated fashion. So the expectation I expect. Q Thank you.
Yeah. If you can if you can actually compute the right hand side of the bounce,
it gives you a principled way to derive new algorithms in the sense that you can just hold the right hand side for.
Q So this gives you say accused of distribution and then you sample new hypothesis from Q Store.
So it's a randomised process, you define the distribution and then you can just sample from the distribution or
look at the expectation of the distribution or modes of the median or whatever.
What it tells you is the optimal distribution to make the bounce the tightest ready.
So by using algorithm sample from that start distribution,
you have the guarantee that the right hand side of the transition plan will be the tightest.
So it's an accumulation problem. There's been a few attempts at trying to solve it over the years,
have been mostly working with multicolour Markov chain techniques and algorithms back in my early days as a student.
There's also a great line of contributions using inference, including lots of people and departments, obviously, and.
Mostly what I would call gradient descent flavoured methods, they come in a variety of of shapes and colours, really,
but there's there's a few techniques out there to try and solve that problem,
or at least approximate one remark, because I think I think it closes very nicely.
The loop in the literature.
There's a few algorithms, very classical, for instance, machine learning, which have all been found to be minimises in based patterns.
And I think it's remarkable because those algorithms actually much more ancient than backspace, which is twenty five years old.
But for example, as VMS or essential waits for that matter, I've found I've been found to be minimises or specific pack based.
And so it gives also a very nice way to justify the use of those algorithms.
And I think I think I'm running a bit low on time, so maybe I'll skip this.
This is a central result, in fact, based on the divergence,
which gives you an explicit formula for the for the distribution, which actually maximises this quantity.
And this is the Gibbs distribution, which is, again, very classical and statistics and mechanical statistics in particular.
So I'll go very briefly on this. This result due to his arms and stronger than in the 70s.
And this actual formulation is due to an getting in one of his funding books, really on bank base and the formalisation of that base.
I'll skip the proof. It's very elementary. But I think I think it's quite an elegant profile.
It's like trying to showcase. It's also because it fits on one slice with another case of that many proofs and papers and the trade.
But just to comment on the last line, so you get at the top the statements of that lemer,
if you pick that five function to be some temperature, time is the empirical risk.
And you write actually, you rewrite actually that formula.
You get that this distribution cue lambda, which is the expansion of minus lambda at times ten times the prior.
So basically a GIPS potential gets distribution. Right, so that this distribution is the end of this problem, this constraint problem.
And this seems very reasonable. Right.
Because what you're doing here is minimising the expectation of the empirical risk with respect to a distribution.
Q Plus a discrepancy between Q and P with some temperature parameter.
But the discrepancy with the cap, this is very much resembles what we're doing in regularisation.
In statistics, for example, LSU is not a different interpretation.
It's exactly the kind of of of criterion that you're optimising.
It fits the data plus a regularisation.
So so again, in a way, this discussed back base into into a series of of well-known techniques, well-known results and literature.
And I think this is an interpretation that is quite nice.
All right, so that closes the first part, that's a small primer on what backbeats is.
Again, the message is that backspace is a way to understand generalisation.
It's it's a way to deal with distribution's of a protector's rather than just protector's.
And in as a discuss source, go learning theories all about this high confidence control of the transition.
And what that allows us to do is really to extend this to a series of frameworks.
There's this really old Tsou of frameworks in which factis has been used and has delivered transition balance,
which often are state of the art transition. It can also lead to new algorithms by minimising right and side of of generalisation bounds.
You might think that this would lead to classical posteriors and classical algorithms is not always the case.
So then there's a Trade-Off between what this actually gives you and how achievable and how computable those objects are.
So again, the Trade-Off between theory and algorithms. Right.
So this couples, as I was saying, a lot of tools from statistics, optimisation, probability theory,
and so then we can back base for 10 years now since since I finished publicity.
And that was much less popular back then than it is now.
And let's say I think one of the explanations for the surge of interest in bank base is that, to my knowledge, is one of the very few,
if not the only one way to address transaction for networks, at least for some architectures of networks,
hence the very strong spotlights that Backpage has received.
All right, what's coming next in the remainder of my talk is what we've been up to for bank base,
maybe now is a good time to take a very short break and ask if there are any questions or comments.
I have a question. So can you comment on I'm not very familiar with this formalism,
but how does how does having a random procedure for breaking your hypothesis relate to practise?
Because it seems that the framework.
Well, you know, you have some kind of search procedure or selection procedure for picking one thing exactly what we do in practise.
Absolutely, that is an excellent point, and this is what I was alluding to when I was talking about the gap between theory and practise,
so there's a lot of the results which actually hold for realisation that the stimulus we proposed,
the randomised balance in the sense that you have that posterior and then you sample from it
and then you have one specific hypothesis for which you want to assess the transition ability.
It doesn't seem very sensible in practise and know anyone who really does that to to solve a particular problem.
So there's also a whole series of bands holding for the expectation with so.
So in that sense, the final muster, the final predictor is not random is the expectation of that distribution, which is called the posterior.
Does that make sense? Yeah, OK, thank you.
And this poses but that that's that's also challenging in some settings because this
might be a distribution which is not so easy to sample from or even to use analytically.
It's a good beating. The mean of the distribution might also be a constitutional challenge.
But I'll touch on this briefly in the following slides. OK, can I have a very quick follow up, please?
Taking a practical example. So you have a neural network and you like use random initialisation and then you train with gradient descent.
What is the plan? Is it the push forward of the initialisation by gradient descent?
So what's the problem? It could be, yeah, it could be that absolutely.
It could be pretty much. It could be anything really, because the back base band will hold water for the prior.
So what you want to do ultimately is take the player which would make the band the tightest.
All right. OK.
All right, thanks. Well, I have a similar question if you have time.
Yeah. So what's the. So, yeah, I'm not quite sure how we do on time anyways, but I'm happy to.
So what's the meaning of this prior distribution in real life.
So this is the most confusing part of baseball's one.
Since we can choose it like anything, pretty much. Yeah, so, OK, let me ask a question by another question.
If I go back to this, for example, some the very classical based bands, this is this is a probabilistic statement.
But if this term over here in the square root.
So let's assume that I mean, a binary classification problem, this is actually the case for this kind of bounce.
So this is a number between zero and one. This is a tester on the binary classification problem.
This is also a number between zero and one. And this might be something that blows up.
So if you pick prior, which is very far from the stereo, for example, the sense then this will blow up and then your band will be with probability.
Close to one digitisation gap is abandoned by 10 to the 30 or something.
This is true. These are probabilistic statement, which is true, but it's absolutely useless.
So then we say that the band is vacuous in some sense.
So the past few works on back base have very much been in the direction of non vacuous
bands trying to ensure that we have something which is ultimately meaningful.
And so the role of the player in that sense is to try and make this complex term as small
as possible so it could be informed by actual prior knowledge as invasion statistics.
It can be just the way you initialise, for example, you or your grid, in a sense, for the way of your network on the push forward phase.
It could be pretty much any anything like that. The band will be valid regardless.
So what's interesting is try and optimise for that prior when you want to make the band tighter and tighter.
I have this answers your question. OK, I think I'm still confused,
but it's better when we translate this into please tell me to stop if you want to tell me when we translate this into practise.
Suppose we're trying to analyse a situation where somebody does a specific training procedure in order to get down to that procedure.
Do we try to find the prior that most represents what they do?
Because it seems like what you've given here is a method to finding a way of a good and a guarantee if you the in such a way.
But then you're going to use this to talk about other things that people do in
the real world where the algorithms are not necessarily derived in this way.
So how do you match the. So that's also a very good point, but I should maybe take a step back and highlight that even though we call this prior,
it doesn't need to be an actual prior in the sense. What I'm saying is because, for example, if you have a classified practises which are, you know,
different on networks and what you want to do is he's learnt the proper weight of that network, it might correspond to prior knowledge.
But what if you had I don't know, you have a finite collection of predictors.
You have one your network, you have one decision tree, one random, first one, whatever.
Then the players just possibly led by, I don't know, the cost of using any of those experts.
Right. So it doesn't correspond to a specific knowledge of your problem.
It's rather driven by, for example, competition constraints could be something like that.
So I think what is always quite confusing and buy, in fact, is that we do call this a fire,
even though it might be an object which is very different from from prior invasion statistics.
And I don't know if that answers your question. I'm just wondering how this bears on on practical situations.
So how do you get how do you put this in a form? So if you take the standard Radovic about right now, like just now we apply it to neural network,
will you calculate the rate of complexity of the neural network and then you put it in?
And so I'm saying we have some algorithms to understand the world and we have to pick the P to correspond to what
they're doing in order for this statement on the screen to give a generalisation guarantee for that procedure.
So the answer the short answer is no, you don't need to do the same.
But if you want this to be a fair comparison with the complexity based bounds, then it's probably a good idea.
That's my sense. OK, thank you. Well, thanks.
All right, sir, I'm going to be touching, actually, very briefly on some of the recent contributions,
because I think I'm running out of time, I'm afraid, but I'll try to wrap up quickly.
So we have been extremely fortunate to work with all those fantastic people over the past few years.
And so I probably just pick one because I want to cover all of those, obviously.
So I probably just briefly touch on on the third one.
Actually, this paper is economising journalise, which is which is a back page in an attempt at understanding binary activity in your network.
So just to briefly also give some context on the other work.
So you had contributions from a theatrical perspective trying to to obtain PANKRATION bands with different divergences, for example.
So this is the first paper with Clarkie, who's with lican in Japan,
where we have PANKRATION bands holding for what we call it as either Heifetz's or Dependent's or relaxing that idea.
Sencion, which obviously is extremely useful in statistics, but not very realistic in the real world, obviously.
And there's also a line of work towards difference between bands either with faster rates.
This is this paper with the comedian Peter Kornfeld and also different measures of the loss.
So, for example, have been touching on the fact that the risk is mostly the expectation of a loss.
But again, this might not correspond to situations where you want to optimise the different quantity for in medical
drugs testing rather than the expectation you'd be much more interested in the quintiles of that loss,
because you want to you want to avoid at all costs costly errors, all leading to death of of patients.
So the expectation just taking the average of a distribution might not be a good idea in those respects.
So we have that paper, at least in Europe, on Backpage for the conditional value at risk in particular.
So different ways to measure the performance. We have all sorts of contributions for structures, predictions.
So echoing what I were saying earlier about the nature and corporate structure and semantic content into learning,
and also this fact based contrastive technique, which I visited last July with German companies, are also from Japan.
So what I'm highlighting here is the range of contributions that are accessible through a bank base.
And again, the only thing to use the bank base machinery is to be able to define a probability distribution, a measure on the space of it.
If you can do that, then it can adapt the bank based framework straightforwardly. All right.
So just take in the three, four minutes. Just give a very brief overview of that one.
So let me just give all of that. Yeah.
So Dippenaar, activity in your network is just a very classical forewords network, fully connected.
The only thing is that the activation function is the same function.
So at each node you output either one or minus one.
This is a challenge because it's obviously non differentiable.
So the way we address this with backspace is with a small trache, which is as in bank base, you're mostly looking at the expectation in many papers.
You're mostly looking at the expectation with respect to the prior. This turns out to actually have a closed form.
So the expectation of the output of your network with a binary activation is actually the arrow function.
And that was the beginning.
So this is a fact pointed out by an earlier paper from Gillette and Jazmyne that nine and this little fact here is that the core of that contribution,
because it allows us to both write back Bayesian generalisation band for that specific architecture, we use also a few tricks in the balance.
So, for example, the divergence term has a closed form if you assume that prior and wistaria with parameters.
So you basically demands to optimising for those parameters to try and control the discrepancy between the prior and the posterior.
This is the very simple case of just one layer is no hidden layer really on your network.
But this is just a teaser. This really is the building block in that paper.
And ultimately we were able to derive a full strategy to ride the gradient descent training of the network and the transition date,
which is right there. So that's the transition.
But so it's a very. Sorry, the questions in the chat.
What is Q W so cute w is the posterior on the weights W as the posterior distribution's.
And so the transition bounce that we have for that specific sector of your network is very much inspired by a band from L.A. and it basically
tells you that the inseparate at first approximation of the insipid risk of the network is far from the outcome of risk by a margin,
which is the care divided by and in that context.
So this really much is consistent with the remaining of the theory.
What was interesting in that is that we're able to derive the actual value of that band in a series of settings.
Let me just briefly zoom in, because I want to leave some time for questions.
I realise we're closing in on the hour. So this is a sample of the few American experiments that we did with that particular architecture.
And so if you look at this column, for example, in me, zoomIn, so this is four variations of Enis as a baseline.
So this is the Back Bays inspired network with the interactive activation function where you
get on the first column is the empirical research on the different instances of data sets.
This is a test server and this is a value of the bands.
And when I was saying earlier that we very much the transition bands are the third full guarantee, which translates into working to for practitioners.
This is very much what we hope to achieve as a practitioner.
If you enter the American error achieved by someone, give them some data set what Watpac Base gives you in return through,
of course, a lot of different steps that would not elaborate too much on.
But what it gives you ultimately is the value of the balance, which is an upper bound with probability, with a confidence level.
So in this case, it's 99 percent. So with 99 percent on this particular problem, I compute the empirical error, which is one point eight percent.
And backspace based theory tells you, all right, if you collect any new instances from that data set, the error will never be more than three percent.
And this occurs with probability 99 percent. So and that is also a very recent paper from my good colleagues and friends at UCLA,
Omar, John and Maria, talking about certificates for machine learning.
And this is very much is the idea of those kind of statements. Judith is asking, what was Delta, so Delta is the temperature.
I'm guessing you were asking, oh, no, sorry you're asking, but this Delta, right,
so the confidence barometer and the confidence barometer is the the probability with which you state the evidence.
So there is something morally very close to zero, the closer to zero it is, and the more likely this is to blow up, obviously.
So this strain of. All right, I think I think I'll wrap up here just to give you a very an attempt at summarising these different lines of research.
So as you understood, the quest was very much for transition guarantees, about half of the contributions from my group with bikeway.
So this is the tip of the iceberg that you see now and in the few directions that have been extremely interested in over the past few years are
providing either generic bounce in the sense that it would be holding under a less restricted set of activities such as I.T. or something like that,
and also tighter bans for specific algorithms. So if you pick a specific algorithm, it's reasonable to hope that you will have tighter rates,
for example, of tighter constants in the balance for that particular algorithm.
Also briefly mentioned the new measures of performance, such as conditions, belayer, trist or contrastive losses.
And again, the interplay between coupling of theory and the algorithm is very much at the centre of all of our team.
Last but not least, the applications that we foresee, that is very much towards.
What I like to call sustainable machine learning is sense that if you have guarantee that with that many data points,
the best you could achieve as a test error or an estimate of the test error with probability 99 percent, for example, is this then?
It is, I think, very valuable guidelines to measure learning practitioners and users of aquifer's.
So I think I'll close up here. Thank you very much. Right on time.
It's four thirty and yeah, I'd welcome any questions if you have any more.
Thank you very much. Stand by for the very nice talk.
So we have a little time for some more quick questions from the audience.
Have a seat.
So you were at the beginning, you were explaining, sort of just justifying the dinner as a strange man to make a link with a double story.
Yeah. Do you pack these bones, give some insight on the double descent or not?
So this is very much the hope in an ongoing work, which is not yet an occasion, I'm afraid.
But so what what we're trying to assess is if you do augments the complexity of the class.
So if you do add layers and layers from your network, do you foresee at some point a decrease in the right answer?
This is very much what we what we're working on. And the answer is yes. But I'm afraid I don't have any more material to share this point.
I see there is a question about. And no, I don't know the question.
You had your hands raised, so it was collapsing, I think.
Thank you. OK. I think we already run out of time.
So let's end this seminar. So thank you, everyone, for coming in.
So let's thank the speaker again. Thanks, everyone.
And I should also mention that the slides will be on my Web page very shortly,
in a few minutes actually, and feel absolutely free to reach out if you have questions or comments.