This lecture is part of a course on "Probability Estimation Theory, and Random Signals". Welcome back to the lecture slide set on multiple random variables. In the last lecture, we looked at the correlation matrix as a key descriptor for multivariate distributions or multiple random variables, or random vectors. And it was defined as the expectation of the outer product of a random vector with itself, which gave combinations of the second moments of each of the individual random variables, but also expectations of products of the random variables which gave a measure for the statistical similarity. In the last video, we also looked at calculating the correlation matrix for a given joint probability density function. And we did that for the example of a uniform distribution over a triangular region. There, we found that the correlation matrix looks like this. In today's video, rather than being given a joint probability density function, we suppose that the correlation matrix has been estimated, for example, from some data and we're being given a correlation matrix. It turns out that in many algorithms, it's crucial for that correlation matrix to satisfy certain properties. If a correlation matrix doesn't satisfy that property, then it could potentially lead to the algorithm breaking down late to point. There are two key properties to consider. The first is that the correlation matrix is equivalent to itself Hermitian; in other words, a correlation matrix and its complex conjugate transpose, are equal. You can see this through the definition of the cross terms and this is very straightforward to prove. For more information, you can have a look at the handout. Perhaps a more important property is that correlation matrices of positive semi-definite. What that means is for any complex vector a, because remember, we're in general considering complex random variables just to keep the theory as general as we possibly can. But for any complex vector a, which we choose, then "a" Hermitian times the correlation matrix times the complex vector "a" has always a positive value. So just to explain that a little bit more clearly, imagine that we have a vector a and it's got coefficients a1 through to an. Then that means that if we take the Hermitian, we take the complex conjugates and I'm going to use bar to denote complex conjugates. And we're going to multiply that by  the correlation matrix, which was R x1 x1 up to R x1 xn, and then R xn  x1 through to R xn xn. That's what we had before. And we fill those in with some dots. And then repays multiplied by the vector a. I must be greater than or equal to 0. So we have a emission times a times a and that must satisfy being greater than or equal to x2, then it's worth noting, just so that you can check that this is indeed a scalar because when you go back to the MHC is multiplying each of it's worth checking. So a is a complex space and is of size n by one. So a Hermitian is one by N, was an m by n matrix. Remember, n is the number of elements in a random vector. A is n by one and you can do domino pattern by issues with checking, but vectors, matrices multiply out quickly. So indeed you can see this is one by one. If you'll matrix does not satisfy this property, then it is not a valid correlation matrix and it's a simple designs. Now, you might wonder why this was all comes along and it's quite interesting really. There are two ways to verify this result. The first is to start off with a definition. So we start off with the end result and we notice that our x by its very definition, is the expectation of X times X commission. So we substituted that into there. But using the linearity properties of expectation, we can rewrite this as being the expected value of a Hermitian times x times x emission times a. And what you'll notice is if I cool this term here, B for example, then this term here is B Hermitian. And then the product to base two is B Hermitian times b, which is the same as taking this new vector b and taking its magnitude and squaring it, because that's the definition. I'll be in a product now, since this term is now a magnitude than the magnitude of anything is always greater than or equal to 0. So we instantly see that lacked must be positive. So that's one approach where we kind of started off with the final result. A similar approach would be deriving the same result is to consider a linear combination of a random variables, any linear combination you want it to. And then like just on a previous slide, we can write this as a coefficient that a times x. Now what we can do is to consider the second moment of y. So we know that the expected value of Y squared must be greater than or equal to 0. So Equivalently, since the variance of y must by definition be positive Ben so must hit second moment. Say by calculating the second moment, as I wrote before, we can come up with this result, which is effectively the same as we had on the previous page, but it's a sort of reverse way of doing it. So we're basically proving the same result coming from two different directions. In this particular proof, I think I've kept it simpler by considering a real vector a. I'm baffled, transpires, but can easily be generalized elsewhere. So let's now look at a couple of examples of determining whether the following matrices are in fact valid correlation matrices. His fs two example. So have a look at that matrix. Decide whether you think it's valid or not. You might want to pause the video while you have a little bit. If I think of when you are ready, please come back to the video. Okay, so welcome back. So I hope you appreciate, but the two properties of matrix must satisfy in order to be valid. Or correlation matrix is the matrix must equal itself, compacts, conjugated, transposed or Hermitian, as it's cool. I'm a second is that for any vector a, it must be positive semi-definite. There are probably some other elements. So we should also consider, which I hope. So elementary. But the first is that the leading diagonal component should always be positive. So if that's because add variances of each of the individual random variables and they must always be positive. So with that in mind, let's have a look at this matrix while we noticed, but it's not Hermitian. And therefore it's not valid. So this is not a valid correlation matrix, is not symmetric, which is a requirement of a valid autocorrelation matrix. Ok, that's fine. Let's take an example. So here's another example. Take some time, have a look at this. Pause the video, try and work for it. Is this matrix a valid correlation matrix or not? Okay, so welcome back. So eight satisfies a couple of tests I had on a previous slide. So it satisfies the fact that x is all that transpose. I'm using transpose here because it's real. We also see that the leading diagonal components here are positive in value. So now we just need to determine, is it positive semi-definite? Now because the elements of the matrix are real, but it's actually only necessary to test this equation for real a. There is a proof of this in the handout. But basically, if you elements of a matrix, a wheel, but when you come to test positive semi-definite property, you only really need to pick a real coefficient vector a. Okay? So how do we do that? Well, what we do is actually literally pick a generic value of a. So for example, a equal to alpha and beta. And show that this must be positive for all values of alpha and beta if it's valid. Or somehow find values of alpha and beta, which make this expression Lesson 0, in which case and venue. No, it's not valid. Let me show you how you do that. So let's wanted to product i is a transpose, I'll X a, and let's define a as alpha and beta. So then we have a transpose times a. And only then today is to be multiplying out these matrices vectors. So let's multiply and AFS Ligua alpha plus two beta to alpha plus beta, and we still have to multiply by a transpose. So this becomes alpha times alpha plus two beta plus beta times two alpha plus beta. And you can combine some terms and you will get a quadratic in alpha and beta. You have a question is, is that quadratic always positive, all candidate take on some negative values. Well, how do you show that? We could guess the pixel values of alpha and beta, and you could plug them in and, and test it. So in fact, I hope you can see, but if I choose alpha equal to, for example, minus one and beta equal to one, then you see if a i is equal to one minus four plus one, which is minus two, which is less than 0. And therefore this is not positive semi-definite and therefore is not valid correlation matrix. Okay, so in that case, I happen to guess that's absolutely fine. But in other situations it might not be so obvious. So how would you then check for it? Well, let me show you one systematic approach. One approach is you try and complete the square. So I see that this is almost quadratic, but not an exact quadratic form. So why did is take the first three terms, make it a quadratic and subtract any term of how to add in order to make it quadratic. So what I mean by that is I could write this as a perfect square by writing alpha square plus two alpha beta plus beta squared. But noticing that when ought to get bouncer had before and even over two alpha beta. There is an alternative way of doing that. And I show you this in a moment. You can see that if you complete square Hague alpha plus beta squared, Now the thing is that's always positive. So now we don't need to worry about that term. We just need to look at the second term. Now the second term is not always positive, so it's a easy. So more generically, select alpha o equal to minus beta and knife or you can shave a i is less than 0. Let me just show you an alternative way. So on a previous slide, we had f squared plus four alpha beta plus beta squared. Let me space if I took the first two terms. If I wanted to make a perfect square, I would need to add on to be two squared. And that's because if you were to factorize this term as alpha plus two beta o squared, you can see that this multiplies as alpha squared plus two times alpha times two b two, which is full alpha-beta plus 2B to o squared. So I would need to subtract two V two squared because I had too many terms on plus I will need to add on innovation or bt squared. So you can assay the i can be written as plus two b two squared minus three v two squared. And so this is always positive, but you could quite easily see. But because of this negative term here, this minus vB squared is not always positive, and therefore it's very easy to see if I can become lessons. Now this second veteran approach is probably much better to do in practice by taking the first two terms, as we've got here, take these first two terms, complete, square, subtract the turn that you don't want, and then combine the last two terms as I've done here. But plenty of ever examples in the handout to mom show where do some more envy hybrid seminars. Now just one more example before we finish. In the previous video, we demonstrated fat, this uniform PDF over this upper triangular region had a correlation matrix which was given by this expression here. Now we know that this should be a valley correlation matrix because it came from and probability density function. Why did we quickly test fat? So we do exactly what we did in the previous example. We set a equal to half a pizza and I'm going to create this term I, which equals a half a visa. A transpose times 16, one-quarter, one-quarter, 1.5 times how from beta. And I'll just multiply that out. So I have got alpha over six plus beta four and alpha over four plus b two over two. And I multiply that out again. And you'll see that we've got alpha squared over six plus alpha v2 over four, plus another alpha-beta over four plus bt squared over two. And we can combine the middle terms, and it's probably easy to take the factor of 106 out to the front. So this will be an alpha beta over two. So I just need a free alpha beta in the numerator effectively. And same here, plus three beta squared. So as we said in the previous example, I'm going to take the first two terms and I am going to effectively make a perfect square. So if I took return alpha plus beta and I were to expand out our alpha squared plus three alpha beta two times two plus three beta two all squared. So I need to subtract that terms. I need to subtract minus p2 squared over four. I've still got the free bt squared term I had before. This is always positive. And a second term, which I am going to rewrite as being beat two squared times three minus nine over four. Well, three minus nine over four is three quarters. And therefore the second term in red is also always greater than 0. And therefore we've proved a i is always greater than 0, is also symmetric. And our second moment terms are also positive. So this is indeed a valid correlation matrix and Babur sort of things that you would like to test. So in today's video, we've looked at properties of correlation matrices. In particular, we've looked at with positive 70 definite property of correlation matrix or they must satisfy free properties. In particular, which refocus dance a positive semi-definite NAS, the leading diagonal term survey. So the second moment must be positive. Boss I've matrix must be symmetric. Now there are some other examples in the tutorial questions and you're strongly encouraged to go work for some advice. And in due course there'll be some self-assessment questions testing your knowledge of valid correlation matrices. In the meantime. Thank you very much.