Numerical Stability Sample Clauses

Numerical Stability. ‌ In this section, we will use insights into numerical stability to improve the stabil- ity of the inference for GPs. Numerical stability is an important issue and has to be accounted for when implementing a framework to be used by a wide range of users. There is often a trade-off between generality of implementation and the nu- ▇▇▇▇▇▇▇ stability of the application itself. In GPy, we chose to split the algorithm at a convenient point, where implementation of new algorithms and ideas is promoted, while keeping numerical improvements and implementation freedom as clear cut as possible. = ( )ij i⋅ j⋅ ( | ) The inference of GPs relies on the inverse of the covariance matrix ▇ ▇ X , X of the covariance function k of the prior p F X . In a general framework implemen- tation of GPs, we need to make sure, the inference is numerically stable for a wide range of covariance matrices. Numerical stability in algebra is a wide field and we will only explain specific parts necessary for the explanation of the implementa- tion shown in this thesis. See Higham [36] for a detailed description of numerical stability and possible solutions. One aspect of the implementation in which the numerical stability can provide erroneous results is the inverse of the covariance matrix, and determinant of a covariance matrix. For both of these tasks, numerical stability can be improved by using the Cholesky decomposition = A LLt , where L is a lower triangular matrix with positive diagonal entries. The Cholesky decomposition requires matrix A to be positive semi-definite [36, Chapter 10], which is also the requirement for kernel functions [70]. | | = | | = We take the log-likelihood of a GP and turn it into a numerically stable equation for use in the actual implementation in a programming language. Now we can have a closer look at the summands. The first summand log 2π can be precomputed as a constant. The second summand can be stabilized numerically by the Cholesky factorization LLt K. log K log LLt log(|L| ⋅ |L|) = 2 log(|L|) i=1 = 2 log (G Lii) L is triangular = Σ ii = 2 log L . t –GP i 1 S