<html lang="en"> <head> <title>Linear Least Squares - GNU Octave</title> <meta http-equiv="Content-Type" content="text/html"> <meta name="description" content="GNU Octave"> <meta name="generator" content="makeinfo 4.13"> <link title="Top" rel="start" href="index.html#Top"> <link rel="up" href="Optimization.html#Optimization" title="Optimization"> <link rel="prev" href="Nonlinear-Programming.html#Nonlinear-Programming" title="Nonlinear Programming"> <link href="http://www.gnu.org/software/texinfo/" rel="generator-home" title="Texinfo Homepage"> <meta http-equiv="Content-Style-Type" content="text/css"> <style type="text/css"><!-- pre.display { font-family:inherit } pre.format { font-family:inherit } pre.smalldisplay { font-family:inherit; font-size:smaller } pre.smallformat { font-family:inherit; font-size:smaller } pre.smallexample { font-size:smaller } pre.smalllisp { font-size:smaller } span.sc { font-variant:small-caps } span.roman { font-family:serif; font-weight:normal; } span.sansserif { font-family:sans-serif; font-weight:normal; } --></style> </head> <body> <div class="node"> <a name="Linear-Least-Squares"></a> <p> Previous: <a rel="previous" accesskey="p" href="Nonlinear-Programming.html#Nonlinear-Programming">Nonlinear Programming</a>, Up: <a rel="up" accesskey="u" href="Optimization.html#Optimization">Optimization</a> <hr> </div> <h3 class="section">25.4 Linear Least Squares</h3> <p>Octave also supports linear least squares minimization. That is, Octave can find the parameter b such that the model y = x*b fits data (x,y) as well as possible, assuming zero-mean Gaussian noise. If the noise is assumed to be isotropic the problem can be solved using the ‘<samp><span class="samp">\</span></samp>’ or ‘<samp><span class="samp">/</span></samp>’ operators, or the <code>ols</code> function. In the general case where the noise is assumed to be anisotropic the <code>gls</code> is needed. <!-- ols scripts/statistics/base/ols.m --> <p><a name="doc_002dols"></a> <div class="defun"> — Function File: [<var>beta</var>, <var>sigma</var>, <var>r</var>] = <b>ols</b> (<var>y, x</var>)<var><a name="index-ols-2392"></a></var><br> <blockquote><p>Ordinary least squares estimation for the multivariate model y = x*b + e<!-- /@w --> with mean (e) = 0 and cov (vec (e)) = kron (s, I). where y is a t by p matrix, x is a t by k matrix, b is a k by p matrix, and e is a t by p matrix. <p>Each row of <var>y</var> and <var>x</var> is an observation and each column a variable. <p>The return values <var>beta</var>, <var>sigma</var>, and <var>r</var> are defined as follows. <dl> <dt><var>beta</var><dd>The OLS estimator for b. <var>beta</var> is calculated directly via <code>inv (x'*x) * x' * y</code> if the matrix <code>x'*x</code> is of full rank. Otherwise, <var>beta</var><code> = pinv (</code><var>x</var><code>) * </code><var>y</var> where <code>pinv (</code><var>x</var><code>)</code> denotes the pseudoinverse of <var>x</var>. <br><dt><var>sigma</var><dd>The OLS estimator for the matrix <var>s</var>, <pre class="example"> <var>sigma</var> = (<var>y</var>-<var>x</var>*<var>beta</var>)' * (<var>y</var>-<var>x</var>*<var>beta</var>) / (<var>t</var>-rank(<var>x</var>)) </pre> <br><dt><var>r</var><dd>The matrix of OLS residuals, <var>r</var><code> = </code><var>y</var><code> - </code><var>x</var><code>*</code><var>beta</var>. </dl> <!-- Texinfo @sp should work but in practice produces ugly results for HTML. --> <!-- A simple blank line produces the correct behavior. --> <!-- @sp 1 --> <p class="noindent"><strong>See also:</strong> <a href="doc_002dgls.html#doc_002dgls">gls</a>, <a href="doc_002dpinv.html#doc_002dpinv">pinv</a>. </p></blockquote></div> <!-- gls scripts/statistics/base/gls.m --> <p><a name="doc_002dgls"></a> <div class="defun"> — Function File: [<var>beta</var>, <var>v</var>, <var>r</var>] = <b>gls</b> (<var>y, x, o</var>)<var><a name="index-gls-2393"></a></var><br> <blockquote><p>Generalized least squares estimation for the multivariate model y = x*b + e<!-- /@w --> with mean (e) = 0 and cov (vec (e)) = (s^2) o, where y is a t by p matrix, x is a t by k matrix, b is a k by p matrix, e is a t by p matrix, and o is a t*p by t*p matrix. <p class="noindent">Each row of <var>y</var> and <var>x</var> is an observation and each column a variable. The return values <var>beta</var>, <var>v</var>, and <var>r</var> are defined as follows. <dl> <dt><var>beta</var><dd>The GLS estimator for b. <br><dt><var>v</var><dd>The GLS estimator for s^2. <br><dt><var>r</var><dd>The matrix of GLS residuals, r = y - x*beta. </dl> <!-- Texinfo @sp should work but in practice produces ugly results for HTML. --> <!-- A simple blank line produces the correct behavior. --> <!-- @sp 1 --> <p class="noindent"><strong>See also:</strong> <a href="doc_002dols.html#doc_002dols">ols</a>. </p></blockquote></div> <!-- lsqnonneg scripts/optimization/lsqnonneg.m --> <p><a name="doc_002dlsqnonneg"></a> <div class="defun"> — Function File: <var>x</var> = <b>lsqnonneg</b> (<var>c, d</var>)<var><a name="index-lsqnonneg-2394"></a></var><br> — Function File: <var>x</var> = <b>lsqnonneg</b> (<var>c, d, x0</var>)<var><a name="index-lsqnonneg-2395"></a></var><br> — Function File: [<var>x</var>, <var>resnorm</var>] = <b>lsqnonneg</b> (<var><small class="dots">...</small></var>)<var><a name="index-lsqnonneg-2396"></a></var><br> — Function File: [<var>x</var>, <var>resnorm</var>, <var>residual</var>] = <b>lsqnonneg</b> (<var><small class="dots">...</small></var>)<var><a name="index-lsqnonneg-2397"></a></var><br> — Function File: [<var>x</var>, <var>resnorm</var>, <var>residual</var>, <var>exitflag</var>] = <b>lsqnonneg</b> (<var><small class="dots">...</small></var>)<var><a name="index-lsqnonneg-2398"></a></var><br> — Function File: [<var>x</var>, <var>resnorm</var>, <var>residual</var>, <var>exitflag</var>, <var>output</var>] = <b>lsqnonneg</b> (<var><small class="dots">...</small></var>)<var><a name="index-lsqnonneg-2399"></a></var><br> — Function File: [<var>x</var>, <var>resnorm</var>, <var>residual</var>, <var>exitflag</var>, <var>output</var>, <var>lambda</var>] = <b>lsqnonneg</b> (<var><small class="dots">...</small></var>)<var><a name="index-lsqnonneg-2400"></a></var><br> <blockquote><p>Minimize <code>norm (</code><var>c</var><code>*</code><var>x</var><code> - d)</code> subject to <var>x</var><code> >= 0</code>. <var>c</var> and <var>d</var> must be real. <var>x0</var> is an optional initial guess for <var>x</var>. <p>Outputs: <ul> <li>resnorm <p>The squared 2-norm of the residual: norm(<var>c</var>*<var>x</var>-<var>d</var>)^2 <li>residual <p>The residual: <var>d</var>-<var>c</var>*<var>x</var> <li>exitflag <p>An indicator of convergence. 0 indicates that the iteration count was exceeded, and therefore convergence was not reached; >0 indicates that the algorithm converged. (The algorithm is stable and will converge given enough iterations.) <li>output <p>A structure with two fields: <ul> <li>"algorithm": The algorithm used ("nnls") <li>"iterations": The number of iterations taken. </ul> <li>lambda <p>Not implemented. </ul> <!-- Texinfo @sp should work but in practice produces ugly results for HTML. --> <!-- A simple blank line produces the correct behavior. --> <!-- @sp 1 --> <p class="noindent"><strong>See also:</strong> <a href="doc_002doptimset.html#doc_002doptimset">optimset</a>, <a href="doc_002dpqpnonneg.html#doc_002dpqpnonneg">pqpnonneg</a>. </p></blockquote></div> <!-- optimset scripts/optimization/optimset.m --> <p><a name="doc_002doptimset"></a> <div class="defun"> — Function File: <b>optimset</b> ()<var><a name="index-optimset-2401"></a></var><br> — Function File: <b>optimset</b> (<var>par, val, <small class="dots">...</small></var>)<var><a name="index-optimset-2402"></a></var><br> — Function File: <b>optimset</b> (<var>old, par, val, <small class="dots">...</small></var>)<var><a name="index-optimset-2403"></a></var><br> — Function File: <b>optimset</b> (<var>old, new</var>)<var><a name="index-optimset-2404"></a></var><br> <blockquote><p>Create options struct for optimization functions. <p>Valid parameters are: <ul> <li>AutoScaling <li>ComplexEqn <li>FinDiffType <li>FunValCheck When enabled, display an error if the objective function returns a complex value or NaN. Must be set to "on" or "off" [default]. <li>GradObj When set to "on", the function to be minimized must return a second argument which is the gradient, or first derivative, of the function at the point <var>x</var>. If set to "off" [default], the gradient is computed via finite differences. <li>Jacobian When set to "on", the function to be minimized must return a second argument which is the Jacobian, or first derivative, of the function at the point <var>x</var>. If set to "off" [default], the Jacobian is computed via finite differences. <li>MaxFunEvals Maximum number of function evaluations before optimization stops. Must be a positive integer. <li>MaxIter Maximum number of algorithm iterations before optimization stops. Must be a positive integer. <li>OutputFcn A user-defined function executed once per algorithm iteration. <li>TolFun Termination criterion for the function output. If the difference in the calculated objective function between one algorithm iteration and the next is less than <code>TolFun</code> the optimization stops. Must be a positive scalar. <li>TolX Termination criterion for the function input. If the difference in <var>x</var>, the current search point, between one algorithm iteration and the next is less than <code>TolX</code> the optimization stops. Must be a positive scalar. <li>TypicalX <li>Updating </ul> </p></blockquote></div> <!-- optimget scripts/optimization/optimget.m --> <p><a name="doc_002doptimget"></a> <div class="defun"> — Function File: <b>optimget</b> (<var>options, parname</var>)<var><a name="index-optimget-2405"></a></var><br> — Function File: <b>optimget</b> (<var>options, parname, default</var>)<var><a name="index-optimget-2406"></a></var><br> <blockquote><p>Return a specific option from a structure created by <code>optimset</code>. If <var>parname</var> is not a field of the <var>options</var> structure, return <var>default</var> if supplied, otherwise return an empty matrix. </p></blockquote></div> <!-- DO NOT EDIT! Generated automatically by munge-texi.pl. --> <!-- Copyright (C) 1996-2012 John W. Eaton --> <!-- This file is part of Octave. --> <!-- Octave is free software; you can redistribute it and/or modify it --> <!-- under the terms of the GNU General Public License as published by the --> <!-- Free Software Foundation; either version 3 of the License, or (at --> <!-- your option) any later version. --> <!-- Octave is distributed in the hope that it will be useful, but WITHOUT --> <!-- ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or --> <!-- FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License --> <!-- for more details. --> <!-- You should have received a copy of the GNU General Public License --> <!-- along with Octave; see the file COPYING. If not, see --> <!-- <http://www.gnu.org/licenses/>. --> </body></html>