<html lang="en">
<head>
<title>Linear Least Squares - GNU Octave</title>
<meta http-equiv="Content-Type" content="text/html">
<meta name="description" content="GNU Octave">
<meta name="generator" content="makeinfo 4.13">
<link title="Top" rel="start" href="index.html#Top">
<link rel="up" href="Optimization.html#Optimization" title="Optimization">
<link rel="prev" href="Nonlinear-Programming.html#Nonlinear-Programming" title="Nonlinear Programming">
<link href="http://www.gnu.org/software/texinfo/" rel="generator-home" title="Texinfo Homepage">
<meta http-equiv="Content-Style-Type" content="text/css">
<style type="text/css"><!--
pre.display { font-family:inherit }
pre.format { font-family:inherit }
pre.smalldisplay { font-family:inherit; font-size:smaller }
pre.smallformat { font-family:inherit; font-size:smaller }
pre.smallexample { font-size:smaller }
pre.smalllisp { font-size:smaller }
span.sc { font-variant:small-caps }
span.roman { font-family:serif; font-weight:normal; }
span.sansserif { font-family:sans-serif; font-weight:normal; }
--></style>
</head>
<body>
<div class="node">
<a name="Linear-Least-Squares"></a>
<p>
Previous: <a rel="previous" accesskey="p" href="Nonlinear-Programming.html#Nonlinear-Programming">Nonlinear Programming</a>,
Up: <a rel="up" accesskey="u" href="Optimization.html#Optimization">Optimization</a>
<hr>
</div>
<h3 class="section">25.4 Linear Least Squares</h3>
<p>Octave also supports linear least squares minimization. That is,
Octave can find the parameter b such that the model
y = x*b
fits data (x,y) as well as possible, assuming zero-mean
Gaussian noise. If the noise is assumed to be isotropic the problem
can be solved using the ‘<samp><span class="samp">\</span></samp>’ or ‘<samp><span class="samp">/</span></samp>’ operators, or the <code>ols</code>
function. In the general case where the noise is assumed to be anisotropic
the <code>gls</code> is needed.
<!-- ols scripts/statistics/base/ols.m -->
<p><a name="doc_002dols"></a>
<div class="defun">
— Function File: [<var>beta</var>, <var>sigma</var>, <var>r</var>] = <b>ols</b> (<var>y, x</var>)<var><a name="index-ols-2392"></a></var><br>
<blockquote><p>Ordinary least squares estimation for the multivariate model
y = x*b + e<!-- /@w --> with
mean (e) = 0 and cov (vec (e)) = kron (s, I).
where
y is a t by p matrix, x is a t by
k matrix, b is a k by p matrix, and
e is a t by p matrix.
<p>Each row of <var>y</var> and <var>x</var> is an observation and each column a
variable.
<p>The return values <var>beta</var>, <var>sigma</var>, and <var>r</var> are defined as
follows.
<dl>
<dt><var>beta</var><dd>The OLS estimator for b.
<var>beta</var> is calculated directly via <code>inv (x'*x) * x' * y</code> if the
matrix <code>x'*x</code> is of full rank.
Otherwise, <var>beta</var><code> = pinv (</code><var>x</var><code>) * </code><var>y</var> where
<code>pinv (</code><var>x</var><code>)</code> denotes the pseudoinverse of <var>x</var>.
<br><dt><var>sigma</var><dd>The OLS estimator for the matrix <var>s</var>,
<pre class="example"> <var>sigma</var> = (<var>y</var>-<var>x</var>*<var>beta</var>)'
* (<var>y</var>-<var>x</var>*<var>beta</var>)
/ (<var>t</var>-rank(<var>x</var>))
</pre>
<br><dt><var>r</var><dd>The matrix of OLS residuals, <var>r</var><code> = </code><var>y</var><code> - </code><var>x</var><code>*</code><var>beta</var>.
</dl>
<!-- Texinfo @sp should work but in practice produces ugly results for HTML. -->
<!-- A simple blank line produces the correct behavior. -->
<!-- @sp 1 -->
<p class="noindent"><strong>See also:</strong> <a href="doc_002dgls.html#doc_002dgls">gls</a>, <a href="doc_002dpinv.html#doc_002dpinv">pinv</a>.
</p></blockquote></div>
<!-- gls scripts/statistics/base/gls.m -->
<p><a name="doc_002dgls"></a>
<div class="defun">
— Function File: [<var>beta</var>, <var>v</var>, <var>r</var>] = <b>gls</b> (<var>y, x, o</var>)<var><a name="index-gls-2393"></a></var><br>
<blockquote><p>Generalized least squares estimation for the multivariate model
y = x*b + e<!-- /@w --> with mean (e) = 0 and
cov (vec (e)) = (s^2) o,
where
y is a t by p matrix, x is a t by
k matrix, b is a k by p matrix, e
is a t by p matrix, and o is a t*p by
t*p matrix.
<p class="noindent">Each row of <var>y</var> and <var>x</var> is an observation and each column a
variable. The return values <var>beta</var>, <var>v</var>, and <var>r</var> are
defined as follows.
<dl>
<dt><var>beta</var><dd>The GLS estimator for b.
<br><dt><var>v</var><dd>The GLS estimator for s^2.
<br><dt><var>r</var><dd>The matrix of GLS residuals, r = y - x*beta.
</dl>
<!-- Texinfo @sp should work but in practice produces ugly results for HTML. -->
<!-- A simple blank line produces the correct behavior. -->
<!-- @sp 1 -->
<p class="noindent"><strong>See also:</strong> <a href="doc_002dols.html#doc_002dols">ols</a>.
</p></blockquote></div>
<!-- lsqnonneg scripts/optimization/lsqnonneg.m -->
<p><a name="doc_002dlsqnonneg"></a>
<div class="defun">
— Function File: <var>x</var> = <b>lsqnonneg</b> (<var>c, d</var>)<var><a name="index-lsqnonneg-2394"></a></var><br>
— Function File: <var>x</var> = <b>lsqnonneg</b> (<var>c, d, x0</var>)<var><a name="index-lsqnonneg-2395"></a></var><br>
— Function File: [<var>x</var>, <var>resnorm</var>] = <b>lsqnonneg</b> (<var><small class="dots">...</small></var>)<var><a name="index-lsqnonneg-2396"></a></var><br>
— Function File: [<var>x</var>, <var>resnorm</var>, <var>residual</var>] = <b>lsqnonneg</b> (<var><small class="dots">...</small></var>)<var><a name="index-lsqnonneg-2397"></a></var><br>
— Function File: [<var>x</var>, <var>resnorm</var>, <var>residual</var>, <var>exitflag</var>] = <b>lsqnonneg</b> (<var><small class="dots">...</small></var>)<var><a name="index-lsqnonneg-2398"></a></var><br>
— Function File: [<var>x</var>, <var>resnorm</var>, <var>residual</var>, <var>exitflag</var>, <var>output</var>] = <b>lsqnonneg</b> (<var><small class="dots">...</small></var>)<var><a name="index-lsqnonneg-2399"></a></var><br>
— Function File: [<var>x</var>, <var>resnorm</var>, <var>residual</var>, <var>exitflag</var>, <var>output</var>, <var>lambda</var>] = <b>lsqnonneg</b> (<var><small class="dots">...</small></var>)<var><a name="index-lsqnonneg-2400"></a></var><br>
<blockquote><p>Minimize <code>norm (</code><var>c</var><code>*</code><var>x</var><code> - d)</code> subject to
<var>x</var><code> >= 0</code>. <var>c</var> and <var>d</var> must be real. <var>x0</var> is an
optional initial guess for <var>x</var>.
<p>Outputs:
<ul>
<li>resnorm
<p>The squared 2-norm of the residual: norm(<var>c</var>*<var>x</var>-<var>d</var>)^2
<li>residual
<p>The residual: <var>d</var>-<var>c</var>*<var>x</var>
<li>exitflag
<p>An indicator of convergence. 0 indicates that the iteration count
was exceeded, and therefore convergence was not reached; >0 indicates
that the algorithm converged. (The algorithm is stable and will
converge given enough iterations.)
<li>output
<p>A structure with two fields:
<ul>
<li>"algorithm": The algorithm used ("nnls")
<li>"iterations": The number of iterations taken.
</ul>
<li>lambda
<p>Not implemented.
</ul>
<!-- Texinfo @sp should work but in practice produces ugly results for HTML. -->
<!-- A simple blank line produces the correct behavior. -->
<!-- @sp 1 -->
<p class="noindent"><strong>See also:</strong> <a href="doc_002doptimset.html#doc_002doptimset">optimset</a>, <a href="doc_002dpqpnonneg.html#doc_002dpqpnonneg">pqpnonneg</a>.
</p></blockquote></div>
<!-- optimset scripts/optimization/optimset.m -->
<p><a name="doc_002doptimset"></a>
<div class="defun">
— Function File: <b>optimset</b> ()<var><a name="index-optimset-2401"></a></var><br>
— Function File: <b>optimset</b> (<var>par, val, <small class="dots">...</small></var>)<var><a name="index-optimset-2402"></a></var><br>
— Function File: <b>optimset</b> (<var>old, par, val, <small class="dots">...</small></var>)<var><a name="index-optimset-2403"></a></var><br>
— Function File: <b>optimset</b> (<var>old, new</var>)<var><a name="index-optimset-2404"></a></var><br>
<blockquote><p>Create options struct for optimization functions.
<p>Valid parameters are:
<ul>
<li>AutoScaling
<li>ComplexEqn
<li>FinDiffType
<li>FunValCheck
When enabled, display an error if the objective function returns a complex
value or NaN. Must be set to "on" or "off" [default].
<li>GradObj
When set to "on", the function to be minimized must return a second argument
which is the gradient, or first derivative, of the function at the point
<var>x</var>. If set to "off" [default], the gradient is computed via finite
differences.
<li>Jacobian
When set to "on", the function to be minimized must return a second argument
which is the Jacobian, or first derivative, of the function at the point
<var>x</var>. If set to "off" [default], the Jacobian is computed via finite
differences.
<li>MaxFunEvals
Maximum number of function evaluations before optimization stops.
Must be a positive integer.
<li>MaxIter
Maximum number of algorithm iterations before optimization stops.
Must be a positive integer.
<li>OutputFcn
A user-defined function executed once per algorithm iteration.
<li>TolFun
Termination criterion for the function output. If the difference in the
calculated objective function between one algorithm iteration and the next
is less than <code>TolFun</code> the optimization stops. Must be a positive
scalar.
<li>TolX
Termination criterion for the function input. If the difference in <var>x</var>,
the current search point, between one algorithm iteration and the next is
less than <code>TolX</code> the optimization stops. Must be a positive scalar.
<li>TypicalX
<li>Updating
</ul>
</p></blockquote></div>
<!-- optimget scripts/optimization/optimget.m -->
<p><a name="doc_002doptimget"></a>
<div class="defun">
— Function File: <b>optimget</b> (<var>options, parname</var>)<var><a name="index-optimget-2405"></a></var><br>
— Function File: <b>optimget</b> (<var>options, parname, default</var>)<var><a name="index-optimget-2406"></a></var><br>
<blockquote><p>Return a specific option from a structure created by
<code>optimset</code>. If <var>parname</var> is not a field of the <var>options</var>
structure, return <var>default</var> if supplied, otherwise return an
empty matrix.
</p></blockquote></div>
<!-- DO NOT EDIT! Generated automatically by munge-texi.pl. -->
<!-- Copyright (C) 1996-2012 John W. Eaton -->
<!-- This file is part of Octave. -->
<!-- Octave is free software; you can redistribute it and/or modify it -->
<!-- under the terms of the GNU General Public License as published by the -->
<!-- Free Software Foundation; either version 3 of the License, or (at -->
<!-- your option) any later version. -->
<!-- Octave is distributed in the hope that it will be useful, but WITHOUT -->
<!-- ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or -->
<!-- FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License -->
<!-- for more details. -->
<!-- You should have received a copy of the GNU General Public License -->
<!-- along with Octave; see the file COPYING. If not, see -->
<!-- <http://www.gnu.org/licenses/>. -->
</body></html>
