<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998) originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds * revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan * with significant contributions from: Jens Lippmann, Marek Rouchal, Martin Wilck and others --> <HTML> <HEAD> <TITLE>Estimation</TITLE> <META NAME="description" CONTENT="Estimation"> <META NAME="keywords" CONTENT="vol2"> <META NAME="resource-type" CONTENT="document"> <META NAME="distribution" CONTENT="global"> <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"> <LINK REL="STYLESHEET" HREF="vol2.css"> <LINK REL="previous" HREF="node13.html"> <LINK REL="up" HREF="node11.html"> <LINK REL="next" HREF="node15.html"> </HEAD> <BODY > <!--Navigation Panel--> <A NAME="tex2html1627" HREF="node15.html"> <IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next" SRC="icons.gif/next_motif.gif"></A> <A NAME="tex2html1624" HREF="node11.html"> <IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up" SRC="icons.gif/up_motif.gif"></A> <A NAME="tex2html1620" HREF="node13.html"> <IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous" SRC="icons.gif/previous_motif.gif"></A> <A NAME="tex2html1626" HREF="node1.html"> <IMG WIDTH="65" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="contents" SRC="icons.gif/contents_motif.gif"></A> <BR> <B> Next:</B> <A NAME="tex2html1628" HREF="node15.html">Raw to Calibrated Data</A> <B> Up:</B> <A NAME="tex2html1625" HREF="node11.html">Basic Concepts</A> <B> Previous:</B> <A NAME="tex2html1621" HREF="node13.html">Noise distributions</A> <BR> <BR> <!--End of Navigation Panel--> <H2><A NAME="SECTION00513000000000000000"> </A> <A NAME="estim"> </A> <BR> Estimation </H2> A number of different statistical methods are used for estimating parameters from a data set. The most commonly used one is the least squares method which estimates a parameter <IMG WIDTH="17" HEIGHT="22" ALIGN="BOTTOM" BORDER="0" SRC="img38.gif" ALT="$\theta$"> by minimizing the function : <BR><P></P> <DIV ALIGN="CENTER"> <!-- MATH: \begin{equation} S(\theta) = \sum_i ( y_i - f(\theta;x_i) )^2 \end{equation} --> <TABLE WIDTH="100%" ALIGN="CENTER"> <TR VALIGN="MIDDLE"><TD ALIGN="CENTER" NOWRAP><A NAME="eq:least-sqr"> </A><IMG WIDTH="239" HEIGHT="61" SRC="img39.gif" ALT="\begin{displaymath}S(\theta) = \sum_i ( y_i - f(\theta;x_i) )^2 \end{displaymath}"></TD> <TD WIDTH=10 ALIGN="RIGHT"> (2.6)</TD></TR> </TABLE> </DIV> <BR CLEAR="ALL"><P></P> where <I>y</I> is the dependent and <I>x</I> the independent variables while <I>f</I> is a given function. Equation <A HREF="node14.html#eq:least-sqr">2.6</A> can be expanded to more parameters if needed. For linear functions <I>f</I> an analytic solution can be derived whereas an iteration scheme must be applied for most non-linear cases. Several conditions must be fulfilled for the method to give a reliable estimate of <IMG WIDTH="17" HEIGHT="22" ALIGN="BOTTOM" BORDER="0" SRC="img40.gif" ALT="$\theta$">. The most important assumptions are that the errors in the dependent variable are normal distributed, the variance is homogeneous, and the independent variables have no errors and are uncorrelated. <P> The other main technique for parameter estimation is the maximum likelihood method where the joint probability of the parameter <IMG WIDTH="16" HEIGHT="22" ALIGN="BOTTOM" BORDER="0" SRC="img41.gif" ALT="$\theta$"> : <BR><P></P> <DIV ALIGN="CENTER"> <!-- MATH: \begin{equation} l(\theta) = \prod_i P(\theta,x_i) \end{equation} --> <TABLE WIDTH="100%" ALIGN="CENTER"> <TR VALIGN="MIDDLE"><TD ALIGN="CENTER" NOWRAP><A NAME="eq:max-like"> </A><IMG WIDTH="163" HEIGHT="61" SRC="img42.gif" ALT="\begin{displaymath}l(\theta) = \prod_i P(\theta,x_i) \end{displaymath}"></TD> <TD WIDTH=10 ALIGN="RIGHT"> (2.7)</TD></TR> </TABLE> </DIV> <BR CLEAR="ALL"><P></P> is maximized. In Equation <A HREF="node14.html#eq:max-like">2.7</A>, <I>P</I> denotes the probability density of the individual data sets. Normally, the logarithm likelihood <!-- MATH: $L = \log(l)$ --> <IMG WIDTH="101" HEIGHT="44" ALIGN="MIDDLE" BORDER="0" SRC="img43.gif" ALT="$L = \log(l)$"> is used to simplify the maximization procedure. This method can be used for any given distribution. For a normal distribution the two methods will give the same result. <P> <HR> <!--Navigation Panel--> <A NAME="tex2html1627" HREF="node15.html"> <IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next" SRC="icons.gif/next_motif.gif"></A> <A NAME="tex2html1624" HREF="node11.html"> <IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up" SRC="icons.gif/up_motif.gif"></A> <A NAME="tex2html1620" HREF="node13.html"> <IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous" SRC="icons.gif/previous_motif.gif"></A> <A NAME="tex2html1626" HREF="node1.html"> <IMG WIDTH="65" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="contents" SRC="icons.gif/contents_motif.gif"></A> <BR> <B> Next:</B> <A NAME="tex2html1628" HREF="node15.html">Raw to Calibrated Data</A> <B> Up:</B> <A NAME="tex2html1625" HREF="node11.html">Basic Concepts</A> <B> Previous:</B> <A NAME="tex2html1621" HREF="node13.html">Noise distributions</A> <!--End of Navigation Panel--> <ADDRESS> <I>Petra Nass</I> <BR><I>1999-06-15</I> </ADDRESS> </BODY> </HTML>