<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998) originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds * revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan * with significant contributions from: Jens Lippmann, Marek Rouchal, Martin Wilck and others --> <HTML> <HEAD> <TITLE>Corrections to the probability distribution</TITLE> <META NAME="description" CONTENT="Corrections to the probability distribution"> <META NAME="keywords" CONTENT="vol2"> <META NAME="resource-type" CONTENT="document"> <META NAME="distribution" CONTENT="global"> <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"> <LINK REL="STYLESHEET" HREF="vol2.css"> <LINK REL="next" HREF="node227.html"> <LINK REL="previous" HREF="node225.html"> <LINK REL="up" HREF="node222.html"> <LINK REL="next" HREF="node227.html"> </HEAD> <BODY > <!--Navigation Panel--> <A NAME="tex2html4237" HREF="node227.html"> <IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next" SRC="icons.gif/next_motif.gif"></A> <A NAME="tex2html4234" HREF="node222.html"> <IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up" SRC="icons.gif/up_motif.gif"></A> <A NAME="tex2html4228" HREF="node225.html"> <IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous" SRC="icons.gif/previous_motif.gif"></A> <A NAME="tex2html4236" HREF="node1.html"> <IMG WIDTH="65" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="contents" SRC="icons.gif/contents_motif.gif"></A> <BR> <B> Next:</B> <A NAME="tex2html4238" HREF="node227.html">Power of test statistics</A> <B> Up:</B> <A NAME="tex2html4235" HREF="node222.html">Basic principles of time</A> <B> Previous:</B> <A NAME="tex2html4229" HREF="node225.html">Test statistics</A> <BR> <BR> <!--End of Navigation Panel--> <H2><A NAME="SECTION001724000000000000000"> </A><A NAME="s:corr"> </A> <BR> Corrections to the probability distribution </H2> <P> In principle, it is possible to compute a value of the statistic <IMG WIDTH="13" HEIGHT="42" ALIGN="MIDDLE" BORDER="0" SRC="img461.gif" ALT="$S(\nu_1)$"> for a single frequency <IMG WIDTH="50" HEIGHT="70" ALIGN="MIDDLE" BORDER="0" SRC="img462.gif" ALT="$\nu_1$"> and to test its consistency with a random signal (<I>H</I><SUB><I>o</I></SUB>). The common procedure of inspecting the whole periodogram for a detected signal corresponds to the <I>N</I>-fold repetition of the single test for a set of trial frequencies, <!-- MATH: $\nu_n, n=1,\dots,N$ --> <IMG WIDTH="115" HEIGHT="39" ALIGN="MIDDLE" BORDER="0" SRC="img463.gif" ALT="$\nu_n, n=1,\dots,N$">. The probability of the whole periodogram being consistent with <I>H</I><SUB><I>o</I></SUB> is <!-- MATH: $1-(1-p)^N \rightarrow Np$ --> <IMG WIDTH="144" HEIGHT="49" ALIGN="MIDDLE" BORDER="0" SRC="img464.gif" ALT="$1-(1-p)^N \rightarrow Np$"> for <!-- MATH: $p\rightarrow 0$ --> <IMG WIDTH="17" HEIGHT="38" ALIGN="MIDDLE" BORDER="0" SRC="img465.gif" ALT="$p\rightarrow 0$">. The factor <I>N</I> means that there is an increased probability of accepting a given value of the statistic as consistent with a random signal. Therefore, increasing the number of trial frequencies decreases the sensitivity for the detection of a significant signal and accordingly is called the penalty factor for multiple trials or for the frequency bandwidth used. The true number of independent frequencies, <I>N</I><SUB><I>t</I></SUB>, remains generally unknown. It is usually less than the number of resolved frequencies <!-- MATH: $N_r=\Delta\nu \Delta t$ --> <IMG WIDTH="72" HEIGHT="41" ALIGN="MIDDLE" BORDER="0" SRC="img466.gif" ALT="$N_r=\Delta\nu \Delta t$"> (Sect. <A HREF="node232.html#s:iftr">12.3.1</A>) because of aliasing and still less than the number of computed frequencies <I>N</I><SUB><I>c</I></SUB>, because of oversampling: <!-- MATH: $N_t \le N_r \le N_c$ --> <IMG WIDTH="96" HEIGHT="39" ALIGN="MIDDLE" BORDER="0" SRC="img467.gif" ALT="$N_t \le N_r \le N_c$">. For a practical and conservative estimate, we recommend to use <I>N</I><SUB><I>r</I></SUB> as the number of trial frequencies, <I>N</I>. <P> According to the standard null hypothesis, <I>H</I><SUB><I>o</I></SUB>, the noise is white noise. This is not the case in many practical cases. For instance, often the noise is a stochastic process with a certain correlation length <!-- MATH: $l_{corr}>0$ --> <I>l</I><SUB><I>corr</I></SUB>>0, so that on average <I>n</I><SUB><I>corr</I></SUB> consecutive observations are correlated. Such noise corresponds to white noise passed through a low pass filter which cuts off all frequencies above <!-- MATH: $1/l_{corr}$ --> 1/<I>l</I><SUB><I>corr</I></SUB>. Such correlation is not usually taken into account by standard test statistics. The effect of this correlation is to reduce the effective number of observations by a factor <I>n</I><SUB><I>corr</I></SUB>(Schwarzenberg-Czerny, 1989). This has to be accounted for by scaling both the statistics <I>S</I> and the number of its degrees of freedom <I>n</I><SUB><I>j</I></SUB>by factors depending on <I>n</I><SUB><I>corr</I></SUB>. <P> In the test statistic, a continuum level which is inconsistent with the expected value of the statistic <IMG WIDTH="14" HEIGHT="44" ALIGN="MIDDLE" BORDER="0" SRC="img468.gif" ALT="$E\{S\}$"> may indicate the presence of such a correlation between consecutive data points. A practical recipe to measure the correlation is to compute the residual time series (e.g. with the <TT>SINEFIT/TSA</TT> command) and to look for its correlation length with <TT>COVAR/TSA</TT> command. The effect of the correlation in the parameter estimation is an underestimation of the uncertainties of the parameters; the true variances of the parameters are a factor <I>n</I><SUB><I>corr</I></SUB> larger than computed. <P> In the command individual descriptions, we often refer to probability distributions of specific statistics. For the properties of these individual distributions see e.g. Eadie <EM>et. al.</EM> (1971), Brandt (1970), and Abramovitz & Stegun (1972). The two latter references contain tables. For a computer code for the computation of the cumulative probabilities see Press <EM>et. al.</EM> (1986). <P> <HR> <!--Navigation Panel--> <A NAME="tex2html4237" HREF="node227.html"> <IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next" SRC="icons.gif/next_motif.gif"></A> <A NAME="tex2html4234" HREF="node222.html"> <IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up" SRC="icons.gif/up_motif.gif"></A> <A NAME="tex2html4228" HREF="node225.html"> <IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous" SRC="icons.gif/previous_motif.gif"></A> <A NAME="tex2html4236" HREF="node1.html"> <IMG WIDTH="65" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="contents" SRC="icons.gif/contents_motif.gif"></A> <BR> <B> Next:</B> <A NAME="tex2html4238" HREF="node227.html">Power of test statistics</A> <B> Up:</B> <A NAME="tex2html4235" HREF="node222.html">Basic principles of time</A> <B> Previous:</B> <A NAME="tex2html4229" HREF="node225.html">Test statistics</A> <!--End of Navigation Panel--> <ADDRESS> <I>Petra Nass</I> <BR><I>1999-06-15</I> </ADDRESS> </BODY> </HTML>