<chapter id="data">
<title>Working With Data</title>
<sect1 id="data-overview">
<title>Data Objects Overview</title>
<para>
We will call Data Objects in &kst; all objects which appear in the <link linkend="data-overview-datamanager">Data Manager</link>. There are ten main kinds of data object in &kst;. The following diagram illustrates the relationships between the different types:
</para>
<para>
<inlinemediaobject>
<imageobject>
<imagedata fileref="Diagram-kst-data-datatypes.png" format="PNG"/>
</imageobject>
<textobject>
<phrase>&kst; Data Types</phrase>
</textobject>
</inlinemediaobject>
</para>
<para>
As can be seen in the diagram, the only truly <quote>plottable</quote> data objects are curves and images. Many data objects contain slave vectors which can be plotted using curves, however — so these data objects are effectively plottable as well.
</para>
<para>
The usefulness of the Data Objects concept in &kst; is that they can be tied together to create pipelines. An Event Monitor object, for instance, can take the output of another object as its input. Updates to objects in the pipeline propagate automatically. This is one of the key features which makes &kst; powerful for realtime plotting.
</para>
<sect2 id="data-overview-datamanager">
<title>The Data Manager</title>
<para>
The Data Manager provides a central location for adding, deleting, and modifying all of the data objects used in &kst;. It can be accessed from <menuchoice><guimenu>Data</guimenu><guimenuitem>Data Manager</guimenuitem></menuchoice>.
</para>
<screenshot>
<screeninfo>The Data Manager</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-datamanager.png" format="PNG" />
</imageobject>
<textobject>
<phrase>The Data Manager</phrase>
</textobject>
</mediaobject>
</screenshot>
<para>
The panel on the left side of the Data Manager can be used to quickly create new data objects. The <guibutton>Purge</guibutton> button automatically deletes Data Objects which are not <link linkend="data-overview-datamanager-used">Used</link> Data Objects. The <guibutton>Edit</guibutton> and <guibutton>Delete</guibutton> buttons allow you to selectively edit or remove particular objects. The <guibutton>Close</guibutton> button exits the Data Manager.
</para>
<tip><para>You can also create new data objects which are based on vectors by right clicking on the corresponding vector and choosing one of the <guimenuitem>Make ...</guimenuitem> options. If you select a curve, on the other hand, then you have the option to add it to an existing plot.</para></tip>
<para>
For its list of Data Objects, the Data Manager displays several pieces of information. These are described below:
</para>
<variablelist>
<varlistentry>
<term>
<guilabel>Name</guilabel>
</term>
<listitem>
<para>
The name of the data object, unique among the set of data objects with the same type.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Type</guilabel>
</term>
<listitem>
<para>
The type of the data object determines how it is created and what its options are. Type can be one of: Data Vector, Curve, Equation, Histogram, Spectrum, Plugin, Event Monitor, Matrix, Image, or Spectrogram.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term id="data-overview-datamanager-used">
<guilabel>Used</guilabel>
</term>
<listitem>
<para>
This field gives the status of the data object. If there is a check mark in the <guilabel>Used</guilabel> column of a data object, then some other object in &kst; is dependent on it.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Samples</guilabel>
</term>
<listitem>
<para>
The number of elements in the data object.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Properties</guilabel>
</term>
<listitem>
<para>
A summary of the data object's key parameters, dependent on its <link linkend="data-types">type</link>.
</para>
</listitem>
</varlistentry>
</variablelist>
<tip>
<para>
If you want to quickly find a vector among many items in the data manager, you may just type part of its name in the search field. Then, the data manager will respond by listing only those items which contain the entered text within their names.
</para>
</tip>
</sect2>
<sect2 id="data-overview-creatingdeleting">
<title>Creating and Deleting Data Objects</title>
<para>
You can create new Data Objects using either the left panel in the <link linkend="data-overview-datamanager">Data Manager</link>, or the <guimenu>Data</guimenu> menu.
</para>
<para>
When you are creating a new data object, you may enter a unique name to identify the object. If you do not enter a custom name then a unique name will be automatically generated.
</para>
<para>
To delete a data object you must use the <link linkend="data-overview-datamanager">Data Manager</link>. Note that if the <guilabel>Used</guilabel> column for a data object has a check mark then some other data or view object in &kst; has a dependency on it. Depending on the strength of these dependencies, &kst; will prompt before an object is deleted. If, for example, other objects have critical dependencies on the one which you are attempting to delete, &kst; will ask if you would like to delete these other objects as well. Some dependencies are listed below:
</para>
<itemizedlist>
<listitem>
<para>
Plots are not critically dependent on the plottable objects which they contain, so if a plotted object is deleted &kst; will automatically remove it from all plots, without prompting.
</para>
</listitem>
<listitem>
<para>
All data objects which use a particular data vector must be deleted before the data vector itself can be deleted.
</para>
</listitem>
<listitem>
<para>
All children of a parent data object must be unused before the parent data object can be deleted.
</para>
</listitem>
</itemizedlist>
<para>
After a sequence of deletions and removals of plottable data objects from plots, you may find that there are numerous unused data objects displayed in the Data Manager. To quickly remove these objects, you can click the <guibutton>Purge</guibutton> button.
</para>
</sect2>
</sect1>
<sect1 id="data-types">
<title>Data Types</title>
<sect2 id="data-types-vectors">
<title>Vectors</title>
<para>
Vectors are one of the most often-used data objects in &kst;. As their name implies, vectors are simply ordered lists of numbers. Usually they contain the x or y coordinates of a set of data points.
</para>
<para>
As vectors can potentially be quite large, it is a good idea to be aware of the amount of memory &kst; has
available for use. The current available memory is displayed in the lower right corner of the status bar
of the main &kst; window. If the status bar is not available, select <menuchoice><guimenu>Settings</guimenu><guimenuitem>Show Statusbar</guimenuitem></menuchoice> to display it.
</para>
<para>
There are two kinds of vector — <link linkend="data-types-vectors-data">data vectors</link> and <link linkend="data-types-vectors-static">static vectors</link>. Data vectors are read into &kst; through a data source. Static vectors, on the other hand, are generated by &kst;. They represent an evenly spaced list of numbers, and can be useful, for example, for generating an x-axis for a plot.
</para>
<sect3 id="data-types-vectors-data">
<title>Data Vectors</title>
<para>
The following is a screenshot of the window displayed when editing data vectors. A new data vector is created if you choose the <guibutton>Read from data source</guibutton> radio button in the New Vector dialog. Explanations of the settings are listed below.
</para>
<screenshot>
<screeninfo>Edit Vectors</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-vectoredit.png" format="PNG" />
</imageobject>
<textobject>
<phrase>Edit Vectors</phrase>
</textobject>
</mediaobject>
</screenshot>
<variablelist>
<varlistentry>
<term>
<guilabel>File name</guilabel>
</term>
<listitem>
<para>
The path to the desired data file. Clicking the button to the right displays a file browser
window that can be used to graphically browse for the file.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Field or column</guilabel>
</term>
<listitem>
<para>
The field or column to use for the vector.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Start frame</guilabel>, <guilabel>Count from end</guilabel>,
<guilabel>Range</guilabel>, and <guilabel>Read to end</guilabel>
</term>
<listitem>
<para>
This section specifies the range of data to use from the selected field for the data vector. This discussion assumes a knowledge of the <link linkend="data-sources-concepts-frames">Frames</link> concept. Using these four settings, the lower and upper boundaries of the data range can be set. For example, to read from frame 10 to frame 900, enter <userinput>10</userinput> for <guilabel>Start frame</guilabel> and <userinput>890</userinput> for <guilabel>Range</guilabel>. To read from frame 500 to the end of the file, enter <userinput>500</userinput> for <guilabel>Start frame</guilabel> and select the <guilabel>Read to end</guilabel> option. To read only the last 450 frames from the file, select the <guilabel>Count from end</guilabel> option and enter <userinput>450</userinput> for <guilabel>Range</guilabel>. The combinations used in the previous two examples are often useful for reading data from a file that is being updated in real time. Subsequent additions to the file are read, causing associated vectors to be updated as well.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Read 1 sample per</guilabel> and <guilabel>Boxcar filter first</guilabel>
</term>
<listitem>
<para>
In addition to the lower and upper range boundaries, the samples to read from the selected range can be set.
If <guilabel>Read 1 sample per</guilabel> is not selected, all samples in the selected range will be read.
If <guilabel>Read 1 sample per</guilabel> is selected, only 1 sample per <literal>N</literal> frames will be read, where <literal>N</literal> is the number entered in selection box to the right.
The value of the 1 sample that is used to represent a frame depends on whether or not
<guilabel>Boxcar filter first</guilabel> is selected. If <guilabel>Boxcar filter first</guilabel> is not selected, the value is the same as the value of the 1st sample in the frame. If <guilabel>Boxcar filter first</guilabel> is selected, the value is the average of all the samples in that particular frame.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect3>
<sect3 id="data-types-vectors-static">
<title>Static Vectors</title>
<para>
The following is a screenshot of the window displayed when editing static vectors. A new static vector is created if you choose the <guibutton>Generate</guibutton> radio button in the New Vector dialog. Explanations of the settings for static vectors are listed below.
</para>
<screenshot>
<screeninfo>Edit Static Vectors</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-staticvectoredit.png" format="PNG" />
</imageobject>
<textobject>
<phrase>Edit Vectors</phrase>
</textobject>
</mediaobject>
</screenshot>
<para>
The entries of the vector will be <guilabel>From</guilabel> + (<guilabel>To</guilabel> - <guilabel>From</guilabel>)*(i-1)/(Number of samples - 1) for i = 1,...,Number of samples.
</para>
</sect3>
<sect3 id="KstScript-vectors">
<title>Generating Vectors from KstScript</title>
<para>
You can also create and initialize vectors by using KstScript.
See details in the <link linkend="extensions-kstscript">KstScript</link> of the Extensions Chapter,
and in the <link linkend="class_Vector">Vector</link> and <link linkend="class_DataVector">DataVector Classes</link> of the Script Chapter.
</para>
<para>
Examples are provided below:
</para>
<example id="kstscript-static-vector">
<title>Example of Using KstScript to Construct a Static Vector</title>
<programlisting>
v = new Vector(); //construct a new vector
v.tagName = "v1"; //give it a name in the Data Manager
v.resize(100); //give it a length of 100
for(var i=0; i<100; i++){v[i]=Math.random()};//intialize this vector
</programlisting>
</example>
<example id="kstscript-data-vector">
<title>Example of Using KstScript to Load a Vector from Source Files</title>
<programlisting>
source = new DataSource("rand.dat");//specify a data source file system. See the <link linkend="class_DataSource">DataSource Class</link> for details
v = new DataVector(s,"1");//construct a new vector v and set its value from field 1 of the source file
v.tagName = "vector" //give the vector a name in the Data Manager
</programlisting>
</example>
</sect3>
</sect2>
<sect2 id="data-types-curves">
<title>Curves</title>
<para>
Curves are primarily used to create plottable objects from vectors. Curves are created from two vectors - an <quote>X axis vector</quote> and a <quote>Y axis vector</quote>, that provide a set of data points. Thus, a curve can be thought
of as a set of data points and the lines that connect them (even though the points or lines may not be visible on plots).
</para>
<para>
The following is a screenshot of the window displayed when creating or editing curves. Explanations of the
settings are listed below.
</para>
<screenshot>
<screeninfo>Edit Curves</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-curveedit.png" format="PNG" />
</imageobject>
<textobject>
<phrase>Edit Curves</phrase>
</textobject>
</mediaobject>
</screenshot>
<variablelist>
<varlistentry>
<term>
<guilabel>Legend Text</guilabel>
</term>
<listitem>
<para>
The string to be used in Plot Legends to describe this curve.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>X axis vector</guilabel>
</term>
<listitem>
<para>
The vector to use for the independent (horizontal) axis.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Y axis vector</guilabel>
</term>
<listitem>
<para>
The vector to use for the dependent (vertical) axis.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>+X error bar</guilabel> and <guilabel>+Y error bar</guilabel>
</term>
<listitem>
<para>
The vectors containing error values corresponding to the X axis vector and Y axis vector, respectively.
The vectors should contain the sizes of the error bars in the same order as the data points.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>Use +<replaceable>X/Y</replaceable> error bar for
-<replaceable>X/Y</replaceable></guilabel></term> <listitem><para>If
this item is checked, the error bars are drawn symmetrically about the
data point. To draw asymmetric error bars in the x or y direction,
uncheck the box, and choose a vector for the length of the error bar
below the data point in the <guilabel>-<replaceable>X/Y</replaceable>
error bar</guilabel> combo box.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>-<replaceable>X/Y</replaceable> error
bar</guilabel></term>
<listitem><para>The vector to use for the error bar below the data
point. See the previous entry for more detail on using asymmetric
error bars.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Color</term>
<listitem><para>Determines the color of the data points of the curve. Clicking the <inlinemediaobject><imageobject><imagedata fileref="Widget-kst-colorchooser.png" format="PNG"/></imageobject></inlinemediaobject> button displays the standard &kde; color chooser.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>Show points</guilabel></term>
<listitem><para>Determines whether or not the data points will be indicated by the selected type of marker.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>Density</guilabel></term>
<listitem><para>If <guilabel>Show points</guilabel> is enabled, <guilabel>Density</guilabel> allows you to have &kst; show only occasional points if they would otherwise clutter the plot.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>Show lines</guilabel></term>
<listitem><para>Determine whether or not the data points will be connected by lines</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>Weight</guilabel></term>
<listitem><para>If <guilabel>Show lines</guilabel> is enabled, this gives the thickness in pixels of the curve.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>Bargraph</guilabel></term>
<listitem><para>Draws lines or rectangles from the data points to the y = 0 line in the plot.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>Ignore in automatic axes range calculations</guilabel></term>
<listitem><para>If this option is checked, the curve will be ignored in determining the automatic y-axis range; if it is unchecked the curve will be used in determining the automatic y-axis range. By default, this option is unchecked.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>Y axis vector offset</guilabel></term>
<listitem><para>If this option is checked, the corresponding selected scalar will be subtracted from the y values of the curve.</para>
</listitem>
</varlistentry>
</variablelist>
<sect3>
<title>Using the Curves Class in KstScript</title>
<para>Plotting a curve can be done by using KstScript. See details in the <link linkend="class_Curve">Curve Class</link> of the Script chapter.</para>
<example id="curve-kstscript-eg">
<title>Example of Using KstScript to plot a curve</title>
<programlisting>
//xv and yv are names of two existing vectors in the Data Manager.
//See example <link linkend="kstscript-static-vector">3.1</link> and <link linkend="kstscript-data-vector">3.2</link> for constructing vectors using KstScript.
c = new Curve("xv", "yv") // to construct a new curve using xv and yv
c.tagName = "xv vs yv"; //define the name for the curve in the data manager
//to view the curve, a <link linkend="class_Window">window object</link> and a <link linkend="class_Plot">plot object</link> need to be constructed.
w = new Window();
p = new Plot(w);
// using the <link linkend="class_Collection">append</link> method in the <link linkend="class_CurveCollection">CurveCollection Class</link> to show the curve in plot
p.cuves.append(c);
</programlisting>
<para>
There is a detailed example of vectors, plots and curves in the Extensions Chapter. Click <link linkend="kstscriptexample">here</link> to view it.
</para>
</example>
</sect3>
</sect2>
<sect2 id="data-types-equations">
<title>Equations</title>
<para>
An equation data object consists of a mathematical expression and an independent variable. The expression
is built using a combination of scalars, vectors, and operators, and usually represents the values of the
dependent variable. The independent variable is generated from an existing vector, and stored as a slave vector along with the evaluated results of the equation.
</para>
<para>
The following is a screenshot of the window displayed when creating or editing equations. Explanations of the
settings are listed below.
</para>
<screenshot>
<screeninfo>Edit Equation</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-equationedit.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
<variablelist>
<varlistentry>
<term>
<guilabel>Operators</guilabel>
</term>
<listitem>
<para>
A list of available operators. Choosing an operator from the list inserts the operator at the current insertion point in the <guilabel>Equation</guilabel> text box.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Vectors</guilabel>
</term>
<listitem>
<para>
A list of the current vector objects in &kst;. Choosing a vector from the list inserts the vector at the current
insertion point in the <guilabel>Equation</guilabel> text box.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Scalars</guilabel>
</term>
<listitem>
<para>
A list of the available scalar values. This list is composed of both the scalar values in the current &kst;
session as well as some built-in constants. Choosing a scalar from the list inserts the scalar at the current
insertion point in the <guilabel>Equation</guilabel> text box.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Equation</guilabel>
</term>
<listitem>
<para>
The mathematical expression representing the independent variable. You may manually type in this text box or
you may select items to insert using the above drop-down lists.
When manually typing equation expressions, remember to <emphasis>put vector and scalar names into square brackets</emphasis>
</para>
<para>
For a full list of the valid Kst operators, functions and constants to express equations, see the <link linkend="equation-expressions">Equation Expression in Kst</link>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>X Vector</guilabel>
</term>
<listitem>
<para>
Select a vector to use as the independent vector. Select a vector from the
drop-down list, or quickly create a new one by clicking the wizard button to the right.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Interpolate to highest resolution vector</guilabel>
</term>
<listitem>
<para>
Select this option to interpolate the selected X vector to the greatest number of samples possible, given the data used in the equation expression. For example, if the data expression acts on a vector Y which contains twice as many variables as the selected X vector, the equation object will create a slave vector of interpolated X values with the same number of points as Y.
</para>
</listitem>
</varlistentry>
</variablelist>
<sect3>
<title>Using the Equation Class in KstScript</title>
<para>
The <link linkend="class_Equation">Equation Class</link> in KstScript provides methods to manipulate Equation objects. An example is shown below.
</para>
<example id="kstscript-equation-eg">
<title>Example of Using the Equation Class</title>
<programlisting>
//construct a vector object
x = new Vector();
x.tagName = "x_vector";//call it x_vector in the Data Manager
x.resize(100);
//initialize x_vector
for(var i=0; i<100; i++){x[i]=i;}
//construct an equation object
eq = new Equation("LN([x_vector])",x);//specify the equation expressions
//check <link linkend="equation-expressions">equation appendix</link> to see
//what expressions are supported by &kst;.
eq.tagName = "Natural Logarithmic Function";
//to check if the equation is valid
eq.valid
//this should return true
//to construct a curve using the resulting vectors of applying this equation
c = new Curve(eq.xVector, eq.yVector);
c.tagName = "eq_curve";
//to view the curve, see <link linkend="curve-kstscript-eg">Example 3.3</link>.
</programlisting>
</example>
</sect3>
</sect2>
<sect2 id="data-types-histograms">
<title>Histograms</title>
<para>
This object calculates the histogram of a chosen vector.
</para>
<para>
The following is a screenshot of the histogram specific options when creating or editing histograms. Explanations of the
settings are listed below.
</para>
<screenshot>
<screeninfo>Edit Histogram</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-histogramedit.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
<variablelist>
<varlistentry>
<term>
<guilabel>Data vector</guilabel>
</term>
<listitem>
<para>
The data vector to create the histogram from. Although a vector is needed to create a histogram, the vector is treated as an unordered set for the purposes of creating a histogram.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>From</guilabel> and <guilabel>To</guilabel>
</term>
<listitem>
<para>
The <guilabel>From</guilabel> field contains the left bound for the leftmost bin in the histogram.
The <guilabel>To</guilabel> field contains the right bound for the rightmost bin in the histogram.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Num bins</guilabel>
</term>
<listitem>
<para>
Enter the total number of bins to use in this field.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guibutton>Auto Bin</guibutton>
</term>
<listitem>
<para>
Instead of specifying values for <guilabel>From</guilabel>, <guilabel>To</guilabel>, and
<guilabel>Num bins</guilabel>, you can click <guibutton>Auto Bin</guibutton> to automatically generate
values for all three fields based on the length and lowest/highest values of the chosen data vector.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Number in bin</guilabel>
</term>
<listitem>
<para>
Selecting this option causes the y axis to represent the number of elements in each bin.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Fraction in bin</guilabel>
</term>
<listitem>
<para>
Selecting this option causes the y axis to represent the fraction of elements
(out of the total number of elements) in each bin.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Percent in bin</guilabel>
</term>
<listitem>
<para>
Selecting this option causes the y axis to represent the percentage of elements (out of the total
number of elements) in each bin.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Peak bin = 1.0</guilabel>
</term>
<listitem>
<para>
Selecting this option causes the y axis to represent the number of elements in each bin divided by
the number of elements in the largest bin (the bin with the greatest number of elements).
</para>
</listitem>
</varlistentry>
</variablelist>
<sect3>
<title>Using the Histogram Class in KstScript</title>
<para>
Histograms can also be constructed using KstScript.
See details in the <link linkend="class_Histogram">Histogram Class</link> of the Working with KstScript appendix.
</para>
<example id = "kstscript-histogram-eg">
<title>Example of Constructing a Histogram through KstScript</title>
<para>The example below uses the data file <filename>normal.dat</filename> generated by the perl script in <link linkend="commontasks-generatehistogram">Generating a Histogram</link> of Chapter 2</para>
<programlisting>
//load the data vector from normal.dat; see details in example 3.2
s = new DataSource("normal.dat");
v = new DataVector(s, "1");
//construct a histogram object using vector v.
hs = new Histogram(v);
//construct a histogram curve; see details in the <link linkend="class_Curve">Curve Class</link>
c = new Curve(hs);
//view the histogram; see <link linkend="curve-kstscript-eg">example 3.3</link>
w = new Window();
p = new Plot(w);
p.curves.append(c);
</programlisting>
</example></sect3>
</sect2>
<sect2 id="data-types-spectrum">
<title>Spectrum</title>
<para>This data object calculates the Discrete Fourier Transform (DFT) of an input signal using a Fast Fourier Transform (FFT) algorithm, and outputs a spectrum of the signal.</para>
<para>The mathematical definition of the DFT is:</para>
<para>
<inlinemediaobject>
<imageobject>
<imagedata fileref="Formula-kst-DFT.png" format="PNG" />
</imageobject>
<textobject>
<phrase>DFT </phrase>
</textobject>
</inlinemediaobject>
</para>
<para>
where <literal>x(n)</literal> is the input signal, and <literal>N</literal> is the number of samples.
</para>
<para>
The following definitions assume basic knowledge of power spectra — we use the conventions and terminology of <quote>Numerical Recipes in C: The Art of Scientific Computing</quote>, published by Cambridge University Press.
</para>
<screenshot>
<screeninfo>Power Spectra Window</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-spectraedit.png" format="PNG" />
</imageobject>
<textobject>
<phrase>Power Spectra Window</phrase>
</textobject>
</mediaobject>
</screenshot>
<variablelist>
<varlistentry>
<term>
<guilabel>Data vector</guilabel>
</term>
<listitem>
<para>
The data vector to create the spectrum from.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Apodize, Function, and Sigma</guilabel>
</term>
<listitem>
<para>
An Apodization Function is a function used to reduce leakage.
</para>
<para>
In the drop-down menu of <guilabel>Function</guilabel>, the apodization function applied to the spectrum can be chosen from:
</para>
<itemizedlist>
<listitem><para>
Default
</para></listitem>
<listitem><para>
<ulink url="http://mathworld.wolfram.com/BartlettFunction.html">Bartlett</ulink>
</para></listitem>
<listitem><para>
<ulink url="http://mathworld.wolfram.com/BlackmanFunction.html">Blackman</ulink>
</para></listitem>
<listitem><para>
<ulink url="http://mathworld.wolfram.com/ConnesFunction.html">Connes</ulink>
</para></listitem>
<listitem><para>
<ulink url="http://mathworld.wolfram.com/CosineApodizationFunction.html">Cosins</ulink>
</para></listitem>
<listitem><para>
<ulink url="http://mathworld.wolfram.com/GaussianFunction.html">Gaussian (custom sigma)</ulink>
</para></listitem>
<listitem><para>
<ulink url="http://mathworld.wolfram.com/HammingFunction.html">Hamming</ulink>
</para></listitem>
<listitem><para>
<ulink url="http://mathworld.wolfram.com/HanningFunction.html">Hann</ulink>
</para></listitem>
<listitem><para>
<ulink url="http://mathworld.wolfram.com/WelchApodizationFunction.html">Welch</ulink>
</para></listitem>
</itemizedlist>
<para>
If the Apodize option is selected, the data is windowed using the function chosen from the drop-down menu at right. Apodization can be used to reduce bin-to-bin leakage but the resolution of the plot will be decreased. If the Gaussian apodization function is chosen, then a <guilabel>sigma</guilabel> value may also be entered to define its effective width.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Remove Mean</guilabel>
</term>
<listitem>
<para>
Select this option to remove the mean from the data while calculating the transform.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Interleaved average</guilabel> and <guilabel>FFT Length</guilabel>
</term>
<listitem>
<para>
Selecting <guilabel>Interleaved average</guilabel> calculates the FFT over segments of the data, which increases the FFT speed and improves accuracy, at the result of less information about low frequency components. The length of the interleaved segments is specified as a power of 2. If <guilabel>Interleaved average</guilabel> is unchecked, &kst; will choose the smallest possible FFT length which is greater than the data vector length.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Interpolate Over Holes</guilabel>
</term>
<listitem>
<para>
If the data vector contains NaN values, then selecting this option will cause &kst; to interpolate through them when calculating the power spectrum.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guibutton>Sample rate</guibutton>
</term>
<listitem>
<para>
This is the number of data vector samples per unit time.
The sample rate is used to generate the X axis of the spectrum for plotting.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Vector units</guilabel> and <guilabel>Rate units</guilabel>
</term>
<listitem>
<para>
The units specified in these textboxes are used for the purpose of auto-generating axes labels for the plots.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Output</guilabel>
</term>
<listitem>
<para>The output spectrum can be chosen from</para>
<itemizedlist>
<listitem>
<para>
Amplitude spectral density (V/Hz^1/2)
</para>
</listitem>
<listitem><para>
Power spectral density (V^2/Hz)
</para></listitem>
<listitem><para>
Amplitude spectrum (V)
</para>
</listitem>
<listitem><para>
Power spectrum (V^2)
</para></listitem>
</itemizedlist>
<para>
The option selected from this drop-down menu determines which variant of the power spectrum should be computed.</para>
<para>
The power spectrum is the plot of |X(k)|^2 vs frequency vector generated by the sample rate.
The amplitude spectrum is calculated by taking the square root of the power spectrum.
Spectral densities are calculated by multiplying the Power or Amplitude Spectra by the factor: N/sample rate, where N is the number of data samples.
</para>
</listitem>
</varlistentry>
</variablelist>
<sect3>
<title>Using the PowerSpectrum Class in Kst</title>
<para>The <link linkend="class_PowerSpectrum">PowerSpectrum Class</link> in KstScript can also be used to construct power spectra.</para>
<example id ="kstscript-spectrum-eg">
<title>Example of Using PowerSpectrum Class</title>
<programlisting>
//construct a random vector whose value is between 0 and 1 with 10000 samples
x = new Vector();
x.resize(10000);
x.tagName = "random" //give it a name: random in Data Manager
for(var i=0; i <10000; i++){x[i]=Math.random()}
//construct a Power Spectrum according to the constructor:
//PowerSpectrum(vector, freq, [,average[,len[,apodize[,removeMean]]])
ps = new PowerSpectrum(x, 10, true, 10, true, false)
ps.tagName ="rand_Spec";
</programlisting>
<para>After executing the above scripts, the Data Manager should look like this:</para>
<screenshot>
<screeninfo>resulting Names in DataManager</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-ch3-spect-eg.png" format="PNG" />
</imageobject>
<textobject>
<phrase>resulting Names in DataManager</phrase>
</textobject>
</mediaobject>
</screenshot>
<para>Later, the properties of this power spectrum can be changed like this: </para>
<programlisting>
ps.removeMean = true; //choose the remove mean option
ps.length = 13; //change the FFT length to be 2^13
</programlisting>
</example></sect3>
</sect2>
<sect2 id="data-types-spectrogram">
<title>Spectrogram</title>
<para>
A spectrogram is a way to plot the frequency spectrum of a data vector as a function of position within the data. It allows you to visualize, for example, how the frequency distribution of a signal changes with time. In &kst; the spectrogram data object takes a data vector as an input and produces an image object which can be displayed as a <quote>waterfall</quote> plot. The waterfall plot is used to show how two-dimensional information changes over time; here, it indicates how power spectrum vs frequency of a data set changes over time. An example is shown in the screenshot below. This is a spectrogram from one day of data from a channel of the WMAP satellite. The size of the intervals over which the component spectra are calculated has been chosen so that each interval corresponds to 1/2 hour of data. The 1h periodic precession of the satellite about its spin axis can be clearly seen. Also, at ~0.007Hz the red lines at the bottom of the image show the much faster (0.5rpm) spin frequency of the satellite.
</para>
<screenshot>
<screeninfo>Spectrogram Example</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-spectrogramexample.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
<para>
Here is the edit dialog for the spectrogram object.
</para>
<screenshot>
<screeninfo>Edit Spectrogram Window</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-spectrogramedit.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
<para>
An explanation of the spectrogram specific options is given below:
</para>
<variablelist>
<varlistentry>
<term>
<guilabel>Window Size</guilabel>
</term>
<listitem>
<para>
The length of the data over which to take the spectra. The data vector will be subdivided into intervals of this length, and a spectrum will be computed for each one.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>FFT Options</guilabel>
</term>
<listitem>
<para>
These are the options for the individual spectra. See the <link linkend="data-types-spectrum">Spectrum</link> data object for more information. The output quantities are indicated by color scalar.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Color Palette</guilabel>
</term>
<listitem>
<para>This option should be accessed from the <guilabel>Edit Image</guilabel> dialog of the image created from the spectrogram.</para>
<para>
Here you can choose the color scheme to use for the spectrogram. The amplitudes of the spectra are color-coded according to the selected scheme.
</para>
</listitem>
</varlistentry>
</variablelist>
<sect3>
<title>Using the Spectrogram Class in KstScript</title>
<para>The spectrogram can also be generated by using the <link linkend="class_Spectrogram">Spectrogram class</link> in KstScript</para>
<example id="Spectrogram-kstscript-eg">
<title>Example of Using the Spectrogram Class</title>
<para>The vector: <quote>random</quote> generated in <link linkend="kstscript-spectrum-eg">Example 3.6</link> will continue to be used in this example</para>
<programlisting>
s = new Spectrogram("random", 10)//generate a spectrogram using vector <quote>random</quote>
//and specify the frequency to be 10 Hz
s.windowsize = 500; // specify the window size to be 500
s.tagName = "rand_Spectrogram";
</programlisting>
<para>Now, you can check that <quote>rand_Spectrogram</quote> is in the Data Manager. You can view the waterfall plot by making an image from the slave matrix</para>
</example>
</sect3>
</sect2>
<sect2 id="data-types-plugins">
<title>Plugins</title>
<para>
A plugin data object represents a &kst; plugin. All plugins have a common format, and show up as type <quote>Plugin</quote> in the Data Manager. For more information about the available plugins and their options, please see the <link linkend="plugins">Plugins</link> chapter.
</para>
</sect2>
<sect2 id="data-types-eventmonitors">
<title>Event Monitors</title>
<para>
An event monitor data object essentially keeps track of one or more vectors
or scalars, and performs an action when a specified condition involving the vectors and scalars is true.
Event monitors are usually used in conjunction with <quote>live</quote>, or streaming data. For example, an event monitor could be created to monitor whether or not elements of a data vector representing temperature exceed a predefined value.
</para>
<para>
You can create a new event monitor from the <menuchoice><guimenu>Data</guimenu><guimenuitem>New Event Monitor...</guimenuitem></menuchoice>, or by using the <link linkend="data-overview-datamanager">Data Manager</link>.
</para>
<para>
The following is a screenshot of the window displayed when creating or editing events. Explanations of the settings are given below.
</para>
<screenshot>
<screeninfo>Edit Event Monitor</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-eventmonitoredit.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
<variablelist>
<varlistentry>
<term>
<guilabel>Expression</guilabel>
</term>
<listitem>
<para>
The expression to monitor. You may type directly in this textbox, or select items to insert using the operator, vector, and scalar drop-down lists. Ensure that the expression entered in this textbox is a boolean expression (i.e. it evaluates to either true or false). This usually entails using an equality or inequality in the expression. Note that vectors entered in the textbox will be monitored according to their individual elements.
</para>
<para>
Whenever this expression is true (evaluates to non-zero), the event will be triggered. The action taken upon an event trigger depends on the settings specified in the other settings.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Description</guilabel>
</term>
<listitem>
<para>
This textbox is used to store a short description of the event. The description is primarily available as a method for keeping track of multiple events. You can enter any text you wish in this textbox.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>Log as:</guilabel></term>
<listitem>
<para>
Enable this checkbox to have entries added to the &kst; <link linkend="settings-debuglog">debug log</link> when this event is triggered.
There are three types of entries in the debug log — notices, warnings, and errors.
Select the desired type of entry using the corresponding radio button.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>E-Mail Notify</guilabel></term>
<listitem>
<para>
Enable this checkbox to have &kst; send e-mail notifications to the specified address when this
event is triggered. General E-mail configuration options are available in the &kst; <link linkend="settings-globalsettings">Settings</link> dialog.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>ELOG</guilabel></term>
<listitem>
<para>
If the <link linkend="extensions-elog">ELOG Extension</link> is activated, then then Event Monitor can provide notification through ELOG. This can be useful for remotely monitoring live data, or for obtaining summaries of event activity. Please see the <link linkend="extensions-elog">ELOG</link> section for more information.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guilabel>KstScript</guilabel></term>
<listitem>
<para>
If this option is activated, then the Event Monitor can execute a KstScript when it triggers. Please see the <link linkend="extensions-kstscript">KstScript</link> documentation for more information.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect2>
<sect2 id="data-types-matrix">
<title>Matrix</title>
<para>
A matrix represents a set of three-dimensional data points (x, y, z) arranged in a two-dimensional grid.
</para>
<screenshot>
<screeninfo>Matrix Structure</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Diagram-kst-data-matrix.png" format="PNG"/>
</imageobject>
</mediaobject>
</screenshot>
<para>
Just as is the case with the <link linkend="data-types-vectors">vector</link> datatype, there are two kinds of matrix: data and static. <link linkend="data-types-matrix-data">Data matrices</link> are read in from data sources, and <link linkend="data-types-matrix-static">static matrices</link> are generated by &kst;.
</para>
<sect3 id="data-types-matrix-data">
<title>Data Matrix</title>
<para>
The following is a screenshot of the window displayed when editing a data matrix. A new data matrix is created if you choose the <guibutton>Read from data source</guibutton> radio button in the New Matrix dialog.
</para>
<screenshot>
<screeninfo>Edit Data Matrix</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-matrixedit.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
<para>
All of the options which are available for data matrices are analogous to the options for <link linkend="data-types-vectors-data">Data Vectors</link>.
</para>
<variablelist>
<varlistentry>
<term><guilabel>Data Source Parameters</guilabel></term>
<listitem><para>
Any field in ascii file that is going to be used as data source for matrix must have its field name in this format: [MATRIX, <replaceable># of rows, x min, y min, x step size, y step size</replaceable>]. A very <link linkend="mx-source-file">simple example</link> of a matrix source file is provided below.
</para></listitem>
</varlistentry>
<varlistentry>
<term><guilabel>Data Range</guilabel></term>
<listitem>
<para>
This option allows you to use only a portion of the matrix by specifying the range of data in both the x and y directions.
<itemizedlist>
<listitem><para>
<guilabel>X starting frame:</guilabel> the starting index in x dimension of the matrix
</para></listitem>
<listitem><para>
<guilabel>Y starting frame:</guilabel> the starting index in y dimension of the matrix
</para></listitem>
<listitem><para>
<guilabel>X number of frames:</guilabel> the number of data points in x dimension
</para></listitem>
<listitem><para>
<guilabel>Y number of frames:</guilabel> the number of data points in y dimension
</para></listitem>
</itemizedlist>
The <link linkend="extract-mx">simple example</link> below shows how to use this option.
</para>
</listitem>
</varlistentry>
</variablelist>
<sect4 id="mx-source-file">
<title>Example of a Simple Matrix Source File</title>
<informalexample>
<programlisting>
[MATRIX,5,0,0,1,1]
0.47
0.12
0.49
0.97
0.67
0.46
0.64
0.9
0.53
0.13
</programlisting>
</informalexample>
<para>
The resulting matrix would be a 2×5 matrix, and the 10 samples are all included:
<screenshot>
<screeninfo>simple_matrix</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-matrixfile.png" format="PNG" />
</imageobject>
<textobject>
<phrase>simple_matrix</phrase>
</textobject>
</mediaobject>
</screenshot>
</para>
<note><para>
Notice that only the number of rows needs to be specified since Kst can figure out the number of columns automatically in order to have the loaded matrix use the largest number of samples in the original data field.
</para>
<para>
If the field name of the matrix is [MATRIX,6,0,0,1,1], the resulting matrix would have dimension 1×6; only 6 samples would be included.
</para></note>
</sect4>
<sect4 id="extract-mx">
<title>Example of Specifying a portion of matrix</title>
<para>
For the 2×5 matrix in the Screenshot above, if only part of the matrix:from index(1,2) to index(2,4), is needed; that is
</para>
<para>0.12 0.64</para>
<para>0.49 0.9</para>
<para>0.97 0.53 </para>
<para> The <guilabel>Data Range </guilabel> needs to be specified in the following way:</para>
<itemizedlist>
<listitem><para>
<guilabel>X starting frame:</guilabel> 0
</para></listitem>
<listitem><para>
<guilabel>Y starting frame:</guilabel> 1
</para></listitem>
<listitem><para>
<guilabel>X number of frames:</guilabel> 2
</para></listitem>
<listitem><para>
<guilabel>Y number of frames:</guilabel> 3
</para></listitem>
</itemizedlist>
<para>The result is this:</para>
<screenshot>
<screeninfo>extract-matrix</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-matrixrange.png" format="PNG" />
</imageobject>
<textobject>
<phrase>extract-matrix</phrase>
</textobject>
</mediaobject>
</screenshot>
</sect4>
</sect3>
<sect3 id="data-types-matrix-static">
<title>Static Matrix</title>
<para>
The following is a screenshot of the window displayed when editing static matrices. Static matrices allow you to create gradient matrices such as the one shown in the background of the annotation object <link linkend="view-overview-concepts-example">example</link>. A new static matrix is created if you choose the <guibutton>Generate gradient</guibutton> radio button in the <guilabel>New Matrix</guilabel> dialog.
</para>
<screenshot>
<screeninfo>Edit Static Matrix</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-staticmatrixedit.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
<para>
These options allow you to construct a matrix which has a linear gradient in Z along either the X or Y axis.
</para>
<para>
The steps options allow you to set the size of the matrix.
<itemizedlist>
<listitem><para>
<guilabel>X</guilabel> steps indicates column numbers
</para></listitem>
<listitem><para>
<guilabel>Y</guilabel> steps indicates row numbers
</para></listitem>
</itemizedlist>
</para>
<para>
The <guilabel>X</guilabel>/<guilabel>Y minimum</guilabel> and <guilabel>X</guilabel>/<guilabel>Y step size</guilabel> options allow you to set the X and Y ranges.
</para>
</sect3>
<sect3>
<title>Using Matrices through KstScript</title>
<para>Matrices can also be constructed by KstScript. See details in the <link linkend="class_Matrix">Matrix class</link> and the <link linkend="class_DataMatrix">DataMatrix Class</link> of the Script Chapter.
</para>
<example id="kstscript-data-matrix-eg">
<title>Example of Using the DataMatrix Class in KstScript</title>
<programlisting>
s = new DataSource("home/vyiwen/mx.dat")//<filename>mx.dat</filename> is the same matrix source file in the example source file above
m = new DataMatrix(s, "[MATRIX,5,0,0,1,1]")//use <filename>mx.dat</filename> to construct a matrix
m.tagName = "Simple_matrix";//Name the matrix <quote>Simple_matrix</quote> and you should be able to see it in the Data Manager.
//if you want to specify a portion of matrix by specifying the data range in the x, y dimension,
//use the method: change(xStart, yStart, xFrames, yFrames)
m.change(0,1,2,3)//you can see the change in <quote>Simple_matrix</quote>
// it should be the same 2 by 3 matrix
// as in the example of specifying a portion of matrix above.
</programlisting>
</example>
<example>
<title>Example of Using the Matrix Class to construct a Static Matrix</title>
<programlisting>
y = new Matrix(); // to construct a new matrix
y.resize(3,5); //define the size of the matrix 3 by 5 matrix
//i.e 3 rows and 5 columns
for(var i=0; i<5;i++){for(var j=0; j<3;j++){y.setValue(i,j,i+j)}}; //to initialize the matrix
y.tagName="mx"; //you should see matrix <quote>mx</quote> in the Data Manager now
</programlisting>
</example>
</sect3>
</sect2>
<sect2 id="data-types-images">
<title>Images</title>
<para>
Images are graphical representations of <link linkend="data-types-matrix">Matrices</link>. Images can be plotted
as color maps, contour maps, or both.
</para>
<screenshot>
<screeninfo>Image Window</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-imageedit.png" format="PNG" />
</imageobject>
<textobject>
<phrase>Image Window</phrase>
</textobject>
</mediaobject>
</screenshot>
<variablelist>
<varlistentry>
<term>
<guilabel>Matrix Selection</guilabel>
</term>
<listitem>
<para>
Select the matrix to use for this image. New matrices can be created or edited by clicking buttons to the right.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Color Map</guilabel>, <guilabel>Contour Map</guilabel>, and <guilabel>Color Map and Contour Map</guilabel>
</term>
<listitem>
<para>
Select the type of image to be plotted. Changing this selection enables or disables sections of the image dialog
as appropriate. A color map represents the Z value of each cell in the matrix using a color. A contour map plots curves which follow lines of constant Z.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Color palette</guilabel>
</term>
<listitem>
<para>
Select the color palette to use for the color map.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Threshold -</guilabel> <guilabel>Lower</guilabel> and <guilabel>Upper</guilabel>
</term>
<listitem>
<para>
Enter the lower and upper thresholds to use for the mapping of colors. Palette colors are evenly distributed
between the lower and upper thresholds. Any cells in the
selected matrix with z values less than the lower threshold are mapped to the first color in the palette.
Any cells in the selected matrix with z values greater than the upper threshold are mapped
to the last color in the palette.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Max/Min</guilabel>
</term>
<listitem>
<para>
Clicking this button causes the lower and upper threshold values to be set to the current minimum and maximum z values of the selected matrix.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Smart</guilabel> and <guilabel>Percentile</guilabel>
</term>
<listitem>
<para>
Clicking the <guibutton>Smart</guibutton> button causes the lower and upper threshold values to be set such that the percentage of z values in the selected matrix contained between them is equal to the value in the <guilabel>Percentile</guilabel> numberbox.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Real-time auto threshold</guilabel>
</term>
<listitem>
<para>
Selecting this option causes the lower and upper threshold values to be always set to the minimum and maximum of the z values for the selected matrix, even when the matrix updates.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Number of contour levels</guilabel>
</term>
<listitem>
<para>
Select the number of contour levels to use. The contour levels will be distributed evenly between the lowest
and highest z values found in the matrix.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guibutton>Color</guibutton>
</term>
<listitem>
<para>
Select the color to use for the contour lines. Clicking this button displays a standard &kde; color chooser
dialog.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Weight</guilabel> and <guilabel>Use variable line weight</guilabel>
</term>
<listitem>
<para>
Select the <guilabel>Weight</guilabel>, or <quote>thickness</quote> of the contour lines. If <guilabel>Use variable line weight</guilabel>
is selected, contour lines representing higher elevations are drawn thicker than those representing lower elevations.
Use discretion when selecting this option, as images with high contour line densities may become unreadable.
</para>
</listitem>
</varlistentry>
</variablelist>
<sect3>
<title>The Image Class in KstScript</title>
<para>See the <link linkend="class_Image">Image Class</link> of the Script Chapter for detail.</para>
</sect3>
</sect2>
<sect2 id="data-types-vector-view">
<title>
Vector View
</title>
<para>
Vector view provides a way to view a portion of the data plot. It excludes any data point which is not in the range of the X/Y limits specified by the user.
</para>
<screenshot>
<screeninfo>
Edit Vector View
</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-vectorviewedit.png" format="PNG"/>
</imageobject>
<textobject>
<phrase>
Edit Vector View
</phrase>
</textobject>
</mediaobject>
</screenshot>
<variablelist>
<varlistentry>
<term>
<guilabel>
Input Vectors
</guilabel>
</term>
<listitem>
<para>
Select an input X/Y axis vector from existing vectors or create one by using the shortcut icons.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>
View Ranges
</guilabel>
</term>
<listitem>
<para>
Adjust the view range of the plot by selecting <guilabel>X min/Y min</guilabel> and <guilabel>X max/Y max</guilabel> values for the input vectors.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>
Flag Vector
</guilabel>
</term>
<listitem>
<para>
This option is used to exclude specific points in the input vector for the vector view. The indices of non-zero points in the flag vector indicate that the data points of the input vector with the same indices should be filtered out for its vector view.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect2>
</sect1>
<sect1 id="data-tools">
<title>Data Tools</title>
<sect2 id="data-tools-wizard">
<title>The Data Wizard</title>
<para>
The Data Wizard provides a graphical quick and easy way of importing data into &kst;- automatically creating vectors, curves, power spectra, and plots. To launch the wizard, select <menuchoice><guimenu>Tools</guimenu><guimenuitem>Data Wizard</guimenuitem></menuchoice> or click on the icon
<inlinemediaobject>
<imageobject>
<imagedata fileref="Icon-kst-datawizard.png" format="PNG" />
</imageobject>
<textobject>
<phrase>data wizard icon</phrase>
</textobject>
</inlinemediaobject>
in the toolbar. The <guilabel>Data Source</guilabel> pane will appear as following and you will need to select a data source file.
</para>
<screenshot>
<screeninfo>data source</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-wizard-datasource.png" format="PNG" />
</imageobject>
<textobject>
<phrase>data source</phrase>
</textobject>
</mediaobject>
</screenshot>
<sect3 id="kst-datatools-configure-data-source">
<title>Configuring Data Source</title>
<para>
After selecting the data source file, you may need to configure the file by clicking on the <guibutton>Configure...</guibutton> button.
For information on how to use the <guilabel>Configure Data Source</guilabel> dialog, see the <link linkend="data-sources-ascii">ASCII datasource</link> section.
</para>
<para>
After configuring the data source file, click on the <guibutton>OK</guibutton> button to close the <guilabel>Configure Data Source</guilabel> dialog, and click on the <guibutton>Next</guibutton> button in the <guilabel>Data Source</guilabel> pane. The <guilabel>Select Data</guilabel> pane will appear as shown below.
</para>
</sect3>
<sect3 id="data-tools-wizard-selectdatafile">
<title>Selecting Data</title>
<screenshot>
<screeninfo>Data Wizard Select Data Dialog</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-wizard-selectdata.png" format="PNG" />
</imageobject>
<textobject>
<phrase>Data Wizard Select Data Dialog</phrase>
</textobject>
</mediaobject>
</screenshot>
<para>
Here you can select the fields you wish to import into &kst;. The right and left arrow buttons allow you to move data vectors back and forth between the <guilabel>Available data</guilabel> and the <guilabel>Selected data</guilabel> lists. The up and down arrow buttons allow you to move the positions of the selected vectors up and down in the <guilabel>Selected data</guilabel> list. You may filter the list of fields by entering a string to match (wildcards such as <literal>*</literal> are supported) into the text box above the list. Click the <guibutton>Next</guibutton> button to advance to the <guilabel>Data Presentation</guilabel> pane.
</para>
</sect3>
<sect3 id="data-tools-wizard-presentation">
<title>Presentation</title>
<screenshot>
<screeninfo>Data Wizard Data Presentation Pane</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-wizard-presentation.png" format="PNG" />
</imageobject>
<textobject>
<phrase>Data Wizard Data Presentation Pane</phrase>
</textobject>
</mediaobject>
</screenshot>
<anchor id="data-tools-wizard-range"></anchor>
<para>
The <guilabel>Data Range</guilabel> section is used to specify the range of data to read from the selected vectors in the input file. For information about these parameters, read the description of the <link linkend="data-types-vectors">Vector</link> data object.
</para>
<para>
Power Spectrum and X axis settings can be specified within the <guilabel>Plot Types</guilabel> section.
These settings are described below.
</para>
<variablelist>
<varlistentry>
<term>
<guilabel>XY</guilabel>, <guilabel>Power Spectrum</guilabel>, and <guilabel>XY and Power Spectrum</guilabel>
</term>
<listitem>
<para>
Select whether to plot the data (<guilabel>XY</guilabel>) only, <guilabel>Power spectrum</guilabel> only, or both. If the power spectrum is selected for
plotting, additional settings in this section become available.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>X Axis Vector</guilabel>
</term>
<listitem>
<para>
The vector to be used as the independent vector for the plots. You may select a field from your
data file, or the INDEX vector. The INDEX
vector is simply a vector containing elements from 0 to N-1, where N is the number of frames in the
data file.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
The <guilabel>FFT Options</guilabel> subsection
in the <guilabel>Plot Types</guilabel> section is available only if a power spectrum
is to be plotted. Detailed information on these options can be found in the <link linkend="data-types-spectrum">Spectrum</link> section.
</para>
<para>
Once you are satisfied with all the settings, click the <guibutton>Next</guibutton> button to advance to the final
window.
</para>
</sect3>
<sect3 id="data-tools-wizard-configureplotlayout">
<title>Configure Plot Layout</title>
<screenshot>
<screeninfo>Data Wizard Configure Plot Layout</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-wizard-configurelayout.png" format="PNG" />
</imageobject>
<textobject>
<phrase>Data Wizard Configure Plot Layout</phrase>
</textobject>
</mediaobject>
</screenshot>
<para>
From here, you can change general plot settings. Most of the settings are self-explanatory.
A brief overview of each section is provided below.
</para>
<variablelist>
<varlistentry>
<term>
<guilabel>Curve Style</guilabel>
</term>
<listitem>
<para>
Select whether to plot data points only, lines connecting the data points only, or both.
By default, lines are continuous with weight 0, and data points are marked using crosses.
Line and data point colors are chosen so that curves with identical colors are minimized.
Note that the curve style settings apply to both Power Spectra
and XY plots.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Curve Placement</guilabel>
</term>
<listitem>
<para>
Select the plots to place the new curves on. <guilabel>Cycle through</guilabel> distributes the
curves on the plots by cycling through the plots. </para>
<para>For example, if you have 4 curves, and you choose to cycle through 2 plots. The result will be that curve 1 is placed in plot 1; curve 2 is in plot 2, and this is the first plot cycle. Curve 3 is back to plot 1; curve 4 goes to plot 2, and this is the second cycle. Note that the curve placement settings apply
to both Power Spectra and XY plots.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Label Generation</guilabel>
</term>
<listitem>
<para>
Select the desired labels and legends to be placed on the plots.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Plot Placement</guilabel>
</term>
<listitem>
<para>
Select the desired window(s) to place the new plots in. New windows can be created for the plots by selecting
<guilabel>In new window</guilabel>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<guilabel>Power Spectrum Axis Mode</guilabel>
</term>
<listitem>
<para>
Select these check-boxes if you would like the automatically created power spectra to have logarithmic X or Y axes.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
Once you are satisfied with all the settings, click the <guibutton>Finish</guibutton> button and the plots will be generated.
</para>
</sect3>
</sect2>
<sect2 id="data-tools-datamenu">
<title>The Data Menu</title>
<para>
The <guimenu>Data</guimenu> menu provides quick access to many features related to data objects in
&kst;. Most of the menu functions duplicate functions found elsewhere, so only brief descriptions are
provided below.
</para>
<variablelist>
<varlistentry>
<term><guimenuitem>Reload</guimenuitem></term>
<listitem>
<para>
Reloads all data vectors from their source files.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guimenuitem>Data Manager</guimenuitem></term>
<listitem>
<para>
Displays the <link linkend="data-overview-datamanager">Data Manager</link>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guimenuitem>View Manager</guimenuitem></term>
<listitem>
<para>
Displays the <link linkend="view-overview-viewmanager">View Manager</link>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><guimenuitem>New</guimenuitem> [data object]</term>
<listitem>
<para>
Displays the corresponding dialog for creating the data object. Refer to
<link linkend="data-types">Data Types</link> for descriptions of
each dialog.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term id="data-tool-view-scalar-vlaues"><guimenuitem>View Scalar Values</guimenuitem></term>
<listitem>
<para>
Displays a dialog from which the values of all the scalars in the current &kst; session
can be viewed. The values are displayed hierarchically, based on which higher level data objects the scalars are determined for. The dialog is updated dynamically if the values change.
</para>
<screenshot>
<screeninfo>View Scalar Values</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-viewscalarvalues.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
</listitem>
</varlistentry>
<varlistentry>
<term id="data-tool-view-vector-vlaues"><guimenuitem>View Vector Values</guimenuitem></term>
<listitem>
<para>
Displays a dialog from which the values in all the current vectors can be viewed. Select
a vector to view using the drop-down list. The dialog is updated dynamically if the values change.
</para>
<screenshot>
<screeninfo>View Vector Values</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-viewvectorvalues.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
</listitem>
</varlistentry>
<varlistentry>
<term><guimenuitem>View Matrix Values</guimenuitem></term>
<listitem>
<para>
Displays a dialog from which the values in all the current matrices can be viewed. Select
a matrix to view using the drop-down list. The dialog
is updated dynamically if the values change.
</para>
<screenshot>
<screeninfo>View Vector Values</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-viewmatrixvalues.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
</listitem>
</varlistentry>
<varlistentry>
<term><guimenuitem>View Fit Results</guimenuitem></term>
<listitem>
<para>
Displays a dialog which shows all the resulting values from fit plugins. Select
a fit result to view using the drop-down list. The dialog
is updated dynamically if the values change.
</para>
<screenshot>
<screeninfo>Curve Fitting - Fit Results</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-commontasks-curvefitting-fitresults.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
</listitem>
</varlistentry>
</variablelist>
</sect2>
<sect2 id="data-tools-datamode">
<title>Data View Mode</title>
<para>Data mode allows precise viewing of the data points used in a plotted data object. To toggle data mode,
select <menuchoice><guimenu>Plots</guimenu><guimenuitem>Data Mode</guimenuitem></menuchoice>, or click the <inlinemediaobject><imageobject><imagedata fileref="Icon-kst-datamode.png" format="PNG"/></imageobject></inlinemediaobject> Data Mode icon on the <link linkend="view-tools-toolbar">Toolbar</link>. Now, when the cursor
is moved over a plot, a colored dot will indicate the closest data point to the cursor, as shown in the
screenshot below. The status bar will
display the coordinates of the data point (in terms of the x and y vectors used to plot the data object) in the status bar at the
lower left corner of the &kst; window.
The status bar will also display the x, y, and z coordinates of any visible image. If images overlap, only the coordinates of the
topmost image will be displayed.
Note that all zooming functions are still available while in data mode.
</para>
<screenshot>
<screeninfo>Data Mode</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-datamode.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
<tip>
<para>
If the status bar is not visible, select <menuchoice><guimenu>Settings</guimenu><guimenuitem>Show Statusbar</guimenuitem></menuchoice>
</para>
</tip>
</sect2>
<sect2 id="data-tools-changedatafile">
<title>Change Data File</title>
<para>
One common mode of using &kst; is to create a <quote>session</quote> of plots and analysis for a particular type of data. Then, the same plots and analysis can be performed easily on another set of data simply by changing the data source file, provided that the two data files are similar. For example, for ASCII data files, similar data files need to have same number of fields.</para>
<para>
The Change Data File dialog can be accessed by choosing <menuchoice><guimenu>Tools</guimenu><guimenuitem>Change Data File</guimenuitem></menuchoice> or clicking on the icon
<inlinemediaobject>
<imageobject>
<imagedata fileref="Icon-kst-changedatafile.png" format="PNG" />
</imageobject>
<textobject>
<phrase>icon changedatafile</phrase>
</textobject>
</inlinemediaobject>
in the toolbar. The dialog which appears is shown below:
</para>
<para>
<inlinemediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-changedatafile.png" format="PNG"/>
</imageobject>
</inlinemediaobject>
</para>
<para>
To use this dialog, select the vectors which you would like to change the data file of using the list in the middle of the dialog. The <guilabel>All From</guilabel>, <guilabel>Clear</guilabel>, <guilabel>Select All</guilabel>, and search field can help you to do this. The use of wildcards is supported in the search field.</para>
<para>
Then, select a new data file. If the <guilabel>Change selected vectors and matrices</guilabel> radio button is selected, then when you press the <guibutton>Apply</guibutton> or <guibutton>OK</guibutton> button, the plots of the selected data vectors will be replaced by the data vectors loaded from the same fields of the new data file that you specified.</para>
<para>
If the <guilabel>Duplicate selected vectors and matrices</guilabel> radio button is selected, vectors coming from the new file will be plotted together with the existing vectors in exactly the same way.
</para>
<para>
If some of the selected vectors have dependents (curves, spectra, histograms, etc.) then the <guilabel>Duplicate dependents</guilabel> option allows you to duplicate these objects as well.
</para>
</sect2>
<sect2 id="data-tools-changesampleranges">
<title>Change Data Sample Ranges</title>
<para>
The <guimenuitem>Change Data Sample Ranges</guimenuitem> dialog can be accessed through the <guimenu>Tools</guimenu> menu or the icon
<inlinemediaobject>
<imageobject>
<imagedata fileref="Icon-kst-changedatarange.png" format="PNG" />
</imageobject>
<textobject>
<phrase>icon_datarange</phrase>
</textobject>
</inlinemediaobject>
in the toolbar. It allows you to modify how a set of vectors are read in from their associated files. The options are detailed in the <guilabel>Data Range</guilabel> option of the <link linkend="data-types-vectors-data">Data Vectors</link> documentation.
</para>
<para>
<inlinemediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-changesampleranges.png" format="PNG"/>
</imageobject>
</inlinemediaobject>
</para>
</sect2>
<sect2 id="common-icons">
<title>Common Icons</title>
<para>
As mentioned above, most data tools have corresponding icons in the <link linkend="view-tools-toolbar">toolbar</link>. There are also some useful icons commonly seen in various dialogs.
</para>
<variablelist>
<varlistentry>
<term>
<inlinemediaobject>
<imageobject>
<imagedata fileref="Icon-kst-edit-creatvec.png" format="PNG" />
</imageobject>
<textobject>
<phrase>create vector icon</phrase>
</textobject>
</inlinemediaobject>
</term>
<listitem><para>
This icon opens the New Vector dialog, the same as choosing <menuchoice><guilabel>Data</guilabel><guimenuitem>New Vector...</guimenuitem></menuchoice>
</para></listitem>
</varlistentry>
<varlistentry>
<term>
<inlinemediaobject>
<imageobject>
<imagedata fileref="Icon-kst-edit-editvec.png" format="PNG" />
</imageobject>
<textobject>
<phrase>edit vector icon</phrase>
</textobject>
</inlinemediaobject>
</term>
<listitem><para>
This icon opens the <link linkend="data-types-vectors-data">Edit Vector</link> dialog where you can edit the properties of the currently selected vector.
</para></listitem>
</varlistentry>
<varlistentry>
<term>
<inlinemediaobject>
<imageobject>
<imagedata fileref="Icon-kst-edit-createmx.png" format="PNG" />
</imageobject>
<textobject>
<phrase>create mx</phrase>
</textobject>
</inlinemediaobject>
</term>
<listitem><para>
This icon opens the New Matrix dialog, the same as choosing <menuchoice><guimenu>Data</guimenu><guimenuitem>New Matrix...</guimenuitem></menuchoice>
</para></listitem>
</varlistentry>
<varlistentry>
<term>
<inlinemediaobject>
<imageobject>
<imagedata fileref="Icon-kst-edit-editmx.png" format="PNG" />
</imageobject>
<textobject>
<phrase>edit matrix icon</phrase>
</textobject>
</inlinemediaobject>
</term>
<listitem><para>
This icon opens the <link linkend="data-types-matrix-data">Edit Matrix</link> dialog where you can edit the properties of the currently selected matrix.
</para></listitem>
</varlistentry>
<varlistentry>
<term>
<inlinemediaobject>
<imageobject>
<imagedata fileref="Icon-kst-edit-clear.png" format="PNG" />
</imageobject>
<textobject>
<phrase>clear icon</phrase>
</textobject>
</inlinemediaobject>
</term>
<listitem>
<para>
This icon is commonly seen in the search field of many dialogs used to list data objects (vectors, curves, etc). For example, it appears in the <link linkend="data-overview-datamanager">Data Manager</link>, <link linkend="data-tool-view-scalar-vlaues">View Scalar Values</link>, and <link linkend="data-tools-changedatafile">Change Data File</link>, etc.
</para>
<para>
It is used to quickly clear the textbox of the <guilabel>Search</guilabel> field and restore the original list of data objects.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<inlinemediaobject>
<imageobject>
<imagedata fileref="Icon-kst-edit-createscalar.png" format="PNG" />
</imageobject>
<textobject>
<phrase>create scalar icon</phrase>
</textobject>
</inlinemediaobject>
</term>
<listitem><para>
This icon opens the New Scalar dialog as shown below:
<screenshot>
<screeninfo>new scalar</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-newscalar.png" format="PNG" />
</imageobject>
<textobject>
<phrase>new scalar</phrase>
</textobject>
</mediaobject>
</screenshot>
</para>
<para>
Here, you can define your own scalar by typing its name and associated value.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<inlinemediaobject>
<imageobject>
<imagedata fileref="Icon-kst-edit-editscalar.png" format="PNG" />
</imageobject>
<textobject>
<phrase>edit scalar icon</phrase>
</textobject>
</inlinemediaobject>
</term>
<listitem><para>
This icon is used to change values of user-defined scalars. Essentially, it opens the same dialog as shown above.
</para></listitem>
</varlistentry>
</variablelist>
</sect2>
</sect1>
<sect1 id="data-sources">
<title>Datasources</title>
<para>
&kst; uses externally linked datasources to read from data files. This means that new data formats can be transparently added to the &kst; repertoire.
Currently, the &kst; distribution includes the following default datasources:
<itemizedlist>
<listitem><para><link linkend="data-sources-ascii">ASCII</link></para></listitem>
<listitem><para><link linkend="data-sources-dirfiles">Dirfiles</link></para></listitem>
<listitem><para><link linkend="data-sources-healpix">HEALPix</link></para></listitem>
<listitem><para><link linkend="data-sources-qimage">QImage</link></para></listitem>
<listitem><para><link linkend="data-sources-indirect">Indirect</link></para></listitem>
<listitem><para><link linkend="data-sources-cdf">CDF</link></para></listitem>
<listitem><para><link linkend="data-sources-netcdf">netCDF</link></para></listitem>
<listitem><para><link linkend="data-sources-lfiio">LFIIO</link></para></listitem>
<listitem><para><link linkend="data-sources-scuba">SCUBA</link></para></listitem>
</itemizedlist>
If you have a data format which you would like &kst; to read, it's easy to <link linkend="supportingadditionalfileformats">create</link> your own.
</para>
<para>
All of the data sources read from files using KIO, which is usually able to load data through common file compression formats (e.g. BZIP2, GZIP, ZIP, TAR) and most popular networking protocols (e.g. HTTP, FTP, SFTP, SMB). For more information on which formats are supported on your system type <userinput><command>kinfocenter</command></userinput> at the command line and inspect the protocols tab. If &kst; detects that you have typed a URL into the filename of a vector or matrix, for example, a <quote><guibutton>Connect</guibutton></quote> button will appear. Clicking the <guibutton>Connect</guibutton> button instructs &kst; to load the indicated data, and channel it to a suitable data source.
</para>
<anchor id="data-sources-concepts"></anchor>
<para>
The following concepts are important in understanding how &kst; works with
data sources. Some terminology is also introduced in this section.
</para>
<variablelist>
<varlistentry>
<term>Samples</term>
<listitem>
<para>
A sample is considered to be the fundamental unit with respect to data files.
Each sample consists of one data value in the file. Note, however, that one sample
may not correspond to one value in a data vector in &kst;, due to the concept of
<link linkend="data-sources-concepts-frames">frames</link>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Fields</term>
<listitem>
<para>
A field usually corresponds to one vector in &kst;. For example, a column in an
ASCII data file is considered to be one field. Each field can have an explicit
or implicit name. Datasource readers provide functions for reading and obtaining
fields and field names.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term id="data-sources-concepts-frames">Frames</term>
<listitem>
<para>
A frame corresponds to a set number of samples, and each field in a data file
can have its own ratio of samples per frame. The size of a data file
is measured by the number of frames it contains. For ASCII data files, the number of samples
per frame is 1, but for some data files, there may be multiple samples per frame. In the below
illustration, the first 3 frames of an imaginary data file are shown. In this particular data
file, Field 1 has a ratio of 5 samples per frame, Field 2 has a ratio of 2 samples per frame,
Field 3 has a ratio of 3 samples per frame, and Field 4 has a ratio of 1 sample per frame.
</para>
<para>
<inlinemediaobject>
<imageobject>
<imagedata fileref="Diagram-kst-data-framesconcept.png" format="PNG"/>
</imageobject>
</inlinemediaobject>
</para>
<para>
Depending on the specific data vector settings in &kst;, data from files may be read as frames instead of samples, with either the first sample in a frame or the mean of all the samples in a frame representing the value of the frame. This capability can be useful for reducing extremely large data-sets to a more manageable size.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term id="data-sources-concepts-index">INDEX field</term>
<listitem>
<para>
Some data files may not have a field that represents the x axis of a plot. However, &kst;
implicitly creates an INDEX field for all data sources. The INDEX field simply contains
integers from 0 to N-1, where N is the number of frames in the data file.
</para>
</listitem>
</varlistentry>
</variablelist>
<sect2 id="data-sources-ascii">
<title>ASCII</title>
<para>
The simplest input file format is the ASCII text file. These files are usually human-readable and
can be easily created by hand or simple scripts if desired. The following is an example of an ASCII input file.
</para>
<informalexample>
<screen>
112.5 3776 428
187.5 5380 429
262.5 5245 345
337.5 2942 184
412.5 1861 119
487.5 2424 138
567.5 2520 162
637.5 1868 144
712.5 1736 211
787.5 1736 211
862.5 2172 292
937.5 1174 377
1000.5 499 623
</screen>
</informalexample>
<screenshot>
<screeninfo>Datasources - Configure ASCII</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-asciiconfig.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
<variablelist>
<varlistentry>
<term>Comment Indicators</term>
<listitem>
<para>
By default, commented lines in ASCII files start with one of the characters in this set
<literal>{#, !, /, ;, c}</literal>. All commented lines and empty lines
are ignored by &kst;. Valid numbers include those with decimals, negative signs,
or <literal>e</literal>, to indicate scientific notation. Invalid numbers (such as
English words) are replaced with 0 by &kst;.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Interpret ... As ...</term>
<listitem>
<para>
See the <link linkend="view-types-plot-interpretas">corresponding section</link> for the Plot Dialog.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Always accept files matching:</term>
<listitem>
<para>
Here you can enter an expression which you want to designate ASCII data files, in case they are being claimed by another data source.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Data starts at line:</term>
<listitem>
<para>
Data before this line will be ignored. Setting this value to zero ensures that all of the data will be read.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Read field names from line:</term>
<listitem>
<para>
If this option is selected, the field names for the file's vectors will be read from the specified line. If the columns do not have labels, field names are assigned by &kst; based on the
order of the columns (with the leftmost column having a field name of <literal>1</literal>).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Data format</term>
<listitem>
<para>
Choose the option which describes how the data is delimited (how the columns and rows are separated). Each column of this file represents a field, while each row represents one frame.
Columns are typically separated by tabs or spaces, and rows are separated by carriage returns.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect2>
<sect2 id="data-sources-dirfiles">
<title>Dirfiles</title>
<para>
Dirfiles are a very simple data source designed to be read efficiently by &kst;. The data
source is a directory which contains each raw field in its own binary file. Because the data
for a given field are stored sequentially on disk in a binary format, reading is very efficient,
typically limited by hard disk speed. Fixed and floating point binary formats are supported, as
well as calibrations, bit fields and linear combinations of channels.
</para>
<para>
The directory contains one file for each field.
These files contain only the raw samples, written sequentially. As in the rest of &kst;, each field in a
datasource must have the
same number of frames, but each field may have a different (fixed) number of samples per frame. Formatting
and calibration information is contained in a file in the Dirfile called
<filename>format</filename>, which lists each field and its properties. Below is example of such a file: In
this file, a <quote>#</quote> specifies a comment, and for all fields, the first column specifies the field name.
</para>
<informalexample>
<screen>
##### Raw fields ####
# data types:
# c: 8 bit signed
# u: 16 bit unsigned. U: 32 bit unsigned
# s: 16 bit signed S: 32 bit signed
# f: 32 bit float d: 64 bit float
# The fieldname is also the name of the raw binary file
# holding the data.
# Fieldname RAW datatype samples/frame
sensor1 RAW s 1
sensor2 RAW U 20
sensor3 RAW c 1
#
#### derived fields ####
# LINCOM fields: (F_out = F1_in*m+b [+ F2_in*m+b [+ ...]])
# Fieldname LINCOM N Field1_in gain offset
S1V LINCOM 1 sensor1 1.52587890625E-4 0.00
#
# LINTERP Fields: Calibrate using an ascii lookup table.
# lookup table format:
# 2 whitespace separated columns
# The first column is raw values, and the second is
# a corresponding calibrated value. The file must be
# sorted by column 1.
# The table is linearly interpolated to the value of
# Field_in.
# Fieldname LINTERP Field_in Calibration_file
S2K LINTERP sensor2 /data/etc/calibration/C2K.cal
#
# BIT values: the value of 1 bit of raw data: 0 or 1
# Fieldname BIT Field_in bit_num (0 to 7 for chars)
RELAY1 BIT sensor3 0
RELAY2 BIT sensor3 1
#
# The next line includes the contents of another format file
# into this one. Any fields referred to in this file will be
# looked for in ../work/, not in ./, so this is essentially
# including an entire other datasource.
INCLUDE ../work/format
</screen>
</informalexample>
<para>
The following code fragment (which foolishly ignores all pretense at error
checking) could be used to create this data source...
</para>
<informalexample>
<screen>
char bits[1000];
short s1[1000];
unsigned int s2[20000];
int fp;
......
/* fill bits, s1, and s2 with data...*/
......
fp = open("/data/example/sensor3", O_WRONLY|O_CREAT, 00644);
write(fp, bits, 1000*sizeof(char));
close(fp);
fp = open("/data/example/sensor2", O_WRONLY|O_CREAT, 00644);
write(fp, bits, 1000*20*sizeof(unsigned int));
close(fp);
fp = open("/data/example/sensor1", O_WRONLY|O_CREAT, 00644);
write(fp, bits, 1000*sizeof(short));
close(fp);
/* create the ascii file /data/example/format, listed above */
/* create the cal file, /data/etc/calibration/C2K.cal,
described above. */
......
</screen>
</informalexample>
<para>
Dirfiles can be used for real time data, but as each field is written separately, this
requires some extra care. First of all, the writing application should avoid all buffering
or caching - raw writes as in the above example work fine. Secondly, the order in which the
fields are written need to be considered: &kst; determines the number of frames available in
the data source from the size of the file containing the first RAW field encountered in the
format file. So the first RAW field in <filename>format</filename> needs to be the last one
written, as in the above example. If the data source is being read over NFS, attribute caching
needs to be turned off.
</para>
<para>
When selecting a dirfile for use in &kst;, the directory containing the field files should be
selected. &kst; will then automatically look for a <filename>format</filename> file, if it exists,
to determine the fields and their properties.
</para>
</sect2>
<sect2 id="data-sources-healpix">
<title>HEALPix FITS files</title>
<para>
HEALPix is a pixelisation scheme for the sphere. More information can be found at <ulink url="http://healpix.jpl.nasa.gov/">http://healpix.jpl.nasa.gov/</ulink>. HEALPix data is typically be loaded into &kst; as a matrix. For this data source to work you must have the CFITSIO libraries installed. An image of a sample HEALPix matrix is shown below:
</para>
<screenshot>
<screeninfo>HEALPix example</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-healpixexample.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
<para>
Here is the configuration dialog for the HEALPix data source, an explanation of the possible options is given underneath:
</para>
<screenshot>
<screeninfo>HEALPix example</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-healpixconfig.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
<variablelist>
<varlistentry>
<term>Dimensions</term>
<listitem>
<para>
The size of the matrix to create based on the HEALPix data. The larger the matrix, the higher the resolution. If you choose dimensions which are larger than the resolution of the data in the HEALPix file, you will be able to see the shape of the HEALPix pixels in the image of the matrix.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Theta / Phi Range</term>
<listitem>
<para>
The colatitude / azimuth range of data to extract.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Theta / Phi Range</term>
<listitem>
<para>
The colatitude / azimuth range of data to extract.
</para>
</listitem>
</varlistentry>
<!--FIXME: how do the polarization options work? -->
</variablelist>
</sect2>
<sect2 id="data-sources-qimage">
<title>QImage</title>
<para>
The QImage datasource allows you to read image data into &kst; matrices from a variety of formats. The collection of supported formats depends on the libraries installed on your system, but most popular formats are generally readable. If you'd just like to add a picutre to a plot, then the <link linkend="view-types-pictures">Picture</link> annotation object is a better choice to use. An example of an image plotted from a QImage sourced matrix is shown below:
</para>
<screenshot>
<screeninfo>QImage example</screeninfo>
<mediaobject>
<imageobject>
<imagedata fileref="Screenshot-kst-data-qimageexample.png" format="PNG" />
</imageobject>
</mediaobject>
</screenshot>
</sect2>
<sect2 id="data-sources-indirect">
<title>Indirect</title>
<para>
This is a <quote>helper</quote> data source, which allows you to access data through a file pointer. If you put the URL or path of a data file on the first line of a text file with the extension <filename class="extension">.cur</filename>, then the Indirect data source is used to load the referenced file. The use of this is that the contents of the <filename class="extension">.cur</filename> file can then be externally modified to change the data file which is loaded by &kst;.
</para>
</sect2>
<sect2 id="data-sources-cdf">
<title>CDF</title>
<para>
The <ulink url="http://cdf.gsfc.nasa.gov/">Common Data Format</ulink> (CDF) was developed by NASA. It is described as <quote>a self-describing data format for the storage and manipulation of scalar and multidimensional data in a platform- and discipline-independent fashion.</quote>. This means that it is a generalized file format for storing and retrieving data, similar to FITS. For this datasource to work you must have the CDF libraries installed.
</para>
</sect2>
<sect2 id="data-sources-netcdf">
<title>netCDF</title>
<para>
The <ulink url="http://www.unidata.ucar.edu/software/netcdf/">network Common Data Form</ulink> (netCDF) was developed by <ulink url="http://www.unidata.ucar.edu/">Unidata</ulink>. They describe it as <quote>a set of interfaces for array-oriented data access and a freely-distributed collection of data access libraries for C, Fortran, C++, Java, and other languages. The netCDF libraries support a machine-independent format for representing scientific data. Together, the interfaces, libraries, and format support the creation, access, and sharing of scientific data.</quote>. In practice, it is similar to the FITS and CDF formats. For this datasource to work you must have the netCDF libraries installed.
</para>
</sect2>
<sect2 id="data-sources-lfiio">
<title>LFIIO</title>
<para>
The LFIIO datasource reads FITS format data files created by the LFI component of the <ulink url="http://www.rssd.esa.int/Planck">Planck</ulink> satellite. It is currently the default reader for FITS files in &kst;. For this datasource to work you must have CFITSIO installed.
</para>
</sect2>
<sect2 id="data-sources-wmap">
<title>WMAP Reader</title>
<para>
This datasource allows you to load vectors from WMAP Time Ordered Data (TOD) files. For more information, see <ulink url="http://lambda.gsfc.nasa.gov/">http://lambda.gsfc.nasa.gov/</ulink>. For this datasource to work you must have CFITSIO installed.
</para>
</sect2>
<sect2 id="data-sources-scuba">
<title>SCUBA File Reader</title>
<para>
This datasource allows you to read the datafiles produced for <ulink url="http://www.jach.hawaii.edu/JCMT/continuum/">SCUBA</ulink>, an instrument on the JCMT telescope in Hawaii.
</para>
</sect2>
</sect1>
</chapter>
<!-- Keep this comment at the end of the file
Local variables:
mode: xml
sgml-omittag:nil
sgml-shorttag:nil
sgml-namecase-general:nil
sgml-general-insert-case:lower
sgml-minimize-attributes:nil
sgml-always-quote-attributes:t
sgml-indent-step:0
sgml-indent-data:true
sgml-parent-document:("index.docbook" "book" "chapter")
sgml-exposed-tags:nil
sgml-local-catalogs:nil
sgml-local-ecat-files:nil
End:
-->
