<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Tutorials — PyTables 3.0.0 documentation</title>
<link rel="stylesheet" href="../_static/cloud.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '3.0.0',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<script type="text/javascript" src="../_static/jquery.cookie.js"></script>
<script type="text/javascript" src="../_static/toggle_sections.js"></script>
<script type="text/javascript" src="../_static/toggle_sidebar.js"></script>
<link rel="shortcut icon" href="../_static/favicon.ico"/>
<link rel="top" title="PyTables 3.0.0 documentation" href="../index.html" />
<link rel="up" title="PyTables User’s Guide" href="index.html" />
<link rel="next" title="Library Reference" href="libref.html" />
<link rel="prev" title="Installation" href="installation.html" />
</head>
<body>
<div class="relbar-top">
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> </li>
<li class="right" >
<a href="../np-modindex.html" title="Python Module Index"
>modules</a> </li>
<li class="right" >
<a href="libref.html" title="Library Reference"
accesskey="N">next</a> </li>
<li class="right" >
<a href="installation.html" title="Installation"
accesskey="P">previous</a> </li>
<li><a href="../index.html">PyTables 3.0.0 documentation</a> »</li>
<li><a href="index.html" accesskey="U">PyTables User’s Guide</a> »</li>
</ul>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="tutorials">
<h1>Tutorials<a class="headerlink" href="#tutorials" title="Permalink to this headline">¶</a></h1>
<blockquote class="epigraph">
<div><p>Seràs la clau que obre tots els panys,
seràs la llum, la llum il.limitada,
seràs confí on l’aurora comença,
seràs forment, escala il.luminada!</p>
<p class="attribution">—Lyrics: Vicent Andrés i Estellés. Music: Ovidi Montllor, Toti Soler, M’aclame a tu</p>
</div></blockquote>
<p>This chapter consists of a series of simple yet comprehensive
tutorials that will enable you to understand PyTables’ main features. If
you would like more information about some particular instance variable,
global function, or method, look at the doc strings or go to the library
reference in <a class="reference internal" href="libref.html#library-reference"><em>Library Reference</em></a>. If you are reading this in PDF or HTML
formats, follow the corresponding hyperlink near each newly introduced
entity.</p>
<p>Please note that throughout this document the terms
<em>column</em> and <em>field</em> will be used
interchangeably, as will the terms <em>row</em> and
<em>record</em>.</p>
<div class="section" id="getting-started">
<h2>Getting started<a class="headerlink" href="#getting-started" title="Permalink to this headline">¶</a></h2>
<p>In this section, we will see how to define our own records in Python and save
collections of them (i.e. a <em>table</em>) into a file. Then we will select some of
the data in the table using Python cuts and create NumPy arrays to store this
selection as separate objects in a tree.</p>
<p>In <em>examples/tutorial1-1.py</em> you will find the working version of all the
code in this section. Nonetheless, this tutorial series has been written to
allow you reproduce it in a Python interactive console. I encourage you to do
parallel testing and inspect the created objects (variables, docs, children
objects, etc.) during the course of the tutorial!</p>
<div class="section" id="importing-tables-objects">
<h3>Importing tables objects<a class="headerlink" href="#importing-tables-objects" title="Permalink to this headline">¶</a></h3>
<p>Before starting you need to import the public objects in the tables package.
You normally do that by executing:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">import</span> <span class="nn">tables</span>
</pre></div>
</div>
<p>This is the recommended way to import tables if you don’t want to pollute
your namespace. However, PyTables has a contained set of first-level
primitives, so you may consider using the alternative:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">tables</span> <span class="kn">import</span> <span class="o">*</span>
</pre></div>
</div>
<p>If you are going to work with NumPy arrays (and normally, you will) you will
also need to import functions from the numpy package. So most PyTables
programs begin with:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">import</span> <span class="nn">tables</span> <span class="c"># but in this tutorial we use "from tables import \*"</span>
<span class="gp">>>> </span><span class="kn">import</span> <span class="nn">numpy</span>
</pre></div>
</div>
</div>
<div class="section" id="declaring-a-column-descriptor">
<h3>Declaring a Column Descriptor<a class="headerlink" href="#declaring-a-column-descriptor" title="Permalink to this headline">¶</a></h3>
<p>Now, imagine that we have a particle detector and we want to create a table
object in order to save data retrieved from it. You need first to define the
table, the number of columns it has, what kind of object is contained in each
column, and so on.</p>
<p>Our particle detector has a TDC (Time to Digital Converter) counter with a
dynamic range of 8 bits and an ADC (Analogical to Digital Converter) with a
range of 16 bits. For these values, we will define 2 fields in our record
object called TDCcount and ADCcount. We also want to save the grid position
in which the particle has been detected, so we will add two new fields called
grid_i and grid_j. Our instrumentation also can obtain the pressure and
energy of the particle. The resolution of the pressure-gauge allows us to use
a single-precision float to store pressure readings, while the energy value
will need a double-precision float. Finally, to track the particle we want to
assign it a name to identify the kind of the particle it is and a unique
numeric identifier. So we will add two more fields: name will be a string of
up to 16 characters, and idnumber will be an integer of 64 bits (to allow us
to store records for extremely large numbers of particles).</p>
<p>Having determined our columns and their types, we can now declare a new
Particle class that will contain all this information:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">tables</span> <span class="kn">import</span> <span class="o">*</span>
<span class="gp">>>> </span><span class="k">class</span> <span class="nc">Particle</span><span class="p">(</span><span class="n">IsDescription</span><span class="p">):</span>
<span class="gp">... </span> <span class="n">name</span> <span class="o">=</span> <span class="n">StringCol</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span> <span class="c"># 16-character String</span>
<span class="gp">... </span> <span class="n">idnumber</span> <span class="o">=</span> <span class="n">Int64Col</span><span class="p">()</span> <span class="c"># Signed 64-bit integer</span>
<span class="gp">... </span> <span class="n">ADCcount</span> <span class="o">=</span> <span class="n">UInt16Col</span><span class="p">()</span> <span class="c"># Unsigned short integer</span>
<span class="gp">... </span> <span class="n">TDCcount</span> <span class="o">=</span> <span class="n">UInt8Col</span><span class="p">()</span> <span class="c"># unsigned byte</span>
<span class="gp">... </span> <span class="n">grid_i</span> <span class="o">=</span> <span class="n">Int32Col</span><span class="p">()</span> <span class="c"># 32-bit integer</span>
<span class="gp">... </span> <span class="n">grid_j</span> <span class="o">=</span> <span class="n">Int32Col</span><span class="p">()</span> <span class="c"># 32-bit integer</span>
<span class="gp">... </span> <span class="n">pressure</span> <span class="o">=</span> <span class="n">Float32Col</span><span class="p">()</span> <span class="c"># float (single-precision)</span>
<span class="gp">... </span> <span class="n">energy</span> <span class="o">=</span> <span class="n">Float64Col</span><span class="p">()</span> <span class="c"># double (double-precision)</span>
<span class="go">>>></span>
</pre></div>
</div>
<p>This definition class is self-explanatory. Basically, you declare a class
variable for each field you need. As its value you assign an instance of the
appropriate Col subclass, according to the kind of column defined (the data
type, the length, the shape, etc). See the <a class="reference internal" href="libref/declarative_classes.html#colclassdescr"><em>The Col class and its descendants</em></a> for a
complete description of these subclasses. See also <a class="reference internal" href="datatypes.html#datatypes"><em>Supported data types in PyTables</em></a> for a
list of data types supported by the Col constructor.</p>
<p>From now on, we can use Particle instances as a descriptor for our detector
data table. We will see later on how to pass this object to construct the
table. But first, we must create a file where all the actual data pushed into
our table will be saved.</p>
</div>
<div class="section" id="creating-a-pytables-file-from-scratch">
<h3>Creating a PyTables file from scratch<a class="headerlink" href="#creating-a-pytables-file-from-scratch" title="Permalink to this headline">¶</a></h3>
<p>Use the top-level <a class="reference internal" href="libref/top_level.html#tables.open_file" title="tables.open_file"><tt class="xref py py-func docutils literal"><span class="pre">open_file()</span></tt></a> function to create a PyTables file:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">h5file</span> <span class="o">=</span> <span class="n">open_file</span><span class="p">(</span><span class="s">"tutorial1.h5"</span><span class="p">,</span> <span class="n">mode</span> <span class="o">=</span> <span class="s">"w"</span><span class="p">,</span> <span class="n">title</span> <span class="o">=</span> <span class="s">"Test file"</span><span class="p">)</span>
</pre></div>
</div>
<p><a class="reference internal" href="libref/top_level.html#tables.open_file" title="tables.open_file"><tt class="xref py py-func docutils literal"><span class="pre">open_file()</span></tt></a> is one of the objects imported by the
<tt class="docutils literal"><span class="pre">`from</span> <span class="pre">tables</span> <span class="pre">import</span> <span class="pre">*`</span></tt> statement. Here, we are saying that we want to
create a new file in the current working directory called “tutorial1.h5” in
“w”rite mode and with an descriptive title string (“Test file”).
This function attempts to open the file, and if successful, returns the File
(see <a class="reference internal" href="libref/file_class.html#fileclassdescr"><em>The File Class</em></a>) object instance h5file. The root of the object
tree is specified in the instance’s root attribute.</p>
</div>
<div class="section" id="creating-a-new-group">
<h3>Creating a new group<a class="headerlink" href="#creating-a-new-group" title="Permalink to this headline">¶</a></h3>
<p>Now, to better organize our data, we will create a group called <em>detector</em>
that branches from the root node. We will save our particle data table in
this group:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">group</span> <span class="o">=</span> <span class="n">h5file</span><span class="o">.</span><span class="n">create_group</span><span class="p">(</span><span class="s">"/"</span><span class="p">,</span> <span class="s">'detector'</span><span class="p">,</span> <span class="s">'Detector information'</span><span class="p">)</span>
</pre></div>
</div>
<p>Here, we have taken the File instance h5file and invoked its
<a class="reference internal" href="libref/file_class.html#tables.File.create_group" title="tables.File.create_group"><tt class="xref py py-meth docutils literal"><span class="pre">File.create_group()</span></tt></a> method to create a new group called <em>detector</em>
branching from “<em>/</em>” (another way to refer to the h5file.root object we
mentioned above). This will create a new Group (see <a class="reference internal" href="libref/hierarchy_classes.html#groupclassdescr"><em>The Group class</em></a>)
object instance that will be assigned to the variable group.</p>
</div>
<div class="section" id="creating-a-new-table">
<h3>Creating a new table<a class="headerlink" href="#creating-a-new-table" title="Permalink to this headline">¶</a></h3>
<p>Let’s now create a Table (see <a class="reference internal" href="libref/structured_storage.html#tableclassdescr"><em>The Table class</em></a>) object as a branch off
the newly-created group. We do that by calling the <a class="reference internal" href="libref/file_class.html#tables.File.create_table" title="tables.File.create_table"><tt class="xref py py-meth docutils literal"><span class="pre">File.create_table()</span></tt></a>
method of the h5file object:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span> <span class="o">=</span> <span class="n">h5file</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span><span class="n">group</span><span class="p">,</span> <span class="s">'readout'</span><span class="p">,</span> <span class="n">Particle</span><span class="p">,</span> <span class="s">"Readout example"</span><span class="p">)</span>
</pre></div>
</div>
<p>We create the Table instance under group. We assign this table the node name
“<em>readout</em>”. The Particle class declared before is the <em>description</em>
parameter (to define the columns of the table) and finally we set
“<em>Readout example</em>” as the Table title. With all this information, a new
Table instance is created and assigned to the variable <em>table</em>.</p>
<p>If you are curious about how the object tree looks right now, simply print
the File instance variable <em>h5file</em>, and examine the output:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">print</span> <span class="n">h5file</span>
<span class="go">tutorial1.h5 (File) 'Test file'</span>
<span class="go">Last modif.: 'Wed Mar 7 11:06:12 2007'</span>
<span class="go">Object Tree:</span>
<span class="go">/ (RootGroup) 'Test file'</span>
<span class="go">/detector (Group) 'Detector information'</span>
<span class="go">/detector/readout (Table(0,)) 'Readout example'</span>
</pre></div>
</div>
<p>As you can see, a dump of the object tree is displayed. It’s easy to see the
Group and Table objects we have just created. If you want more information,
just type the variable containing the File instance:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">h5file</span>
<span class="go">File(filename='tutorial1.h5', title='Test file', mode='w', root_uep='/', filters=Filters(complevel=0, shuffle=False, fletcher32=False))</span>
<span class="go">/ (RootGroup) 'Test file'</span>
<span class="go">/detector (Group) 'Detector information'</span>
<span class="go">/detector/readout (Table(0,)) 'Readout example'</span>
<span class="go">description := {</span>
<span class="go"> "ADCcount": UInt16Col(shape=(), dflt=0, pos=0),</span>
<span class="go"> "TDCcount": UInt8Col(shape=(), dflt=0, pos=1),</span>
<span class="go"> "energy": Float64Col(shape=(), dflt=0.0, pos=2),</span>
<span class="go"> "grid_i": Int32Col(shape=(), dflt=0, pos=3),</span>
<span class="go"> "grid_j": Int32Col(shape=(), dflt=0, pos=4),</span>
<span class="go"> "idnumber": Int64Col(shape=(), dflt=0, pos=5),</span>
<span class="go"> "name": StringCol(itemsize=16, shape=(), dflt='', pos=6),</span>
<span class="go"> "pressure": Float32Col(shape=(), dflt=0.0, pos=7)}</span>
<span class="go"> byteorder := 'little'</span>
<span class="go"> chunkshape := (87,)</span>
</pre></div>
</div>
<p>More detailed information is displayed about each object in the tree. Note
how Particle, our table descriptor class, is printed as part of the <em>readout</em>
table description information. In general, you can obtain much more
information about the objects and their children by just printing them. That
introspection capability is very useful, and I recommend that you use it
extensively.</p>
<p>The time has come to fill this table with some values. First we will get a
pointer to the Row (see <a class="reference internal" href="libref/structured_storage.html#rowclassdescr"><em>The Row class</em></a>) instance of this table
instance:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">particle</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">row</span>
</pre></div>
</div>
<p>The row attribute of table points to the Row instance that will be used to
write data rows into the table. We write data simply by assigning the Row
instance the values for each row as if it were a dictionary (although it is
actually an <em>extension class</em>), using the column names as keys.</p>
<p>Below is an example of how to write rows:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'name'</span><span class="p">]</span> <span class="o">=</span> <span class="s">'Particle: </span><span class="si">%6d</span><span class="s">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'TDCcount'</span><span class="p">]</span> <span class="o">=</span> <span class="n">i</span> <span class="o">%</span> <span class="mi">256</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'ADCcount'</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span> <span class="o">*</span> <span class="mi">256</span><span class="p">)</span> <span class="o">%</span> <span class="p">(</span><span class="mi">1</span> <span class="o"><<</span> <span class="mi">16</span><span class="p">)</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'grid_i'</span><span class="p">]</span> <span class="o">=</span> <span class="n">i</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'grid_j'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">10</span> <span class="o">-</span> <span class="n">i</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'pressure'</span><span class="p">]</span> <span class="o">=</span> <span class="nb">float</span><span class="p">(</span><span class="n">i</span><span class="o">*</span><span class="n">i</span><span class="p">)</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'energy'</span><span class="p">]</span> <span class="o">=</span> <span class="nb">float</span><span class="p">(</span><span class="n">particle</span><span class="p">[</span><span class="s">'pressure'</span><span class="p">]</span> <span class="o">**</span> <span class="mi">4</span><span class="p">)</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'idnumber'</span><span class="p">]</span> <span class="o">=</span> <span class="n">i</span> <span class="o">*</span> <span class="p">(</span><span class="mi">2</span> <span class="o">**</span> <span class="mi">34</span><span class="p">)</span>
<span class="gp">... </span> <span class="c"># Insert a new particle record</span>
<span class="gp">... </span> <span class="n">particle</span><span class="o">.</span><span class="n">append</span><span class="p">()</span>
<span class="go">>>></span>
</pre></div>
</div>
<p>This code should be easy to understand. The lines inside the loop just assign
values to the different columns in the Row instance particle (see
<a class="reference internal" href="libref/structured_storage.html#rowclassdescr"><em>The Row class</em></a>). A call to its append() method writes this information
to the table I/O buffer.</p>
<p>After we have processed all our data, we should flush the table’s I/O buffer
if we want to write all this data to disk. We achieve that by calling the
table.flush() method:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
</pre></div>
</div>
<p>Remember, flushing a table is a <em>very important</em> step as it will not only
help to maintain the integrity of your file, but also will free valuable
memory resources (i.e. internal buffers) that your program may need for other
things.</p>
</div>
<div class="section" id="reading-and-selecting-data-in-a-table">
<span id="readingandselectingusage"></span><h3>Reading (and selecting) data in a table<a class="headerlink" href="#reading-and-selecting-data-in-a-table" title="Permalink to this headline">¶</a></h3>
<p>Ok. We have our data on disk, and now we need to access it and select from
specific columns the values we are interested in. See the example below:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span> <span class="o">=</span> <span class="n">h5file</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">detector</span><span class="o">.</span><span class="n">readout</span>
<span class="gp">>>> </span><span class="n">pressure</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="p">[</span><span class="s">'pressure'</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">iterrows</span><span class="p">()</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="s">'TDCcount'</span><span class="p">]</span> <span class="o">></span> <span class="mi">3</span> <span class="ow">and</span> <span class="mi">20</span> <span class="o"><=</span> <span class="n">x</span><span class="p">[</span><span class="s">'pressure'</span><span class="p">]</span> <span class="o"><</span> <span class="mi">50</span><span class="p">]</span>
<span class="gp">>>> </span><span class="n">pressure</span>
<span class="go">[25.0, 36.0, 49.0]</span>
</pre></div>
</div>
<p>The first line creates a “shortcut” to the <em>readout</em> table deeper on the
object tree. As you can see, we use the <em>natural naming</em> schema to access it.
We also could have used the h5file.get_node() method, as we will do later on.</p>
<p>You will recognize the last two lines as a Python list comprehension.
It loops over the rows in <em>table</em> as they are provided by the
<a class="reference internal" href="libref/structured_storage.html#tables.Table.iterrows" title="tables.Table.iterrows"><tt class="xref py py-meth docutils literal"><span class="pre">Table.iterrows()</span></tt></a> iterator. The iterator returns values until all the
data in table is exhausted. These rows are filtered using the expression:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">x</span><span class="p">[</span><span class="s">'TDCcount'</span><span class="p">]</span> <span class="o">></span> <span class="mi">3</span> <span class="ow">and</span> <span class="mi">20</span> <span class="o"><=</span> <span class="n">x</span><span class="p">[</span><span class="s">'pressure'</span><span class="p">]</span> <span class="o"><</span> <span class="mi">50</span>
</pre></div>
</div>
<p>So, we are selecting the values of the pressure
column from filtered records to create the final list and assign it to
pressure variable.</p>
<p>We could have used a normal for loop to accomplish the same purpose, but I
find comprehension syntax to be more compact and elegant.</p>
<p>PyTables do offer other, more powerful ways of performing selections which
may be more suitable if you have very large tables or if you need very high
query speeds. They are called <em>in-kernel</em> and <em>indexed</em> queries, and you can
use them through <a class="reference internal" href="libref/structured_storage.html#tables.Table.where" title="tables.Table.where"><tt class="xref py py-meth docutils literal"><span class="pre">Table.where()</span></tt></a> and other related methods.</p>
<p>Let’s use an in-kernel selection to query the name column for the same set of
cuts:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">names</span> <span class="o">=</span> <span class="p">[</span> <span class="n">x</span><span class="p">[</span><span class="s">'name'</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="s">"""(TDCcount > 3) & (20 <= pressure) & (pressure < 50)"""</span><span class="p">)</span> <span class="p">]</span>
<span class="gp">>>> </span><span class="n">names</span>
<span class="go">['Particle: 5', 'Particle: 6', 'Particle: 7']</span>
</pre></div>
</div>
<p>In-kernel and indexed queries are not only much faster, but as you can see,
they also look more compact, and are among the greatests features for
PyTables, so be sure that you use them a lot. See <a class="reference internal" href="condition_syntax.html#condition-syntax"><em>Condition Syntax</em></a> and
<a class="reference internal" href="optimization.html#searchoptim"><em>Accelerating your searches</em></a> for more information on in-kernel and indexed selections.</p>
<p>That’s enough about selections for now. The next section will show you how to
save these selected results to a file.</p>
</div>
<div class="section" id="creating-new-array-objects">
<h3>Creating new array objects<a class="headerlink" href="#creating-new-array-objects" title="Permalink to this headline">¶</a></h3>
<p>In order to separate the selected data from the mass of detector data, we
will create a new group columns branching off the root group. Afterwards,
under this group, we will create two arrays that will contain the selected
data. First, we create the group:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">gcolumns</span> <span class="o">=</span> <span class="n">h5file</span><span class="o">.</span><span class="n">create_group</span><span class="p">(</span><span class="n">h5file</span><span class="o">.</span><span class="n">root</span><span class="p">,</span> <span class="s">"columns"</span><span class="p">,</span> <span class="s">"Pressure and Name"</span><span class="p">)</span>
</pre></div>
</div>
<p>Note that this time we have specified the first parameter using <em>natural
naming</em> (h5file.root) instead of with an absolute path string (“/”).</p>
<p>Now, create the first of the two Array objects we’ve just mentioned:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">h5file</span><span class="o">.</span><span class="n">create_array</span><span class="p">(</span><span class="n">gcolumns</span><span class="p">,</span> <span class="s">'pressure'</span><span class="p">,</span> <span class="n">array</span><span class="p">(</span><span class="n">pressure</span><span class="p">),</span>
<span class="go">"Pressure column selection")</span>
<span class="go">/columns/pressure (Array(3,)) 'Pressure column selection'</span>
<span class="go"> atom := Float64Atom(shape=(), dflt=0.0)</span>
<span class="go"> maindim := 0</span>
<span class="go"> flavor := 'numpy'</span>
<span class="go"> byteorder := 'little'</span>
<span class="go"> chunkshape := None</span>
</pre></div>
</div>
<p>We already know the first two parameters of the <a class="reference internal" href="libref/file_class.html#tables.File.create_array" title="tables.File.create_array"><tt class="xref py py-meth docutils literal"><span class="pre">File.create_array()</span></tt></a>
methods (these are the same as the first two in create_table): they are the
parent group <em>where</em> Array will be created and the Array instance <em>name</em>.
The third parameter is the <em>object</em> we want to save to disk. In this case, it
is a NumPy array that is built from the selection list we created before.
The fourth parameter is the <em>title</em>.</p>
<p>Now, we will save the second array. It contains the list of strings we
selected before: we save this object as-is, with no further conversion:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">h5file</span><span class="o">.</span><span class="n">create_array</span><span class="p">(</span><span class="n">gcolumns</span><span class="p">,</span> <span class="s">'name'</span><span class="p">,</span> <span class="n">names</span><span class="p">,</span> <span class="s">"Name column selection"</span><span class="p">)</span>
<span class="go">/columns/name (Array(3,)) 'Name column selection'</span>
<span class="go"> atom := StringAtom(itemsize=16, shape=(), dflt='')</span>
<span class="go"> maindim := 0</span>
<span class="go"> flavor := 'python'</span>
<span class="go"> byteorder := 'irrelevant'</span>
<span class="go"> chunkshape := None</span>
</pre></div>
</div>
<p>As you can see, <a class="reference internal" href="libref/file_class.html#tables.File.create_array" title="tables.File.create_array"><tt class="xref py py-meth docutils literal"><span class="pre">File.create_array()</span></tt></a> accepts <em>names</em> (which is a regular
Python list) as an <em>object</em> parameter. Actually, it accepts a variety of
different regular objects (see <tt class="xref py py-func docutils literal"><span class="pre">create_array()</span></tt>) as parameters. The flavor
attribute (see the output above) saves the original kind of object that was
saved. Based on this <em>flavor</em>, PyTables will be able to retrieve exactly the
same object from disk later on.</p>
<p>Note that in these examples, the create_array method returns an Array instance
that is not assigned to any variable. Don’t worry, this is intentional to
show the kind of object we have created by displaying its representation. The
Array objects have been attached to the object tree and saved to disk, as you
can see if you print the complete object tree:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">print</span> <span class="n">h5file</span>
<span class="go">tutorial1.h5 (File) 'Test file'</span>
<span class="go">Last modif.: 'Wed Mar 7 19:40:44 2007'</span>
<span class="go">Object Tree:</span>
<span class="go">/ (RootGroup) 'Test file'</span>
<span class="go">/columns (Group) 'Pressure and Name'</span>
<span class="go">/columns/name (Array(3,)) 'Name column selection'</span>
<span class="go">/columns/pressure (Array(3,)) 'Pressure column selection'</span>
<span class="go">/detector (Group) 'Detector information'</span>
<span class="go">/detector/readout (Table(10,)) 'Readout example'</span>
</pre></div>
</div>
</div>
<div class="section" id="closing-the-file-and-looking-at-its-content">
<h3>Closing the file and looking at its content<a class="headerlink" href="#closing-the-file-and-looking-at-its-content" title="Permalink to this headline">¶</a></h3>
<p>To finish this first tutorial, we use the close method of the h5file File
object to close the file before exiting Python:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">h5file</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="gp">>>> </span><span class="o">^</span><span class="n">D</span>
<span class="go">$</span>
</pre></div>
</div>
<p>You have now created your first PyTables file with a table and two arrays.
You can examine it with any generic HDF5 tool, such as h5dump or h5ls. Here is
what the tutorial1.h5 looks like when read with the h5ls program.</p>
<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span>h5ls -rd tutorial1.h5
/columns Group
/columns/name Dataset <span class="o">{</span>3<span class="o">}</span>
Data:
<span class="o">(</span>0<span class="o">)</span> <span class="s2">"Particle: 5"</span>, <span class="s2">"Particle: 6"</span>, <span class="s2">"Particle: 7"</span>
/columns/pressure Dataset <span class="o">{</span>3<span class="o">}</span>
Data:
<span class="o">(</span>0<span class="o">)</span> 25, 36, 49
/detector Group
/detector/readout Dataset <span class="o">{</span>10/Inf<span class="o">}</span>
Data:
<span class="o">(</span>0<span class="o">)</span> <span class="o">{</span>0, 0, 0, 0, 10, 0, <span class="s2">"Particle: 0"</span>, 0<span class="o">}</span>,
<span class="o">(</span>1<span class="o">)</span> <span class="o">{</span>256, 1, 1, 1, 9, 17179869184, <span class="s2">"Particle: 1"</span>, 1<span class="o">}</span>,
<span class="o">(</span>2<span class="o">)</span> <span class="o">{</span>512, 2, 256, 2, 8, 34359738368, <span class="s2">"Particle: 2"</span>, 4<span class="o">}</span>,
<span class="o">(</span>3<span class="o">)</span> <span class="o">{</span>768, 3, 6561, 3, 7, 51539607552, <span class="s2">"Particle: 3"</span>, 9<span class="o">}</span>,
<span class="o">(</span>4<span class="o">)</span> <span class="o">{</span>1024, 4, 65536, 4, 6, 68719476736, <span class="s2">"Particle: 4"</span>, 16<span class="o">}</span>,
<span class="o">(</span>5<span class="o">)</span> <span class="o">{</span>1280, 5, 390625, 5, 5, 85899345920, <span class="s2">"Particle: 5"</span>, 25<span class="o">}</span>,
<span class="o">(</span>6<span class="o">)</span> <span class="o">{</span>1536, 6, 1679616, 6, 4, 103079215104, <span class="s2">"Particle: 6"</span>, 36<span class="o">}</span>,
<span class="o">(</span>7<span class="o">)</span> <span class="o">{</span>1792, 7, 5764801, 7, 3, 120259084288, <span class="s2">"Particle: 7"</span>, 49<span class="o">}</span>,
<span class="o">(</span>8<span class="o">)</span> <span class="o">{</span>2048, 8, 16777216, 8, 2, 137438953472, <span class="s2">"Particle: 8"</span>, 64<span class="o">}</span>,
<span class="o">(</span>9<span class="o">)</span> <span class="o">{</span>2304, 9, 43046721, 9, 1, 154618822656, <span class="s2">"Particle: 9"</span>, 81<span class="o">}</span>
</pre></div>
</div>
<p>Here’s the output as displayed by the “ptdump” PyTables utility (located in
utils/ directory).</p>
<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span>ptdump tutorial1.h5
/ <span class="o">(</span>RootGroup<span class="o">)</span> <span class="s1">'Test file'</span>
/columns <span class="o">(</span>Group<span class="o">)</span> <span class="s1">'Pressure and Name'</span>
/columns/name <span class="o">(</span>Array<span class="o">(</span>3,<span class="o">))</span> <span class="s1">'Name column selection'</span>
/columns/pressure <span class="o">(</span>Array<span class="o">(</span>3,<span class="o">))</span> <span class="s1">'Pressure column selection'</span>
/detector <span class="o">(</span>Group<span class="o">)</span> <span class="s1">'Detector information'</span>
/detector/readout <span class="o">(</span>Table<span class="o">(</span>10,<span class="o">))</span> <span class="s1">'Readout example'</span>
</pre></div>
</div>
<p>You can pass the <em class="xref std std-option">-v</em> or <em class="xref std std-option">-d</em> options to ptdump if you want
more verbosity. Try them out!</p>
<p>Also, in <a class="reference internal" href="#tutorial1-1-tableview"><em>Figure 1</em></a>, you can admire how the
tutorial1.h5 looks like using the <a class="reference external" href="http://vitables.org">ViTables</a> graphical
interface.</p>
<div class="figure align-center" id="tutorial1-1-tableview">
<img alt="../_images/tutorial1-1-tableview.png" src="../_images/tutorial1-1-tableview.png" />
<p class="caption"><strong>Figure 1. The initial version of the data file for tutorial 1, with a
view of the data objects.</strong></p>
</div>
</div>
</div>
<div class="section" id="browsing-the-object-tree">
<h2>Browsing the <em>object tree</em><a class="headerlink" href="#browsing-the-object-tree" title="Permalink to this headline">¶</a></h2>
<p>In this section, we will learn how to browse the tree and retrieve data and
also meta-information about the actual data.</p>
<p>In <em>examples/tutorial1-2.py</em> you will find the working version of all the
code in this section. As before, you are encouraged to use a python shell and
inspect the object tree during the course of the tutorial.</p>
<div class="section" id="traversing-the-object-tree">
<h3>Traversing the object tree<a class="headerlink" href="#traversing-the-object-tree" title="Permalink to this headline">¶</a></h3>
<p>Let’s start by opening the file we created in last tutorial section:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">h5file</span> <span class="o">=</span> <span class="n">open_file</span><span class="p">(</span><span class="s">"tutorial1.h5"</span><span class="p">,</span> <span class="s">"a"</span><span class="p">)</span>
</pre></div>
</div>
<p>This time, we have opened the file in “a”ppend mode. We use this mode to add
more information to the file.</p>
<p>PyTables, following the Python tradition, offers powerful introspection
capabilities, i.e. you can easily ask information about any component of the
object tree as well as search the tree.</p>
<p>To start with, you can get a preliminary overview of the object tree by
simply printing the existing File instance:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">print</span> <span class="n">h5file</span>
<span class="go">tutorial1.h5 (File) 'Test file'</span>
<span class="go">Last modif.: 'Wed Mar 7 19:50:57 2007'</span>
<span class="go">Object Tree:</span>
<span class="go">/ (RootGroup) 'Test file'</span>
<span class="go">/columns (Group) 'Pressure and Name'</span>
<span class="go">/columns/name (Array(3,)) 'Name column selection'</span>
<span class="go">/columns/pressure (Array(3,)) 'Pressure column selection'</span>
<span class="go">/detector (Group) 'Detector information'</span>
<span class="go">/detector/readout (Table(10,)) 'Readout example'</span>
</pre></div>
</div>
<p>It looks like all of our objects are there. Now let’s make use of the File
iterator to see how to list all the nodes in the object tree:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">h5file</span><span class="p">:</span>
<span class="gp">... </span> <span class="k">print</span> <span class="n">node</span>
<span class="go">/ (RootGroup) 'Test file'</span>
<span class="go">/columns (Group) 'Pressure and Name'</span>
<span class="go">/detector (Group) 'Detector information'</span>
<span class="go">/columns/name (Array(3,)) 'Name column selection'</span>
<span class="go">/columns/pressure (Array(3,)) 'Pressure column selection'</span>
<span class="go">/detector/readout (Table(10,)) 'Readout example'</span>
</pre></div>
</div>
<p>We can use the <a class="reference internal" href="libref/file_class.html#tables.File.walk_groups" title="tables.File.walk_groups"><tt class="xref py py-meth docutils literal"><span class="pre">File.walk_groups()</span></tt></a> method of the File class to list only
the <em>groups</em> on tree:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">for</span> <span class="n">group</span> <span class="ow">in</span> <span class="n">h5file</span><span class="o">.</span><span class="n">walk_groups</span><span class="p">():</span>
<span class="gp">... </span> <span class="k">print</span> <span class="n">group</span>
<span class="go">/ (RootGroup) 'Test file'</span>
<span class="go">/columns (Group) 'Pressure and Name'</span>
<span class="go">/detector (Group) 'Detector information'</span>
</pre></div>
</div>
<p>Note that <a class="reference internal" href="libref/file_class.html#tables.File.walk_groups" title="tables.File.walk_groups"><tt class="xref py py-meth docutils literal"><span class="pre">File.walk_groups()</span></tt></a> actually returns an <em>iterator</em>, not a list
of objects. Using this iterator with the list_nodes() method is a powerful
combination. Let’s see an example listing of all the arrays in the tree:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">for</span> <span class="n">group</span> <span class="ow">in</span> <span class="n">h5file</span><span class="o">.</span><span class="n">walk_groups</span><span class="p">(</span><span class="s">"/"</span><span class="p">):</span>
<span class="gp">... </span> <span class="k">for</span> <span class="n">array</span> <span class="ow">in</span> <span class="n">h5file</span><span class="o">.</span><span class="n">list_nodes</span><span class="p">(</span><span class="n">group</span><span class="p">,</span> <span class="n">classname</span><span class="o">=</span><span class="s">'Array'</span><span class="p">):</span>
<span class="gp">... </span> <span class="k">print</span> <span class="n">array</span>
<span class="go">/columns/name (Array(3,)) 'Name column selection'</span>
<span class="go">/columns/pressure (Array(3,)) 'Pressure column selection'</span>
</pre></div>
</div>
<p><a class="reference internal" href="libref/file_class.html#tables.File.list_nodes" title="tables.File.list_nodes"><tt class="xref py py-meth docutils literal"><span class="pre">File.list_nodes()</span></tt></a> returns a list containing all the nodes hanging off a
specific Group. If the <em>classname</em> keyword is specified, the method will
filter out all instances which are not descendants of the class. We have
asked for only Array instances. There exist also an iterator counterpart
called <a class="reference internal" href="libref/file_class.html#tables.File.iter_nodes" title="tables.File.iter_nodes"><tt class="xref py py-meth docutils literal"><span class="pre">File.iter_nodes()</span></tt></a> that might be handy is some situations, like
for example when dealing with groups with a large number of nodes behind it.</p>
<p>We can combine both calls by using the <a class="reference internal" href="libref/file_class.html#tables.File.walk_nodes" title="tables.File.walk_nodes"><tt class="xref py py-meth docutils literal"><span class="pre">File.walk_nodes()</span></tt></a> special method
of the File object. For example:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">for</span> <span class="n">array</span> <span class="ow">in</span> <span class="n">h5file</span><span class="o">.</span><span class="n">walk_nodes</span><span class="p">(</span><span class="s">"/"</span><span class="p">,</span> <span class="s">"Array"</span><span class="p">):</span>
<span class="gp">... </span> <span class="k">print</span> <span class="n">array</span>
<span class="go">/columns/name (Array(3,)) 'Name column selection'</span>
<span class="go">/columns/pressure (Array(3,)) 'Pressure column selection'</span>
</pre></div>
</div>
<p>This is a nice shortcut when working interactively.</p>
<p>Finally, we will list all the Leaf, i.e. Table and Array instances (see
<a class="reference internal" href="libref/hierarchy_classes.html#leafclassdescr"><em>The Leaf class</em></a> for detailed information on Leaf class), in the
/detector group. Note that only one instance of the Table class (i.e.
readout) will be selected in this group (as should be the case):</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">for</span> <span class="n">leaf</span> <span class="ow">in</span> <span class="n">h5file</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">detector</span><span class="o">.</span><span class="n">_f_walknodes</span><span class="p">(</span><span class="s">'Leaf'</span><span class="p">):</span>
<span class="gp">... </span> <span class="k">print</span> <span class="n">leaf</span>
<span class="go">/detector/readout (Table(10,)) 'Readout example'</span>
</pre></div>
</div>
<p>We have used a call to the <a class="reference internal" href="libref/hierarchy_classes.html#tables.Group._f_walknodes" title="tables.Group._f_walknodes"><tt class="xref py py-meth docutils literal"><span class="pre">Group._f_walknodes()</span></tt></a> method, using the
<em>natural naming</em> path specification.</p>
<p>Of course you can do more sophisticated node selections using these powerful
methods. But first, let’s take a look at some important PyTables object
instance variables.</p>
</div>
<div class="section" id="setting-and-getting-user-attributes">
<h3>Setting and getting user attributes<a class="headerlink" href="#setting-and-getting-user-attributes" title="Permalink to this headline">¶</a></h3>
<p>PyTables provides an easy and concise way to complement the meaning of your
node objects on the tree by using the AttributeSet class (see
<a class="reference internal" href="libref/declarative_classes.html#attributesetclassdescr"><em>The AttributeSet class</em></a>). You can access this object through the
standard attribute attrs in Leaf nodes and _v_attrs in Group nodes.</p>
<p>For example, let’s imagine that we want to save the date indicating when the
data in /detector/readout table has been acquired, as well as the temperature
during the gathering process:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span> <span class="o">=</span> <span class="n">h5file</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">detector</span><span class="o">.</span><span class="n">readout</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">gath_date</span> <span class="o">=</span> <span class="s">"Wed, 06/12/2003 18:33"</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">temperature</span> <span class="o">=</span> <span class="mf">18.4</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">temp_scale</span> <span class="o">=</span> <span class="s">"Celsius"</span>
</pre></div>
</div>
<p>Now, let’s set a somewhat more complex attribute in the /detector group:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">detector</span> <span class="o">=</span> <span class="n">h5file</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">detector</span>
<span class="gp">>>> </span><span class="n">detector</span><span class="o">.</span><span class="n">_v_attrs</span><span class="o">.</span><span class="n">stuff</span> <span class="o">=</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="p">(</span><span class="mf">2.3</span><span class="p">,</span> <span class="mf">4.5</span><span class="p">),</span> <span class="s">"Integer and tuple"</span><span class="p">]</span>
</pre></div>
</div>
<p>Note how the AttributeSet instance is accessed with the _v_attrs attribute
because detector is a Group node. In general, you can save any standard
Python data structure as an attribute node. See <a class="reference internal" href="libref/declarative_classes.html#attributesetclassdescr"><em>The AttributeSet class</em></a>
for a more detailed explanation of how they are serialized for export to
disk.</p>
<p>Retrieving the attributes is equally simple:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">gath_date</span>
<span class="go">'Wed, 06/12/2003 18:33'</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">temperature</span>
<span class="go">18.399999999999999</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">temp_scale</span>
<span class="go">'Celsius'</span>
<span class="gp">>>> </span><span class="n">detector</span><span class="o">.</span><span class="n">_v_attrs</span><span class="o">.</span><span class="n">stuff</span>
<span class="go">[5, (2.2999999999999998, 4.5), 'Integer and tuple']</span>
</pre></div>
</div>
<p>You can probably guess how to delete attributes:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">del</span> <span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">gath_date</span>
</pre></div>
</div>
<p>If you want to examine the current user attribute set of /detector/table, you
can print its representation (try hitting the TAB key twice if you are on a
Unix Python console with the rlcompleter module active):</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span>
<span class="go">/detector/readout._v_attrs (AttributeSet), 23 attributes:</span>
<span class="go"> [CLASS := 'TABLE',</span>
<span class="go"> FIELD_0_FILL := 0,</span>
<span class="go"> FIELD_0_NAME := 'ADCcount',</span>
<span class="go"> FIELD_1_FILL := 0,</span>
<span class="go"> FIELD_1_NAME := 'TDCcount',</span>
<span class="go"> FIELD_2_FILL := 0.0,</span>
<span class="go"> FIELD_2_NAME := 'energy',</span>
<span class="go"> FIELD_3_FILL := 0,</span>
<span class="go"> FIELD_3_NAME := 'grid_i',</span>
<span class="go"> FIELD_4_FILL := 0,</span>
<span class="go"> FIELD_4_NAME := 'grid_j',</span>
<span class="go"> FIELD_5_FILL := 0,</span>
<span class="go"> FIELD_5_NAME := 'idnumber',</span>
<span class="go"> FIELD_6_FILL := '',</span>
<span class="go"> FIELD_6_NAME := 'name',</span>
<span class="go"> FIELD_7_FILL := 0.0,</span>
<span class="go"> FIELD_7_NAME := 'pressure',</span>
<span class="go"> FLAVOR := 'numpy',</span>
<span class="go"> NROWS := 10,</span>
<span class="go"> TITLE := 'Readout example',</span>
<span class="go"> VERSION := '2.6',</span>
<span class="go"> temp_scale := 'Celsius',</span>
<span class="go"> temperature := 18.399999999999999]</span>
</pre></div>
</div>
<p>We’ve got all the attributes (including the <em>system</em> attributes). You can get
a list of <em>all</em> attributes or only the <em>user</em> or <em>system</em> attributes with the
_f_list() method:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">print</span> <span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">_f_list</span><span class="p">(</span><span class="s">"all"</span><span class="p">)</span>
<span class="go">['CLASS', 'FIELD_0_FILL', 'FIELD_0_NAME', 'FIELD_1_FILL', 'FIELD_1_NAME',</span>
<span class="go">'FIELD_2_FILL', 'FIELD_2_NAME', 'FIELD_3_FILL', 'FIELD_3_NAME', 'FIELD_4_FILL',</span>
<span class="go">'FIELD_4_NAME', 'FIELD_5_FILL', 'FIELD_5_NAME', 'FIELD_6_FILL', 'FIELD_6_NAME',</span>
<span class="go">'FIELD_7_FILL', 'FIELD_7_NAME', 'FLAVOR', 'NROWS', 'TITLE', 'VERSION',</span>
<span class="go">'temp_scale', 'temperature']</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">_f_list</span><span class="p">(</span><span class="s">"user"</span><span class="p">)</span>
<span class="go">['temp_scale', 'temperature']</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">_f_list</span><span class="p">(</span><span class="s">"sys"</span><span class="p">)</span>
<span class="go">['CLASS', 'FIELD_0_FILL', 'FIELD_0_NAME', 'FIELD_1_FILL', 'FIELD_1_NAME',</span>
<span class="go">'FIELD_2_FILL', 'FIELD_2_NAME', 'FIELD_3_FILL', 'FIELD_3_NAME', 'FIELD_4_FILL',</span>
<span class="go">'FIELD_4_NAME', 'FIELD_5_FILL', 'FIELD_5_NAME', 'FIELD_6_FILL', 'FIELD_6_NAME',</span>
<span class="go">'FIELD_7_FILL', 'FIELD_7_NAME', 'FLAVOR', 'NROWS', 'TITLE', 'VERSION']</span>
</pre></div>
</div>
<p>You can also rename attributes:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">_f_rename</span><span class="p">(</span><span class="s">"temp_scale"</span><span class="p">,</span><span class="s">"tempScale"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">_f_list</span><span class="p">()</span>
<span class="go">['tempScale', 'temperature']</span>
</pre></div>
</div>
<p>And, from PyTables 2.0 on, you are allowed also to set, delete or rename
system attributes:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">_f_rename</span><span class="p">(</span><span class="s">"VERSION"</span><span class="p">,</span> <span class="s">"version"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">VERSION</span>
<span class="gt">Traceback (most recent call last):</span>
<span class="n">File</span> <span class="s">"<stdin>"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">1</span><span class="p">,</span> <span class="ow">in</span> <span class="o"><</span><span class="n">module</span><span class="o">></span>
<span class="n">File</span> <span class="s">"tables/attributeset.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">222</span><span class="p">,</span> <span class="ow">in</span> <span class="n">__getattr__</span>
<span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">_v__nodepath</span><span class="p">)</span>
<span class="gr">AttributeError</span>: <span class="n">Attribute 'VERSION' does not exist in node: '/detector/readout'</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">version</span>
<span class="go">'2.6'</span>
</pre></div>
</div>
<p><em>Caveat emptor:</em> you must be careful when modifying system attributes because
you may end fooling PyTables and ultimately getting unwanted behaviour. Use
this only if you know what are you doing.</p>
<p>So, given the caveat above, we will proceed to restore the original name of
VERSION attribute:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">_f_rename</span><span class="p">(</span><span class="s">"version"</span><span class="p">,</span> <span class="s">"VERSION"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">attrs</span><span class="o">.</span><span class="n">VERSION</span>
<span class="go">'2.6'</span>
</pre></div>
</div>
<p>Ok. that’s better. If you would terminate your session now, you would be able
to use the h5ls command to read the /detector/readout attributes from the
file written to disk.</p>
<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span>h5ls -vr tutorial1.h5/detector/readout
Opened <span class="s2">"tutorial1.h5"</span> with sec2 driver.
/detector/readout Dataset <span class="o">{</span>10/Inf<span class="o">}</span>
Attribute: CLASS scalar
Type: 6-byte null-terminated ASCII string
Data: <span class="s2">"TABLE"</span>
Attribute: VERSION scalar
Type: 4-byte null-terminated ASCII string
Data: <span class="s2">"2.6"</span>
Attribute: TITLE scalar
Type: 16-byte null-terminated ASCII string
Data: <span class="s2">"Readout example"</span>
Attribute: NROWS scalar
Type: native long long
Data: 10
Attribute: FIELD_0_NAME scalar
Type: 9-byte null-terminated ASCII string
Data: <span class="s2">"ADCcount"</span>
Attribute: FIELD_1_NAME scalar
Type: 9-byte null-terminated ASCII string
Data: <span class="s2">"TDCcount"</span>
Attribute: FIELD_2_NAME scalar
Type: 7-byte null-terminated ASCII string
Data: <span class="s2">"energy"</span>
Attribute: FIELD_3_NAME scalar
Type: 7-byte null-terminated ASCII string
Data: <span class="s2">"grid_i"</span>
Attribute: FIELD_4_NAME scalar
Type: 7-byte null-terminated ASCII string
Data: <span class="s2">"grid_j"</span>
Attribute: FIELD_5_NAME scalar
Type: 9-byte null-terminated ASCII string
Data: <span class="s2">"idnumber"</span>
Attribute: FIELD_6_NAME scalar
Type: 5-byte null-terminated ASCII string
Data: <span class="s2">"name"</span>
Attribute: FIELD_7_NAME scalar
Type: 9-byte null-terminated ASCII string
Data: <span class="s2">"pressure"</span>
Attribute: FLAVOR scalar
Type: 5-byte null-terminated ASCII string
Data: <span class="s2">"numpy"</span>
Attribute: tempScale scalar
Type: 7-byte null-terminated ASCII string
Data: <span class="s2">"Celsius"</span>
Attribute: temperature scalar
Type: native double
Data: 18.4
Location: 0:1:0:1952
Links: 1
Modified: 2006-12-11 10:35:13 CET
Chunks: <span class="o">{</span>85<span class="o">}</span> 3995 bytes
Storage: 470 logical bytes, 3995 allocated bytes, 11.76% utilization
Type: struct <span class="o">{</span>
<span class="s2">"ADCcount"</span> +0 native unsigned short
<span class="s2">"TDCcount"</span> +2 native unsigned char
<span class="s2">"energy"</span> +3 native double
<span class="s2">"grid_i"</span> +11 native int
<span class="s2">"grid_j"</span> +15 native int
<span class="s2">"idnumber"</span> +19 native long long
<span class="s2">"name"</span> +27 16-byte null-terminated ASCII string
<span class="s2">"pressure"</span> +43 native float
<span class="o">}</span> 47 bytes
</pre></div>
</div>
<p>Attributes are a useful mechanism to add persistent (meta) information to
your data.</p>
</div>
<div class="section" id="getting-object-metadata">
<h3>Getting object metadata<a class="headerlink" href="#getting-object-metadata" title="Permalink to this headline">¶</a></h3>
<p>Each object in PyTables has <em>metadata</em> information about the data in the
file. Normally this <em>meta-information</em> is accessible through the node
instance variables. Let’s take a look at some examples:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">print</span> <span class="s">"Object:"</span><span class="p">,</span> <span class="n">table</span>
<span class="go">Object: /detector/readout (Table(10,)) 'Readout example'</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Table name:"</span><span class="p">,</span> <span class="n">table</span><span class="o">.</span><span class="n">name</span>
<span class="go">Table name: readout</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Table title:"</span><span class="p">,</span> <span class="n">table</span><span class="o">.</span><span class="n">title</span>
<span class="go">Table title: Readout example</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Number of rows in table:"</span><span class="p">,</span> <span class="n">table</span><span class="o">.</span><span class="n">nrows</span>
<span class="go">Number of rows in table: 10</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Table variable names with their type and shape:"</span>
<span class="go">Table variable names with their type and shape:</span>
<span class="gp">>>> </span><span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">colnames</span><span class="p">:</span>
<span class="gp">... </span> <span class="k">print</span> <span class="n">name</span><span class="p">,</span> <span class="s">':= </span><span class="si">%s</span><span class="s">, </span><span class="si">%s</span><span class="s">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">coldtypes</span><span class="p">[</span><span class="n">name</span><span class="p">],</span> <span class="n">table</span><span class="o">.</span><span class="n">coldtypes</span><span class="p">[</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
<span class="go">ADCcount := uint16, ()</span>
<span class="go">TDCcount := uint8, ()</span>
<span class="go">energy := float64, ()</span>
<span class="go">grid_i := int32, ()</span>
<span class="go">grid_j := int32, ()</span>
<span class="go">idnumber := int64, ()</span>
<span class="go">name := |S16, ()</span>
<span class="go">pressure := float32, ()</span>
</pre></div>
</div>
<p>Here, the name, title, nrows, colnames and coldtypes attributes (see
<a class="reference internal" href="libref/structured_storage.html#tables.Table" title="tables.Table"><tt class="xref py py-class docutils literal"><span class="pre">Table</span></tt></a> for a complete attribute list) of the Table object gives us
quite a bit of information about the table data.</p>
<p>You can interactively retrieve general information about the public objects
in PyTables by asking for help:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">help</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
<span class="go">Help on Table in module tables.table:</span>
<span class="go">class Table(tableextension.Table, tables.leaf.Leaf)</span>
<span class="go">| This class represents heterogeneous datasets in an HDF5 file.</span>
<span class="go">|</span>
<span class="go">| Tables are leaves (see the `Leaf` class) whose data consists of a</span>
<span class="go">| unidimensional sequence of *rows*, where each row contains one or</span>
<span class="go">| more *fields*. Fields have an associated unique *name* and</span>
<span class="go">| *position*, with the first field having position 0. All rows have</span>
<span class="go">| the same fields, which are arranged in *columns*.</span>
<span class="go">[snip]</span>
<span class="go">|</span>
<span class="go">| Instance variables</span>
<span class="go">| ------------------</span>
<span class="go">|</span>
<span class="go">| The following instance variables are provided in addition to those</span>
<span class="go">| in `Leaf`. Please note that there are several `col` dictionaries</span>
<span class="go">| to ease retrieving information about a column directly by its path</span>
<span class="go">| name, avoiding the need to walk through `Table.description` or</span>
<span class="go">| `Table.cols`.</span>
<span class="go">|</span>
<span class="go">| autoindex</span>
<span class="go">| Automatically keep column indexes up to date?</span>
<span class="go">|</span>
<span class="go">| Setting this value states whether existing indexes should be</span>
<span class="go">| automatically updated after an append operation or recomputed</span>
<span class="go">| after an index-invalidating operation (i.e. removal and</span>
<span class="go">| modification of rows). The default is true.</span>
<span class="go">[snip]</span>
<span class="go">| rowsize</span>
<span class="go">| The size in bytes of each row in the table.</span>
<span class="go">|</span>
<span class="go">| Public methods -- reading</span>
<span class="go">| -------------------------</span>
<span class="go">|</span>
<span class="go">| * col(name)</span>
<span class="go">| * iterrows([start][, stop][, step])</span>
<span class="go">| * itersequence(sequence)</span>
<span class="go">* itersorted(sortby[, checkCSI][, start][, stop][, step])</span>
<span class="go">| * read([start][, stop][, step][, field][, coords])</span>
<span class="go">| * read_coordinates(coords[, field])</span>
<span class="go">* read_sorted(sortby[, checkCSI][, field,][, start][, stop][, step])</span>
<span class="go">| * __getitem__(key)</span>
<span class="go">| * __iter__()</span>
<span class="go">|</span>
<span class="go">| Public methods -- writing</span>
<span class="go">| -------------------------</span>
<span class="go">|</span>
<span class="go">| * append(rows)</span>
<span class="go">| * modify_column([start][, stop][, step][, column][, colname])</span>
<span class="go">[snip]</span>
</pre></div>
</div>
<p>Try getting help with other object docs by yourself:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">help</span><span class="p">(</span><span class="n">h5file</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">help</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">remove_rows</span><span class="p">)</span>
</pre></div>
</div>
<p>To examine metadata in the <em>/columns/pressure</em> Array object:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">pressureObject</span> <span class="o">=</span> <span class="n">h5file</span><span class="o">.</span><span class="n">get_node</span><span class="p">(</span><span class="s">"/columns"</span><span class="p">,</span> <span class="s">"pressure"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Info on the object:"</span><span class="p">,</span> <span class="nb">repr</span><span class="p">(</span><span class="n">pressureObject</span><span class="p">)</span>
<span class="go">Info on the object: /columns/pressure (Array(3,)) 'Pressure column selection'</span>
<span class="go"> atom := Float64Atom(shape=(), dflt=0.0)</span>
<span class="go"> maindim := 0</span>
<span class="go"> flavor := 'numpy'</span>
<span class="go"> byteorder := 'little'</span>
<span class="go"> chunkshape := None</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">" shape: ==>"</span><span class="p">,</span> <span class="n">pressureObject</span><span class="o">.</span><span class="n">shape</span>
<span class="go"> shape: ==> (3,)</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">" title: ==>"</span><span class="p">,</span> <span class="n">pressureObject</span><span class="o">.</span><span class="n">title</span>
<span class="go"> title: ==> Pressure column selection</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">" atom: ==>"</span><span class="p">,</span> <span class="n">pressureObject</span><span class="o">.</span><span class="n">atom</span>
<span class="go"> atom: ==> Float64Atom(shape=(), dflt=0.0)</span>
</pre></div>
</div>
<p>Observe that we have used the <a class="reference internal" href="libref/file_class.html#tables.File.get_node" title="tables.File.get_node"><tt class="xref py py-meth docutils literal"><span class="pre">File.get_node()</span></tt></a> method of the File class
to access a node in the tree, instead of the natural naming method. Both are
useful, and depending on the context you will prefer one or the other.
<a class="reference internal" href="libref/file_class.html#tables.File.get_node" title="tables.File.get_node"><tt class="xref py py-meth docutils literal"><span class="pre">File.get_node()</span></tt></a> has the advantage that it can get a node from the
pathname string (as in this example) and can also act as a filter to show
only nodes in a particular location that are instances of class <em>classname</em>.
In general, however, I consider natural naming to be more elegant and easier
to use, especially if you are using the name completion capability present in
interactive console. Try this powerful combination of natural naming and
completion capabilities present in most Python consoles, and see how pleasant
it is to browse the object tree (well, as pleasant as such an activity can
be).</p>
<p>If you look at the type attribute of the pressureObject object, you can
verify that it is a “<em>float64</em>” array. By looking at its shape attribute, you
can deduce that the array on disk is unidimensional and has 3 elements.
See <a class="reference internal" href="libref/homogenous_storage.html#tables.Array" title="tables.Array"><tt class="xref py py-class docutils literal"><span class="pre">Array</span></tt></a> or the internal doc strings for the complete Array
attribute list.</p>
</div>
<div class="section" id="reading-data-from-array-objects">
<h3>Reading data from Array objects<a class="headerlink" href="#reading-data-from-array-objects" title="Permalink to this headline">¶</a></h3>
<p>Once you have found the desired Array, use the read() method of the Array
object to retrieve its data:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">pressureArray</span> <span class="o">=</span> <span class="n">pressureObject</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">pressureArray</span>
<span class="go">array([ 25., 36., 49.])</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"pressureArray is an object of type:"</span><span class="p">,</span> <span class="nb">type</span><span class="p">(</span><span class="n">pressureArray</span><span class="p">)</span>
<span class="go">pressureArray is an object of type: <type 'numpy.ndarray'></span>
<span class="gp">>>> </span><span class="n">nameArray</span> <span class="o">=</span> <span class="n">h5file</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">columns</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"nameArray is an object of type:"</span><span class="p">,</span> <span class="nb">type</span><span class="p">(</span><span class="n">nameArray</span><span class="p">)</span>
<span class="go">nameArray is an object of type: <type 'list'></span>
<span class="go">>>></span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Data on arrays nameArray and pressureArray:"</span>
<span class="go">Data on arrays nameArray and pressureArray:</span>
<span class="gp">>>> </span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">pressureObject</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]):</span>
<span class="gp">... </span> <span class="k">print</span> <span class="n">nameArray</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="s">"-->"</span><span class="p">,</span> <span class="n">pressureArray</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
<span class="go">Particle: 5 --> 25.0</span>
<span class="go">Particle: 6 --> 36.0</span>
<span class="go">Particle: 7 --> 49.0</span>
</pre></div>
</div>
<p>You can see that the <a class="reference internal" href="libref/homogenous_storage.html#tables.Array.read" title="tables.Array.read"><tt class="xref py py-meth docutils literal"><span class="pre">Array.read()</span></tt></a> method returns an authentic NumPy
object for the pressureObject instance by looking at the output of the type()
call. A read() of the nameArray object instance returns a native Python list
(of strings). The type of the object saved is stored as an HDF5 attribute
(named FLAVOR) for objects on disk. This attribute is then read as Array
meta-information (accessible through in the Array.attrs.FLAVOR variable),
enabling the read array to be converted into the original object. This
provides a means to save a large variety of objects as arrays with the
guarantee that you will be able to later recover them in their original form.
See <a class="reference internal" href="libref/file_class.html#tables.File.create_array" title="tables.File.create_array"><tt class="xref py py-meth docutils literal"><span class="pre">File.create_array()</span></tt></a> for a complete list of supported objects for the
Array object class.</p>
</div>
</div>
<div class="section" id="commiting-data-to-tables-and-arrays">
<h2>Commiting data to tables and arrays<a class="headerlink" href="#commiting-data-to-tables-and-arrays" title="Permalink to this headline">¶</a></h2>
<p>We have seen how to create tables and arrays and how to browse both data and
metadata in the object tree. Let’s examine more closely now one of the most
powerful capabilities of PyTables, namely, how to modify already created
tables and arrays <a class="footnote-reference" href="#id5" id="id1">[1]</a></p>
<div class="section" id="appending-data-to-an-existing-table">
<h3>Appending data to an existing table<a class="headerlink" href="#appending-data-to-an-existing-table" title="Permalink to this headline">¶</a></h3>
<p>Now, let’s have a look at how we can add records to an existing table on
disk. Let’s use our well-known <em>readout</em> Table object and append some new
values to it:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span> <span class="o">=</span> <span class="n">h5file</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">detector</span><span class="o">.</span><span class="n">readout</span>
<span class="gp">>>> </span><span class="n">particle</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">row</span>
<span class="gp">>>> </span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">15</span><span class="p">):</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'name'</span><span class="p">]</span> <span class="o">=</span> <span class="s">'Particle: </span><span class="si">%6d</span><span class="s">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'TDCcount'</span><span class="p">]</span> <span class="o">=</span> <span class="n">i</span> <span class="o">%</span> <span class="mi">256</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'ADCcount'</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span> <span class="o">*</span> <span class="mi">256</span><span class="p">)</span> <span class="o">%</span> <span class="p">(</span><span class="mi">1</span> <span class="o"><<</span> <span class="mi">16</span><span class="p">)</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'grid_i'</span><span class="p">]</span> <span class="o">=</span> <span class="n">i</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'grid_j'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">10</span> <span class="o">-</span> <span class="n">i</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'pressure'</span><span class="p">]</span> <span class="o">=</span> <span class="nb">float</span><span class="p">(</span><span class="n">i</span><span class="o">*</span><span class="n">i</span><span class="p">)</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'energy'</span><span class="p">]</span> <span class="o">=</span> <span class="nb">float</span><span class="p">(</span><span class="n">particle</span><span class="p">[</span><span class="s">'pressure'</span><span class="p">]</span> <span class="o">**</span> <span class="mi">4</span><span class="p">)</span>
<span class="gp">... </span> <span class="n">particle</span><span class="p">[</span><span class="s">'idnumber'</span><span class="p">]</span> <span class="o">=</span> <span class="n">i</span> <span class="o">*</span> <span class="p">(</span><span class="mi">2</span> <span class="o">**</span> <span class="mi">34</span><span class="p">)</span>
<span class="gp">... </span> <span class="n">particle</span><span class="o">.</span><span class="n">append</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
</pre></div>
</div>
<p>It’s the same method we used to fill a new table. PyTables knows that this
table is on disk, and when you add new records, they are appended to the end
of the table <a class="footnote-reference" href="#id6" id="id2">[2]</a>.</p>
<p>If you look carefully at the code you will see that we have used the
table.row attribute to create a table row and fill it with the new values.
Each time that its append() method is called, the actual row is committed to
the output buffer and the row pointer is incremented to point to the next
table record. When the buffer is full, the data is saved on disk, and the
buffer is reused again for the next cycle.</p>
<p><em>Caveat emptor</em>: Do not forget to always call the flush() method after a
write operation, or else your tables will not be updated!</p>
<p>Let’s have a look at some rows in the modified table and verify that our new
data has been appended:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">iterrows</span><span class="p">():</span>
<span class="gp">... </span> <span class="k">print</span> <span class="s">"</span><span class="si">%-16s</span><span class="s"> | </span><span class="si">%11.1f</span><span class="s"> | </span><span class="si">%11.4g</span><span class="s"> | </span><span class="si">%6d</span><span class="s"> | </span><span class="si">%6d</span><span class="s"> | </span><span class="si">%8d</span><span class="s"> \|"</span> <span class="o">%</span> \\
<span class="gp">... </span> <span class="p">(</span><span class="n">r</span><span class="p">[</span><span class="s">'name'</span><span class="p">],</span> <span class="n">r</span><span class="p">[</span><span class="s">'pressure'</span><span class="p">],</span> <span class="n">r</span><span class="p">[</span><span class="s">'energy'</span><span class="p">],</span> <span class="n">r</span><span class="p">[</span><span class="s">'grid_i'</span><span class="p">],</span> <span class="n">r</span><span class="p">[</span><span class="s">'grid_j'</span><span class="p">],</span>
<span class="gp">... </span> <span class="n">r</span><span class="p">[</span><span class="s">'TDCcount'</span><span class="p">])</span>
<span class="go">Particle: 0 | 0.0 | 0 | 0 | 10 | 0 |</span>
<span class="go">Particle: 1 | 1.0 | 1 | 1 | 9 | 1 |</span>
<span class="go">Particle: 2 | 4.0 | 256 | 2 | 8 | 2 |</span>
<span class="go">Particle: 3 | 9.0 | 6561 | 3 | 7 | 3 |</span>
<span class="go">Particle: 4 | 16.0 | 6.554e+04 | 4 | 6 | 4 |</span>
<span class="go">Particle: 5 | 25.0 | 3.906e+05 | 5 | 5 | 5 |</span>
<span class="go">Particle: 6 | 36.0 | 1.68e+06 | 6 | 4 | 6 |</span>
<span class="go">Particle: 7 | 49.0 | 5.765e+06 | 7 | 3 | 7 |</span>
<span class="go">Particle: 8 | 64.0 | 1.678e+07 | 8 | 2 | 8 |</span>
<span class="go">Particle: 9 | 81.0 | 4.305e+07 | 9 | 1 | 9 |</span>
<span class="go">Particle: 10 | 100.0 | 1e+08 | 10 | 0 | 10 |</span>
<span class="go">Particle: 11 | 121.0 | 2.144e+08 | 11 | -1 | 11 |</span>
<span class="go">Particle: 12 | 144.0 | 4.3e+08 | 12 | -2 | 12 |</span>
<span class="go">Particle: 13 | 169.0 | 8.157e+08 | 13 | -3 | 13 |</span>
<span class="go">Particle: 14 | 196.0 | 1.476e+09 | 14 | -4 | 14 |</span>
</pre></div>
</div>
</div>
<div class="section" id="modifying-data-in-tables">
<h3>Modifying data in tables<a class="headerlink" href="#modifying-data-in-tables" title="Permalink to this headline">¶</a></h3>
<p>Ok, until now, we’ve been only reading and writing (appending) values to our
tables. But there are times that you need to modify your data once you have
saved it on disk (this is specially true when you need to modify the real
world data to adapt your goals ;).
Let’s see how we can modify the values that were saved in our existing tables.
We will start modifying single cells in the first row of the Particle table:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">print</span> <span class="s">"Before modif-->"</span><span class="p">,</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="go">Before modif--> (0, 0, 0.0, 0, 10, 0L, 'Particle: 0', 0.0)</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">cols</span><span class="o">.</span><span class="n">TDCcount</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="mi">1</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"After modifying first row of ADCcount-->"</span><span class="p">,</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="go">After modifying first row of ADCcount--> (0, 1, 0.0, 0, 10, 0L, 'Particle: 0', 0.0)</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">cols</span><span class="o">.</span><span class="n">energy</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="mi">2</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"After modifying first row of energy-->"</span><span class="p">,</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="go">After modifying first row of energy--> (0, 1, 2.0, 0, 10, 0L, 'Particle: 0', 0.0)</span>
</pre></div>
</div>
<p>We can modify complete ranges of columns as well:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">cols</span><span class="o">.</span><span class="n">TDCcount</span><span class="p">[</span><span class="mi">2</span><span class="p">:</span><span class="mi">5</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">]</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"After modifying slice [2:5] of TDCcount-->"</span><span class="p">,</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">5</span><span class="p">]</span>
<span class="go">After modifying slice [2:5] of TDCcount--></span>
<span class="go">[(0, 1, 2.0, 0, 10, 0L, 'Particle: 0', 0.0)</span>
<span class="go"> (256, 1, 1.0, 1, 9, 17179869184L, 'Particle: 1', 1.0)</span>
<span class="go"> (512, 2, 256.0, 2, 8, 34359738368L, 'Particle: 2', 4.0)</span>
<span class="go"> (768, 3, 6561.0, 3, 7, 51539607552L, 'Particle: 3', 9.0)</span>
<span class="go"> (1024, 4, 65536.0, 4, 6, 68719476736L, 'Particle: 4', 16.0)]</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">cols</span><span class="o">.</span><span class="n">energy</span><span class="p">[</span><span class="mi">1</span><span class="p">:</span><span class="mi">9</span><span class="p">:</span><span class="mi">3</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">]</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"After modifying slice [1:9:3] of energy-->"</span><span class="p">,</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">9</span><span class="p">]</span>
<span class="go">After modifying slice [1:9:3] of energy--></span>
<span class="go">[(0, 1, 2.0, 0, 10, 0L, 'Particle: 0', 0.0)</span>
<span class="go"> (256, 1, 2.0, 1, 9, 17179869184L, 'Particle: 1', 1.0)</span>
<span class="go"> (512, 2, 256.0, 2, 8, 34359738368L, 'Particle: 2', 4.0)</span>
<span class="go"> (768, 3, 6561.0, 3, 7, 51539607552L, 'Particle: 3', 9.0)</span>
<span class="go"> (1024, 4, 3.0, 4, 6, 68719476736L, 'Particle: 4', 16.0)</span>
<span class="go"> (1280, 5, 390625.0, 5, 5, 85899345920L, 'Particle: 5', 25.0)</span>
<span class="go"> (1536, 6, 1679616.0, 6, 4, 103079215104L, 'Particle: 6', 36.0)</span>
<span class="go"> (1792, 7, 4.0, 7, 3, 120259084288L, 'Particle: 7', 49.0)</span>
<span class="go"> (2048, 8, 16777216.0, 8, 2, 137438953472L, 'Particle: 8', 64.0)]</span>
</pre></div>
</div>
<p>Check that the values have been correctly modified!</p>
<div class="admonition hint">
<p class="first admonition-title">Hint</p>
<p class="last">remember that column TDCcount is the second one, and that energy is the
third. Look for more info on modifying columns in
<a class="reference internal" href="libref/structured_storage.html#tables.Column.__setitem__" title="tables.Column.__setitem__"><tt class="xref py py-meth docutils literal"><span class="pre">Column.__setitem__()</span></tt></a>.</p>
</div>
<p>PyTables also lets you modify complete sets of rows at the same time. As a
demonstration of these capability, see the next example:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">modify_rows</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">step</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span>
<span class="gp">... </span> <span class="n">rows</span><span class="o">=</span><span class="p">[(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mf">3.0</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="il">6L</span><span class="p">,</span> <span class="s">'Particle: None'</span><span class="p">,</span> <span class="mf">8.0</span><span class="p">),</span>
<span class="gp">... </span> <span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mf">6.0</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="il">12L</span><span class="p">,</span> <span class="s">'Particle: None*2'</span><span class="p">,</span> <span class="mf">16.0</span><span class="p">)])</span>
<span class="go">2</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"After modifying the complete third row-->"</span><span class="p">,</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">5</span><span class="p">]</span>
<span class="go">After modifying the complete third row--></span>
<span class="go">[(0, 1, 2.0, 0, 10, 0L, 'Particle: 0', 0.0)</span>
<span class="go"> (1, 2, 3.0, 4, 5, 6L, 'Particle: None', 8.0)</span>
<span class="go"> (512, 2, 256.0, 2, 8, 34359738368L, 'Particle: 2', 4.0)</span>
<span class="go"> (768, 3, 6561.0, 3, 7, 51539607552L, 'Particle: 3', 9.0)</span>
<span class="go"> (2, 4, 6.0, 8, 10, 12L, 'Particle: None*2', 16.0)]</span>
</pre></div>
</div>
<p>As you can see, the modify_rows() call has modified the rows second and fifth,
and it returned the number of modified rows.</p>
<p>Apart of <a class="reference internal" href="libref/structured_storage.html#tables.Table.modify_rows" title="tables.Table.modify_rows"><tt class="xref py py-meth docutils literal"><span class="pre">Table.modify_rows()</span></tt></a>, there exists another method, called
<a class="reference internal" href="libref/structured_storage.html#tables.Table.modify_column" title="tables.Table.modify_column"><tt class="xref py py-meth docutils literal"><span class="pre">Table.modify_column()</span></tt></a> to modify specific columns as well.</p>
<p>Finally, it exists another way of modifying tables that is generally more
handy than the described above. This new way uses the method
<tt class="xref py py-meth docutils literal"><span class="pre">Row.update()</span></tt> of the Row instance that is attached to every table, so it
is meant to be used in table iterators. Look at the next example:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="s">'TDCcount <= 2'</span><span class="p">):</span>
<span class="gp">... </span> <span class="n">row</span><span class="p">[</span><span class="s">'energy'</span><span class="p">]</span> <span class="o">=</span> <span class="n">row</span><span class="p">[</span><span class="s">'TDCcount'</span><span class="p">]</span><span class="o">*</span><span class="mi">2</span>
<span class="gp">... </span> <span class="n">row</span><span class="o">.</span><span class="n">update</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"After modifying energy column (where TDCcount <=2)-->"</span><span class="p">,</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">4</span><span class="p">]</span>
<span class="go">After modifying energy column (where TDCcount <=2)--></span>
<span class="go">[(0, 1, 2.0, 0, 10, 0L, 'Particle: 0', 0.0)</span>
<span class="go"> (1, 2, 4.0, 4, 5, 6L, 'Particle: None', 8.0)</span>
<span class="go"> (512, 2, 4.0, 2, 8, 34359738368L, 'Particle: 2', 4.0)</span>
<span class="go"> (768, 3, 6561.0, 3, 7, 51539607552L, 'Particle: 3', 9.0)]</span>
</pre></div>
</div>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">The authors find this way of updating tables (i.e. using Row.update())
to be both convenient and efficient. Please make sure to use it
extensively.</p>
</div>
</div>
<div class="section" id="modifying-data-in-arrays">
<h3>Modifying data in arrays<a class="headerlink" href="#modifying-data-in-arrays" title="Permalink to this headline">¶</a></h3>
<p>We are going now to see how to modify data in array objects.
The basic way to do this is through the use of <a class="reference internal" href="libref/homogenous_storage.html#tables.Array.__setitem__" title="tables.Array.__setitem__"><tt class="xref py py-meth docutils literal"><span class="pre">Array.__setitem__()</span></tt></a>
special method. Let’s see at how modify data on the pressureObject array:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">pressureObject</span> <span class="o">=</span> <span class="n">h5file</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">columns</span><span class="o">.</span><span class="n">pressure</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Before modif-->"</span><span class="p">,</span> <span class="n">pressureObject</span><span class="p">[:]</span>
<span class="go">Before modif--> [ 25. 36. 49.]</span>
<span class="gp">>>> </span><span class="n">pressureObject</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="mi">2</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"First modif-->"</span><span class="p">,</span> <span class="n">pressureObject</span><span class="p">[:]</span>
<span class="go">First modif--> [ 2. 36. 49.]</span>
<span class="gp">>>> </span><span class="n">pressureObject</span><span class="p">[</span><span class="mi">1</span><span class="p">:</span><span class="mi">3</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="mf">2.1</span><span class="p">,</span> <span class="mf">3.5</span><span class="p">]</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Second modif-->"</span><span class="p">,</span> <span class="n">pressureObject</span><span class="p">[:]</span>
<span class="go">Second modif--> [ 2. 2.1 3.5]</span>
<span class="gp">>>> </span><span class="n">pressureObject</span><span class="p">[::</span><span class="mi">2</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">]</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Third modif-->"</span><span class="p">,</span> <span class="n">pressureObject</span><span class="p">[:]</span>
<span class="go">Third modif--> [ 1. 2.1 2. ]</span>
</pre></div>
</div>
<p>So, in general, you can use any combination of (multidimensional) extended
slicing.</p>
<p>With the sole exception that you cannot use negative values for step to refer
to indexes that you want to modify. See <a class="reference internal" href="libref/homogenous_storage.html#tables.Array.__getitem__" title="tables.Array.__getitem__"><tt class="xref py py-meth docutils literal"><span class="pre">Array.__getitem__()</span></tt></a> for more
examples on how to use extended slicing in PyTables objects.</p>
<p>Similarly, with and array of strings:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">nameObject</span> <span class="o">=</span> <span class="n">h5file</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">columns</span><span class="o">.</span><span class="n">name</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Before modif-->"</span><span class="p">,</span> <span class="n">nameObject</span><span class="p">[:]</span>
<span class="go">Before modif--> ['Particle: 5', 'Particle: 6', 'Particle: 7']</span>
<span class="gp">>>> </span><span class="n">nameObject</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="s">'Particle: None'</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"First modif-->"</span><span class="p">,</span> <span class="n">nameObject</span><span class="p">[:]</span>
<span class="go">First modif--> ['Particle: None', 'Particle: 6', 'Particle: 7']</span>
<span class="gp">>>> </span><span class="n">nameObject</span><span class="p">[</span><span class="mi">1</span><span class="p">:</span><span class="mi">3</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="s">'Particle: 0'</span><span class="p">,</span> <span class="s">'Particle: 1'</span><span class="p">]</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Second modif-->"</span><span class="p">,</span> <span class="n">nameObject</span><span class="p">[:]</span>
<span class="go">Second modif--> ['Particle: None', 'Particle: 0', 'Particle: 1']</span>
<span class="gp">>>> </span><span class="n">nameObject</span><span class="p">[::</span><span class="mi">2</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="s">'Particle: -3'</span><span class="p">,</span> <span class="s">'Particle: -5'</span><span class="p">]</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Third modif-->"</span><span class="p">,</span> <span class="n">nameObject</span><span class="p">[:]</span>
<span class="go">Third modif--> ['Particle: -3', 'Particle: 0', 'Particle: -5']</span>
</pre></div>
</div>
</div>
<div class="section" id="and-finally-how-to-delete-rows-from-a-table">
<h3>And finally... how to delete rows from a table<a class="headerlink" href="#and-finally-how-to-delete-rows-from-a-table" title="Permalink to this headline">¶</a></h3>
<p>We’ll finish this tutorial by deleting some rows from the table we have.
Suppose that we want to delete the the 5th to 9th rows (inclusive):</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">remove_rows</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span><span class="mi">10</span><span class="p">)</span>
<span class="go">5</span>
</pre></div>
</div>
<p><a class="reference internal" href="libref/structured_storage.html#tables.Table.remove_rows" title="tables.Table.remove_rows"><tt class="xref py py-meth docutils literal"><span class="pre">Table.remove_rows()</span></tt></a> deletes the rows in the range (start, stop). It
returns the number of rows effectively removed.</p>
<p>We have reached the end of this first tutorial. Don’t forget to close the
file when you finish:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">h5file</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="gp">>>> </span><span class="o">^</span><span class="n">D</span>
<span class="go">$</span>
</pre></div>
</div>
<p>In <a class="reference internal" href="#tutorial1-2-tableview"><em>Figure 2</em></a> you can see a graphical view of the
PyTables file with the datasets we have just created. In
<a class="reference internal" href="#tutorial1-general"><em>Figure 3. General properties of the /detector/readout table.</em></a> are displayed the general properties of the table
/detector/readout.</p>
<div class="figure align-center" id="tutorial1-2-tableview">
<img alt="../_images/tutorial1-2-tableview.png" src="../_images/tutorial1-2-tableview.png" />
<p class="caption"><strong>Figure 2. The final version of the data file for tutorial 1.</strong></p>
</div>
<div class="figure align-center" id="tutorial1-general">
<img alt="../_images/tutorial1-general.png" src="../_images/tutorial1-general.png" />
<p class="caption"><strong>Figure 3. General properties of the /detector/readout table.</strong></p>
</div>
</div>
</div>
<div class="section" id="multidimensional-table-cells-and-automatic-sanity-checks">
<span id="secondexample"></span><h2>Multidimensional table cells and automatic sanity checks<a class="headerlink" href="#multidimensional-table-cells-and-automatic-sanity-checks" title="Permalink to this headline">¶</a></h2>
<p>Now it’s time for a more real-life example (i.e. with errors in the code). We
will create two groups that branch directly from the root node, Particles and
Events. Then, we will put three tables in each group. In Particles we will
put tables based on the Particle descriptor and in Events, the tables based
the Event descriptor.</p>
<p>Afterwards, we will provision the tables with a number of records. Finally,
we will read the newly-created table /Events/TEvent3 and select some values
from it, using a comprehension list.</p>
<p>Look at the next script (you can find it in <tt class="file docutils literal"><span class="pre">examples/tutorial2.py</span></tt>).
It appears to do all of the above, but it contains some small bugs. Note that
this Particle class is not directly related to the one defined in
last tutorial; this class is simpler (note, however, the <em>multidimensional</em>
columns called pressure and temperature).</p>
<p>We also introduce a new manner to describe a Table as a structured NumPy
dtype (or even as a dictionary), as you can see in the Event description. See
<a class="reference internal" href="libref/file_class.html#tables.File.create_table" title="tables.File.create_table"><tt class="xref py py-meth docutils literal"><span class="pre">File.create_table()</span></tt></a> about the different kinds of descriptor objects that
can be passed to this method:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">tables</span> <span class="kn">import</span> <span class="o">*</span>
<span class="kn">from</span> <span class="nn">numpy</span> <span class="kn">import</span> <span class="o">*</span>
<span class="c"># Describe a particle record</span>
<span class="k">class</span> <span class="nc">Particle</span><span class="p">(</span><span class="n">IsDescription</span><span class="p">):</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">StringCol</span><span class="p">(</span><span class="n">itemsize</span><span class="o">=</span><span class="mi">16</span><span class="p">)</span> <span class="c"># 16-character string</span>
<span class="n">lati</span> <span class="o">=</span> <span class="n">Int32Col</span><span class="p">()</span> <span class="c"># integer</span>
<span class="n">longi</span> <span class="o">=</span> <span class="n">Int32Col</span><span class="p">()</span> <span class="c"># integer</span>
<span class="n">pressure</span> <span class="o">=</span> <span class="n">Float32Col</span><span class="p">(</span><span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">))</span> <span class="c"># array of floats (single-precision)</span>
<span class="n">temperature</span> <span class="o">=</span> <span class="n">Float64Col</span><span class="p">(</span><span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">))</span> <span class="c"># array of doubles (double-precision)</span>
<span class="c"># Native NumPy dtype instances are also accepted</span>
<span class="n">Event</span> <span class="o">=</span> <span class="n">dtype</span><span class="p">([</span>
<span class="p">(</span><span class="s">"name"</span> <span class="p">,</span> <span class="s">"S16"</span><span class="p">),</span>
<span class="p">(</span><span class="s">"TDCcount"</span> <span class="p">,</span> <span class="n">uint8</span><span class="p">),</span>
<span class="p">(</span><span class="s">"ADCcount"</span> <span class="p">,</span> <span class="n">uint16</span><span class="p">),</span>
<span class="p">(</span><span class="s">"xcoord"</span> <span class="p">,</span> <span class="n">float32</span><span class="p">),</span>
<span class="p">(</span><span class="s">"ycoord"</span> <span class="p">,</span> <span class="n">float32</span><span class="p">)</span>
<span class="p">])</span>
<span class="c"># And dictionaries too (this defines the same structure as above)</span>
<span class="c"># Event = {</span>
<span class="c"># "name" : StringCol(itemsize=16),</span>
<span class="c"># "TDCcount" : UInt8Col(),</span>
<span class="c"># "ADCcount" : UInt16Col(),</span>
<span class="c"># "xcoord" : Float32Col(),</span>
<span class="c"># "ycoord" : Float32Col(),</span>
<span class="c"># }</span>
<span class="c"># Open a file in "w"rite mode</span>
<span class="n">fileh</span> <span class="o">=</span> <span class="n">open_file</span><span class="p">(</span><span class="s">"tutorial2.h5"</span><span class="p">,</span> <span class="n">mode</span> <span class="o">=</span> <span class="s">"w"</span><span class="p">)</span>
<span class="c"># Get the HDF5 root group</span>
<span class="n">root</span> <span class="o">=</span> <span class="n">fileh</span><span class="o">.</span><span class="n">root</span>
<span class="c"># Create the groups:</span>
<span class="k">for</span> <span class="n">groupname</span> <span class="ow">in</span> <span class="p">(</span><span class="s">"Particles"</span><span class="p">,</span> <span class="s">"Events"</span><span class="p">):</span>
<span class="n">group</span> <span class="o">=</span> <span class="n">fileh</span><span class="o">.</span><span class="n">create_group</span><span class="p">(</span><span class="n">root</span><span class="p">,</span> <span class="n">groupname</span><span class="p">)</span>
<span class="c"># Now, create and fill the tables in Particles group</span>
<span class="n">gparticles</span> <span class="o">=</span> <span class="n">root</span><span class="o">.</span><span class="n">Particles</span>
<span class="c"># Create 3 new tables</span>
<span class="k">for</span> <span class="n">tablename</span> <span class="ow">in</span> <span class="p">(</span><span class="s">"TParticle1"</span><span class="p">,</span> <span class="s">"TParticle2"</span><span class="p">,</span> <span class="s">"TParticle3"</span><span class="p">):</span>
<span class="c"># Create a table</span>
<span class="n">table</span> <span class="o">=</span> <span class="n">fileh</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span><span class="s">"/Particles"</span><span class="p">,</span> <span class="n">tablename</span><span class="p">,</span> <span class="n">Particle</span><span class="p">,</span> <span class="s">"Particles: "</span><span class="o">+</span><span class="n">tablename</span><span class="p">)</span>
<span class="c"># Get the record object associated with the table:</span>
<span class="n">particle</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">row</span>
<span class="c"># Fill the table with 257 particles</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="mi">257</span><span class="p">):</span>
<span class="c"># First, assign the values to the Particle record</span>
<span class="n">particle</span><span class="p">[</span><span class="s">'name'</span><span class="p">]</span> <span class="o">=</span> <span class="s">'Particle: </span><span class="si">%6d</span><span class="s">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="n">particle</span><span class="p">[</span><span class="s">'lati'</span><span class="p">]</span> <span class="o">=</span> <span class="n">i</span>
<span class="n">particle</span><span class="p">[</span><span class="s">'longi'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">10</span> <span class="o">-</span> <span class="n">i</span>
<span class="c">########### Detectable errors start here. Play with them!</span>
<span class="n">particle</span><span class="p">[</span><span class="s">'pressure'</span><span class="p">]</span> <span class="o">=</span> <span class="n">array</span><span class="p">(</span><span class="n">i</span><span class="o">*</span><span class="n">arange</span><span class="p">(</span><span class="mi">2</span><span class="o">*</span><span class="mi">3</span><span class="p">))</span><span class="o">.</span><span class="n">reshape</span><span class="p">((</span><span class="mi">2</span><span class="p">,</span><span class="mi">4</span><span class="p">))</span> <span class="c"># Incorrect</span>
<span class="c">#particle['pressure'] = array(i*arange(2*3)).reshape((2,3)) # Correct</span>
<span class="c">########### End of errors</span>
<span class="n">particle</span><span class="p">[</span><span class="s">'temperature'</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span> <span class="c"># Broadcasting</span>
<span class="c"># This injects the Record values</span>
<span class="n">particle</span><span class="o">.</span><span class="n">append</span><span class="p">()</span>
<span class="c"># Flush the table buffers</span>
<span class="n">table</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="c"># Now, go for Events:</span>
<span class="k">for</span> <span class="n">tablename</span> <span class="ow">in</span> <span class="p">(</span><span class="s">"TEvent1"</span><span class="p">,</span> <span class="s">"TEvent2"</span><span class="p">,</span> <span class="s">"TEvent3"</span><span class="p">):</span>
<span class="c"># Create a table in Events group</span>
<span class="n">table</span> <span class="o">=</span> <span class="n">fileh</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span><span class="n">root</span><span class="o">.</span><span class="n">Events</span><span class="p">,</span> <span class="n">tablename</span><span class="p">,</span> <span class="n">Event</span><span class="p">,</span> <span class="s">"Events: "</span><span class="o">+</span><span class="n">tablename</span><span class="p">)</span>
<span class="c"># Get the record object associated with the table:</span>
<span class="n">event</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">row</span>
<span class="c"># Fill the table with 257 events</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="mi">257</span><span class="p">):</span>
<span class="c"># First, assign the values to the Event record</span>
<span class="n">event</span><span class="p">[</span><span class="s">'name'</span><span class="p">]</span> <span class="o">=</span> <span class="s">'Event: </span><span class="si">%6d</span><span class="s">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="n">event</span><span class="p">[</span><span class="s">'TDCcount'</span><span class="p">]</span> <span class="o">=</span> <span class="n">i</span> <span class="o">%</span> <span class="p">(</span><span class="mi">1</span><span class="o"><<</span><span class="mi">8</span><span class="p">)</span> <span class="c"># Correct range</span>
<span class="c">########### Detectable errors start here. Play with them!</span>
<span class="n">event</span><span class="p">[</span><span class="s">'xcoor'</span><span class="p">]</span> <span class="o">=</span> <span class="nb">float</span><span class="p">(</span><span class="n">i</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span> <span class="c"># Wrong spelling</span>
<span class="c">#event['xcoord'] = float(i**2) # Correct spelling</span>
<span class="n">event</span><span class="p">[</span><span class="s">'ADCcount'</span><span class="p">]</span> <span class="o">=</span> <span class="s">"sss"</span> <span class="c"># Wrong type</span>
<span class="c">#event['ADCcount'] = i * 2 # Correct type</span>
<span class="c">########### End of errors</span>
<span class="n">event</span><span class="p">[</span><span class="s">'ycoord'</span><span class="p">]</span> <span class="o">=</span> <span class="nb">float</span><span class="p">(</span><span class="n">i</span><span class="p">)</span><span class="o">**</span><span class="mi">4</span>
<span class="c"># This injects the Record values</span>
<span class="n">event</span><span class="o">.</span><span class="n">append</span><span class="p">()</span>
<span class="c"># Flush the buffers</span>
<span class="n">table</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="c"># Read the records from table "/Events/TEvent3" and select some</span>
<span class="n">table</span> <span class="o">=</span> <span class="n">root</span><span class="o">.</span><span class="n">Events</span><span class="o">.</span><span class="n">TEvent3</span>
<span class="n">e</span> <span class="o">=</span> <span class="p">[</span> <span class="n">p</span><span class="p">[</span><span class="s">'TDCcount'</span><span class="p">]</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">table</span> <span class="k">if</span> <span class="n">p</span><span class="p">[</span><span class="s">'ADCcount'</span><span class="p">]</span> <span class="o"><</span> <span class="mi">20</span> <span class="ow">and</span> <span class="mi">4</span> <span class="o"><=</span> <span class="n">p</span><span class="p">[</span><span class="s">'TDCcount'</span><span class="p">]</span> <span class="o"><</span> <span class="mi">15</span> <span class="p">]</span>
<span class="k">print</span> <span class="s">"Last record ==>"</span><span class="p">,</span> <span class="n">p</span>
<span class="k">print</span> <span class="s">"Selected values ==>"</span><span class="p">,</span> <span class="n">e</span>
<span class="k">print</span> <span class="s">"Total selected records ==> "</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
<span class="c"># Finally, close the file (this also will flush all the remaining buffers!)</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
</div>
<div class="section" id="shape-checking">
<h3>Shape checking<a class="headerlink" href="#shape-checking" title="Permalink to this headline">¶</a></h3>
<p>If you look at the code carefully, you’ll see that it won’t work. You will
get the following error.</p>
<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span>python tutorial2.py
Traceback <span class="o">(</span>most recent call last<span class="o">)</span>:
File <span class="s2">"tutorial2.py"</span>, line 60, in <module>
particle<span class="o">[</span><span class="s1">'pressure'</span><span class="o">]</span> <span class="o">=</span> array<span class="o">(</span>i*arange<span class="o">(</span>2*3<span class="o">))</span>.reshape<span class="o">((</span>2,4<span class="o">))</span> <span class="c"># Incorrect</span>
ValueError: total size of new array must be unchanged
Closing remaining open files: tutorial2.h5... <span class="k">done</span>
</pre></div>
</div>
<p>This error indicates that you are trying to assign an array with an
incompatible shape to a table cell. Looking at the source, we see that we
were trying to assign an array of shape (2,4) to a pressure element, which
was defined with the shape (2,3).</p>
<p>In general, these kinds of operations are forbidden, with one valid
exception: when you assign a <em>scalar</em> value to a multidimensional column
cell, all the cell elements are populated with the value of the scalar.
For example:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">particle</span><span class="p">[</span><span class="s">'temperature'</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span> <span class="c"># Broadcasting</span>
</pre></div>
</div>
<p>The value i**2 is assigned to all the elements of the temperature table cell.
This capability is provided by the NumPy package and is known as
<em>broadcasting</em>.</p>
</div>
<div class="section" id="field-name-checking">
<h3>Field name checking<a class="headerlink" href="#field-name-checking" title="Permalink to this headline">¶</a></h3>
<p>After fixing the previous error and rerunning the program, we encounter
another error.</p>
<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span>python tutorial2.py
Traceback <span class="o">(</span>most recent call last<span class="o">)</span>:
File <span class="s2">"tutorial2.py"</span>, line 73, in ?
event<span class="o">[</span><span class="s1">'xcoor'</span><span class="o">]</span> <span class="o">=</span> float<span class="o">(</span>i**2<span class="o">)</span> <span class="c"># Wrong spelling</span>
File <span class="s2">"tableextension.pyx"</span>, line 1094, in tableextension.Row.__setitem__
File <span class="s2">"tableextension.pyx"</span>, line 127, in tableextension.get_nested_field_cache
File <span class="s2">"utilsextension.pyx"</span>, line 331, in utilsextension.get_nested_field
KeyError: <span class="s1">'no such column: xcoor'</span>
</pre></div>
</div>
<p>This error indicates that we are attempting to assign a value to a
non-existent field in the <em>event</em> table object. By looking carefully at the
Event class attributes, we see that we misspelled the xcoord field (we wrote
xcoor instead). This is unusual behavior for Python, as normally when you
assign a value to a non-existent instance variable, Python creates a new
variable with that name. Such a feature can be dangerous when dealing with an
object that contains a fixed list of field names. PyTables checks that the
field exists and raises a KeyError if the check fails.</p>
</div>
<div class="section" id="data-type-checking">
<h3>Data type checking<a class="headerlink" href="#data-type-checking" title="Permalink to this headline">¶</a></h3>
<p>Finally, the last issue which we will find here is a TypeError exception.</p>
<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span>python tutorial2.py
Traceback <span class="o">(</span>most recent call last<span class="o">)</span>:
File <span class="s2">"tutorial2.py"</span>, line 75, in ?
event<span class="o">[</span><span class="s1">'ADCcount'</span><span class="o">]</span> <span class="o">=</span> <span class="s2">"sss"</span> <span class="c"># Wrong type</span>
File <span class="s2">"tableextension.pyx"</span>, line 1111, in tableextension.Row.__setitem__
TypeError: invalid <span class="nb">type</span> <span class="o">(</span><<span class="nb">type</span> <span class="s1">'str'</span>><span class="o">)</span> <span class="k">for </span>column <span class="sb">``</span>ADCcount<span class="sb">``</span>
</pre></div>
</div>
<p>And, if we change the affected line to read:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">event</span><span class="o">.</span><span class="n">ADCcount</span> <span class="o">=</span> <span class="n">i</span> <span class="o">*</span> <span class="mi">2</span> <span class="c"># Correct type</span>
</pre></div>
</div>
<p>we will see that the script ends well.</p>
<p>You can see the structure created with this (corrected) script in
<a class="reference internal" href="#tutorial2-tableview"><em>Figure 4</em></a>.
In particular, note the multidimensional column cells in table
/Particles/TParticle2.</p>
<div class="figure align-center" id="tutorial2-tableview">
<img alt="../_images/tutorial2-tableview.png" src="../_images/tutorial2-tableview.png" />
<p class="caption"><strong>Figure 4. Table hierarchy for tutorial 2.</strong></p>
</div>
</div>
</div>
<div class="section" id="using-links-for-more-convenient-access-to-nodes">
<span id="linkstutorial"></span><h2>Using links for more convenient access to nodes<a class="headerlink" href="#using-links-for-more-convenient-access-to-nodes" title="Permalink to this headline">¶</a></h2>
<p>Links are special nodes that can be used to create additional paths to your
existing nodes. PyTables supports three kinds of links: hard links, soft
links (aka symbolic links) and external links.</p>
<p>Hard links let the user create additional paths to access another node in the
same file, and once created, they are indistinguishable from the referred
node object, except that they have different paths in the object tree. For
example, if the referred node is, say, a Table object, then the new hard link
will become a Table object itself. From this point on, you will be able to
access the same Table object from two different paths: the original one and
the new hard link path. If you delete one path to the table, you will be
able to reach it via the other path.</p>
<p>Soft links are similar to hard links, but they keep their own personality.
When you create a soft link to another node, you will get a new SoftLink
object that <em>refers</em> to that node. However, in order to access the referred
node, you need to <em>dereference</em> it.</p>
<p>Finally, external links are like soft links, with the difference that these
are meant to point to nodes in <em>external</em> files instead of nodes in the same
file. They are represented by the ExternalLink class and, like soft links,
you need to dereference them in order to get access to the pointed node.</p>
<div class="section" id="interactive-example">
<h3>Interactive example<a class="headerlink" href="#interactive-example" title="Permalink to this headline">¶</a></h3>
<p>Now we are going to learn how to deal with links. You can find the code used
in this section in <tt class="file docutils literal"><span class="pre">examples/links.py</span></tt>.</p>
<p>First, let’s create a file with some group structure:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">import</span> <span class="nn">tables</span> <span class="kn">as</span> <span class="nn">tb</span>
<span class="gp">>>> </span><span class="n">f1</span> <span class="o">=</span> <span class="n">tb</span><span class="o">.</span><span class="n">open_file</span><span class="p">(</span><span class="s">'links1.h5'</span><span class="p">,</span> <span class="s">'w'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">g1</span> <span class="o">=</span> <span class="n">f1</span><span class="o">.</span><span class="n">create_group</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="s">'g1'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">g2</span> <span class="o">=</span> <span class="n">f1</span><span class="o">.</span><span class="n">create_group</span><span class="p">(</span><span class="n">g1</span><span class="p">,</span> <span class="s">'g2'</span><span class="p">)</span>
</pre></div>
</div>
<p>Now, we will put some datasets on the /g1 and /g1/g2 groups:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">a1</span> <span class="o">=</span> <span class="n">f1</span><span class="o">.</span><span class="n">create_carray</span><span class="p">(</span><span class="n">g1</span><span class="p">,</span> <span class="s">'a1'</span><span class="p">,</span> <span class="n">tb</span><span class="o">.</span><span class="n">Int64Atom</span><span class="p">(),</span> <span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="mi">10000</span><span class="p">,))</span>
<span class="gp">>>> </span><span class="n">t1</span> <span class="o">=</span> <span class="n">f1</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span><span class="n">g2</span><span class="p">,</span> <span class="s">'t1'</span><span class="p">,</span> <span class="p">{</span><span class="s">'f1'</span><span class="p">:</span> <span class="n">tb</span><span class="o">.</span><span class="n">IntCol</span><span class="p">(),</span> <span class="s">'f2'</span><span class="p">:</span> <span class="n">tb</span><span class="o">.</span><span class="n">FloatCol</span><span class="p">()})</span>
</pre></div>
</div>
<p>We can start the party now. We are going to create a new group, say /gl,
where we will put our links and will start creating one hard link too:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">gl</span> <span class="o">=</span> <span class="n">f1</span><span class="o">.</span><span class="n">create_group</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="s">'gl'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">ht</span> <span class="o">=</span> <span class="n">f1</span><span class="o">.</span><span class="n">create_hard_link</span><span class="p">(</span><span class="n">gl</span><span class="p">,</span> <span class="s">'ht'</span><span class="p">,</span> <span class="s">'/g1/g2/t1'</span><span class="p">)</span> <span class="c"># ht points to t1</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"``</span><span class="si">%s</span><span class="s">`` is a hard link to: ``</span><span class="si">%s</span><span class="s">``"</span> <span class="o">%</span> <span class="p">(</span><span class="n">ht</span><span class="p">,</span> <span class="n">t1</span><span class="p">)</span>
<span class="go">``/gl/ht (Table(0,)) `` is a hard link to: ``/g1/g2/t1 (Table(0,)) ``</span>
</pre></div>
</div>
<p>You can see how we’ve created a hard link in /gl/ht which is pointing to the
existing table in /g1/g2/t1. Have look at how the hard link is represented;
it looks like a table, and actually, it is an <em>real</em> table. We have two
different paths to access that table, the original /g1/g2/t1 and the new one
/gl/ht. If we remove the original path we still can reach the table by using
the new path:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">t1</span><span class="o">.</span><span class="n">remove</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"table continues to be accessible in: ``</span><span class="si">%s</span><span class="s">``"</span> <span class="o">%</span> <span class="n">f1</span><span class="o">.</span><span class="n">get_node</span><span class="p">(</span><span class="s">'/gl/ht'</span><span class="p">)</span>
<span class="go">table continues to be accessible in: ``/gl/ht (Table(0,)) ``</span>
</pre></div>
</div>
<p>So far so good. Now, let’s create a couple of soft links:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">la1</span> <span class="o">=</span> <span class="n">f1</span><span class="o">.</span><span class="n">create_soft_link</span><span class="p">(</span><span class="n">gl</span><span class="p">,</span> <span class="s">'la1'</span><span class="p">,</span> <span class="s">'/g1/a1'</span><span class="p">)</span> <span class="c"># la1 points to a1</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"``</span><span class="si">%s</span><span class="s">`` is a soft link to: ``</span><span class="si">%s</span><span class="s">``"</span> <span class="o">%</span> <span class="p">(</span><span class="n">la1</span><span class="p">,</span> <span class="n">la1</span><span class="o">.</span><span class="n">target</span><span class="p">)</span>
<span class="go">``/gl/la1 (SoftLink) -> /g1/a1`` is a soft link to: ``/g1/a1``</span>
<span class="gp">>>> </span><span class="n">lt</span> <span class="o">=</span> <span class="n">f1</span><span class="o">.</span><span class="n">create_soft_link</span><span class="p">(</span><span class="n">gl</span><span class="p">,</span> <span class="s">'lt'</span><span class="p">,</span> <span class="s">'/g1/g2/t1'</span><span class="p">)</span> <span class="c"># lt points to t1</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"``</span><span class="si">%s</span><span class="s">`` is a soft link to: ``</span><span class="si">%s</span><span class="s">``"</span> <span class="o">%</span> <span class="p">(</span><span class="n">lt</span><span class="p">,</span> <span class="n">lt</span><span class="o">.</span><span class="n">target</span><span class="p">)</span>
<span class="go">``/gl/lt (SoftLink) -> /g1/g2/t1 (dangling)`` is a soft link to: ``/g1/g2/t1``</span>
</pre></div>
</div>
<p>Okay, we see how the first link /gl/la1 points to the array /g1/a1. Notice
how the link prints as a SoftLink, and how the referred node is stored in the
target instance attribute. The second link (/gt/lt) pointing to /g1/g2/t1
also has been created successfully, but by better inspecting the string
representation of it, we see that is labeled as ‘(dangling)’. Why is this?
Well, you should remember that we recently removed the /g1/g2/t1 path to
access table t1. When printing it, the object knows that it points to
<em>nowhere</em> and reports this.
This is a nice way to quickly know whether a soft link points to an exiting
node or not.</p>
<p>So, let’s re-create the removed path to t1 table:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">t1</span> <span class="o">=</span> <span class="n">f1</span><span class="o">.</span><span class="n">create_hard_link</span><span class="p">(</span><span class="s">'/g1/g2'</span><span class="p">,</span> <span class="s">'t1'</span><span class="p">,</span> <span class="s">'/gl/ht'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"``</span><span class="si">%s</span><span class="s">`` is not dangling anymore"</span> <span class="o">%</span> <span class="p">(</span><span class="n">lt</span><span class="p">,)</span>
<span class="go">``/gl/lt (SoftLink) -> /g1/g2/t1`` is not dangling anymore</span>
</pre></div>
</div>
<p>and the soft link is pointing to an existing node now.</p>
<p>Of course, for soft links to serve any actual purpose we need a way to get
the pointed node. It happens that soft links are callable, and that’s the
way to get the referred nodes back:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">plt</span> <span class="o">=</span> <span class="n">lt</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"dereferred lt node: ``</span><span class="si">%s</span><span class="s">``"</span> <span class="o">%</span> <span class="n">plt</span>
<span class="go">dereferred lt node: ``/g1/g2/t1 (Table(0,)) ``</span>
<span class="gp">>>> </span><span class="n">pla1</span> <span class="o">=</span> <span class="n">la1</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"dereferred la1 node: ``</span><span class="si">%s</span><span class="s">``"</span> <span class="o">%</span> <span class="n">pla1</span>
<span class="go">dereferred la1 node: ``/g1/a1 (CArray(10000,)) ``</span>
</pre></div>
</div>
<p>Now, plt is a Python reference to the t1 table while pla1 refers to the a1
array. Easy, uh?</p>
<p>Let’s suppose now that a1 is an array whose access speed is critical for our
application. One possible solution is to move the entire file into a faster
disk, say, a solid state disk so that access latencies can be reduced quite a
lot. However, it happens that our file is too big to fit into our shiny new
(although small in capacity) SSD disk. A solution is to copy just the a1
array into a separate file that would fit into our SSD disk. However, our
application would be able to handle two files instead of only one, adding
significantly more complexity, which is not a good thing.</p>
<p>External links to the rescue! As we’ve already said, external links are like
soft links, but they are designed to link objects in external files. Back to
our problem, let’s copy the a1 array into a different file:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">f2</span> <span class="o">=</span> <span class="n">tb</span><span class="o">.</span><span class="n">open_file</span><span class="p">(</span><span class="s">'links2.h5'</span><span class="p">,</span> <span class="s">'w'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">new_a1</span> <span class="o">=</span> <span class="n">a1</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">f2</span><span class="o">.</span><span class="n">root</span><span class="p">,</span> <span class="s">'a1'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">f2</span><span class="o">.</span><span class="n">close</span><span class="p">()</span> <span class="c"># close the other file</span>
</pre></div>
</div>
<p>And now, we can remove the existing soft link and create the external link in
its place:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">la1</span><span class="o">.</span><span class="n">remove</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">la1</span> <span class="o">=</span> <span class="n">f1</span><span class="o">.</span><span class="n">create_external_link</span><span class="p">(</span><span class="n">gl</span><span class="p">,</span> <span class="s">'la1'</span><span class="p">,</span> <span class="s">'links2.h5:/a1'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"``</span><span class="si">%s</span><span class="s">`` is an external link to: ``</span><span class="si">%s</span><span class="s">``"</span> <span class="o">%</span> <span class="p">(</span><span class="n">la1</span><span class="p">,</span> <span class="n">la1</span><span class="o">.</span><span class="n">target</span><span class="p">)</span>
<span class="go">``/gl/la1 (ExternalLink) -> links2.h5:/a1`` is an external link to: ``links2.h5:/a1``</span>
</pre></div>
</div>
<p>Let’s try dereferring it:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">new_a1</span> <span class="o">=</span> <span class="n">la1</span><span class="p">()</span> <span class="c"># dereferrencing la1 returns a1 in links2.h5</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"dereferred la1 node: ``</span><span class="si">%s</span><span class="s">``"</span> <span class="o">%</span> <span class="n">new_a1</span>
<span class="go">dereferred la1 node: ``/a1 (CArray(10000,)) ``</span>
</pre></div>
</div>
<p>Well, it seems like we can access the external node. But just to make sure
that the node is in the other file:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">print</span> <span class="s">"new_a1 file:"</span><span class="p">,</span> <span class="n">new_a1</span><span class="o">.</span><span class="n">_v_file</span><span class="o">.</span><span class="n">filename</span>
<span class="go">new_a1 file: links2.h5</span>
</pre></div>
</div>
<p>Okay, the node is definitely in the external file. So, you won’t have to
worry about your application: it will work exactly the same no matter the
link is internal (soft) or external.</p>
<p>Finally, here it is a dump of the objects in the final file, just to get a
better idea of what we ended with:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">f1</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="gp">>>> </span><span class="nb">exit</span><span class="p">()</span>
<span class="go">$ ptdump links1.h5</span>
<span class="go">/ (RootGroup) ''</span>
<span class="go">/g1 (Group) ''</span>
<span class="go">/g1/a1 (CArray(10000,)) ''</span>
<span class="go">/gl (Group) ''</span>
<span class="go">/gl/ht (Table(0,)) ''</span>
<span class="go">/gl/la1 (ExternalLink) -> links2.h5:/a1</span>
<span class="go">/gl/lt (SoftLink) -> /g1/g2/t1</span>
<span class="go">/g1/g2 (Group) ''</span>
<span class="go">/g1/g2/t1 (Table(0,)) ''</span>
</pre></div>
</div>
<p>This ends this tutorial. I hope it helped you to appreciate how useful links
can be. I’m sure you will find other ways in which you can use links that
better fit your own needs.</p>
</div>
</div>
<div class="section" id="exercising-the-undo-redo-feature">
<h2>Exercising the Undo/Redo feature<a class="headerlink" href="#exercising-the-undo-redo-feature" title="Permalink to this headline">¶</a></h2>
<p>PyTables has integrated support for undoing and/or redoing actions. This
functionality lets you put marks in specific places of your hierarchy
manipulation operations, so that you can make your HDF5 file pop back
(<em>undo</em>) to a specific mark (for example for inspecting how your hierarchy
looked at that point). You can also go forward to a more recent marker
(<em>redo</em>). You can even do jumps to the marker you want using just one
instruction as we will see shortly.</p>
<p>You can undo/redo all the operations that are related to object tree
management, like creating, deleting, moving or renaming nodes (or complete
sub-hierarchies) inside a given object tree. You can also undo/redo
operations (i.e. creation, deletion or modification) of persistent node
attributes. However, when actions include <em>internal</em> modifications of
datasets (that includes Table.append, Table.modify_rows or Table.remove_rows
among others), they cannot be undone/redone currently.</p>
<p>This capability can be useful in many situations, like for example when doing
simulations with multiple branches. When you have to choose a path to follow
in such a situation, you can put a mark there and, if the simulation is not
going well, you can go back to that mark and start another path. Other
possible application is defining coarse-grained operations which operate in a
transactional-like way, i.e. which return the database to its previous state
if the operation finds some kind of problem while running. You can probably
devise many other scenarios where the Undo/Redo feature can be useful to you
<a class="footnote-reference" href="#id7" id="id3">[3]</a>.</p>
<div class="section" id="a-basic-example">
<h3>A basic example<a class="headerlink" href="#a-basic-example" title="Permalink to this headline">¶</a></h3>
<p>In this section, we are going to show the basic behavior of the Undo/Redo
feature. You can find the code used in this example in
<tt class="file docutils literal"><span class="pre">examples/tutorial3-1.py</span></tt>. A somewhat more complex example will be
explained in the next section.</p>
<p>First, let’s create a file:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">import</span> <span class="nn">tables</span>
<span class="gp">>>> </span><span class="n">fileh</span> <span class="o">=</span> <span class="n">tables</span><span class="o">.</span><span class="n">open_file</span><span class="p">(</span><span class="s">"tutorial3-1.h5"</span><span class="p">,</span> <span class="s">"w"</span><span class="p">,</span> <span class="n">title</span><span class="o">=</span><span class="s">"Undo/Redo demo 1"</span><span class="p">)</span>
</pre></div>
</div>
<p>And now, activate the Undo/Redo feature with the method
<a class="reference internal" href="libref/file_class.html#tables.File.enable_undo" title="tables.File.enable_undo"><tt class="xref py py-meth docutils literal"><span class="pre">File.enable_undo()</span></tt></a> of File:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">fileh</span><span class="o">.</span><span class="n">enable_undo</span><span class="p">()</span>
</pre></div>
</div>
<p>From now on, all our actions will be logged internally by PyTables. Now, we
are going to create a node (in this case an Array object):</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">one</span> <span class="o">=</span> <span class="n">fileh</span><span class="o">.</span><span class="n">create_array</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="s">'anarray'</span><span class="p">,</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">],</span> <span class="s">"An array"</span><span class="p">)</span>
</pre></div>
</div>
<p>Now, mark this point:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">fileh</span><span class="o">.</span><span class="n">mark</span><span class="p">()</span>
<span class="go">1</span>
</pre></div>
</div>
<p>We have marked the current point in the sequence of actions.
In addition, the mark() method has returned the identifier assigned to this
new mark, that is 1 (mark #0 is reserved for the implicit mark at the
beginning of the action log). In the next section we will see that you can
also assign a <em>name</em> to a mark (see <a class="reference internal" href="libref/file_class.html#tables.File.mark" title="tables.File.mark"><tt class="xref py py-meth docutils literal"><span class="pre">File.mark()</span></tt></a> for more info on
mark()).
Now, we are going to create another array:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">another</span> <span class="o">=</span> <span class="n">fileh</span><span class="o">.</span><span class="n">create_array</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="s">'anotherarray'</span><span class="p">,</span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span><span class="mi">5</span><span class="p">],</span> <span class="s">"Another array"</span><span class="p">)</span>
</pre></div>
</div>
<p>Right. Now, we can start doing funny things. Let’s say that we want to pop
back to the previous mark (that whose value was 1, do you remember?). Let’s
introduce the undo() method (see <a class="reference internal" href="libref/file_class.html#tables.File.undo" title="tables.File.undo"><tt class="xref py py-meth docutils literal"><span class="pre">File.undo()</span></tt></a>):</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">fileh</span><span class="o">.</span><span class="n">undo</span><span class="p">()</span>
</pre></div>
</div>
<p>Fine, what do you think it happened? Well, let’s have a look at the object
tree:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">print</span> <span class="n">fileh</span>
<span class="go">tutorial3-1.h5 (File) 'Undo/Redo demo 1'</span>
<span class="go">Last modif.: 'Tue Mar 13 11:43:55 2007'</span>
<span class="go">Object Tree:</span>
<span class="go">/ (RootGroup) 'Undo/Redo demo 1'</span>
<span class="go">/anarray (Array(2,)) 'An array'</span>
</pre></div>
</div>
<p>What happened with the /anotherarray node we’ve just created? You guess it,
it has disappeared because it was created <em>after</em> the mark 1. If you are
curious enough you may well ask where it has gone. Well, it has not been
deleted completely; it has been just moved into a special, hidden, group of
PyTables that renders it invisible and waiting for a chance to be reborn.</p>
<p>Now, unwind once more, and look at the object tree:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">fileh</span><span class="o">.</span><span class="n">undo</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">fileh</span>
<span class="go">tutorial3-1.h5 (File) 'Undo/Redo demo 1'</span>
<span class="go">Last modif.: 'Tue Mar 13 11:43:55 2007'</span>
<span class="go">Object Tree:</span>
<span class="go">/ (RootGroup) 'Undo/Redo demo 1'</span>
</pre></div>
</div>
<p>Oops, /anarray has disappeared as well!.
Don’t worry, it will revisit us very shortly. So, you might be somewhat lost
right now; in which mark are we?. Let’s ask the <a class="reference internal" href="libref/file_class.html#tables.File.get_current_mark" title="tables.File.get_current_mark"><tt class="xref py py-meth docutils literal"><span class="pre">File.get_current_mark()</span></tt></a>
method in the file handler:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">print</span> <span class="n">fileh</span><span class="o">.</span><span class="n">get_current_mark</span><span class="p">()</span>
<span class="go">0</span>
</pre></div>
</div>
<p>So we are at mark #0, remember? Mark #0 is an implicit mark that is created
when you start the log of actions when calling File.enable_undo(). Fine, but
you are missing your too-young-to-die arrays. What can we do about that?
<a class="reference internal" href="libref/file_class.html#tables.File.redo" title="tables.File.redo"><tt class="xref py py-meth docutils literal"><span class="pre">File.redo()</span></tt></a> to the rescue:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">fileh</span><span class="o">.</span><span class="n">redo</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">fileh</span>
<span class="go">tutorial3-1.h5 (File) 'Undo/Redo demo 1'</span>
<span class="go">Last modif.: 'Tue Mar 13 11:43:55 2007'</span>
<span class="go">Object Tree:</span>
<span class="go">/ (RootGroup) 'Undo/Redo demo 1'</span>
<span class="go">/anarray (Array(2,)) 'An array'</span>
</pre></div>
</div>
<p>Great! The /anarray array has come into life again. Just check that it is
alive and well:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">anarray</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="go">[3, 4]</span>
<span class="gp">>>> </span><span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">anarray</span><span class="o">.</span><span class="n">title</span>
<span class="go">'An array'</span>
</pre></div>
</div>
<p>Well, it looks pretty similar than in its previous life;
what’s more, it is exactly the same object!:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">anarray</span> <span class="ow">is</span> <span class="n">one</span>
<span class="go">True</span>
</pre></div>
</div>
<p>It just was moved to the the hidden group and back again, but that’s all!
That’s kind of fun, so we are going to do the same with /anotherarray:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">fileh</span><span class="o">.</span><span class="n">redo</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">fileh</span>
<span class="go">tutorial3-1.h5 (File) 'Undo/Redo demo 1'</span>
<span class="go">Last modif.: 'Tue Mar 13 11:43:55 2007'</span>
<span class="go">Object Tree:</span>
<span class="go">/ (RootGroup) 'Undo/Redo demo 1'</span>
<span class="go">/anarray (Array(2,)) 'An array'</span>
<span class="go">/anotherarray (Array(2,)) 'Another array'</span>
</pre></div>
</div>
<p>Welcome back, /anotherarray! Just a couple of sanity checks:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">assert</span> <span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">anotherarray</span><span class="o">.</span><span class="n">read</span><span class="p">()</span> <span class="o">==</span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span><span class="mi">5</span><span class="p">]</span>
<span class="gp">>>> </span><span class="k">assert</span> <span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">anotherarray</span><span class="o">.</span><span class="n">title</span> <span class="o">==</span> <span class="s">"Another array"</span>
<span class="gp">>>> </span><span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">anotherarray</span> <span class="ow">is</span> <span class="n">another</span>
<span class="go">True</span>
</pre></div>
</div>
<p>Nice, you managed to turn your data back into life.
Congratulations! But wait, do not forget to close your action log when you
don’t need this feature anymore:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">fileh</span><span class="o">.</span><span class="n">disable_undo</span><span class="p">()</span>
</pre></div>
</div>
<p>That will allow you to continue working with your data without actually
requiring PyTables to keep track of all your actions, and more importantly,
allowing your objects to die completely if they have to, not requiring to
keep them anywhere, and hence saving process time and space in your database
file.</p>
</div>
<div class="section" id="a-more-complete-example">
<h3>A more complete example<a class="headerlink" href="#a-more-complete-example" title="Permalink to this headline">¶</a></h3>
<p>Now, time for a somewhat more sophisticated demonstration of the Undo/Redo
feature. In it, several marks will be set in different parts of the code flow
and we will see how to jump between these marks with just one method call.
You can find the code used in this example in <tt class="file docutils literal"><span class="pre">examples/tutorial3-2.py</span></tt></p>
<p>Let’s introduce the first part of the code:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">tables</span>
<span class="c"># Create an HDF5 file</span>
<span class="n">fileh</span> <span class="o">=</span> <span class="n">tables</span><span class="o">.</span><span class="n">open_file</span><span class="p">(</span><span class="s">'tutorial3-2.h5'</span><span class="p">,</span> <span class="s">'w'</span><span class="p">,</span> <span class="n">title</span><span class="o">=</span><span class="s">'Undo/Redo demo 2'</span><span class="p">)</span>
<span class="c">#'-**-**-**-**-**-**- enable undo/redo log -**-**-**-**-**-**-**-'</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">enable_undo</span><span class="p">()</span>
<span class="c"># Start undoable operations</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">create_array</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="s">'otherarray1'</span><span class="p">,</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">],</span> <span class="s">'Another array 1'</span><span class="p">)</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">create_group</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="s">'agroup'</span><span class="p">,</span> <span class="s">'Group 1'</span><span class="p">)</span>
<span class="c"># Create a 'first' mark</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">mark</span><span class="p">(</span><span class="s">'first'</span><span class="p">)</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">create_array</span><span class="p">(</span><span class="s">'/agroup'</span><span class="p">,</span> <span class="s">'otherarray2'</span><span class="p">,</span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span><span class="mi">5</span><span class="p">],</span> <span class="s">'Another array 2'</span><span class="p">)</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">create_group</span><span class="p">(</span><span class="s">'/agroup'</span><span class="p">,</span> <span class="s">'agroup2'</span><span class="p">,</span> <span class="s">'Group 2'</span><span class="p">)</span>
<span class="c"># Create a 'second' mark</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">mark</span><span class="p">(</span><span class="s">'second'</span><span class="p">)</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">create_array</span><span class="p">(</span><span class="s">'/agroup/agroup2'</span><span class="p">,</span> <span class="s">'otherarray3'</span><span class="p">,</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span><span class="mi">6</span><span class="p">],</span> <span class="s">'Another array 3'</span><span class="p">)</span>
<span class="c"># Create a 'third' mark</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">mark</span><span class="p">(</span><span class="s">'third'</span><span class="p">)</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">create_array</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="s">'otherarray4'</span><span class="p">,</span> <span class="p">[</span><span class="mi">6</span><span class="p">,</span><span class="mi">7</span><span class="p">],</span> <span class="s">'Another array 4'</span><span class="p">)</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">create_array</span><span class="p">(</span><span class="s">'/agroup'</span><span class="p">,</span> <span class="s">'otherarray5'</span><span class="p">,</span> <span class="p">[</span><span class="mi">7</span><span class="p">,</span><span class="mi">8</span><span class="p">],</span> <span class="s">'Another array 5'</span><span class="p">)</span>
</pre></div>
</div>
<p>You can see how we have set several marks interspersed in the code flow,
representing different states of the database. Also, note that we have
assigned <em>names</em> to these marks, namely ‘first’, ‘second’ and ‘third’.</p>
<p>Now, start doing some jumps back and forth in the states of the database:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># Now go to mark 'first'</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">goto</span><span class="p">(</span><span class="s">'first'</span><span class="p">)</span>
<span class="k">assert</span> <span class="s">'/otherarray1'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/agroup2'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/otherarray2'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/agroup2/otherarray3'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/otherarray4'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/otherarray5'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="c"># Go to mark 'third'</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">goto</span><span class="p">(</span><span class="s">'third'</span><span class="p">)</span>
<span class="k">assert</span> <span class="s">'/otherarray1'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/agroup2'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/otherarray2'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/agroup2/otherarray3'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/otherarray4'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/otherarray5'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="c"># Now go to mark 'second'</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">goto</span><span class="p">(</span><span class="s">'second'</span><span class="p">)</span>
<span class="k">assert</span> <span class="s">'/otherarray1'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/agroup2'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/otherarray2'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/agroup2/otherarray3'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/otherarray4'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/otherarray5'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">fileh</span>
</pre></div>
</div>
<p>Well, the code above shows how easy is to jump to a certain mark in the
database by using the <a class="reference internal" href="libref/file_class.html#tables.File.goto" title="tables.File.goto"><tt class="xref py py-meth docutils literal"><span class="pre">File.goto()</span></tt></a> method.</p>
<p>There are also a couple of implicit marks for going to the beginning or the
end of the saved states: 0 and -1. Going to mark #0 means go to the beginning
of the saved actions, that is, when method fileh.enable_undo() was called.
Going to mark #-1 means go to the last recorded action, that is the last
action in the code flow.</p>
<p>Let’s see what happens when going to the end of the action log:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># Go to the end</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">goto</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
<span class="k">assert</span> <span class="s">'/otherarray1'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/agroup2'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/otherarray2'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/agroup2/otherarray3'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/otherarray4'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="k">assert</span> <span class="s">'/agroup/otherarray5'</span> <span class="ow">in</span> <span class="n">fileh</span>
<span class="c"># Check that objects have come back to life in a sane state</span>
<span class="k">assert</span> <span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">otherarray1</span><span class="o">.</span><span class="n">read</span><span class="p">()</span> <span class="o">==</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">]</span>
<span class="k">assert</span> <span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">agroup</span><span class="o">.</span><span class="n">otherarray2</span><span class="o">.</span><span class="n">read</span><span class="p">()</span> <span class="o">==</span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span><span class="mi">5</span><span class="p">]</span>
<span class="k">assert</span> <span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">agroup</span><span class="o">.</span><span class="n">agroup2</span><span class="o">.</span><span class="n">otherarray3</span><span class="o">.</span><span class="n">read</span><span class="p">()</span> <span class="o">==</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span><span class="mi">6</span><span class="p">]</span>
<span class="k">assert</span> <span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">otherarray4</span><span class="o">.</span><span class="n">read</span><span class="p">()</span> <span class="o">==</span> <span class="p">[</span><span class="mi">6</span><span class="p">,</span><span class="mi">7</span><span class="p">]</span>
<span class="k">assert</span> <span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="o">.</span><span class="n">agroup</span><span class="o">.</span><span class="n">otherarray5</span><span class="o">.</span><span class="n">read</span><span class="p">()</span> <span class="o">==</span> <span class="p">[</span><span class="mi">7</span><span class="p">,</span><span class="mi">8</span><span class="p">]</span>
</pre></div>
</div>
<p>Try yourself going to the beginning of the action log (remember, the mark #0)
and check the contents of the object tree.</p>
<p>We have nearly finished this demonstration. As always, do not forget to close
the action log as well as the database:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c">#'-**-**-**-**-**-**- disable undo/redo log -**-**-**-**-**-**-**-'</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">disable_undo</span><span class="p">()</span>
<span class="c"># Close the file</span>
<span class="n">fileh</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
</div>
<p>You might want to check other examples on Undo/Redo feature that appear in
<tt class="file docutils literal"><span class="pre">examples/undo-redo.py</span></tt>.</p>
</div>
</div>
<div class="section" id="using-enumerated-types">
<h2>Using enumerated types<a class="headerlink" href="#using-enumerated-types" title="Permalink to this headline">¶</a></h2>
<p>PyTables includes support for handling enumerated types. Those types are
defined by providing an exhaustive <em>set</em> or <em>list</em> of possible, named values
for a variable of that type. Enumerated variables of the same type are
usually compared between them for equality and sometimes for order, but are
not usually operated upon.</p>
<p>Enumerated values have an associated <em>name</em> and <em>concrete value</em>. Every name
is unique and so are concrete values. An enumerated variable always takes the
concrete value, not its name. Usually, the concrete value is not used
directly, and frequently it is entirely irrelevant. For the same reason, an
enumerated variable is not usually compared with concrete values out of its
enumerated type. For that kind of use, standard variables and constants are
more adequate.</p>
<p>PyTables provides the Enum (see <a class="reference internal" href="libref/helper_classes.html#enumclassdescr"><em>The Enum class</em></a>) class to provide
support for enumerated types. Each instance of Enum is an enumerated type (or
<em>enumeration</em>). For example, let us create an enumeration of colors</p>
<p>All these examples can be found in <tt class="file docutils literal"><span class="pre">examples/enum.py</span></tt>:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">import</span> <span class="nn">tables</span>
<span class="gp">>>> </span><span class="n">colorList</span> <span class="o">=</span> <span class="p">[</span><span class="s">'red'</span><span class="p">,</span> <span class="s">'green'</span><span class="p">,</span> <span class="s">'blue'</span><span class="p">,</span> <span class="s">'white'</span><span class="p">,</span> <span class="s">'black'</span><span class="p">]</span>
<span class="gp">>>> </span><span class="n">colors</span> <span class="o">=</span> <span class="n">tables</span><span class="o">.</span><span class="n">Enum</span><span class="p">(</span><span class="n">colorList</span><span class="p">)</span>
</pre></div>
</div>
<p>Here we used a simple list giving the names of enumerated values, but we left
the choice of concrete values up to the Enum class. Let us see the enumerated
pairs to check those values:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">print</span> <span class="s">"Colors:"</span><span class="p">,</span> <span class="p">[</span><span class="n">v</span> <span class="k">for</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">colors</span><span class="p">]</span>
<span class="go">Colors: [('blue', 2), ('black', 4), ('white', 3), ('green', 1), ('red', 0)]</span>
</pre></div>
</div>
<p>Names have been given automatic integer concrete values. We can iterate over
the values in an enumeration, but we will usually be more interested in
accessing single values. We can get the concrete value associated with a name
by accessing it as an attribute or as an item (the later can be useful for
names not resembling Python identifiers):</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">print</span> <span class="s">"Value of 'red' and 'white':"</span><span class="p">,</span> <span class="p">(</span><span class="n">colors</span><span class="o">.</span><span class="n">red</span><span class="p">,</span> <span class="n">colors</span><span class="o">.</span><span class="n">white</span><span class="p">)</span>
<span class="go">Value of 'red' and 'white': (0, 3)</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Value of 'yellow':"</span><span class="p">,</span> <span class="n">colors</span><span class="o">.</span><span class="n">yellow</span>
<span class="go">Value of 'yellow':</span>
<span class="gt">Traceback (most recent call last):</span>
File <span class="nb">"<stdin>"</span>, line <span class="m">1</span>, in <span class="n">?</span>
File <span class="nb">".../tables/misc/enum.py"</span>, line <span class="m">230</span>, in <span class="n">__getattr__</span>
<span class="k">raise</span> <span class="ne">AttributeError</span><span class="p">(</span>\<span class="o">*</span><span class="n">ke</span><span class="o">.</span><span class="n">args</span><span class="p">)</span>
<span class="gr">AttributeError</span>: <span class="n">no enumerated value with that name: 'yellow'</span>
<span class="go">>>></span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Value of 'red' and 'white':"</span><span class="p">,</span> <span class="p">(</span><span class="n">colors</span><span class="p">[</span><span class="s">'red'</span><span class="p">],</span> <span class="n">colors</span><span class="p">[</span><span class="s">'white'</span><span class="p">])</span>
<span class="go">Value of 'red' and 'white': (0, 3)</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Value of 'yellow':"</span><span class="p">,</span> <span class="n">colors</span><span class="p">[</span><span class="s">'yellow'</span><span class="p">]</span>
<span class="go">Value of 'yellow':</span>
<span class="gt">Traceback (most recent call last):</span>
File <span class="nb">"<stdin>"</span>, line <span class="m">1</span>, in <span class="n">?</span>
File <span class="nb">".../tables/misc/enum.py"</span>, line <span class="m">189</span>, in <span class="n">__getitem__</span>
<span class="k">raise</span> <span class="ne">KeyError</span><span class="p">(</span><span class="s">"no enumerated value with that name: </span><span class="si">%r</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">name</span><span class="p">,))</span>
<span class="gr">KeyError</span>: <span class="n">"no enumerated value with that name: 'yellow'"</span>
</pre></div>
</div>
<p>See how accessing a value that is not in the enumeration raises the
appropriate exception. We can also do the opposite action and get the name
that matches a concrete value by using the __call__() method of Enum:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">print</span> <span class="s">"Name of value </span><span class="si">%s</span><span class="s">:"</span> <span class="o">%</span> <span class="n">colors</span><span class="o">.</span><span class="n">red</span><span class="p">,</span> <span class="n">colors</span><span class="p">(</span><span class="n">colors</span><span class="o">.</span><span class="n">red</span><span class="p">)</span>
<span class="go">Name of value 0: red</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="s">"Name of value 1234:"</span><span class="p">,</span> <span class="n">colors</span><span class="p">(</span><span class="mi">1234</span><span class="p">)</span>
<span class="go">Name of value 1234:</span>
<span class="gt">Traceback (most recent call last):</span>
File <span class="nb">"<stdin>"</span>, line <span class="m">1</span>, in <span class="n">?</span>
File <span class="nb">".../tables/misc/enum.py"</span>, line <span class="m">320</span>, in <span class="n">__call__</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
<span class="gr">ValueError</span>: <span class="n">no enumerated value with that concrete value: 1234</span>
</pre></div>
</div>
<p>You can see what we made as using the enumerated type to <em>convert</em> a concrete
value into a name in the enumeration. Of course, values out of the
enumeration can not be converted.</p>
<div class="section" id="enumerated-columns">
<h3>Enumerated columns<a class="headerlink" href="#enumerated-columns" title="Permalink to this headline">¶</a></h3>
<p>Columns of an enumerated type can be declared by using the EnumCol (see
<a class="reference internal" href="libref/declarative_classes.html#colclassdescr"><em>The Col class and its descendants</em></a>) class. To see how this works, let us open a new
PyTables file and create a table to collect the simulated results of a
probabilistic experiment. In it, we have a bag full of colored balls; we take
a ball out and annotate the time of extraction and the color of the ball:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">h5f</span> <span class="o">=</span> <span class="n">tables</span><span class="o">.</span><span class="n">open_file</span><span class="p">(</span><span class="s">'enum.h5'</span><span class="p">,</span> <span class="s">'w'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="k">class</span> <span class="nc">BallExt</span><span class="p">(</span><span class="n">tables</span><span class="o">.</span><span class="n">IsDescription</span><span class="p">):</span>
<span class="gp">... </span> <span class="n">ballTime</span> <span class="o">=</span> <span class="n">tables</span><span class="o">.</span><span class="n">Time32Col</span><span class="p">()</span>
<span class="gp">... </span> <span class="n">ballColor</span> <span class="o">=</span> <span class="n">tables</span><span class="o">.</span><span class="n">EnumCol</span><span class="p">(</span><span class="n">colors</span><span class="p">,</span> <span class="s">'black'</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="s">'uint8'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">tbl</span> <span class="o">=</span> <span class="n">h5f</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="s">'extractions'</span><span class="p">,</span> <span class="n">BallExt</span><span class="p">,</span> <span class="n">title</span><span class="o">=</span><span class="s">"Random ball extractions"</span><span class="p">)</span>
<span class="go">>>></span>
</pre></div>
</div>
<p>We declared the ballColor column to be of the enumerated type colors, with a
default value of black. We also stated that we are going to store concrete
values as unsigned 8-bit integer values <a class="footnote-reference" href="#id8" id="id4">[4]</a>.</p>
<p>Let us use some random values to fill the table:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">import</span> <span class="nn">time</span>
<span class="gp">>>> </span><span class="kn">import</span> <span class="nn">random</span>
<span class="gp">>>> </span><span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">row</span> <span class="o">=</span> <span class="n">tbl</span><span class="o">.</span><span class="n">row</span>
<span class="gp">>>> </span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
<span class="gp">... </span> <span class="n">row</span><span class="p">[</span><span class="s">'ballTime'</span><span class="p">]</span> <span class="o">=</span> <span class="n">now</span> <span class="o">+</span> <span class="n">i</span>
<span class="gp">... </span> <span class="n">row</span><span class="p">[</span><span class="s">'ballColor'</span><span class="p">]</span> <span class="o">=</span> <span class="n">colors</span><span class="p">[</span><span class="n">random</span><span class="o">.</span><span class="n">choice</span><span class="p">(</span><span class="n">colorList</span><span class="p">)]</span> <span class="c"># notice this</span>
<span class="gp">... </span> <span class="n">row</span><span class="o">.</span><span class="n">append</span><span class="p">()</span>
<span class="go">>>></span>
</pre></div>
</div>
<p>Notice how we used the __getitem__() call of colors to get the concrete value
to store in ballColor. You should know that this way of appending values to a
table does automatically check for the validity on enumerated values.
For instance:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">row</span><span class="p">[</span><span class="s">'ballTime'</span><span class="p">]</span> <span class="o">=</span> <span class="n">now</span> <span class="o">+</span> <span class="mi">42</span>
<span class="gp">>>> </span><span class="n">row</span><span class="p">[</span><span class="s">'ballColor'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">1234</span>
<span class="gt">Traceback (most recent call last):</span>
File <span class="nb">"<stdin>"</span>, line <span class="m">1</span>, in <span class="n"><module></span>
File <span class="nb">"tableextension.pyx"</span>, line <span class="m">1086</span>, in <span class="n">tableextension.Row.__setitem__</span>
File <span class="nb">".../tables/misc/enum.py"</span>, line <span class="m">320</span>, in <span class="n">__call__</span>
<span class="s">"no enumerated value with that concrete value: </span><span class="si">%r</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">value</span><span class="p">,))</span>
<span class="gr">ValueError</span>: <span class="n">no enumerated value with that concrete value: 1234</span>
</pre></div>
</div>
<p>But take care that this check is <em>only</em> performed here and not in other
methods such as tbl.append() or tbl.modify_rows(). Now, after flushing the
table we can see the results of the insertions:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">tbl</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">tbl</span><span class="p">:</span>
<span class="gp">... </span> <span class="n">ballTime</span> <span class="o">=</span> <span class="n">r</span><span class="p">[</span><span class="s">'ballTime'</span><span class="p">]</span>
<span class="gp">... </span> <span class="n">ballColor</span> <span class="o">=</span> <span class="n">colors</span><span class="p">(</span><span class="n">r</span><span class="p">[</span><span class="s">'ballColor'</span><span class="p">])</span> <span class="c"># notice this</span>
<span class="gp">... </span> <span class="k">print</span> <span class="s">"Ball extracted on </span><span class="si">%d</span><span class="s"> is of color </span><span class="si">%s</span><span class="s">."</span> <span class="o">%</span> <span class="p">(</span><span class="n">ballTime</span><span class="p">,</span> <span class="n">ballColor</span><span class="p">)</span>
<span class="go">Ball extracted on 1173785568 is of color green.</span>
<span class="go">Ball extracted on 1173785569 is of color black.</span>
<span class="go">Ball extracted on 1173785570 is of color white.</span>
<span class="go">Ball extracted on 1173785571 is of color black.</span>
<span class="go">Ball extracted on 1173785572 is of color black.</span>
<span class="go">Ball extracted on 1173785573 is of color red.</span>
<span class="go">Ball extracted on 1173785574 is of color green.</span>
<span class="go">Ball extracted on 1173785575 is of color red.</span>
<span class="go">Ball extracted on 1173785576 is of color white.</span>
<span class="go">Ball extracted on 1173785577 is of color white.</span>
</pre></div>
</div>
<p>As a last note, you may be wondering how to have access to the enumeration
associated with ballColor once the file is closed and reopened. You can call
tbl.get_enum(‘ballColor’) (see <a class="reference internal" href="libref/structured_storage.html#tables.Table.get_enum" title="tables.Table.get_enum"><tt class="xref py py-meth docutils literal"><span class="pre">Table.get_enum()</span></tt></a>) to get the enumeration
back.</p>
</div>
<div class="section" id="enumerated-arrays">
<h3>Enumerated arrays<a class="headerlink" href="#enumerated-arrays" title="Permalink to this headline">¶</a></h3>
<p>EArray and VLArray leaves can also be declared to store enumerated values by
means of the EnumAtom (see <a class="reference internal" href="libref/declarative_classes.html#atomclassdescr"><em>The Atom class and its descendants</em></a>) class, which works very
much like EnumCol for tables. Also, Array leaves can be used to open native
HDF enumerated arrays.</p>
<p>Let us create a sample EArray containing ranges of working days as
bidimensional values:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">workingDays</span> <span class="o">=</span> <span class="p">{</span><span class="s">'Mon'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="s">'Tue'</span><span class="p">:</span> <span class="mi">2</span><span class="p">,</span> <span class="s">'Wed'</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span> <span class="s">'Thu'</span><span class="p">:</span> <span class="mi">4</span><span class="p">,</span> <span class="s">'Fri'</span><span class="p">:</span> <span class="mi">5</span><span class="p">}</span>
<span class="gp">>>> </span><span class="n">dayRange</span> <span class="o">=</span> <span class="n">tables</span><span class="o">.</span><span class="n">EnumAtom</span><span class="p">(</span><span class="n">workingDays</span><span class="p">,</span> <span class="s">'Mon'</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="s">'uint16'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">earr</span> <span class="o">=</span> <span class="n">h5f</span><span class="o">.</span><span class="n">create_earray</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="s">'days'</span><span class="p">,</span> <span class="n">dayRange</span><span class="p">,</span> <span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">),</span> <span class="n">title</span><span class="o">=</span><span class="s">"Working day ranges"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">earr</span><span class="o">.</span><span class="n">flavor</span> <span class="o">=</span> <span class="s">'python'</span>
</pre></div>
</div>
<p>Nothing surprising, except for a pair of details. In the first place, we use
a <em>dictionary</em> instead of a list to explicitly set concrete values in the
enumeration. In the second place, there is no explicit Enum instance created!
Instead, the dictionary is passed as the first argument to the constructor of
EnumAtom. If the constructor gets a list or a dictionary instead of an
enumeration, it automatically builds the enumeration from it.</p>
<p>Now let us feed some data to the array:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">wdays</span> <span class="o">=</span> <span class="n">earr</span><span class="o">.</span><span class="n">get_enum</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">earr</span><span class="o">.</span><span class="n">append</span><span class="p">([(</span><span class="n">wdays</span><span class="o">.</span><span class="n">Mon</span><span class="p">,</span> <span class="n">wdays</span><span class="o">.</span><span class="n">Fri</span><span class="p">),</span> <span class="p">(</span><span class="n">wdays</span><span class="o">.</span><span class="n">Wed</span><span class="p">,</span> <span class="n">wdays</span><span class="o">.</span><span class="n">Fri</span><span class="p">)])</span>
<span class="gp">>>> </span><span class="n">earr</span><span class="o">.</span><span class="n">append</span><span class="p">([(</span><span class="n">wdays</span><span class="o">.</span><span class="n">Mon</span><span class="p">,</span> <span class="mi">1234</span><span class="p">)])</span>
</pre></div>
</div>
<p>Please note that, since we had no explicit Enum instance, we were forced to
use get_enum() (see <a class="reference internal" href="libref/homogenous_storage.html#earraymethodsdescr"><em>EArray methods</em></a>) to get it from the array (we
could also have used dayRange.enum). Also note that we were able to append
an invalid value (1234). Array methods do not check the validity of
enumerated values.</p>
<p>Finally, we will print the contents of the array:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">for</span> <span class="p">(</span><span class="n">d1</span><span class="p">,</span> <span class="n">d2</span><span class="p">)</span> <span class="ow">in</span> <span class="n">earr</span><span class="p">:</span>
<span class="gp">... </span> <span class="k">print</span> <span class="s">"From </span><span class="si">%s</span><span class="s"> to </span><span class="si">%s</span><span class="s"> (</span><span class="si">%d</span><span class="s"> days)."</span> <span class="o">%</span> <span class="p">(</span><span class="n">wdays</span><span class="p">(</span><span class="n">d1</span><span class="p">),</span> <span class="n">wdays</span><span class="p">(</span><span class="n">d2</span><span class="p">),</span> <span class="n">d2</span><span class="o">-</span><span class="n">d1</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
<span class="go">From Mon to Fri (5 days).</span>
<span class="go">From Wed to Fri (3 days).</span>
<span class="gt">Traceback (most recent call last):</span>
File <span class="nb">"<stdin>"</span>, line <span class="m">2</span>, in <span class="n"><module></span>
File <span class="nb">".../tables/misc/enum.py"</span>, line <span class="m">320</span>, in <span class="n">__call__</span>
<span class="s">"no enumerated value with that concrete value: </span><span class="si">%r</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">value</span><span class="p">,))</span>
<span class="gr">ValueError</span>: <span class="n">no enumerated value with that concrete value: 1234</span>
</pre></div>
</div>
<p>That was an example of operating on concrete values. It also showed how the
value-to-name conversion failed because of the value not belonging to the
enumeration.</p>
<p>Now we will close the file, and this little tutorial on enumerated types is
done:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">h5f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
</div>
</div>
</div>
<div class="section" id="dealing-with-nested-structures-in-tables">
<h2>Dealing with nested structures in tables<a class="headerlink" href="#dealing-with-nested-structures-in-tables" title="Permalink to this headline">¶</a></h2>
<p>PyTables supports the handling of nested structures (or nested datatypes, as
you prefer) in table objects, allowing you to define arbitrarily nested
columns.</p>
<p>An example will clarify what this means. Let’s suppose that you want to group
your data in pieces of information that are more related than others pieces
in your table, So you may want to tie them up together in order to have your
table better structured but also be able to retrieve and deal with these
groups more easily.</p>
<p>You can create such a nested substructures by just nesting subclasses of
IsDescription. Let’s see one example (okay, it’s a bit silly, but will serve
for demonstration purposes):</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">tables</span> <span class="kn">import</span> <span class="o">*</span>
<span class="k">class</span> <span class="nc">Info</span><span class="p">(</span><span class="n">IsDescription</span><span class="p">):</span>
<span class="sd">"""A sub-structure of Test"""</span>
<span class="n">_v_pos</span> <span class="o">=</span> <span class="mi">2</span> <span class="c"># The position in the whole structure</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">StringCol</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
<span class="n">value</span> <span class="o">=</span> <span class="n">Float64Col</span><span class="p">(</span><span class="n">pos</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="n">colors</span> <span class="o">=</span> <span class="n">Enum</span><span class="p">([</span><span class="s">'red'</span><span class="p">,</span> <span class="s">'green'</span><span class="p">,</span> <span class="s">'blue'</span><span class="p">])</span>
<span class="k">class</span> <span class="nc">NestedDescr</span><span class="p">(</span><span class="n">IsDescription</span><span class="p">):</span>
<span class="sd">"""A description that has several nested columns"""</span>
<span class="n">color</span> <span class="o">=</span> <span class="n">EnumCol</span><span class="p">(</span><span class="n">colors</span><span class="p">,</span> <span class="s">'red'</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="s">'uint32'</span><span class="p">)</span>
<span class="n">info1</span> <span class="o">=</span> <span class="n">Info</span><span class="p">()</span>
<span class="k">class</span> <span class="nc">info2</span><span class="p">(</span><span class="n">IsDescription</span><span class="p">):</span>
<span class="n">_v_pos</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">StringCol</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
<span class="n">value</span> <span class="o">=</span> <span class="n">Float64Col</span><span class="p">(</span><span class="n">pos</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">info3</span><span class="p">(</span><span class="n">IsDescription</span><span class="p">):</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">Float64Col</span><span class="p">(</span><span class="n">dflt</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">UInt8Col</span><span class="p">(</span><span class="n">dflt</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
</pre></div>
</div>
<p>The root class is NestedDescr and both info1 and info2 are <em>substructures</em> of
it. Note how info1 is actually an instance of the class Info that was defined
prior to NestedDescr. Also, there is a third substructure, namely info3 that
hangs from the substructure info2. You can also define positions of
substructures in the containing object by declaring the special class
attribute _v_pos.</p>
<div class="section" id="nested-table-creation">
<h3>Nested table creation<a class="headerlink" href="#nested-table-creation" title="Permalink to this headline">¶</a></h3>
<p>Now that we have defined our nested structure, let’s create a <em>nested</em> table,
that is a table with columns that contain other subcolumns:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">fileh</span> <span class="o">=</span> <span class="n">open_file</span><span class="p">(</span><span class="s">"nested-tut.h5"</span><span class="p">,</span> <span class="s">"w"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">table</span> <span class="o">=</span> <span class="n">fileh</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span><span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="p">,</span> <span class="s">'table'</span><span class="p">,</span> <span class="n">NestedDescr</span><span class="p">)</span>
</pre></div>
</div>
<p>Done! Now, we have to feed the table with some values. The problem is how we
are going to reference to the nested fields. That’s easy, just use a ‘/’
character to separate names in different nested levels. Look at this:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">row</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">row</span>
<span class="gp">>>> </span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
<span class="gp">... </span> <span class="n">row</span><span class="p">[</span><span class="s">'color'</span><span class="p">]</span> <span class="o">=</span> <span class="n">colors</span><span class="p">[[</span><span class="s">'red'</span><span class="p">,</span> <span class="s">'green'</span><span class="p">,</span> <span class="s">'blue'</span><span class="p">][</span><span class="n">i</span><span class="o">%</span><span class="mi">3</span><span class="p">]]</span>
<span class="gp">... </span> <span class="n">row</span><span class="p">[</span><span class="s">'info1/name'</span><span class="p">]</span> <span class="o">=</span> <span class="s">"name1-</span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">i</span>
<span class="gp">... </span> <span class="n">row</span><span class="p">[</span><span class="s">'info2/name'</span><span class="p">]</span> <span class="o">=</span> <span class="s">"name2-</span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">i</span>
<span class="gp">... </span> <span class="n">row</span><span class="p">[</span><span class="s">'info2/info3/y'</span><span class="p">]</span> <span class="o">=</span> <span class="n">i</span>
<span class="gp">... </span> <span class="c"># All the rest will be filled with defaults</span>
<span class="gp">... </span> <span class="n">row</span><span class="o">.</span><span class="n">append</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">nrows</span>
<span class="go">10</span>
</pre></div>
</div>
<p>You see? In order to fill the fields located in the substructures, we just
need to specify its full path in the table hierarchy.</p>
</div>
<div class="section" id="reading-nested-tables">
<h3>Reading nested tables<a class="headerlink" href="#reading-nested-tables" title="Permalink to this headline">¶</a></h3>
<p>Now, what happens if we want to read the table? What kind oft data container
will we get? Well, it’s worth trying it:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">nra</span> <span class="o">=</span> <span class="n">table</span><span class="p">[::</span><span class="mi">4</span><span class="p">]</span>
<span class="gp">>>> </span><span class="n">nra</span>
<span class="go">array([(((1.0, 0), 'name2-0', 0.0), ('name1-0', 0.0), 0L),</span>
<span class="go"> (((1.0, 4), 'name2-4', 0.0), ('name1-4', 0.0), 1L),</span>
<span class="go"> (((1.0, 8), 'name2-8', 0.0), ('name1-8', 0.0), 2L)],</span>
<span class="go"> dtype=[('info2', [('info3', [('x', '>f8'), ('y', '\|u1')]),</span>
<span class="go"> ('name', '\|S10'), ('value', '>f8')]),</span>
<span class="go"> ('info1', [('name', '\|S10'), ('value', '>f8')]),</span>
<span class="go"> ('color', '>u4')])</span>
</pre></div>
</div>
<p>What we got is a NumPy array with a <em>compound, nested datatype</em> (its dtype is
a list of name-datatype tuples). We read one row for each four in the table,
giving a result of three rows.</p>
<p>You can make use of the above object in many different ways.
For example, you can use it to append new data to the existing table object:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">nra</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">nrows</span>
<span class="go">13</span>
</pre></div>
</div>
<p>Or, to create new tables:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table2</span> <span class="o">=</span> <span class="n">fileh</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span><span class="n">fileh</span><span class="o">.</span><span class="n">root</span><span class="p">,</span> <span class="s">'table2'</span><span class="p">,</span> <span class="n">nra</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">table2</span><span class="p">[:]</span>
<span class="go">array([(((1.0, 0), 'name2-0', 0.0), ('name1-0', 0.0), 0L),</span>
<span class="go"> (((1.0, 4), 'name2-4', 0.0), ('name1-4', 0.0), 1L),</span>
<span class="go"> (((1.0, 8), 'name2-8', 0.0), ('name1-8', 0.0), 2L)],</span>
<span class="go"> dtype=[('info2', [('info3', [('x', '<f8'), ('y', '\|u1')]),</span>
<span class="go"> ('name', '\|S10'), ('value', '<f8')]),</span>
<span class="go"> ('info1', [('name', '\|S10'), ('value', '<f8')]),</span>
<span class="go"> ('color', '<u4')])</span>
</pre></div>
</div>
<p>Finally, we can select nested values that fulfill some condition:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">names</span> <span class="o">=</span> <span class="p">[</span> <span class="n">x</span><span class="p">[</span><span class="s">'info2/name'</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">table</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="s">'color'</span><span class="p">]</span> <span class="o">==</span> <span class="n">colors</span><span class="o">.</span><span class="n">red</span> <span class="p">]</span>
<span class="gp">>>> </span><span class="n">names</span>
<span class="go">['name2-0', 'name2-3', 'name2-6', 'name2-9', 'name2-0']</span>
</pre></div>
</div>
<p>Note that the row accessor does not provide the natural naming feature, so
you have to completely specify the path of your desired columns in order to
reach them.</p>
</div>
<div class="section" id="using-cols-accessor">
<h3>Using Cols accessor<a class="headerlink" href="#using-cols-accessor" title="Permalink to this headline">¶</a></h3>
<p>We can use the cols attribute object (see <a class="reference internal" href="libref/structured_storage.html#colsclassdescr"><em>The Cols class</em></a>) of the table
so as to quickly access the info located in the interesting substructures:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">cols</span><span class="o">.</span><span class="n">info2</span><span class="p">[</span><span class="mi">1</span><span class="p">:</span><span class="mi">5</span><span class="p">]</span>
<span class="go">array([((1.0, 1), 'name2-1', 0.0), ((1.0, 2), 'name2-2', 0.0),</span>
<span class="go"> ((1.0, 3), 'name2-3', 0.0), ((1.0, 4), 'name2-4', 0.0)],</span>
<span class="go"> dtype=[('info3', [('x', '<f8'), ('y', '\|u1')]), ('name', '\|S10'),</span>
<span class="go"> ('value', '<f8')])</span>
</pre></div>
</div>
<p>Here, we have made use of the cols accessor to access to the <em>info2</em>
substructure and an slice operation to get access to the subset of data we
were interested in; you probably have recognized the natural naming approach
here. We can continue and ask for data in <em>info3</em> substructure:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">cols</span><span class="o">.</span><span class="n">info2</span><span class="o">.</span><span class="n">info3</span><span class="p">[</span><span class="mi">1</span><span class="p">:</span><span class="mi">5</span><span class="p">]</span>
<span class="go">array([(1.0, 1), (1.0, 2), (1.0, 3), (1.0, 4)],</span>
<span class="go"> dtype=[('x', '<f8'), ('y', '\|u1')])</span>
</pre></div>
</div>
<p>You can also use the _f_col method to get a handler for a column:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">cols</span><span class="o">.</span><span class="n">_f_col</span><span class="p">(</span><span class="s">'info2'</span><span class="p">)</span>
<span class="go">/table.cols.info2 (Cols), 3 columns</span>
<span class="go"> info3 (Cols(), Description)</span>
<span class="go"> name (Column(), \|S10)</span>
<span class="go"> value (Column(), float64)</span>
</pre></div>
</div>
<p>Here, you’ve got another Cols object handler because <em>info2</em> was a nested
column. If you select a non-nested column, you will get a regular Column
instance:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">cols</span><span class="o">.</span><span class="n">_f_col</span><span class="p">(</span><span class="s">'info2/info3/y'</span><span class="p">)</span>
<span class="go">/table.cols.info2.info3.y (Column(), uint8, idx=None)</span>
</pre></div>
</div>
<p>To sum up, the cols accessor is a very handy and powerful way to access data
in your nested tables. Don’t be afraid of using it, specially when doing
interactive work.</p>
</div>
<div class="section" id="accessing-meta-information-of-nested-tables">
<h3>Accessing meta-information of nested tables<a class="headerlink" href="#accessing-meta-information-of-nested-tables" title="Permalink to this headline">¶</a></h3>
<p>Tables have an attribute called description which points to an instance of
the Description class (see <a class="reference internal" href="libref/structured_storage.html#descriptionclassdescr"><em>The Description class</em></a>) and is useful to
discover different meta-information about table data.</p>
<p>Let’s see how it looks like:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">description</span>
<span class="go">{</span>
<span class="go"> "info2": {</span>
<span class="go"> "info3": {</span>
<span class="go"> "x": Float64Col(shape=(), dflt=1.0, pos=0),</span>
<span class="go"> "y": UInt8Col(shape=(), dflt=1, pos=1)},</span>
<span class="go"> "name": StringCol(itemsize=10, shape=(), dflt='', pos=1),</span>
<span class="go"> "value": Float64Col(shape=(), dflt=0.0, pos=2)},</span>
<span class="go"> "info1": {</span>
<span class="go"> "name": StringCol(itemsize=10, shape=(), dflt='', pos=0),</span>
<span class="go"> "value": Float64Col(shape=(), dflt=0.0, pos=1)},</span>
<span class="go"> "color": EnumCol(enum=Enum({'blue': 2, 'green': 1, 'red': 0}), dflt='red',</span>
<span class="go"> base=UInt32Atom(shape=(), dflt=0), shape=(), pos=2)}</span>
</pre></div>
</div>
<p>As you can see, it provides very useful information on both the formats and
the structure of the columns in your table.</p>
<p>This object also provides a natural naming approach to access to subcolumns
metadata:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">description</span><span class="o">.</span><span class="n">info1</span>
<span class="go">{"name": StringCol(itemsize=10, shape=(), dflt='', pos=0),</span>
<span class="go"> "value": Float64Col(shape=(), dflt=0.0, pos=1)}</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">description</span><span class="o">.</span><span class="n">info2</span><span class="o">.</span><span class="n">info3</span>
<span class="go">{"x": Float64Col(shape=(), dflt=1.0, pos=0),</span>
<span class="go"> "y": UInt8Col(shape=(), dflt=1, pos=1)}</span>
</pre></div>
</div>
<p>There are other variables that can be interesting for you:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">description</span><span class="o">.</span><span class="n">_v_nested_names</span>
<span class="go">[('info2', [('info3', ['x', 'y']), 'name', 'value']),</span>
<span class="go"> ('info1', ['name', 'value']), 'color']</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">description</span><span class="o">.</span><span class="n">info1</span><span class="o">.</span><span class="n">_v_nested_names</span>
<span class="go">['name', 'value']</span>
</pre></div>
</div>
<p>_v_nested_names provides the names of the columns as well as its structure.
You can see that there are the same attributes for the different levels of
the Description object, because the levels are <em>also</em> Description objects
themselves.</p>
<p>There is a special attribute, called _v_nested_descr, that can be useful to
create nested structured arrays that imitate the structure of the table (or a
subtable thereof):</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">import</span> <span class="nn">numpy</span>
<span class="gp">>>> </span><span class="n">table</span><span class="o">.</span><span class="n">description</span><span class="o">.</span><span class="n">_v_nested_descr</span>
<span class="go">[('info2', [('info3', [('x', '()f8'), ('y', '()u1')]), ('name', '()S10'),</span>
<span class="go"> ('value', '()f8')]), ('info1', [('name', '()S10'), ('value', '()f8')]),</span>
<span class="go"> ('color', '()u4')]</span>
<span class="gp">>>> </span><span class="n">numpy</span><span class="o">.</span><span class="n">rec</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="bp">None</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span>
<span class="go"> dtype=table.description._v_nested_descr)</span>
<span class="go">recarray([],</span>
<span class="go"> dtype=[('info2', [('info3', [('x', '>f8'), ('y', '|u1')]),</span>
<span class="go"> ('name', '|S10'), ('value', '>f8')]),</span>
<span class="go"> ('info1', [('name', '|S10'), ('value', '>f8')]),</span>
<span class="go"> ('color', '>u4')])</span>
<span class="gp">>>> </span><span class="n">numpy</span><span class="o">.</span><span class="n">rec</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="bp">None</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span>
<span class="go"> dtype=table.description.info2._v_nested_descr)</span>
<span class="go">recarray([],</span>
<span class="go"> dtype=[('info3', [('x', '>f8'), ('y', '|u1')]), ('name', '|S10'),</span>
<span class="go"> ('value', '>f8')])</span>
</pre></div>
</div>
<p>You can see a simple example on how to create an array with NumPy.</p>
<p>Finally, there is a special iterator of the Description class, called _f_walk
that is able to return you the different columns of the table:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">for</span> <span class="n">coldescr</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">description</span><span class="o">.</span><span class="n">_f_walk</span><span class="p">():</span>
<span class="gp">... </span> <span class="k">print</span> <span class="s">"column-->"</span><span class="p">,</span><span class="n">coldescr</span>
<span class="go">column--> Description([('info2', [('info3', [('x', '()f8'), ('y', '()u1')]),</span>
<span class="go"> ('name', '()S10'), ('value', '()f8')]),</span>
<span class="go"> ('info1', [('name', '()S10'), ('value', '()f8')]),</span>
<span class="go"> ('color', '()u4')])</span>
<span class="go">column--> EnumCol(enum=Enum({'blue': 2, 'green': 1, 'red': 0}), dflt='red',</span>
<span class="go"> base=UInt32Atom(shape=(), dflt=0), shape=(), pos=2)</span>
<span class="go">column--> Description([('info3', [('x', '()f8'), ('y', '()u1')]), ('name', '()S10'),</span>
<span class="go"> ('value', '()f8')])</span>
<span class="go">column--> StringCol(itemsize=10, shape=(), dflt='', pos=1)</span>
<span class="go">column--> Float64Col(shape=(), dflt=0.0, pos=2)</span>
<span class="go">column--> Description([('name', '()S10'), ('value', '()f8')])</span>
<span class="go">column--> StringCol(itemsize=10, shape=(), dflt='', pos=0)</span>
<span class="go">column--> Float64Col(shape=(), dflt=0.0, pos=1)</span>
<span class="go">column--> Description([('x', '()f8'), ('y', '()u1')])</span>
<span class="go">column--> Float64Col(shape=(), dflt=1.0, pos=0)</span>
<span class="go">column--> UInt8Col(shape=(), dflt=1, pos=1)</span>
</pre></div>
</div>
<p>See the <a class="reference internal" href="libref/structured_storage.html#descriptionclassdescr"><em>The Description class</em></a> for the complete listing of attributes
and methods of Description.</p>
<p>Well, this is the end of this tutorial. As always, do not forget to close
your files:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">fileh</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
</div>
<p>Finally, you may want to have a look at your resulting data file.</p>
<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span>ptdump -d nested-tut.h5
/ <span class="o">(</span>RootGroup<span class="o">)</span> <span class="s1">''</span>
/table <span class="o">(</span>Table<span class="o">(</span>13,<span class="o">))</span> <span class="s1">''</span>
Data dump:
<span class="o">[</span>0<span class="o">]</span> <span class="o">(((</span>1.0, 0<span class="o">)</span>, <span class="s1">'name2-0'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-0'</span>, 0.0<span class="o">)</span>, 0L<span class="o">)</span>
<span class="o">[</span>1<span class="o">]</span> <span class="o">(((</span>1.0, 1<span class="o">)</span>, <span class="s1">'name2-1'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-1'</span>, 0.0<span class="o">)</span>, 1L<span class="o">)</span>
<span class="o">[</span>2<span class="o">]</span> <span class="o">(((</span>1.0, 2<span class="o">)</span>, <span class="s1">'name2-2'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-2'</span>, 0.0<span class="o">)</span>, 2L<span class="o">)</span>
<span class="o">[</span>3<span class="o">]</span> <span class="o">(((</span>1.0, 3<span class="o">)</span>, <span class="s1">'name2-3'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-3'</span>, 0.0<span class="o">)</span>, 0L<span class="o">)</span>
<span class="o">[</span>4<span class="o">]</span> <span class="o">(((</span>1.0, 4<span class="o">)</span>, <span class="s1">'name2-4'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-4'</span>, 0.0<span class="o">)</span>, 1L<span class="o">)</span>
<span class="o">[</span>5<span class="o">]</span> <span class="o">(((</span>1.0, 5<span class="o">)</span>, <span class="s1">'name2-5'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-5'</span>, 0.0<span class="o">)</span>, 2L<span class="o">)</span>
<span class="o">[</span>6<span class="o">]</span> <span class="o">(((</span>1.0, 6<span class="o">)</span>, <span class="s1">'name2-6'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-6'</span>, 0.0<span class="o">)</span>, 0L<span class="o">)</span>
<span class="o">[</span>7<span class="o">]</span> <span class="o">(((</span>1.0, 7<span class="o">)</span>, <span class="s1">'name2-7'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-7'</span>, 0.0<span class="o">)</span>, 1L<span class="o">)</span>
<span class="o">[</span>8<span class="o">]</span> <span class="o">(((</span>1.0, 8<span class="o">)</span>, <span class="s1">'name2-8'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-8'</span>, 0.0<span class="o">)</span>, 2L<span class="o">)</span>
<span class="o">[</span>9<span class="o">]</span> <span class="o">(((</span>1.0, 9<span class="o">)</span>, <span class="s1">'name2-9'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-9'</span>, 0.0<span class="o">)</span>, 0L<span class="o">)</span>
<span class="o">[</span>10<span class="o">]</span> <span class="o">(((</span>1.0, 0<span class="o">)</span>, <span class="s1">'name2-0'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-0'</span>, 0.0<span class="o">)</span>, 0L<span class="o">)</span>
<span class="o">[</span>11<span class="o">]</span> <span class="o">(((</span>1.0, 4<span class="o">)</span>, <span class="s1">'name2-4'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-4'</span>, 0.0<span class="o">)</span>, 1L<span class="o">)</span>
<span class="o">[</span>12<span class="o">]</span> <span class="o">(((</span>1.0, 8<span class="o">)</span>, <span class="s1">'name2-8'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-8'</span>, 0.0<span class="o">)</span>, 2L<span class="o">)</span>
/table2 <span class="o">(</span>Table<span class="o">(</span>3,<span class="o">))</span> <span class="s1">''</span>
Data dump:
<span class="o">[</span>0<span class="o">]</span> <span class="o">(((</span>1.0, 0<span class="o">)</span>, <span class="s1">'name2-0'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-0'</span>, 0.0<span class="o">)</span>, 0L<span class="o">)</span>
<span class="o">[</span>1<span class="o">]</span> <span class="o">(((</span>1.0, 4<span class="o">)</span>, <span class="s1">'name2-4'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-4'</span>, 0.0<span class="o">)</span>, 1L<span class="o">)</span>
<span class="o">[</span>2<span class="o">]</span> <span class="o">(((</span>1.0, 8<span class="o">)</span>, <span class="s1">'name2-8'</span>, 0.0<span class="o">)</span>, <span class="o">(</span><span class="s1">'name1-8'</span>, 0.0<span class="o">)</span>, 2L<span class="o">)</span>
</pre></div>
</div>
<p>Most of the code in this section is also available in examples/nested-tut.py.</p>
<p>All in all, PyTables provides a quite comprehensive toolset to cope with
nested structures and address your classification needs.
However, caveat emptor, be sure to not nest your data too deeply or you will
get inevitably messed interpreting too intertwined lists, tuples and
description objects.</p>
</div>
</div>
<div class="section" id="other-examples-in-pytables-distribution">
<h2>Other examples in PyTables distribution<a class="headerlink" href="#other-examples-in-pytables-distribution" title="Permalink to this headline">¶</a></h2>
<p>Feel free to examine the rest of examples in directory <tt class="file docutils literal"><span class="pre">examples/</span></tt>, and
try to understand them. We have written several practical sample scripts to
give you an idea of the PyTables capabilities, its way of dealing with HDF5
objects, and how it can be used in the real world.</p>
<hr class="docutils" />
<table class="docutils footnote" frame="void" id="id5" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id1">[1]</a></td><td>Appending data to arrays is also supported, but you need to create
special objects called EArray (see <a class="reference internal" href="libref/homogenous_storage.html#earrayclassdescr"><em>The EArray class</em></a> for more
info).</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="id6" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id2">[2]</a></td><td>Note that you can append not only scalar values to tables,
but also fully multidimensional array objects.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="id7" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id3">[3]</a></td><td>You can even <em>hide</em> nodes temporarily. Will you be able to find out how?</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="id8" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id4">[4]</a></td><td>In fact, only integer values are supported right now, but
this may change in the future.</td></tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../index.html">
<img class="logo" src="../_static/logo-pytables-small.png" alt="Logo"/>
</a></p>
<h3><a href="../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Tutorials</a><ul>
<li><a class="reference internal" href="#getting-started">Getting started</a><ul>
<li><a class="reference internal" href="#importing-tables-objects">Importing tables objects</a></li>
<li><a class="reference internal" href="#declaring-a-column-descriptor">Declaring a Column Descriptor</a></li>
<li><a class="reference internal" href="#creating-a-pytables-file-from-scratch">Creating a PyTables file from scratch</a></li>
<li><a class="reference internal" href="#creating-a-new-group">Creating a new group</a></li>
<li><a class="reference internal" href="#creating-a-new-table">Creating a new table</a></li>
<li><a class="reference internal" href="#reading-and-selecting-data-in-a-table">Reading (and selecting) data in a table</a></li>
<li><a class="reference internal" href="#creating-new-array-objects">Creating new array objects</a></li>
<li><a class="reference internal" href="#closing-the-file-and-looking-at-its-content">Closing the file and looking at its content</a></li>
</ul>
</li>
<li><a class="reference internal" href="#browsing-the-object-tree">Browsing the <em>object tree</em></a><ul>
<li><a class="reference internal" href="#traversing-the-object-tree">Traversing the object tree</a></li>
<li><a class="reference internal" href="#setting-and-getting-user-attributes">Setting and getting user attributes</a></li>
<li><a class="reference internal" href="#getting-object-metadata">Getting object metadata</a></li>
<li><a class="reference internal" href="#reading-data-from-array-objects">Reading data from Array objects</a></li>
</ul>
</li>
<li><a class="reference internal" href="#commiting-data-to-tables-and-arrays">Commiting data to tables and arrays</a><ul>
<li><a class="reference internal" href="#appending-data-to-an-existing-table">Appending data to an existing table</a></li>
<li><a class="reference internal" href="#modifying-data-in-tables">Modifying data in tables</a></li>
<li><a class="reference internal" href="#modifying-data-in-arrays">Modifying data in arrays</a></li>
<li><a class="reference internal" href="#and-finally-how-to-delete-rows-from-a-table">And finally... how to delete rows from a table</a></li>
</ul>
</li>
<li><a class="reference internal" href="#multidimensional-table-cells-and-automatic-sanity-checks">Multidimensional table cells and automatic sanity checks</a><ul>
<li><a class="reference internal" href="#shape-checking">Shape checking</a></li>
<li><a class="reference internal" href="#field-name-checking">Field name checking</a></li>
<li><a class="reference internal" href="#data-type-checking">Data type checking</a></li>
</ul>
</li>
<li><a class="reference internal" href="#using-links-for-more-convenient-access-to-nodes">Using links for more convenient access to nodes</a><ul>
<li><a class="reference internal" href="#interactive-example">Interactive example</a></li>
</ul>
</li>
<li><a class="reference internal" href="#exercising-the-undo-redo-feature">Exercising the Undo/Redo feature</a><ul>
<li><a class="reference internal" href="#a-basic-example">A basic example</a></li>
<li><a class="reference internal" href="#a-more-complete-example">A more complete example</a></li>
</ul>
</li>
<li><a class="reference internal" href="#using-enumerated-types">Using enumerated types</a><ul>
<li><a class="reference internal" href="#enumerated-columns">Enumerated columns</a></li>
<li><a class="reference internal" href="#enumerated-arrays">Enumerated arrays</a></li>
</ul>
</li>
<li><a class="reference internal" href="#dealing-with-nested-structures-in-tables">Dealing with nested structures in tables</a><ul>
<li><a class="reference internal" href="#nested-table-creation">Nested table creation</a></li>
<li><a class="reference internal" href="#reading-nested-tables">Reading nested tables</a></li>
<li><a class="reference internal" href="#using-cols-accessor">Using Cols accessor</a></li>
<li><a class="reference internal" href="#accessing-meta-information-of-nested-tables">Accessing meta-information of nested tables</a></li>
</ul>
</li>
<li><a class="reference internal" href="#other-examples-in-pytables-distribution">Other examples in PyTables distribution</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="installation.html"
title="previous chapter">Installation</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="libref.html"
title="next chapter">Library Reference</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/usersguide/tutorials.txt"
rel="nofollow">Show Source</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="relbar-bottom">
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> </li>
<li class="right" >
<a href="../np-modindex.html" title="Python Module Index"
>modules</a> </li>
<li class="right" >
<a href="libref.html" title="Library Reference"
>next</a> </li>
<li class="right" >
<a href="installation.html" title="Installation"
>previous</a> </li>
<li><a href="../index.html">PyTables 3.0.0 documentation</a> »</li>
<li><a href="index.html" >PyTables User’s Guide</a> »</li>
</ul>
</div>
</div>
<div class="footer">
© Copyright 2011-2013, PyTables maintainers.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3.
</div>
<!-- cloud_sptheme 1.3 -->
</body>
</html>
