<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Image Data — Astropy v0.2.4</title>
<link rel="stylesheet" href="../../../_static/bootstrap-astropy.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../../../',
VERSION: '0.2.4',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/sidebar.js"></script>
<link rel="shortcut icon" href="../../../_static/astropy_logo.ico"/>
<link rel="top" title="Astropy v0.2.4" href="../../../index.html" />
<link rel="up" title="FITS File handling (astropy.io.fits)" href="../index.html" />
<link rel="next" title="Table Data" href="table.html" />
<link rel="prev" title="FITS Headers" href="headers.html" />
</head>
<body>
<div class="topbar">
<a class="brand" title="Documentation Home" href="../../../index.html"></a>
<ul>
<li><a class="homelink" title="AstroPy Homepage" href="http://www.astropy.org"></a></li>
<li><a title="General Index" href="../../../genindex.html">Index</a></li>
<li><a title="Python Module Index" href="../../../py-modindex.html">Modules</a></li>
<li>
<form action="../../../search.html" method="get">
<input type="text" name="q" placeholder="Search" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</li>
</ul>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right">
<a href="table.html" title="Table Data">
next »
</a>
</li>
<li class="right">
<a href="headers.html" title="FITS Headers">
« previous
</a>
|
</li>
<li>
<a href="../../../index.html">Astropy v0.2.4</a>
»
</li>
<li><a href="../index.html" accesskey="U">FITS File handling (<tt class="docutils literal"><span class="pre">astropy.io.fits</span></tt>)</a> »</li>
<li>Image Data</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="image-data">
<h1>Image Data<a class="headerlink" href="#image-data" title="Permalink to this headline">¶</a></h1>
<p>In this chapter, we’ll discuss the data component in an image HDU.</p>
<div class="section" id="image-data-as-an-array">
<h2>Image Data as an Array<a class="headerlink" href="#image-data-as-an-array" title="Permalink to this headline">¶</a></h2>
<p>A FITS primary HDU or an image extension HDU may contain image data. The
following discussions apply to both of these HDU classes. In Astropy, for most
cases, it is just a simple numpy array, having the shape specified by the NAXIS
keywords and the data type specified by the BITPIX keyword - unless the data is
scaled, see next section. Here is a quick cross reference between allowed
BITPIX values in FITS images and the numpy data types:</p>
<pre class="literal-block">
<strong>BITPIX</strong> <strong>Numpy Data Type</strong>
8 numpy.uint8 (note it is UNsigned integer)
16 numpy.int16
32 numpy.int32
-32 numpy.float32
-64 numpy.float64
</pre>
<p>To recap the fact that in numpy the arrays are 0-indexed and the axes are
ordered from slow to fast. So, if a FITS image has NAXIS1=300 and NAXIS2=400,
the numpy array of its data will have the shape of (400, 300).</p>
<p>Here is a summary of reading and updating image data values:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">f</span> <span class="o">=</span> <span class="n">fits</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s">'image.fits'</span><span class="p">)</span> <span class="c"># open a FITS file</span>
<span class="gp">>>> </span><span class="n">scidata</span> <span class="o">=</span> <span class="n">f</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">data</span> <span class="c"># assume the first extension is an image</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">scidata</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">4</span><span class="p">]</span> <span class="c"># get the pixel value at x=5, y=2</span>
<span class="gp">>>> </span><span class="n">scidata</span><span class="p">[</span><span class="mi">30</span><span class="p">:</span><span class="mi">40</span><span class="p">,</span> <span class="mi">10</span><span class="p">:</span><span class="mi">20</span><span class="p">]</span> <span class="c"># get values of the subsection</span>
<span class="go"> # from x=11 to 20, y=31 to 40 (inclusive)</span>
<span class="gp">>>> </span><span class="n">scidata</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">4</span><span class="p">]</span> <span class="o">=</span> <span class="mi">999</span> <span class="c"># update a pixel value</span>
<span class="gp">>>> </span><span class="n">scidata</span><span class="p">[</span><span class="mi">30</span><span class="p">:</span><span class="mi">40</span><span class="p">,</span> <span class="mi">10</span><span class="p">:</span><span class="mi">20</span><span class="p">]</span> <span class="o">=</span> <span class="mi">0</span> <span class="c"># update values of a subsection</span>
<span class="gp">>>> </span><span class="n">scidata</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span> <span class="o">=</span> <span class="n">scidata</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="c"># copy the 3rd row to the 4th row</span>
</pre></div>
</div>
<p>Here are some more complicated examples by using the concept of the “mask
array”. The first example is to change all negative pixel values in scidata to
zero. The second one is to take logarithm of the pixel values which are
positive:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">scidata</span><span class="p">[</span><span class="n">scidata</span><span class="o"><</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="mi">0</span>
<span class="gp">>>> </span><span class="n">scidata</span><span class="p">[</span><span class="n">scidata</span><span class="o">></span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">numpy</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">scidata</span><span class="p">[</span><span class="n">scidata</span><span class="o">></span><span class="mi">0</span><span class="p">])</span>
</pre></div>
</div>
<p>These examples show the concise nature of numpy array operations.</p>
</div>
<div class="section" id="scaled-data">
<h2>Scaled Data<a class="headerlink" href="#scaled-data" title="Permalink to this headline">¶</a></h2>
<p>Sometimes an image is scaled, i.e. the data stored in the file is not the
image’s physical (true) values, but linearly transformed according to the
equation:</p>
<pre class="literal-block">
physical value = BSCALE*(storage value) + BZERO
</pre>
<p>BSCALE and BZERO are stored as keywords of the same names in the header of the
same HDU. The most common use of scaled image is to store unsigned 16-bit
integer data because FITS standard does not allow it. In this case, the stored
data is signed 16-bit integer (BITPIX=16) with BZERO=32768 (2**15), BSCALE=1.</p>
<div class="section" id="reading-scaled-image-data">
<h3>Reading Scaled Image Data<a class="headerlink" href="#reading-scaled-image-data" title="Permalink to this headline">¶</a></h3>
<p>Images are scaled only when either of the BSCALE/BZERO keywords are present in
the header and either of their values is not the default value (BSCALE=1,
BZERO=0).</p>
<p>For unscaled data, the data attribute of an HDU in Astropy is a numpy array of
the same data type as specified by the BITPIX keyword. For scaled image, the
.data attribute will be the physical data, i.e. already transformed from the
storage data and may not be the same data type as prescribed in BITPIX. This
means an extra step of copying is needed and thus the corresponding memory
requirement. This also means that the advantage of memory mapping is reduced
for scaled data.</p>
<p>For floating point storage data, the scaled data will have the same data type.
For integer data type, the scaled data will always be single precision floating
point (numpy.float32). Here is an example of what happens to such a file,
before and after the data is touched</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">f</span> <span class="o">=</span> <span class="n">fits</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s">'scaled_uint16.fits'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">hdu</span> <span class="o">=</span> <span class="n">f</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">hdu</span><span class="o">.</span><span class="n">header</span><span class="p">[</span><span class="s">'bitpix'</span><span class="p">],</span> <span class="n">hdu</span><span class="o">.</span><span class="n">header</span><span class="p">[</span><span class="s">'bzero'</span><span class="p">]</span>
<span class="go">16 32768</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">hdu</span><span class="o">.</span><span class="n">data</span> <span class="c"># once data is touched, it is scaled</span>
<span class="go">[ 11. 12. 13. 14. 15.]</span>
<span class="gp">>>> </span><span class="n">hdu</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">dtype</span><span class="o">.</span><span class="n">name</span>
<span class="go">'float32'</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">hdu</span><span class="o">.</span><span class="n">header</span><span class="p">[</span><span class="s">'bitpix'</span><span class="p">]</span> <span class="c"># BITPIX is also updated</span>
<span class="go">-32</span>
<span class="go"># BZERO and BSCALE are removed after the scaling</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">hdu</span><span class="o">.</span><span class="n">header</span><span class="p">[</span><span class="s">'bzero'</span><span class="p">]</span>
<span class="go">KeyError: "Keyword 'bzero' not found."</span>
</pre></div>
</div>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p>An important caveat to be aware of when dealing with scaled data in PyFITS,
is that when accessing the data via the .data attribute, the data is
automatically scaled with the BZERO and BSCALE parameters. If the file was
opened in “update” mode, it will be saved with the rescaled data. This
surprising behavior is a compromise to err on the side of not losing data:
If some floating point calculations were made on the data, rescaling it
when saving could result in a loss of information.</p>
<p>To prevent this automatic scaling, open the file with the
<tt class="docutils literal"><span class="pre">do_not_scale_image_data=True</span></tt> argument to <tt class="docutils literal"><span class="pre">fits.open()</span></tt>. This is
especially useful for updating some header values, while ensuring that the
data is not modified.</p>
<p class="last">One may also manually reapply scale parameters by using <tt class="docutils literal"><span class="pre">hdu.scale()</span></tt>
(see below). Alternately, one may open files with the <tt class="docutils literal"><span class="pre">scale_back=True</span></tt>
argument. This assures that the original scaling is preserved when saving.</p>
</div>
</div>
<div class="section" id="writing-scaled-image-data">
<h3>Writing Scaled Image Data<a class="headerlink" href="#writing-scaled-image-data" title="Permalink to this headline">¶</a></h3>
<p>With the extra processing and memory requirement, we discourage users to use
scaled data as much as possible. However, Astropy does provide ways to write
scaled data with the scale(type, option, bscale, bzero) method. Here are a few
examples:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="c"># scale the data to Int16 with user specified bscale/bzero</span>
<span class="gp">>>> </span><span class="n">hdu</span><span class="o">.</span><span class="n">scale</span><span class="p">(</span><span class="s">'int16'</span><span class="p">,</span> <span class="n">bzero</span><span class="o">=</span><span class="mi">32768</span><span class="p">)</span>
<span class="gp">>>> </span><span class="c"># scale the data to Int32 with the min/max of the data range</span>
<span class="gp">>>> </span><span class="n">hdu</span><span class="o">.</span><span class="n">scale</span><span class="p">(</span><span class="s">'int32'</span><span class="p">,</span> <span class="s">'minmax'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="c"># scale the data, using the original BSCALE/BZERO</span>
<span class="gp">>>> </span><span class="n">hdu</span><span class="o">.</span><span class="n">scale</span><span class="p">(</span><span class="s">'int32'</span><span class="p">,</span> <span class="s">'old'</span><span class="p">)</span>
</pre></div>
</div>
<p>The first example above shows how to store an unsigned short integer array.</p>
<p>Great caution must be exercised when using the <a class="reference internal" href="../api/images.html#astropy.io.fits.ImageHDU.scale" title="astropy.io.fits.ImageHDU.scale"><tt class="xref py py-meth docutils literal"><span class="pre">scale()</span></tt></a> method.
The <tt class="xref py py-attr docutils literal"><span class="pre">data</span></tt> attribute of an image HDU, after the
<a class="reference internal" href="../api/images.html#astropy.io.fits.ImageHDU.scale" title="astropy.io.fits.ImageHDU.scale"><tt class="xref py py-meth docutils literal"><span class="pre">scale()</span></tt></a> call, will become the storage values, not the physical
values. So, only call <a class="reference internal" href="../api/images.html#astropy.io.fits.ImageHDU.scale" title="astropy.io.fits.ImageHDU.scale"><tt class="xref py py-meth docutils literal"><span class="pre">scale()</span></tt></a> just before writing out to FITS
files, i.e. calls of <a class="reference internal" href="../api/hdulists.html#astropy.io.fits.HDUList.writeto" title="astropy.io.fits.HDUList.writeto"><tt class="xref py py-meth docutils literal"><span class="pre">writeto()</span></tt></a>, <a class="reference internal" href="../api/hdulists.html#astropy.io.fits.HDUList.flush" title="astropy.io.fits.HDUList.flush"><tt class="xref py py-meth docutils literal"><span class="pre">flush()</span></tt></a>, or
<a class="reference internal" href="../api/hdulists.html#astropy.io.fits.HDUList.close" title="astropy.io.fits.HDUList.close"><tt class="xref py py-meth docutils literal"><span class="pre">close()</span></tt></a>. No further use of the data should be exercised. Here is
an example of what happens to the <tt class="xref py py-attr docutils literal"><span class="pre">data</span></tt> attribute after the
<a class="reference internal" href="../api/images.html#astropy.io.fits.ImageHDU.scale" title="astropy.io.fits.ImageHDU.scale"><tt class="xref py py-meth docutils literal"><span class="pre">scale()</span></tt></a> call:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">hdu</span> <span class="o">=</span> <span class="n">fits</span><span class="o">.</span><span class="n">PrimaryHDU</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">0.</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]))</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">hdu</span><span class="o">.</span><span class="n">data</span>
<span class="go">[ 0. 1. 2. 3.]</span>
<span class="gp">>>> </span><span class="n">hdu</span><span class="o">.</span><span class="n">scale</span><span class="p">(</span><span class="s">'int16'</span><span class="p">,</span> <span class="n">bzero</span><span class="o">=</span><span class="mi">32768</span><span class="p">)</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">hdu</span><span class="o">.</span><span class="n">data</span> <span class="c"># now the data has storage values</span>
<span class="go">[-32768 -32767 -32766 -32765]</span>
<span class="gp">>>> </span><span class="n">hdu</span><span class="o">.</span><span class="n">writeto</span><span class="p">(</span><span class="s">'new.fits'</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
<div class="section" id="data-sections">
<span id="id1"></span><h2>Data Sections<a class="headerlink" href="#data-sections" title="Permalink to this headline">¶</a></h2>
<p>When a FITS image HDU’s .data is accessed, either the whole data is copied into
memory (in cases of NOT using memory mapping or if the data is scaled) or a
virtual memory space equivalent to the data size is allocated (in the case of
memory mapping of non-scaled data). If there are several very large image HDUs
being accessed at the same time, the system may run out of memory.</p>
<p>If a user does not need the entire image(s) at the same time, e.g. processing
images(s) ten rows at a time, the <a class="reference internal" href="../api/images.html#astropy.io.fits.ImageHDU.section" title="astropy.io.fits.ImageHDU.section"><tt class="xref py py-attr docutils literal"><span class="pre">section</span></tt></a> attribute of an
HDU can be used to alleviate such memory problems.</p>
<p>With PyFITS’ improved support for memory-mapping, the sections feature is not
as necessary as it used to be for handling very large images. However, if the
image’s data is scaled with non-trivial BSCALE/BZERO values, accessing the data
in sections may still be necessary under the current implementation. Memmap is
also insufficient for loading images large than ~4 GB on a 32-bit system–in
such cases it may be necessary to use sections.</p>
<p>Here is an example of getting the median image from 3 input images of the size
5000x5000:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">f1</span> <span class="o">=</span> <span class="n">fits</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s">'file1.fits'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">f2</span> <span class="o">=</span> <span class="n">fits</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s">'file2.fits'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">f3</span> <span class="o">=</span> <span class="n">fits</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s">'file3.fits'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">output</span> <span class="o">=</span> <span class="n">numpy</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="mi">5000</span> <span class="o">*</span> <span class="mi">5000</span><span class="p">)</span>
<span class="gp">>>> </span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">50</span><span class="p">):</span>
<span class="gp">... </span><span class="n">j</span> <span class="o">=</span> <span class="n">i</span> <span class="o">*</span> <span class="mi">100</span>
<span class="gp">... </span><span class="n">k</span> <span class="o">=</span> <span class="n">j</span> <span class="o">+</span> <span class="mi">100</span>
<span class="gp">... </span><span class="n">x1</span> <span class="o">=</span> <span class="n">f1</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">section</span><span class="p">[</span><span class="n">j</span><span class="p">:</span><span class="n">k</span><span class="p">,:]</span>
<span class="gp">... </span><span class="n">x2</span> <span class="o">=</span> <span class="n">f2</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">section</span><span class="p">[</span><span class="n">j</span><span class="p">:</span><span class="n">k</span><span class="p">,:]</span>
<span class="gp">... </span><span class="n">x3</span> <span class="o">=</span> <span class="n">f3</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">section</span><span class="p">[</span><span class="n">j</span><span class="p">:</span><span class="n">k</span><span class="p">,:]</span>
<span class="gp">... </span><span class="c"># use scipy.stsci.image's median function</span>
<span class="gp">... </span><span class="n">output</span><span class="p">[</span><span class="n">j</span><span class="p">:</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">image</span><span class="o">.</span><span class="n">median</span><span class="p">([</span><span class="n">x1</span><span class="p">,</span> <span class="n">x2</span><span class="p">,</span> <span class="n">x3</span><span class="p">])</span>
</pre></div>
</div>
<p>Data in each <a class="reference internal" href="../api/images.html#astropy.io.fits.ImageHDU.section" title="astropy.io.fits.ImageHDU.section"><tt class="xref py py-attr docutils literal"><span class="pre">section</span></tt></a> does not need to be contiguous for
memory savings to be possible. PyFITS will do its best to join together
discontiguous sections of the array while reading as little as possible into
memory.</p>
<p>Sections cannot be assigned to. Any modifications made to a data section are
not saved back to the original file.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper"><h3>Page Contents</h3>
<ul>
<li><a class="reference internal" href="#">Image Data</a><ul>
<li><a class="reference internal" href="#image-data-as-an-array">Image Data as an Array</a></li>
<li><a class="reference internal" href="#scaled-data">Scaled Data</a><ul>
<li><a class="reference internal" href="#reading-scaled-image-data">Reading Scaled Image Data</a></li>
<li><a class="reference internal" href="#writing-scaled-image-data">Writing Scaled Image Data</a></li>
</ul>
</li>
<li><a class="reference internal" href="#data-sections">Data Sections</a></li>
</ul>
</li>
</ul>
</div>
</div>
<div class="clearer"></div>
</div>
<footer class="footer">
<p class="pull-right">
<a href="http://github.com/astropy/astropy/tree/v0.2.4/docs/io/fits/usage/image.rst">Edit This Page on Github</a>
<a href="../../../_sources/io/fits/usage/image.txt"
rel="nofollow">Page Source</a>
<a href="#">Back to Top</a></p>
<p>
© Copyright 2011-2013, The Astropy Developers.<br/>
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3.
Last built 22 Oct 2013. <br/>
</p>
</footer>
</body>
</html>
