IRAF (Mar02) V2.12EXPORT Release Notes IRAF (Mar02)
IRAF V2.12EXPORT Release Notes
January 25, 2002
Updated: March 25, 2002
These release notes provide a summary of the major changes in V2.12.
This is a major release of IRAF and will be available for all supported
platforms. More detailed technical documentation of all system changes
will be found in the 'notes.v211' and 'notes.v212' files in the
iraf$doc and iraf$local directories. Detailed revisions notes for each
application package are in the package directories in a file called
Revisions, e.g. apphot$Revisions.
1. Highlights of This Release
2. IRAF System Revisions Summary
3. Core IRAF Revisions Summary
3.1 New Tasks
3.2 Existing tasks with new parameters/defaults
3.3 Existing tasks with new capabilities
4. NOAO Package Revisions Summary
4.1 New NOAO Packages
4.2 New NOAO Package Tasks
4.3 Existing tasks with new parameters/defaults
4.4 Existing tasks with new capabilities
5. General Package Changes
6. General Task Changes
7. Parameter File Changes
8. Details of Major System Changes
8.1 FITS Kernel Changes
8.2 Large Image Support
8.3 Virtual Memory Cache
8.4 New Developer libraries
9. System Changes Which May Affect You
9.1 Shared Library Version Incremented
9.2 External Package Recompilation
9.3 Parameter File Changes
9.4 Installation Script Changes
9.5 Help System Changes
9.6 Image Display Changes
9.7 PLIO Changes
9.8 New Environment Variables Changes
1. Highlights of This Release
o Pixel Mask Support Added to FITS Kernel
The FITS kernel was modified to add support for storing images in
extensions as compressed pixel masks using the binary table extension.
These masks may be accessed like any other image and allow for tasks
to more easily store bad-pixel masks, regions masks, or error arrays
in the same image file as the science data.
o New Pixel Mask Tasks
Several new tasks have been added to the system PROTO package for
manipulating pixel masks:
o MIMSTATISTICS allows image statistics to be computed while
rejecting pixels specified by an input mask.
o MSKEXPR task is a general-purpose mask expression evaluator
similar to IMEXPR for images, but has builtin boolean region
functions which can be used to set values inside or outside
of certain geometric shapes.
o MSKREGIONS creates an output mask based on an input text de-
scription. Region descriptions can be composed of geometric
shapes and logical operation on mask regions.
o OBJMASKS in the NPROTO package is a new task for detecting objects
in an image and creating an output catalog or pixel mask of
found objects.
o Shared Memory Limitations Eased
The IMAGES and DAOPHOT packages executables are now linked statically
to remove per-process memory limitations imposed by the IRAF shared
library on Sun and Dec Alpha systems. Previously tasks such as
DAOFIND and IMCOMBINE were limited to 268Mb on Sun systems, these
tasks can now use up to the machine memory limits.
o Image I/O Buffer Sizes Increased
Support for large image I/O was improved by changes to the internal
buffer sizes. These buffers may automatically adjust to optimal
values for the image being accessed, however new environment variables
may be set to further tune the buffers at the user level. Where
needed applications tasks were modified to take advantage of these
buffer size changes to force the imio buffer to be the size of the
input image.
o Simplified Installation Script
The install script was rewritten to clarify the output and provide
some basic checking of the IRAF system setup prior to installation,
and to do some of the most common post-install configuration. The
script will print an explanation of any errors it finds and suggest
corrective action, the hope is this will lead the user past some of
the most common installation errors.
In addition, the SYSINFO diagnostic script which does more extensive
checking of the system is also now part of the distribution. This
script can be used to verify the system once the install is complete,
or to generate a report of the system configuration if needed by
site support. An UNINSTALL script to remove iraf command links and
files created by the INSTALL script is also available to remove IRAF
from a machine. All scripts are now installed in the hlib$ directory.
o New HELP GUI and Output Options
The HELP task was enhanced to have a new GUI option for XGterm
users. This is essentially the XHELP task which has been available
in the GUIAPPS external package for some time, however the task is
fully backwards compatible and the text-mode output is still the
default. As part of this work, help pages may also now be formatted
as either HTML or Postscript for web presentation or pretty-printing
to a hardcopy device. The LROFF task was similarly modified to
provide direct conversion of lroff text sources.
o DISPLAY Task Changes
As part of the recent X11IRAF enhancements, the DISPLAY task and
others such as IMEXAMINE which interact with the display server were
modified to take advantage of the new features in XImtool V1.3.
These include support for 16 frame buffers (increased from 4 in
previous versions), and enhanced WCS readout capabilities. The
changes are fully backwards compatible for use with older XImtool
versions or display servers such as SAOimage, SAOtng, or DS9 which
have not yet been updated.
X11IRAF V1.3 is being released simultaneously (but still separately)
with IRAF V2.12. While V2.12 is fully compatible with older versions
of X11IRAF, however users will need to upgrade both systems to take
full advantage of all the new features. Users should consult the
X11IRAF Release Notes for details on what has changed there.
o New Packages
Several new packages are available in this release (see the NOAO
package change notes below for details):
- A new ASTCAT package for extracting astrometric and photometric
calibration data from remote or local catalogs was added to NOAO.
- A new CRUTIL package for doing cosmic ray detection and removal
package was installed in the IMRED package.
- A new QUADRED reduction package for QUAD format data was installed
in the IMRED package. This is a generalized replacement for the
ARED.QUAD and XCCDRED external packages for processing CTIO and
ESO FORS1 multi-amplifier data.
- A new OBSUTIL package was installed in NOAO. This is a collection
of tasks from various external packages which are useful to plan or
carry out observations.
o New Developer Libraries.
Several new libraries are available for SPP developers:
- PSIO is a new Postscript text generation library installed in
sys$psio.
- CATQUERY is a remote astrometric/photometric catalog access lib
installed in the XTOOLS utility library.
- SKYWCS is a sky coordinate transformation library installed in
the XTOOLS utility library.
2. IRAF System Revisions Summary
o The IRAF shared library version number was incremented for SunOS
and Solaris systems. See below for details on how this change
will affect external packages and locally-compiled software.
o The maximum number of nodes in a local iraf network was increased
from 320 to 512.
o The max number of open files in FIO, FIO_MAXFD, was increased from
256 to 4096. This is the "hard limit" on the maximum number of
open files in an IRAF process.
o The maximum number of host level open files, MAXOFILES, was increased
from 64 to 256. This is the maximum number of files that can be
simultaneously open at the host level. It determines the maximum
number of files that can be simultaneously open by an IRAF process
in the usual case.
o The number of keywords in a group header block for STF images (i.e.
the MAX_PCOUNT) was increased from 50 to 99 in the STF image kernel.
o Added support for the bitwise boolean operators: '&' (and), '|' (or),
'^' (xor), and '~' (not/complement), to vectory expression evaluator
fmtio$evvexpr.gy. The IMEXPR task was modified to allow these new
bitwise operations.
o Added new vector operators to VOPS library: alan, alank (logical AND)
and alor, alork (logical OR). These take any integer data as input
(short, int, long) and return a logical (expressed as int) result.
o The 'imextn' environment variable will now accept upper-case extensions
to specify image types.
o Host Command Execution: The way command line arguments are parsed
was modified to make it easier to set the value of a string parameter
to the null string. Whitespace is still skipped in @par files
as before, however null strings are valid parameter values and will
no longer cause a parameter prompt.
o The MKPKG special file list link support was enhanced to allow replacing
LFLAGS (the link flags variable) as well as the entire link line.
This makes is possible to write special-file list entries for packages
which need e.g. to be compiled nonshared on certain platforms without
creating a platform specific mkpkg file for the package itself.
o The HSI zawset.c routine which controls a process working set size was
modified to automatically detect the physical size of system memory
(with a maximum return value of 2Gb). The hard upper limit on memory
utilization defined by the unix kernel can be limited either by the
value return by the IRAF kernel (up to 90% of physical memory), or by
the value set in the user environment variable MAXWORKSET (given in
units of Mb).
o New stdimage display devices were added to support the display of Gemini
GMOS CCD data. These devices are named 'imt45' thru 'imt49' and
correspond to the following frame buffer sizes:
imt45 2080 x 4644 # imt45|imtgmosccd
imt46 6400 x 4644 # imt46|imtgmos
imt47 3200 x 2322 # imt47|imtgmos2
imt48 1600 x 1161 # imt48|imtgmos4
imt49 800 x 581 # imt49|imtgmos8
3. CORE IRAF REVISIONS SUMMARY
3.1 New Tasks
imcoords.ccget - extract objects from a test file catalog
imcoords.ccstd - transform to and from standard astrometric coordinates
proto.mimstatistics - do image statistics through a mask
proto.rskysub - sky subtract images using running mean or median
proto.mskexpr - general mask expression evaluator
proto.mskregions - create a mask from a list of region specifications
3.2 Existing Tasks with New Parameters or New Parameter Defaults
immatch.imcentroid - new parameter maxshift
immatch.imalign - new parameter maxshift
immatch.geomap - new parameter maxiter, default reject = 3.0 not INDEF
imcoords.ccmap - new parameter maxiter, default reject = 3.0 not INDEF
imcoords.imcctran - new parameter longpole
imutil.hedit - new parameter addonly
imutil.imstatistics - new parameters nclip, lsigma, usigma, cache
3.3 Existing Tasks with New Capabilities
immatch.imcentroid - optionally rejects objects whose centers wander too much
immatch.imalign - optionally rejects objects whose centers wander too much
immatch.geomap - iterative rejection capability added
imcoords.ccmap - iterative rejection capability added
imcoords.imcctran - support for non-zenithal projections added
imutil.hedit - support for add keyword only if new option
imutil.imstatistics - support for iterative rejection and memory caching added
imutil.imexpr - support for bitwise operators or, and, xor, and not added
4. NOAO PACKAGE REVISIONS SUMMARY
4.1 New NOAO Packages
astcat - Astronomical catalog and surveys access package
crutil - Cosmic ray detection and removal package
obsutil - Observing utilities package
4.2 New NOAO Package Tasks
apphot.pcalc - Do arithmetic operations on a list of apphot databases
apphot.pconvert - Convert a text database to a tables database
apphot.pdump - Print selected fields from a list of apphot databases
apphot.pexamine - Interactively examine and edit an apphot database
apphot.prenumber - Renumber stars in an apphot database
apphot.pselect - Select records from an apphot database
apphot.psort - Sort an apphot database
astcat.aclist - List the supported astrometric catalogs
astcat.agetcat - Extract astrometry files from astrometric catalogs
astcat.afiltcat - Filter astrometry files derived from astrometric catalogs
astcat.adumpcat - Catalog access debugging task
astcat.aslist - List the supported image surveys
astcat.agetim - Extract FITS images from image surveys
astcat.ahedit - Initialize the image wcs and set standard keywords
astcat.aimfind - Select images containing catalog objects
astcat.adumpim - Image survey access debugging task
astcat.aregpars - Default region parameter set
astcat.acatpars - Default astrometry file format parameter set
astcat.afiltpars - Default astrometry file filtering parameters
astcat.aimpars - Default image data parameters
astcat.awcspars - Default image wcs parameters
crutil.cosmicrays - Remove cosmic rays using flux ratio algorithm
crutil.craverage - Detect CRs against average and avoid objects
crutil.crcombine - Combine multiple exposures to eliminate cosmic rays
crutil.credit - Interactively edit cosmic rays using an image display
crutil.crfix - Fix cosmic rays in images using cosmic ray masks
crutil.crgrow - Grow cosmic rays in cosmic ray masks
crutil.crmedian - Detect and replace cosmic rays with median filter
crutil.crnebula - Detect and replace cosmic rays in nebular data
obsutil.psfmeasure - Measure PSF sizes from stellar images
obsutil.specfocus - Determine spectral focus and alignment variations
obsutil.starfocus - Determine direct focus variations from stellar images
obsutil.ccdtime - CCD photometry exposure time calculator
obsutil.pairmass - Plot airmass vs time for a given coordinate
obsutil.sptime - Spectroscopic exposure time calculator
obsutil.specpars - Spectrograph instrument parameters for sptime
obsutil.bitcount - Accumulate the bit statistics for a list of images
obsutil.findgain - Estimate the gain and readnoise of a CCD
obsutil.shutcor - Shutter correction from images of varying exposure times
nproto.objmasks - detect and catalog objects in image
4.3 Existing Packages and Tasks with New Parameters or New Parameter Defaults
apphot - new package parameters wcsin, wcsout, and cache
apphot.center - new parameters wcsin, wcsout, cache
apphot.daofind - new parameters wcsout, cache
apphot.fitpsf - new parameters wcsin, wcsout, cache
apphot.fitsky - new parameters wcsin, wcsout, cache
apphot.phot - new parameters wcsin, wcsout, cache
apphot.polymark - new parameters wcsin, wcsout, cache
apphot.polyphot - new parameters wcsin, wcsout, cache
apphot.qphot - new parameters wcsin, wcsout, cache
apphot.radprof - new parameters wcsin, wcsout, cache
apphot.wphot - new parameters wcsin, wcsout, cache
apphot.txdump - replaced by pdump, available as a hidden task
astutil.setairmass - new parameters ra, dec, equinox, st, ut, scale
daophot - new package parameters wcsin, wcsout, wcspsf, and cache
daophot.addstar - new parameters wcsin, wcsout, wcspsf, and cache
daophot.allstar - new parameters wcsin, wcsout, and wcspsf
daophot.daoedit - new parameters cache
daophot.daofind - new parameters wcsout, and cache
daophot.group - new parameters wcsin, wcsout, wcspsf, and cache
daophot.nstar - new parameters wcsin, wcsout, wcspsf, and cache
daophot.peak - new parameters wcsin, wcsout, wcspsf, and cache
daophot.phot - new parameters wcsin, wcsout, and cache
daophot.psf - new parameters wcsin, wcsout, and cache
daophot.substar - new parameters wcsin, wcsout, and cache
4.4 Existing Tasks with New Capabilities
apphot.center - coordinate system support, optional image cacheing
apphot.daofind - coordinate system support, optional image cacheing
apphot.fitpsf - coordinate system support, optional image cacheing
apphot.fitsky - coordinate system support, optional image cacheing
apphot.phot - coordinate system support, optional image cacheing
apphot.polymark - coordinate system support, optional image cacheing
apphot.polyphot - coordinate system support, optional image cacheing
apphot.qphot - coordinate system support, optional image cacheing
apphot.radprof - coordinate system support, optional image cacheing
apphot.wphot - coordinate system support, optional image cacheing
astutil.setairmass - ra, dec, equinox, st, ut, scale are no longer hardwired
astutil.rvcorrect - more flexibility in setting ut
daophot.addstar - coordinate system support, optional image cacheing
daophot.allstar - coordinate system support
daophot.daoedit - optional image cacheing
daophot.daofind - coordinate system support, optional image cacheing
daophot.group - coordinate system support, optional image cacheing
daophot.nstar - coordinate system support, optional image cacheing
daophot.peak - coordinate system support, optional image cacheing
daophot.phot - coordinate system support, optional image cacheing
daophot.psf - coordinate system support, optional image cacheing
daophot.substar - coordinate system support, optional image cacheing
5. General Package Changes
NOAO
ONEDSPEC
More than 999 apertures are now allowed.
APPHOT
Coordinate Support:
All the apphot tasks have been modified to accept input coordinates
in the logical, tv, physical, or world systems, and to write output
coordinates in the logical, tv, or physical coordinate systems. One
consequence of this is that the apphot tasks will now work correctly
on image sections in interactive mode. Another is that users can now
work directly on image sections while preserving the coordinate
system of the parent image.
Image Cacheing Support:
All the apphot tasks which accept image pixel input have been mod-
ified to optional cache the entire input image in memory. Cacheing
may significantly improve the performance of tasks where many random
access operations are performed.
File and image name directory information removed from output files
All the apphot tasks have been modified to strip directory infor-
mation from the image and coordinate file names written to the output
files, to the terminal, and to the plot headers. The colon commands
will still read and write full image and coordinate file path names.
New PTOOLS Tasks Added
The ptools package tasks pcalc, pconvert, pdump, prenumber, pselect
and psort were added to the apphot package. The functionality of the
old txdump task as been replaced by the pdump. TXDUMP is still avail-
able as a hidden task.
ASTCAT
The astcat package is a set of tasks for extracting astrometric and
photometric calibration data from remote or local catalogs, filtering
the data, extracting FITS images from remote or local surveys, and adding
standard keywords to the extracted images. There is also a task for
selecting images which contain catalog objects and locating the catalog
objects in the image.
IMRED.CRUTIL
Cosmic ray detection and removal package. This package includes new
tasks and links to tasks from other package. It replaces the CRUTIL
external package.
IMRED.QUADRED
Reduction package for QUAD format data. This replaces the ARED.QUAD
and XCCDRED external packages for processing CTIO and ESO FORS1 multi-
amplifier data.
DAOPHOT
Coordinate Support
All the daophot tasks have been modified to accept input coordinates
in the logical, tv, physical, or world systems, and to write the output
coordinates in the logical, tv, or physical coordinate systems. One
consequence of this is that the daophot tasks will now work correctly
on image sections in interactive mode. Another is that users can now
work directly on image sections while preserving the coordinate
system of the parent image.
Image Cacheing Support
All the daophot tasks which accept image pixel input have been modified
to optionally cache the entire input image in memory. Cacheing signif-
icantly improves the performance of the tasks when many random access
operations are performed. The cacheing already performed by the
ALLSTAR task is unchanged.
File and image name directory information removed from output files
All the daophot tasks have been modified to strip directory information
from the image and coordinate file names written to the output files,
to the terminal, and to the plot headers. The colon commands will still
read and write full image and coordinate file path names.
OBSUTIL
New observing utilities package. This collects tasks from the NMISC,
SPECTIME, PROTO, and NLOCAL external package which are useful to
plan or carry out observations. The new tasks are:
PSFMEASURE STARFOCUS SPECFOCUS CCDTIME
PAIRMASS SPTIME BITCOUNT FINDGAIN
SHUTCOR
OBSOLETE
o Added tasks OIMCOMBINE and OIMSTATISTICS which are the previous
versions from V2.113b system
o Deleted the ODISPLAY task
6. General Task Changes
NOAO
ONEDSPEC.SPLOT
Rather than refusing to evaluate errors when there is negative data,
negative data is treated as zero.
ASTUTIL.SETAIRMASS
Modified to have greater flexibility in selecting the keyword defining
the universal time. New parameters define the keywords for RA, dec,
equinox, siderial time, universal time, and astrospheric scale height.
ASTUTIL.RVCORRECT
Modified to have greater flexibility in selecting the keyword defining
the universal time.
IMRED.ECHELLE.ECIDENTIFY
Help page describes how to externally evaluate the dispersion fcns.
IMRED.CCREDRED.COSMISRAYS
Task was removed (see CRUTIL)
NPROTO.FINDGAIN
Task was removed (see OBSUTIL)
NPROTO.OBJMASKS
This is a new task for detecting objects in an image and creating
an output catalog or pixel mask of found objects.
TWODSPEC.LONGSLIT.FITCOORDS
- Help page describes the contents of the database and how to ext-
ernally evaluate the fits.
- The RMS is shown in the graph title and in the :show output.
TWODSPEC.APEXTRACT.APEDIT
When there is just one aperture the background regions are shown
on the graph without needing to enter the 'b' background mode.
IMAGES
TV.DISPLAY
- The mask overlay feature when the displayed image is a reduction of
mask (e.g. a block average) now uses the maximum of all mask pixels
within the display pixel.
- The task will now allow up to 16 frame buffers to be used for the
display if allowed by the server. (Currently requires XIMtool V1.3).
TV.IMEXAMINE
- A new key 't' allows output of a region centered on the cursor as an
image for further analysis by other programs.
- The task will now allow up to 16 frame buffers to be used for the
display if allowed by the server. (Currently requires XIMtool V1.3).
- Cursor readback will now properly detect the correct image when more
than one image is displayed per frame, e.g. in a mosaic. (Currently
requires XIMtool V1.3).
IMMATCH.IMCOMBINE
- New parameters "headers", "bpmasks", "rejmasks", "nrejmasks", and
"expmasks" provide additional types of output. The old parameters
"rejmask" and "plfile" were removed. The new "nrejmasks" parameter
corresponds to the old "plfile" and the new "rejmasks" parameter
corresponds to the old "rejmask".
- There is a new "combine" type "sum" for summing instead of averaging
the final set of offset, scaled, and weighted pixels.
- There is a new parameter "outlimits" to allow output of a subregion
of the full output. This is useful for raster surveys with large
numbers of images.
- Additional keywords may appear in the output headers.
- Scaling is now done relative to the first image rather than an average
over the images. This is done so that flux related keywords such as
exposure time and airmass remain representative.
- A median calculation was made faster.
- The previous version is available in the OBSOLETE package.
IMMATCH.IMCENTROID
IMMATCH.IMALIGN
A new parameter maxshift has been added to the imcentroid and imalign
tasks. Maxshift defines the maximum permitted difference between the
predicted and computed shifts. It is used to reject objects whose
positions have wandered too far from the predicted positions.
IMMATCH.GEOMAP
IMCOORDS.CCMAP
An iterative rejection capability has been added to the geomap and
ccmap tasks. The new parameter maxiter in combination with the existing
parameter reject define the rejection parameter. The default value of
the reject parameter has been changed from INDEF to 3.0.
The colon command ":order <value>" has been added to the geomap and ccmap
tasks. The new command enables the user to change all the order parameters
simultaneously when experimenting with different fitting functions.
IMCOORDS.STARFIND
The starfind task background estimation algorithm has been modified so
that it no longer depends on the value and density of the central pixel.
IMCOORDS.IMCCTRAN
Support for non-zenithal projections has been added to the imcctran task.
The previous technique of rotating the cd matrix does not work properly
for these functions. The new parameter longpole was added to imcctran.
Longpole enables the user to select either the cd matrix or longpole /
latpole method for transforming zenithal projections.
IMCOORDS.CCGET
The new task ccget was added to the imcoords package. Ccget extracts
objects in a user specified region from a a simple text file catalog.
IMCOORDS.CCSTD
The task ccstd was added to the imcoords package. Ccstd transforms pixel
and celestial coordinates to standard coordinates and vice versa.
IMUTIL.HEDIT
The new parameter addonly was added to hedit task. The addonly switch
is used to add a parameter to the image header only if it does not
already exist. The addonly switch has a precedence intermediate between
the add and delete switches.
IMUTIL.IMSTATISTICS
An interative rejection capability has been added to the imstatistics
task. The new parameters nclip, lsigma, and usigma define the rejection
parameters. A memory cacheing option was also added to imstatistics in
order to optionally speed up performance if iterative rejection is en-
abled or the midpt/mode is computed.
IMUTIL.IMEXPR
Support for the bitwise operators or (|), and (&), exclusive or (^), and
not (~) has been added to the imexpr task. The logical operators or (||)
and and (&&) have ben made truly logical i.e. they return 0's or 1's,
rather than results of a bitwise or and and.
PROTO
MIMSTATISTICS
The new task mimstatistics has been added to the proto package.
Mimstatistics does image statistics through a mask.
RSKYSUB
The new task rskysub was added to the proto package. Rskysub does a
running mean or median sky subtraction on an ordered list of images
using optional background scaling and object masking.
MSKEXPR
The new task mskexpr has been added to the proto package. Mskexpr
creates a new mask from a user supplied expression, an optional
reference image, and an optional reference mask.
MSKREGIONS
The new task mskregions has been added to the proto package. Mskregions
creates a new mask or modifies an existing mask using a list of region
definitions or region expressions.
XTOOLS
SKYWCS
A new library skywcs has been added to the xtools package. The skywcs
library is a set of routines for managing image and catalog celestial
coordinate systems and for transforming from one celestial coordinate
system to another. Skywcs is layered on the Starlink Positional
Astronomy library slalib which is installed in the iraf math package.
CATQUERY
A new library catquery was added to the xtools package. The catquery
library is a set of routines for doing local and remote catalog and
image survey access.
SYSTEM
HELP
Task was modified to call the XHELP code to run the GUI version of
the task if requested. Task output is the same if the device
remains the default 'terminal' value, however resetting the 'device'
parameter to one of 'gui', 'html', or 'ps' will either spawn the GUI
task under xgterm or print the converted help page to the stdout.
LROFF
The task was enhanced with a new 'format' parameter that allows the
text to be formatted as one of: plain-text, HTML, or Postscript.
7. Parameter File Changes
In the tables below each parameter change is identified with one of the
following codes followed by task name and the description of the change.
* n = new parameter
* c = changed/modified parameter
* d = deleted parameter
CL
n cl Added the new CL parameter "release". This
is a string valued parameter with values such
as "2.11.3a", "2.12", "3.0" etc. This differs
from "version" which is a descriptive string
such as "NOAO/IRAF V2.11.3 EXPORT". There can
be multiple releases of one version of the
software, and "release" specifies exactly what
build the software is. The release strings are
composed in such a way that they can be used
in expressions, e.g. (release >= 2.11.3) would
be true for IRAF V2.11.3 and all subsequent
releases.
DATAIO
c dataio.export Made the 'format' parameter automatic mode
c dataio.import Made the 'format' parameter automatic mode
IMAGES
n imcoords.imcctran Added a new parameter longpole to the imcctran
task. If longpole=yes then coordinate transfor-
mations with zenithal projections will be rot-
ated using longpole rather than the CD matrix.
c immatch.wregister Fixed boundary option typo, "refect" to "reflect".
c immatch.sregister Fixed boundary option typo, "refect" to "reflect".
n immatch.imcentroid Added a new parameter maxshift to the imcentroid
immatch.imalign and imalign tasks. Maxshift is the maximum perm-
itted difference between the computed and predicted
shifts. Maxshift can be used to reject objects whose
centers have wandered too far from the expected
center. By default maxshift is undefined.
n immatch.geomap Added a new parameter maxiter to the geomap and
immatch.ccmap ccmap tasks. Maxiter defines the maximum number of
rejection iterations and has a default value of 0
for no rejection.
c immatch.geomap Changed the default value of the ccmap and geomap
c immatch.ccmap parameter reject from INDEF to 3.0.
c immatch.imcombine Numerous changes, see details above
c imgeom.imlintran Changed the nrows argument names to nlines
n imutil.hedit Added a new addonly parameter to the hedit task. If
addonly is set a new field will only be added to
the image header if it does not already exist.
n tv.imexamine Added new parameters 'output', 'ncoutput', and
'nloutput' used by the new 't' keystroke when
outputting an image section centered on the cursor.
SYSTEM
n help New parameters required for GUI options, output
formats for HTML/PS, printer, etc.
n lroff Added new 'format' parameter for HTML/PS output
UTILITIES
c utilities.surfit Added support for the half cross-terms option to
the surfit task. This involved changing the type
of the xterms parameter from boolen (yes/no) to
string (none,half,full).
NOAO
ASTUTIL
n astutil.setairmass new parameters ra, dec, equinox, st, ut, scale
DIGIPHOT
n apphot new package parameters wcsin, wcsout, and cache
n apphot.center new parameters wcsin, wcsout, cache
n apphot.daofind new parameters wcsout, cache
n apphot.fitpsf new parameters wcsin, wcsout, cache
n apphot.fitsky new parameters wcsin, wcsout, cache
n apphot.phot new parameters wcsin, wcsout, cache
n apphot.polymark new parameters wcsin, wcsout, cache
n apphot.polyphot new parameters wcsin, wcsout, cache
n apphot.qphot new parameters wcsin, wcsout, cache
n apphot.radprof new parameters wcsin, wcsout, cache
n apphot.wphot new parameters wcsin, wcsout, cache
n daophot new package params wcsin, wcsout, wcspsf, and cache
n daophot.addstar new parameters wcsin, wcsout, wcspsf, and cache
n daophot.allstar new parameters wcsin, wcsout, and wcspsf
n daophot.daoedit new parameters cache
n daophot.daofind new parameters wcsout, and cache
n daophot.group new parameters wcsin, wcsout, wcspsf, and cache
n daophot.nstar new parameters wcsin, wcsout, wcspsf, and cache
n daophot.peak new parameters wcsin, wcsout, wcspsf, and cache
n daophot.phot new parameters wcsin, wcsout, and cache
n daophot.psf new parameters wcsin, wcsout, and cache
n daophot.substar new parameters wcsin, wcsout, and cache
ONEDSPEC
n standard new parameter mag, magband, and teff. These
n splot params can be use to specify calibration files
n lcalib as blackbody curves scale to a specified magnitude
TWODSPEC
c apextract.apall1 Reduced the 'polysep' parameter.
c apextract.apdebug Reduced the 'polysep' parameter.
c apextract.apfit1 Reduced the 'polysep' parameter.
c apextract.apnoise1 Reduced the 'polysep' parameter.
c apextract.apnorm1 Reduced the 'polysep' parameter.
c apextract.apparams Reduced the 'polysep' parameter.
8. Details of Major System Changes
8.1 FITS kernel changes
The FITS kernel was modified to add support for storing images in
extensions as compressed pixel masks. The mask is stored as a binary
table using the "ZIMAGE" (compressed image) convention proposed by White,
Greenfield, Pence, and Tody in 1999:
http://heasarc.gsfc.nasa.gov
/docs/software/fitsio/compression/compress_image.html
In the current implementation only the "PLIO_1" compression
algorithm is implemented. Mask extensions may be read or written directly
by the kernel. When writing a new extension it will be appended to the
MEF file. To append an image to a MEF file as a mask, include "type=mask"
in the image kernel section when the output image is opened.
Masks are interfaced to the system as images and may be read and
written like any other image via IMIO. They have a normal image header
and can be manipulated with any program that deals with images. The pixel
type is INT.
It is also possible to access a mask image as a PLIO mask. An
IMSTATI query for IM_PLDES parameter will return the PLIO mask descriptor.
While a mask extension is opened under IMIO it is represented as a PLIO
mask and may be accessed in this form like any other mask.
The mask image is stored in the FITS binary table (BINTABLE)
extension when the image is closed, and is loaded from the extension when
the image is opened. The compression representation used to store the
mask in the binary table is the same as is used within PLIO. The new
(V2.12) encoding is used, allowing very large masks to be stored.
Currently masks up to 3D are supported. Data on each 2D mask plane
will be compressed in both X and Y as with PLIO. The depth of the mask
is preserved.
Although a mask is stored as a binary table the format of the
table is not completely general. In the current implementation there
can be only one column in the table (COMPRESSED_DATA). This is an
integer-valued variable length array column containing, for each line of
the N-dimensional image, the PLIO compressed version of that image line.
The actual compressed data is stored in the heap area of the table.
Multiple image lines may point to the same compressed line list, e.g.,
to store the empty line or to compression in Y.
8.2 Large Image Support
The following changes were made to enable IMIO to use larger
buffer sizes to optimize i/o for large images:
The default file buffer size set by IMIO is unchanged: it is still
about 512 MB, the value set for V2.11.2. However, a new parameter
IM_BUFFRAC was added. Both IM_BUFSIZE and IM_BUFFRAC are used to help
determine the FIO buffer size set when an image is opened. The logic
for this is implemented in imsetbuf.x.
Backwards compatibility. If you do nothing about IMIO/FIO buffers
in your program, the system may transparently use a larger buffer for
larger images. If you set BUFSIZE in your program, the system will
by default use the value you give, or possibly a larger value, if the
image you are accessing is very large. If you set BUFSIZE and you want
to guarantee that the value you set is used (even for very large images)
then you should also set BUFFRAC=0 to ensure that only BUFSIZE is used.
How it works. BUFFRAC specifies the default FIO buffer size used to
access an image, expressed as a percentage of the size of the image.
For example, the builtin default value of BUFFRAC=10 will try to make a
FIO buffer 10% of the size of the image. The actual value used will be
given by BUFFRAC, but will be at least BUFSIZE, and no more than a builtin
default maximum value, currently 32 MB. Given the builtin defaults,
the buffer size will range from 0.5 to 32 MB depending upon the size of
the image being accessed. As noted above, BUFSIZE and BUFFRAC can be
set to force the buffer size to a specific value if desired.
Environment variables for both parameters are provided. The names
are "IMIO_BUFSIZE" (specified as bytes) and "IMIO_BUFFRAC" (specified
as a decimal fraction). If defined, these override (at image open time)
the builtin default values for both parameters. An IMSET call by the
application will override all such defaults.
The FIO buffer allocated will not be larger than the size of the
image. The FIO buffer will also not exceed the maximum size set by
the file driver being accessed. For example, for PLIO images the file
buffer will not exceed about 2KB, even for a very large mask. This is
because the "pixel file" for a PLIO image is dev$null, the driver for
which specifies a maximum i/o buffer size of 2K (the real file to load
or save the mask will use a different descriptor).
The intent here is to provide an adaptive mechanism for setting the
FIO buffer size used to access an image, which automatically adapts to
the size of the image being accessed. If you access a lot of small
images you will get smaller buffers - everything will be as before.
If you access very large images, you may get large buffers up to the
builtin maximum value of (currently) 32 MB.
Using large buffers could cause a machine to run out of memory.
However, it is likely that if someone is working on 300 MB images
that they are using a machine which has a memory at least that large
- probably larger. If there are problems, the environment variable
overrides can be used to tune IMIO.
The reason for large file buffers is to limit the number of disk
data transfers, and hence the number of disk seeks. Using buffers larger
than a certain amount (32 MB is generous) is probably counterproductive.
If the i/o system provides 20 MB/sec i/o transfers, 32 MB will take
1.6 seconds. This should be more than a large enough granularity to
provide efficient i/o, hence is a reasonable limit (at this point paging
effects are likely to dominate).
8.3 Virtual Memory Cache
The VMcache client interface and daemon provide a method by
which data-intensive IRAF tasks (or non-IRAF tasks for that matter) can
manage how files/images are maintained in virtual memory to avoid
excessive system paging. In essence it's a way to "lock" a specific
image in memory to improve performance. As of this release no tasks in
the system have been modified to make use of the VMcache daemon,
however installing it in the system at this point provides a framework
for future applications and systems development.
The following notes summarize the changes made for this feature
and describe it's function in more detail. A more complete description
of the interface, environment variables which control it, etc can be
found in the main systems revisions file iraf$local/notes.v211.
The source for the developmental version of the VMcache library
and the VMcache daemon (vmcached) have been installed in the
unix$boot tree and the HSI binary file driver was modified to add VMcache
client support. This adds two new capabilities to the driver: 1)
built-in support for direct i/o (on systems that support it), and 2)
a client interface to the VMcache daemon to permit the daemon to
optimally manage binary file i/o if a VMcache daemon is present.
The vmcached code is complete but only enough debug/testing was done to
support development of the VMcache client interface for IRAF (the vmcached
code is debugged but the new version of the vmcache library code has
not been tested). Since the daemon can be utilized outside the normal
IRAF release we do not have to fully develop it for the release.
It should be stressed that VMcache is only useful or warranted for
systems that are very data intensive. The standard host operating
system file access heuristics are best for "normal" processing where
either the system is not really busy, or the datafiles are not
excessively large. On systems with very large physical memories
where massive amounts of data are being processed, VMcache can make
a significant difference in overall system performance.
VMcache is too complex to document here. Without going into the
details, its function is to manage a cache of files in system
virtual memory. Files can be explicitly cached or uncached, or they
can be "accessed", and VMcache will decide whether or not to cache
the file in virtual memory. This is what the VMcache client
interface does: every time it accesses (opens or extends) a file
larger then the VM threshold it sends an "access" directive to the
VMcache daemon. The daemon sends back a response of 0 (file not
cached; use direct i/o to access the file), or 1 (file cached in VM;
use normal VM-buffered i/o to access the file). Even if a file is
not cached the daemon keeps track of all accesses. Files which are
frequently accessed will have a higher prority and are more likely
to be cached in memory.
The VMcache daemon is a separate system-level program outside of
IRAF. This is necessary to provide a central system-wide cache
controller. It also provides flexibility, allowing multiple
versions of the daemon to exist, e.g., to allow experimentation with
different types of caching algorithms. It also allows easy
customization of the daemon independently of the IRAF applications
using the VMcache client interface.
8.4 New Developer Libraries
o Several new libraries are now available for developers:
PSIO New Postscript text generation library installed in the
sys$psio. The PSIO interface is used to format a block of
text as Postscript output on a page of a given size (Letter,
Legal, A4 or B5). See the psio$README file for details.
CATQUERY Remote astrometric/photometric catalog access lib installed
in the XTOOLS utility library.
The catquery package provides a set of routines for local
and remote catalog and image survey server access. The sup-
ported catalogs and image surveys are described in records
stored in a catalog and image survey configuration file
respectively. The catalog and image survey records specify
the network address, the query format, and the output format
for each supported catalog or image display server. See
"help catalogs" and "help surveys" for details.
SKYWCS Sky coordinate transformation library installed in the XTOOLS
utility library.
The skywcs package contains a simple set of routines for
managing sky coordinate information and for transforming from
one sky coordinate system to another. The sky coordinate
system is defined either by a system name, e.g. "J2000",
"galactic", etc., or by an image system name, e.g. "dev$ypix"
or "dev$ypix world".
9. System Changes Which May Affect You
* SHARED LIBRARY VERSION INCREMENTED (Sun/IRAF only)
The IRAF shared library for SunOS and Solaris platforms has been
incremented with this release due to the nature of various system
changes. Existing IRAF binaries (e.g. locally written software
or external packages) will continue to run using the old shared
image, however they will need to be recompiled against V2.12 in order
to pick up the numerous system bug fixes and features in this release.
In particular, pixel masks produced by V2.12 IRAF tasks may be
incompatible with external packages which have not been recompiled.
* EXTERNAL PACKAGE RECOMPILATION
The V2.12 release contains changes to the FIO and PLIO/PMIO interface
header files used by numerous applications. Relinking of an external
package may fail to pick up these changes and not recompile a source
file which uses one of these header files if the mkpkg file doesn't
correctly list all of the dependencies (nearly all packages have one
or more mkpkg files which have this problem). In the worst cases
this could lead to a runtime error due to the incompatibilities.
For this reason we recommend that all packages and local tasks be
recompiled (completely from source* (rather than simply relinked
against the new version) to assure that all changes and new features
will be included. Recompilation also guarantees that packages can
take advantage of some of the larger buffer sizes and optimizations
in this release. Site support can supply a list of missing mkpkg
dependencies for most external packages being developed outside NOAO
that wish to fix these files for a future release.
* PARAMETER FILE CHANGES
As with all major releases, we recommend that you do a MKIRAF and
delete all your old parameter files after the IRAF upgrade. You may
choose not to do this if you are in the midst of a project and have
setups that may be difficult to reproduce.
The automatic parameter file update/merge mechanism, which is used
if you do not initialize your parameters with MKIRAF, is based on
file date comparisons. If you run IRAF V2.11 after V2.12 has been
installed, the file dates on your uparm parameter files will be
more recent than the V2.12 installation date. If you then try to
run V2.12, the automatic parameter file merge/update will fail due
to the file dates. The system only updates personal parameter
files which are older than the update date of the system. A
MKIRAF avoids the problem if you delete your parameter files,
causing them to be updated from the system default versions.
* INSTALLATION SCRIPT CHANGES
As the first step of an ongoing effort to simplify the installation
and system configuration, the IRAF install script was rewritten to
do some error-checking of the iraf setup, present a simplified and
easier to read output, and do some common post-install configuration
of the system. Additionally, the SYSINFO diagnostic script for
finding system errors and reporting on the configuration, and a new
UNINSTALL script for removing IRAF files/links from the system have
also been installed. The old install script is still available as
a fallback in case problems with the new script are found.
* HELP SYSTEM CHANGES
The HELP task was modified with several new parameters controlling
the display and formatting of the help pages. Help may now be
presented as formatted text (as before), HTML, or fully formatted
Postscript. Additionally, users running under an XGterm window can
use the task in a new GUI mode. The help GUI allows users to browse
the help system and easily search for tasks/topics using a familiar
web-like interface. The GUI mode is not the default, but can be
enabled easily using the 'device' parameter.
* IMAGE DISPLAY CHANGES
Tasks which display images or interact with the image display were
modified to take advantage of new features added to XImtool V1.3
(e.g. the multiple WCS and pixel-value readouts and 16 display frame
buffers). These changes were done in a backwards compatible way so
interaction with display servers such as SAOimage, SAOtng, DS9, or
older XImtool versions should be unaffected. If problems are dis-
covered a CL environment variable 'disable_wcs_maps' can be defined
to force all of the old behaviors. These changes do not add any new
functionality to the tasks themselves, only the underlying display
protocols.
* PLIO Changes
The LEN and BLEN fields of the encoded line list (LL) descriptor
would limit the length of a pixel area (and hence the size of a
pixel mask) to the max size of a signed short, 32768. This was due
to the use of a simple array of type short to encode the line list
(which simplifies handling considerably). Nonetheless the limit to
32K was unacceptable. The fix adopted was to increase the LL header
from 3 to 7 words. Two 16 bit words are now used to encode each of
LEN and BLEN. A "version" word was added to allow the old, new, and
future encodings to be distinguished. A "hdrlen" word was added to
parameterize the length of the LL header, rather than fix it at
compile time as in the initial version. With this change, the
maximum length of an image line under PLIO is increased from 32768
to 1073741824 (32768*32768). All the higher level PLIO code is
integer, so should already support larger masks.
This was done in such a way that old line lists can still be read,
although PLIO will always write out new format line lists (pixel
mask files and images, QPOE, and MWCS all store encoded line lists
in external storage, so backwards compatibility is important; also
existing complied programs will continue to generate the old
format). The cost is 8 bytes per encoded line list. For most masks
this should only increase the size of the mask by a few percent at
most.
* NEW ENVIRONMENT VARIABLES
The following new environment variables may be defined to tune the
size of the system file i/o buffers used by the image i/o system.
The system will automatically adjust to use larger buffers when
accessing larger images, these variables may be set to further
optimize the buffers
IMIO_BUFSIZE Size of the FIO buffer size in bytes.
IMIO_BUFFRAC FIO buffer size expressed as a percentage of
the image size. Actual value will be at least
BUFSIZE and no more than BUFMAX.
IMIO_BUFMAX Max size of FIO buffer which will override the
32Mb default.
Other miscellaneous environment variables:
disable_wcs_maps If defined or set to 'yes', this variable will
force any tasks which interact with the image
display to use the old protocols.
pspage Variable which is used by the PSIO interface to
set the default page size. Acceptable values
include "letter" (the default) for US Letter,
"legal" for US Legal, and "a4" and "b5" for the
most common European sizes. Pspage can be used
by the new HELP and LROFF tasks to automatically
set the desired Postscript page size.
IRAF (Aug97) V2.11 Revisions Summary IRAF (Aug97)
IRAF V2.11 Revisions Summary
August 27, 1997
Introduction
Modifications and additions to IRAF V2.11EXPORT, compiled since the last
documented release of IRAF, V2.10.3, are summarized below. V2.11EXPORT is
a major release of IRAF and will be available for all supported
platforms. These release notes provide a summary of the major changes in
V2.11. More detailed technical documentation of all system changes will
be found in the "notes.v210" and "notes.v211" files in the iraf$doc and
iraf$local directories.
1. Things to be aware of or watch out for
1.1. Parameter file changes
1.2. Networking change
1.3. Image format change
1.4. FITS kernel
1.5. RFITS/wfits changes
1.6. Merged SunOS and Solaris IRAF systems
1.7. Tape access
1.8. Default magnitude zero changed
2. IMAGES package changes
3. Major system changes
3.1. New FITS image kernel (FXF)
3.2. Changes to the IRAF native image format (OIF)
3.3. IMFORT changes
3.4. Environment variables
3.5. New intrinsic functions
3.6. System default modifications
3.7. Libraries added
3.8. Graphics changes
3.9. FITS-related core-level task changes
3.10. General changes
4. New tasks, or old tasks moved to new packages
5. Task and package deletions
6. Modifications to old tasks
7. Parameter file changes
The IRAF V2.11 release notes can also be found online on the Internet at
the URL http://iraf.noao.edu/v211revs.html.
1. Things to be aware of or watch out for
1.1 Parameter file changes
Since this is a major release we recommend that you do a mkiraf and
delete all your old parameter files. You may choose not to do this if
you are in the midst of a project and have setups that may be difficult
to reproduce. Old IMAGES package parameter files will no longer be
recognized, however, because of the package reorganization mentioned
below. Generally, old parameter files are merged automatically with
any new parameter files if there have been any changes, but if you do
have problems you will need to unlearn the task before you can proceed.
A list of the parameter file changes appears below and you may wish to
check this list to see how this will affect your setups.
The automatic parameter file update/merge mechanism, which is used if
you do not initialize your parameters with mkiraf, is based on file date
comparisons. If you run IRAF V2.10 after V2.11 has been installed, the
file dates on your uparm parameter files will be more recent than the
V2.11 installation date. If you then try to run V2.11, the automatic
parameter file merge/update will fail due to the file dates. The system
only updates personal parameter files which are older than the update
date of the system. A mkiraf avoids the problem if you delete your
parameter files, causing them to be updated from the system default
versions.
1.2 Networking change
The "set node = foo" syntax, used to enable remote image display under
IRAF networking, has changed. The new syntax requires that an
exclamation be appended to the node name as in the example below (this
dates back to V2.10.4 so many users will already be familiar with the
feature).
cl> set node = "orion!"
1.3 Image format change
The internal IRAF image format (.imh images) has changed. V2.11EXPORT
can read the old image format but the new image format is not readable
by V2.10.4 or earlier versions. This means that you can not easily go
from the new IRAF system (V2.11) to an old one (V2.10.4 or earlier)
unless you first convert the V2.11 IRAF images to FITS files. All your
old V2.10.4 or earlier images are readable by V2.11EXPORT. The benefit
is that the new image format is machine independent, slightly more
storage efficient, and supports long file pathnames. If it is
necessary to be able to read images written by V2.11 with older
software, V2.11 can be made to write the old IRAF image format by
setting the oifversion environment variable, e.g., "set oifversion = 1"
(the default is version 2). See below for details.
1.4 FITS kernel
A FITS image kernel is available in V2.11, allowing runtime read and
write access to FITS files on disk. There are many related changes to
IRAF image i/o and FITS support. More information on the new image
kernel, and on the expanded FITS support available in V2.11, is given
below.
1.5 RFITS/WFITS changes
rfits and wfits have been modified to support multi-extension FITS
files. The extension numbering convention used is the same as in the
FITS image kernel. As a result, users who read simple FITS files on
disk with a command such as "rfits diskfilename 1 foo" will find that
this no longer works - instead use "rfits diskfilename 0 foo". See
below for details.
1.6 Merged SunOS and Solaris IRAF systems
A single installation of Sun/IRAF will now simultaneously support both
SunOS and Solaris (previously separate IRAF distributions were required
for each).
1.7 Tape access
The "tapecap" mechanism has changed. The system now looks for the
filename "tapecap.<node>" followed by the default "tapecap". <node>:
should be the hostname (as used by IRAF networking) of the server
hosting the tape drives described by the tapecap file. For example if
host "gemini" serves up some tape drives it's tapecap file is named
"tapecap.gemini". If a server-specific tapecap file is not found the
default "tapecap" (on the possibly remote server node) is used. This
feature allows a single IRAF installation to be shared by multiple
servers.
1.8 Default magnitude zero changed
The default APPHOT magnitude zero point has been changed from 26.0 to
25.0 to bring it into agreement with the DAOPHOT package default value
and thereby avoid confusion for users who switch back and forth between
packages. The affected APPHOT tasks are phot, photpars, polypars,
polyphot, qphot, radprof, and wphot. The APPHOTX package in the addon
DIGIPHOTX package will retain the old zero point values until IRAF 2.11
is released after which they will be updated.
The default value of the magzero parameter in imexamine was changed
from 30.0 to 25.0 for consistency with the DIGIPHOT package.
2. IMAGES package changes
The IMAGES package has been reorganized by function into the 7
subpackages listed below.
imcoords - Image coordinates package
imfilter - Image filtering package
imfit - Image fitting package
imgeom - Image geometric transformation package
immatch - Image matching and combining package
imutil - Image utilities package
tv - Image display utilities package
The new IMAGES package contains a total of 82 tasks, including 26 new
tasks from the IMMATCH and VOL external addon packages, 6 existing
PROTO package tasks, and 1 existing NOAO.PROTO package task. The
undocumented IMAGES package IMDEBUG and its 6 tasks have been deleted
from the IMAGES package. User should use the tasks in the ARTDATA
package instead.
This reorganization of the IMAGES package should be mostly transparent
to the user and not affect any existing scripts, unless you were using
any of the 6 deleted tasks. By default, the IMAGES subpackages are
automatically loaded when you log in to the CL. Old parameter files
will not be recognized since the package names have changed.
3. Major system changes
3.1 New FITS image kernel (FXF)
V2.11 introduces a new image kernel providing runtime access to FITS
multi-extension image datafiles. What this means is that IRAF tasks
can now read and write FITS images directly at runtime. The native IRAF
image format (used by images with the .imh extension), remains the
default as it is the most efficient and well-tested format. IMH, FITS,
and the other types of images supported by IRAF can be used
interchangeably in most IRAF tasks. Although we have extensively tested
the new FITS image kernel, it is still evolving, is complex, and still
has some bugs. Users should use it with caution. Please let us know of
any problems.
Besides support for classical FITS images, the new FITS kernel also
supports multi-extension FITS files: several FITS files packed into one
large file with a PHU (Primary Header Unit) that contains global header
information shared by the other files. Multi-extension FITS files are
0-indexed, with the PHU being 0 and the first image extension (or other
data extension) at index 1. If there is no PHU then the first FITS
image is 0 and there is no global header.
For further details about the FITS kernel please see the new FITS Kernel
User's Guide by Nelson Zarate.
3.2 Changes to the IRAF native image format (OIF)
o It was necessary to change the IRAF image format to increase the
maximum path length for header and pixel files. This made it necessary
to change the disk image format, since the old format only allowed 80
characters for the pixel file pathname. The path lengths can now be up
to 255 characters.
Support for two versions of the image and pixel file headers was added.
V2.11 will read both the old image format (V1) and the new image format
(V2). But the new image format is not readable by older versions of IRAF.
o Native format IRAF images (OIF type or extension ".imh") are now machine
independent, for example, a PC and a Sun can now access the same images.
o Support was added for byte swapping pixels. With the machine independent
image header, this allows .imh images to be read on any node (integer)
or any IEEE-compatible node (floating).
o Some pointers: "strings foo.imh" (or other tools like the "less" file
viewer) can be used at the Unix level to look at the text contained in
the new V2 OIF image headers.
3.3 IMFORT changes
o IMFORT was brought up to date to read and write the new V2 ".imh" images.
The old V1 format is still supported but new images are written using
the new machine independent V2 format by default.
o Image headers can now be any size (the old IMFORT had a fixed, relatively
low, limit on the image header size).
o The "min_lenuserarea" variable is now supported by IMFORT (since IMFORT
is host level the variable must be defined in the host environment).
The builtin default header buffer is 64000 chars, which is about 800
card images.
3.4 Environment variables
Several new environment variable have been added to the system.
o The new environment variable imextn determines the image kernels
(image file formats) recognized by IRAF and defines the mapping of
imagefile extensions to these image formats. A file that does not have
an extension listed in imextn may not be recognized as an image by all
IRAF tasks. The default value of imextn is as follows:
imextn = "oif:imh fxf:fits,fit plf:pl qpf:qp stf:hhh,??h"
IRAF tasks will not recognize a file as an image unless it has an
extension (except rfits which will read FITS files on disk that
have no extensions). The rename task can be used to add
extensions to image files if needed. "imextn" can be redefined (use
reset imextn = "new-value") to modify the mapping of extensions to
image types.
The meaning of the fields of the default "imextn" are as follows. Each
substring corresponds to a single kernel. The "xxx:" is the internal
name of the image kernel, i.e. "oif", "fxf", "plf", etc. A comma
delimited list of the extensions, or extension patterns, associated with
that image format follows the colon. For example, for the FITS image
kernel, the internal kernel name is "fxf" and the system default file
extensions are ".fits" and ".fit".
- oif:imh - The original (native) IRAF image format.
- fxf:fits,fit - The FITS image extension format, which supports
classical FITS images as well as multi-extension FITS files.
- plf:pl - The pixel list format used for compressed pixel masks.
- qpf:qp - The QPOE image format for event list data (typically
X-ray or other high energy data).
- stf:hhh,??h - The Space Telescope runtime GEIS image format.
Users can define extensions for the fxf and stf kernels. For example,
if you have FITS files on disk that have a .ft extension then you can
reset imextn so that IRAF will recognize these image extensions:
cl> reset imextn="fxf:ft"
The new .ft extension for the FITS kernel images will now override the
default values - the other kernels remain unchanged. ".fits" will no
longer be recognized as a FITS file unless you include it in the list
of extensions for the "fxf" kernel.
The first extension given for a kernel defines the default file
extension for new images of that type (rather than e.g. the value of
imtype, or a builtin default).
The value of "imextn" is only read once when a process starts up. Hence
it is advisable to do a "flpr" (flush process cache) after changing
this variable, to force all processes to re-read it.
o The environment variable imtype defines the default image type for
new images created by IRAF. If a file extension is given explicitly
when creating a new image then this file extension, in combination with
the "imextn" mappings, determines the type of the new image. Otherwise
the type is determined by the value of "imtype". Typical values are
"imh" for native IRAF images, or "fits" for FITS images. The internal
kernel name documented above for "imextn" can be used instead of a file
extension to ensure that the desired image format is used regardless of
what extensions are assigned to that type by imextn.
The default value of imtype is "oif,noinherit" which means that the
IRAF native format is the default for all new images, regardless of the
type of the input image (i.e. do not "inherit" the input image type).
"inherit" was the default in V2.10 and earlier versions of IRAF.
For example, if you wish to use FITS as the default for new images you
can set imtype=fits as follows:
cl> reset imtype="fits"
cl> flpr % needed before the next task execution
Assuming "imextn" defines ".fits" as a valid file extension for FITS
imagefiles (kernel "fxf"), all new images produced by IRAF will be FITS
files with the extension .fits unless some other extension is given
explicitly when creating a new image.
cl> reset imtype = "imh,inherit"
This example sets the default type for new images to ".imh" for native
format images, but enables image type inheritance. By default new
images will be of the same type as the input image. For example if a
FITS image is being read another FITS image will be output, or if a
pixel mask is being read a pixel mask will be created.
You can override the default output image type specified by imtype by
giving an image extension (as defined by imextn) explicitly in the image
name, e.g. "pix.imh", "pix.fits", "pix.pl" and so on.
o A new environment variable called imclobber has been added.
The default value is set to no. If imclobber is set to yes, images
can and will be overwritten, without warning, when you create new
images.
o The original IRAF image format (OIF) kernel now supports an environment
variable oifversion which, if defined, specifies the file
version for new OIF images (for example, oifversion=2 produces the
new format, or version 2 images). By default the variable is undefined,
the builtin OIF default will be in effect, and new-format images will
be generated. This variable is provided only for backwards
compatibility, e.g., when using V2.11 IRAF with old software which
cannot easily be updated.
3.5 New intrinsic functions
Two new intrinsic functions were added to the CL, imaccess and defvar.
imaccess tests whether an image exists, e.g., (imaccess("dev$pix")) or
(imaccess(foo.fits[3])). defvar tests whether an environment variable
exists, e.g. (defvar("imextn")).
3.6 System default modifications
o The default size of the user area (min_lenuserarea) was increased
to 64000 (about 800 80 character cards). There was some ambiguity about
units for min_lenuserarea; it should be consistently characters now.
o The maximum number of open IRAF logical files was increased from 128 to
256. This is good news for imcombine users.
o Various buffer limits were increased:
- The IRAF line length was increased from 161 to 1023 characters.
One often ran into this lower limit when entering a long list of
input image names, for example.
- CL commands can now be 2047 characters long (the old limit was
1024) - this is particularly useful for scripts.
- IRAF file names can now be up to 255 characters long.
- Expanded file names (pathnames) can be up to 511 characters long.
3.7 Libraries added
The Starlink positional astronomy library SLALIB was added to the math
package.
3.8 Graphics changes
o SGI Translator change: Modified the header ID string produced by
sgi2uapl to be "%!PS", this is required by certain models of printers.
o Installed graphcap support for the STSDAS PostScript graphics kernel.
o The SGI graphics kernel was upgraded with a better roman font, and a
new greek font was added. To use the new high-quality greek fonts use
the "\fG" escape sequence when you use the "T" keystroke to mark text
in a plot, e.g., \fGa Hydra would produce " Hydra". The greek font
may also be used in label parameters for tasks like GRAPH, for example
cl> graph dev$pix xlabel="Wavelength\\fG(A)"
would produce an Angstrom symbol in parenthesis. The double backslash
is necessary to pass the escape string thru the CL. A file containing
the mappings for the greek fonts and other special characters can be
found at http://iraf.noao.edu/greek.ps.
3.9 FITS-related core-level task changes
3.9.1 IMHEADER
The behavior of imheader has changed a bit - typed with no arguments it
will list all the images in the current directory, as in the following
example:
cl> imhead
pix4.imh[512,512][short]: m51 B 600s
boc.fits: FXF: must specify which FITS extension (boc.fits)
fits.fits[512,512][short]: m51 B 600s
pix.fits[512,512][short]: m51 B 600s
pix3.fits[512,512][short]: m51 B 600s
pix5.fits: FXF: must specify which FITS extension (pix5.fits)
zero.fits: FXF: must specify which FITS extension (zero.fits)
mask.pl[512,512][int]: m51 B 600s
foo.qp[1024,811][int]:
The multi-extensions FITS files show up in the list with the message
"FXF: must specify which FITS extension", since these files contain
multiple images and the task does not know which image to open to get
header information.
At present imheader does not use "imextn" to determine what is and is
not an image. The parameter "imlist" defines the valid imagefile
extensions. If imextn is modified "imlist" may need to be modified as
well.
3.9.2 RENAME
The rename task was modified to allow a destination directory to be
specified without changing the filename. A new "field" parameter option
"all" was added and made the default so the entire filename is changed
(or moved in the case of a destination directory). When field is set
to "extn" filenames without an extension will not have the new extension
appended so a filename isn't generated which can get overwritten.
3.9.3 rfits/wfits
Some changes were also implemented in rfits and wfits to add support
for multi-extension FITS files.
o The default action of wfits when writing to tape is unchanged for
single image files.
wfits now has a new parameter called "fextn" and it is set to
"fits". This parameter only affects FITS files written by wfits
to disk - the extension .fits will automatically be added to the
filenames, so that the files will be automatically recognized by
the FITS image kernel.
wfits also has two additional parameters called "extensions" and
"global_hdr" that deal with writing multi-extension FITS files.
o The default action of rfits from tape to disk remains unchanged.
If rfits is used to read FITS files on disk then users need to
be aware of a change to the behavior of the "file-list" parameter.
File-list is now used to define the list of files on the tape as
well as the list of extensions in a multi-extension FITS image.
To read single FITS images on disk use "" as the value for
"file-list". Some users have been using "1" for this value but now
that value will try to read the first extension which doesn't
exist - use "0" if you wish to put something there.
rfits will unpack multi-extension FITS files upon a read. If you
wish to retain the multi-extension FITS format then use T2D and
RENAME.
The help pages have been updated to reflect these changes.
wfits now writes ushort (16 bit unsigned short) images to FITS images
with bitpix=16, bscale=1.0, and bzero=32768.0 by default. rfits will
read these images back as type ushort.
3.10 General changes
o The GSURFIT package has been updated to include support for the "half"
cross terms option useful in computing plate solutions. Tasks that can
make use of this feature have been updated although their default
behaviors have not changed.
o The code which computes the CD matrix from CDELT/CROTA was modified.
The old code computed the diagonal (scale) terms correctly but the
rotation terms were evidently incorrect. The old code was based on
the 1988 Hanisch and Wells WCS paper and the new code is based on a
more recent paper by Mark Calabretta et al, which supercedes the
1988 representation. The affect of this change should be limited
as it only affects rotated images for which CDELT is given but no
CD matrix is defined. (V2.10.4p2)
o Several new celestial coordinate projection functions have been added.
Users with IPAC data that use the CAR projection function should now be
able to read the image coordinates directly with LISTPIXELS, etc.
4. New tasks, or old tasks moved to new packages
Task Name Package Function
CCXYMATCH IMCOORDS Four new tasks for computing and evaluating
CCMAP simple astrometric plate solutions and storing
CCSETWC them in the image headers in fits-compatible
CCTRAN format.
CCFIND IMCOORDS CCFIND locate objects in an image given a
celestial coordinate list and the image wcs.
IMCCTRAN IMCOORDS Two new tasks for transforming celestial
SKYCTRAN coordinate lists and image celestial
coordinate systems from one celestial
coordinate system to another.
STARFIND IMCOORDS STARFIND automatically detects stellar objects
in a list of images.
WCSCTRAN IMCOORDS A new task for transforming between IRAF image
coordinate systems.
WCSEDIT IMCOORDS Two unaltered former PROTO package tasks for
WCSRESET editing IRAF image coordinate systems.
FRMEDIAN IMFILTER Four new tasks for doing circular/elliptical/
FRMODE ring image median filtering.
RMEDIAN
RMODE
IM3DTRAN IMGEOM The former addon VOL package task IM3DTRAN for
doing 3D image transposes.
GEOXYTRAN IMMATCH A new task for transforming coordinate lists
using the results of the GEOMAP task.
IMCENTROID IMMATCH Four new tasks for computing matched pixel
SKYXYMATCH lists. IMCENTROID is a modified version of the
WCSXYMATCH PROTO package task of the same name.
XYXYMATCH
SKYMAP IMMATCH Two new tasks for computing geometric
WCSMAP transforms using the image coordinate system
information.
IMALIGN IMMATCH Three new tasks for doing automated image
SREGISTER registration. IMALIGN is a modified version
WREGISTER of the PROTO package task of the same name.
WCSCOPY IMMATCH A new task for copying the coordinate system
of a reference image to a set of input images.
PSFMATCH IMMATCH A new task for matching the PSFs of a set of
input images to the PSF of a reference image
using Fourier techniques.
LINMATCH IMMATCH A new task for matching the linear intensity a
scale of a set of input images to the
intensity scale of a reference image.
IMFUNCTION IMUTIL The former PROTO package task for applying a
single argument function to an image.
IMJOIN IMUTIL The former addon VOL package task for joining
same-dimensioned images along a specified
dimension.
IMREPLACE IMUTIL The former PROTO package task IMREPLACE for
replacing bad pixels based on their value.
IMTILE IMUTIL A modified version of the NOAO.PROTO IRMOSAIC
task for tiling same sized 2D images into a
single mosaiced image.
EXPORT DATAIO Two new tasks from the external IMCNV package
IMPORT for exporting IRAF images to binary formats
and for importing binary files into IRAF
images.
TEXT2MASK PROTO This new task appears in conjunction with a
new pixel mask based version of FIXPIX.
IMEXTENSIONS PROTO This task selects and lists image extensions
in files. Image extensions currently occur
in multi-extension FITS files and multi-group
Geiss (STF format) files.
CCDMASK CCDRED This new task creates a pixel mask from a
CCD image.
AIDPAR ONEDSPEC New parameter set for automatic line
identification for the tasks AUTOIDENTIFY,
IDENTIFY and REIDENTIFY.
AUTOIDENTIFY ONEDSPEC A new task to automatically identify lines
and fit the dispersion function.
SKYTWEAK ONEDSPEC Sky spectra are shifted and scaled to best
subtract sky features from data spectra.
TELLURIC ONEDSPEC Telluric calibration spectra are shifted and
scaled to best divide out telluric features
from data spectra.
ASTCALC ASTUTIL An astronomical calculator.
ASTRADIUS ASTUTIL Finds images within a circle on the sky.
5. Task and package deletions
CUBE, DUMP, GSUBRAS, MAXMIN, MKIMAGE, MKTEST: These tasks have been
superseded by the equivalent tasks in the IMAGES or ARTDATA packages.
The functionality of these tasks still exists in the
iraf$pkg/images/lib/zzdebug.x file.
REGISTER: This task has been renamed to GREGISTER.
IMALIGN, IMCENTROID, IMFUNCTION, IMREPLACE, WCSEDIT, WCSRESET: Moved to
the IMAGES package.
6. Modifications to old tasks
Grouped by package, a list of modifications to old tasks in IRAF are
summarized below. We have included modifications since the V2.10.3
release.
IMFILTER:
FMEDIAN, FMODE, MEDIAN, MODE
Minimum and maximum good data value parameters zloreject and zhireject
and a verbose option parameter were added.
MEDIAN, MODE
The 64 x 64 maximum kernel size limit was removed from these tasks.
IMMATCH:
GEOMAP
Renamed the output parameter to database for the sake of consistency
with other new images tasks.
Changed the default value of the reject parameter from 0.0 to INDEF.
Added the transforms parameter. Transforms permits the user to specify
the names of the output database records explicitly.
Added the parameter results. Results permits the user to save a summary
of the results including a description of the transform geometry, and a
listing of the input coordinates, the fitted coordinates, and the fit
residuals.
Added the fitgeometry parameter. Fitgeometry permits the user to
constrain the linear part of the fit to: 1) x and y shifts only, 2) x
and y shifts and rotation only, 3) x and y shifts and x and y scale
changes only, 4) x and y shifts, rotation, and a scale change only, 5)
x and y shifts, rotation, x and y scale changes only, and 5) x and
shifts, rotation, skew, and x and y scale changes.
GREGISTER
Renamed the register task gregister to emphasize that it is paired with
the geomap task and to avoid confusion with the new registration tasks.
GEOTRAN, GREGISTER
Renamed the transform parameter to transforms.
Added the verbose parameter.
GEOTRAN
Added the ability to write to a section of an existing image.
IMCOMBINE
The limit of the number of images that may be combined has been
removed. If the number of images exceeds the maximum number of
open images permitted then the images are stacked in a single
temporary image and then combined with the project option.
Note that this will double the amount of diskspace
temporarily. There is also a limitation in this case that the
bad pixel mask from the first image in the list will be applied
to all the images.
Integer offsets may be determined from the image world
coordinate system.
A combination of ushort and short images now defaults to
integer.
Added support for type ushort images.
Added a new options for computing offsets using the image wcs.
Removed the limit on the maximum number of images that can be combined.
IMALIGN, IMCENTROID
Renamed the images parameter to input, changed the reference parameter
mode from hidden to automatic, and reversed the order of the reference
and coords parameters.
IMUTILS:
IMEXPR
Modified the task so that it will accept an image name that looks like
a number in the first few characters, but which is really an image
name. For example, "123.imh" or "../foo.imh". The previous version
of imexpr was treating any string which looked like a number in the
first few characters as a numeric constant. (V2.10.4p2)
IMREPLACE
The lower value is now rounded up for integer images so that a
range like 5.1-9.9 affects pixels 6-9 instead of 5-9.
IMSUM
Now allows "ushort" data types.
TV:
DISPLAY
The bad pixel mask, overlay mask, sample mask, and overlay
colors parameters and functionality have been added. The
"nsample_lines" parameter is now an "nsample" parameter.
Bugs in the coordinate system sent to the image display for
cursor readback were fixed.
IMEDIT
If xorder or yorder are zero then a median background is
computed for the 'a' and 'b' keys.
IMEXAMINE
('a' and 'r'): The fit to the radial profile points now
includes both a Gaussian and a Moffat profile. The Moffat
profile exponent parameter, beta, may be fixed or left free to
be fit.
('a' and 'r'): New estimates fo the FWHM were added to the 'a'
and 'r' keys. These include the Moffat profile fit noted
above, a direct measurement of the FWHM from the radially
binned profile, and a Gaussian or Moffat fit to the radial
enclosed flux profile. The output from these keys was modified
to include the new information.
('a' and 'r'): The direct FWHM may be used to iteratively
adjust the fitting radius to lessen the dependence on the
initial fitting radius value.
('k'): Added a kimexam parameter set.
The default value of rimexam.magzero parameter was changed from
30.0 to 25.0 for consistency with the digiphot package.
PROTO:
FIELDS
The default value for the lines parameter was changed to an open
upper limit.
Changed the default value of the lines parameter from "1-999" to
"1-" so that the upper bound would be open ended.
FIXPIX
This task replaces the old task (now obsolete.ofixpix) and
works with the more general pixel mask facilities. It also
provides greater flexibility in chosing the interpolation
direction.
ICFIT used in various tasks:
ICFIT
The :xyshow output was modified to 1) not include colon labels,
2) print (X, Y, Y fit, Weight) instead of (X, Y fit, Y), and 3)
the printed values are those actually used in the fit when using
composite points (naverage not 1).
ARTDATA:
MK1DSPEC
Lorentzian and Voigt profiles were added and the parameters and
input line list format were changed. The widths are now FWHM
instead of gaussian sigmas.
MKOBJECTS, MKNOISE
The default value of "ranbuf" was changed to zero.
GALLIST
The default value for the minimum elliptical galaxy axial ratio
was changed to 0.3 to cover the range E0-E7 uniformly.
MKPATTERN
Now allows ndim=0 to make an dataless header.
Now allows ushort pixel type.
ASTUTIL:
SETJD
The checking of the epoch keyword value was improved.
Previously if there was a problem with the keyword value
(missing or malformed) the task would use the epoch of the
observation. Now it is an error if an epoch keyword is
specified but the epoch value can't be determined. Also a
leading 'B' or 'J' is allowed and a warning will be given if
the epoch value is unlikely.
ASTHEDIT
There are new astronomical functions and input/output functions.
The command syntax may now use "=" as a delimiter as well as
the whitespace.
A new parameter "update" allows protecting images and accessing
read-only images for the purpose of calculating and printing
quantities.
The special variable name "$I" has the value of the image name,
$D the current date, and $T the current time.
The case of no image name creates and deletes a temporary image
so the task can be used purely as a calculator (but see
astcalc).
CCDRED:
CCDPROC
The bad pixel fixing was modified to allow use of pixel masks,
images, or the text file description. Bad pixel masks are the
desired description and use of text files is only supported for
backward compatibility. Note that support for the trimmed or
untrimmed conversion from text files has been eliminated.
Line-by-line overscan/prescan subtraction is now provided with
three simple algorithms.
COMBINE
The limit of the number of images that may be combined has been
removed. If the number of images exceeds the maximum number of
open images permitted then the images are stacked in a single
temporary image and then combined with the project option.
Note that this will double the amount of diskspace
temporarily. There is also a limitation in this case that the
bad pixel mask from the first image in the list will be applied
to all the images.
Integer offsets may be determined from the image world
coordinate system.
Fixed a bug where a variable was improperly used for two different
purposes causing the algorithm to fail. This also affected IMCOMBINE
and SCOMBINE. See bug 316 for details. (V2.10.4p2)
COSMICRAYS
The output bad pixel data accidentally included some extra fields
making it incorrect to use the file directly with BADPIXIMAGE.
The extra diagnostic fields were removed. For details, see bug 311
in the buglog. (V2.10.4p2)
ECHELLE:
ECIDENTIFY
The dispersion units are now determined from a user parameter,
the coordinate list, or the database entry.
IMRED Spectral Packages:
DOARGUS, DOFIBERS, DOHYDRA
A sky alignment option was added.
The aperture identification can new be taken from image header
keywords.
The initial arc line identifications is done with the automatic
line identification algorithm.
DOSLIT, DO3FIBER
The initial arc line identifications is done with the automatic
line identification algorithm.
ONEDSPEC:
Support for the Sloan Sky Survey was added by allowing multifiber
reductions with up to 500 fibers with non-linear dispersions. (V2.10.4p2)
BPLOT
Fixed a general problem in BPLOT and SLIST that caused a segmentation
violation error. See buglog 312 for details. (V2.10.4p2)
FITPROFS
Modified to include lorentzian and voigt profiles. The
parameters and positions file format have changed in this
version. A new parameter controls the number of Monte-Carlo
samples used in the error estimates.
IDENTIFY
The dispersion units are now determined from a user parameter,
the coordinate list, or the database entry.
A new key, 'e', has been added to add features from a line list
without doing any fits. This is like the 'l' but without the
automatic fitting before and after adding new features.
A new key, 'b', has been added to apply an automatic line
identification algorithm.
The 'x' key has been changed to use the automatic line
identification algorithm. The allows finding much larger
shifts.
The match parameter may now be specified either in user
coordinates or in pixels. The default is now 3 pixels.
The default threshold value has been changed to 0.
REIDENTIFY
A new parameter, "search", was added to specify a search radius
for the automatic line identification algorithm.
The "nlost" parameter now also applies when not tracing; i.e. it
will issue a warning and not record a solution if the specified
number of features is lost when reidentifying against a specific
reference spectrum as is done with multispec data.
The task will now work with only a warning if the reference
image is absent; i.e. it is possible to reidentify given only
the database.
The addfeatures function will now add features before a fit if
there are no reference database features. Previously features
could only be added after an initial fit using the reference
features and, so, required the reference database to contain
features for reidentification. This new feature is useful if
one wants to uses a dispersion function from one type of
calibration but wants to add features for a different kind of
calibration.
SAPERTURES
This task has been modified to allow use of image header
keywords as done in the APEXTRACT package.
SARITH
Previously both w1 and w2 had to be specified to select a range
to be used. Now if only one is specified the second endpoint
defaults to the first or last pixel.
The noise band in multispec data is only copied from the primary
spectrum and not modified. This is a kludge until the noise is
handled properly.
Fixed a problem in SARITH and SCOPY where a segmentation error
occurred when a wavelength range was specified in the reverse sense
of the data and without rebinning. See buglog 323 for details.
(V2.10.4p2)
SBANDS
Fixed a problem in SBANDS that caused a segmentation violation when
the number of input bandpasses was greater than 10. See buglog 298
for details. (V2.10.4p2)
SCOPY
Previously both w1 and w2 had to be specified to select a range
to copy. Now if only one is specified the second endpoint
defaults to the first or last pixel.
SPECPLOT
The scale and offset parameters may now be a value, a filename,
or and image header keyword.
The 'f' key was added to toggle between world and logical pixel
coordinates.
SPLOT
The profile fitting and deblending was expanded to include
lorentzian and voigt profiles. A new parameter controls the
number of Monte-Carlo samples used in the error estimates.
RSPECTEXT
The task now automatically senses the presence of a header.
APEXTRACT:
APALL, APSUM, APNOISE, APNORMALIZE, APSCATTER, APTRACE,
APRECENTER, APRESIZE, APMASK, APFIND, APFLATTEN
The "apertures" parameter can be used to select apertures for
resizing, recentering, tracing, and extraction. This parameter
name was previously used for selecting apertures in the
recentering algorithm. The new parameter name for this is now
"aprecenter".
APALL, APSUM
The "nsubaps" parameter now allows onedspec and echelle output
formats. The echelle format is appropriate for treating each
subaperture as a full echelle extraction.
APALL
The aperture ID table information may now be contained in the
image header under the keywords SLFIBnnn.
APSUM
The dispersion axis parameter was moved to purely a package
parameter.
As a final step when computing a weighted/cleaned spectrum the
total fluxes from the weighted spectrum and the simple
unweighted spectrum (excluding any deviant and saturated
pixels) are computed and a "bias" factor of the ratio of the
two fluxes is multiplied into the weighted spectrum and the
sigma estimate. This makes the total fluxes the same. In this
version the bias factor is recorded in the logfile if one is
kept. Also a check is made for unusual bias factors. If the
two fluxes disagree by more than a factor of two a warning is
given on the standard output and the logfile with the individual
total fluxes as well as the bias factor. If the bias factor is
negative a warning is also given and no bias factor is applied.
In the previous version a negative (inverted) spectrum would
result.
RV:
RVIDLINES, RVREIDLINES
These tasks now work in the units of the input spectra.
FXCOR
The input spectra are converted to Angstroms for the
crosscorrelation and analysis. Thus the velocities will
be correctly computed regardless of the units of the
input spectra.
DAOPHOT:
DAOFIND
A major bug in the DAOFIND task centering code that affects the
computed x and y positions was fixed. Users should refer to bug
log entry number 332 for details. (V2.10.4p2)
A new roundness statistic was added to the DAOFIND output to
bring the task into conformance with standalone DAOPHOT II. The new
statistic is sensitive to and tries to eliminate detected objects
which are significantly elongated in directions other than 0, 90,
180, and 270 degrees. The original roundness statistic is stored in
the ground column, the new one in the sround column. The same
default roundness limits apply to both statistics. (V2.10.4p2)
DAOPARS
Added a new parameter "mergerad" to the DAOPARS parameter set.
Mergerad permits the user to control the merging process. If
mergerad is INDEF (the default setting) the default merging radius
is used. If mergerad is 0 object merging is turned off altogether.
Otherwise the user can set the merging radius to a specific value.
Mergerad is used by the nstar and allstar tasks, but their default
behavior is unchanged. (V2.10.4p2)
Changed the name of the "critovlap" parameter to "critsnratio" to
avoid confusion with the the meaning of the parameter especially
with regard the mergerad parameter. The behavior of the parameter is
unchanged. (V2.10.4p2)
7. Parameter file changes
The parameter file changes below are for modifications between V2.10.4
and V2.11. For a list of parameter file changes between V2.10.3 and
V2.10.4 see the file iraf/v210/params.v2104.Z in the IRAF FTP archives.
In the tables below each parameter change is identified with one of the
following codes followed by task_name.parameter_name and the
description of the change.
n = new parameter
c = changed/modified parameter
d = deleted parameter
TV:
n display.nsample: replaces nsample_lines
d display.nsample_lines: replaced by nsample
n display.bpmask: specify a bad pixel mask
n display.bpdisplay: specify display mode for bad pixel mask
n display.bpcolors: specify overlay colors for bad pixel mask
n display.overlay: specify an overlay mask
n display.ocolors: specify overlay colors for overlay mask
n display.zmask: specify a sample mask for the zscale calculation
c imedit.xorder: now allows a value of zero for median background
c imedit.yorder: now allows a value of zero for median background
n rimexam.fittype: specify a profile type to fit - default is now moffat
n rimexam.iterations: specify number of iterations to adjust fitting radius
n rimexam.beta: specify beta value for moffat fitting
c rimexam.buffer: default value changed from 1 to 5
c rimexam.width: default value changed from 2 to 5
c rimexam.magzero: default value changed from 30 to 25
n wcslab.overplot: specify overplotting
DATAIO:
n wfits.fextn: extension to append to output disk FITS filename - default is
fits
n wfits.extensions: write all images to a single FITS file ? - default is no
n wfits.global_hdr: prepend a global header to the FITS extensions - default
is yes
IMAGES:
n fmedian.zloreject: good data minimum
n fmedian.zhireject: good data maximum
n fmedian.verbose: verbose option
n fmode.zloreject: good data minimum
n fmode.zhireject: good data maximum
n fmode.verbose: verbose option
n median.zloreject: good data minimum
n median.zhireject: good data maximum
n median.verbose: verbose option
n mode.zloreject: good data minimum
n mode.zhireject: good data maximum
n mode.verbose: verbose option
n geomap.transforms: list of record names
n geomap.results: results summary file
n geomap.fitgeometry: fitting geometry
d geomap.output: renamed to database
c geomap.reject: changed from 0.0 to INDEF
n [g]register.verbose: verbose option
d [g]register.transform: renamed to transfo
n geotran.verbose: verbose option
d geotran.transform: renamed to transforms
c imcombine.offsets: now allows specifying "wcs" to compute offsets from WCS
d imalign.images: renamed to input
c imalign.reference: went from hidden to auto
c imalign.coords: reversed places with reference
d imcentroid.images: renamed to input
c imcentroid.reference: went from hidden to auto
c imcentroid.coords: reversed places with reference
n imheader.imlist: default image names
PLOT:
n graph.ltypes: specify the line types
n graph.colors: specify the colors
PROTO:
n fixpix.masks: new version specifies bad pixel masks
n fixpix.linterp: specify mask values for line interpolation
n fixpix.cinterp: specify mask values for column interpolation
n fixpix.pixels: list pixels that are modified
d fixpix.badpixels: new version does not use bad pixel region description
c fields.lines: default value changed from "1-9999" to "1-"
ARTDATA:
n mk1dspec.profile: define the profile type
n mk1dspec.gfwhm: replaces sigma for gaussian profile width
n mk1dspec.lfwhm: width for lorentzian profile
c artdata.randbuf: default value changed from 1000 to 0.
c mkpattern.ndim: allows a value of 0 for a dataless header.
c mkpattern.pixtype: allows ushort.
c gallist.ar: default value changed to 0.3.
d mk1dspec.sigma: replaced by gfwhm
ASTUTIL:
n rvcorrect.keywpars: added to define keywords used
n asthedit.prompt: used for new calculator option
n asthedit.update: update the header
n asthedit.oldstyle: to allow backward compatibility
APEXTRACT, IMRED spectral packages:
n apnoise.apertures: select apertures to be used
n apfit.apertures: select apertures to be used
n apedit.apertures: select apertures to be used
n apfind.apertures: select apertures to be used
n apnormalize.apertures: select apertures to be used
n apscatter.apertures: select apertures to be used
n apsum.apertures: select apertures to be used
n aptrace.apertures: select apertures to be used
n apresize.apertures: select apertures to be used
n apmask.apertures: select apertures to be used
n apflatten.apertures: select apertures to be used
n aprecenter.apertures: select apertures to be used
n aprecenter.aprecenter: was what was previously the apertures parameter
n apall.apertures: select apertures to be used
n apall.aprecenter: was what was previously the apertures parameter
ARGUS, HYDRA, SPECRED:
n doargus.crval: for automatic line identifications
n doargus.crdelt: for automatic line identifications
n doargus.skyalign: night sky alignment option
n dohydra.crval: for automatic line identifications
n dohydra.crdelt: for automatic line identifications
n dohydra.skyalign: night sky alignment option
n dofibers.crval: for automatic line identifications
n dofibers.crdelt: for automatic line identifications
n dofibers.skyalign: night sky alignment option
n do3fiber.crval: for automatic line identifications
n do3fiber.crdelt: for automatic line identifications
ARGUS, HYDRA, KPNOCOUDE, KPNOSLIT, SPECRED:
c params.match: default value changed to -3 (3 pixels instead of Angstroms)
c sparams.match: default value changed to to -3 (3 pixels instead of Angs)
ONEDSPEC, IMRED spectral packages:
d fitprofs.sigma: replaced by gfwhm
d fitprofs.function: replaced by profile
d fitprofs.fitsigmas: replaced by fitgfwhm
d rspectext.header: removed since the task now senses the header information
ONEDSPEC, LONGSLIT, IMRED spectral packages:
n identify.units: specify the desired units for the dispersion function
n identify.crval: for automatic line identifications
n identify.crdelt: for automatic line identifications
n identify.aidpars: parameter set for automatic line identifications
c identify.match: default value changed to -3 (3 pixels instead of Angstroms)
c identify.threshold: default value changed from 10 to 0.
c reidentify.match: default value changed to -3 (3 pixels instead of Angs)
c reidentify.threshold: default value changed from 10 to 0.
n reidentify.crval: for automatic line identifications
n reidentify.crdelt: for automatic line identifications
n reidentify.aidpars: parameter set for automatic line identifications
n reidentify.search: specify a search radius for the automatic line
identification algorithm
n ecidentify.units: specify the desired units for the dispersion function
n fitprofs.profile: define the profile type
n fitprofs.gfwhm: replaces sigma for gaussian profile width
n fitprofs.lfwhm: width for lorentzian profile
n fitprofs.fitgfwhm: replaces fitsigmas
n fitprofs.fitlfwhm: select whether to fit lorentzian profile widths
n fitprofs.nerrsample: allows control of the error calculation accuracy
n splot.nerrsample: allows control of the error calculation accuracy
CCDRED:
c ccdproc.fixfile: this now specifies a bad pixel mask
c combine.offsets: now allows specifying "wcs" to compute from WCS
RV:
n rvcorrect.par: Added the KEYWPARS pset declaration
DAOPHOT:
c daopars.critsnratio: critical S/N ratio for group membership - changed
the name only from critovlap (V2.10.4p2)
n daopars.mergerad: critical object merging radius in scale units
(V2.10.4p2)
IRAF (Jul92) V2.10 Revisions Summary IRAF (Jul92)
IRAF Version 2.10 Revisions Summary
July 1992
1. Introduction
This document summarizes the changes made to IRAF in version 2.10.
V2.10 is a major release of IRAF, meaning that there were significant
changes to both the system and applications software, and the release
will eventually be made available on all supported platforms.
By IRAF version 2.10 we refer only to the core IRAF system and NOAO
packages. Numerous external or "layered" packages are also available
for IRAF, for example the NSO package (solar astronomy), the ICE
package (data acquisition), the STSDAS package from STScI (HST data
reduction and analysis), the TABLES package from STScI (tabular data),
the XRAY package from SAO (X-ray data analysis), and so on. These
packages are layered upon the IRAF core system, and once installed, are
indistinguishable from the rest of IRAF. Layered packages are
developed and maintained separately from the core IRAF release and
follow independent release schedules, hence we will not discuss them
further here. Contact the IRAF project or any of the outside groups
supporting IRAF layered packages for additional information on what is
available.
This document is intended only as an overview of what is new in version
2.10 IRAF. More detailed documentation will be found in the systems and
applications notes files (files sysnotes.v210.Z and pkgnotes.v210.Z in
the network archive), in the online help pages, and in any reference
papers or user's manuals distributed with the software. The IRAF
Newsletter is a good source of information for new IRAF software.
This revisions summary is organized as follows:
1. Introduction
2. IRAF System Revisions
3. IRAF and NOAO Package Revisions
3.1. Changes to the System Packages
3.2. Changes to the NOAO Packages
4. Programming Environment Revisions
4.1. Compatibility Issues
4.2. Portability Issues
4.3. Software Tools
4.4. Programming Interface Changes
5. Getting Copies of this Revisions Summary
The reader is assumed to already be familiar with the basic concepts and
operation of the IRAF system. In particular, familiarity with V2.9
IRAF is assumed.
2. IRAF System Revisions
2.1 Starting up V2.10 IRAF
Before attempting to start V2.10 IRAF each user should run the
mkiraf command in their IRAF login directory. This will create a new
login.cl file and uparm (user parameter) directory. mkiraf should be
allowed to delete any existing parameter files, as there have been many
changes to task parameter sets in the new version of IRAF.
2.1.1 LOGIN.CL changes
The login.cl file is read by the CL during startup to perform some
runtime initialization and to customize IRAF for each user. A standard
login.cl file is created and initialized by the mkiraf command when the
user's IRAF login directory is configured. For V2.10 IRAF the login.cl
file has undergone the following changes.
o The IRAF version number is now checked automatically whenever
you login, and a warning message will be printed if your
login.cl file is out of date. If you see this message it means
that important changes have been made to the login.cl file and
you need to rerun mkiraf to update this file.
o Most of core IRAF system packages are now loaded automatically
at login time by the login.cl file. If you use a personal
loginuser.cl file and you previously loaded any core system
packages in this file, you should edit the file and remove
those entries.
o A "quiet" login option is now provided. If a file named
.hushiraf exists in the login directory when you start up the
CL, printing of the usual login messages will be disabled (the
only output seen will be the "cl>" prompt).
o On UNIX/IRAF systems the login script now looks at the host
system environment variable TERM and checks for the common
terminal types "sun" and "xterm", configuring the IRAF stty
accordingly if either terminal type is seen (note that the
default number of lines set for an xterm terminal window may
need to be modified). The logic used to do this is not
foolproof however, and is provided only as an example
illustrating how to make use of the host system terminal
definitions. You may need to customize this portion of the
script, or override it in your loginuser.cl file.
o The CL hidden parameter showtype is now set to "yes" by default.
This will cause a character to be printed after the name of
each package or named pset in CL package menus to allow these
objects to be easily distinguished from ordinary tasks.
Packages are marked with a trailing period (".") and psets with
an ampersand ("@"). This feature can be disabled with a
"showtype=no" statement.
o The V2.10 login script contains a call to a new internal
(non-user) IRAF task mtclean. Be sure to leave this alone, it
is required for the correct operation of the new magtape i/o
system.
The USER package defined in the template login.cl has been
extensively revised, adding many new tasks. These are mainly used
to make common UNIX commands available from within the IRAF
environment. Type "?user" in the CL to see what is available, e.g.:
cl> ?user
adb cp fc lpq mv rlogin spell top
bc csh find ls nbugs rsh sps touch
buglog date finger mail nm rtar strings vi
cal df ftp make od ruptime su w
cat diff generic man ps rusers sync wc
cls du grep mkpkg pwd rwho telnet wtar
comm emacs less mon rcp sh tip xc
Note that since the USER package is defined in the user's login
file it is easily customized to add new tasks. Refer to the
existing package for examples illustrating how to do this.
2.1.2 Compatibility Issues
Version 2.10 IRAF requires the new login.cl file; if the CL does not
start up correctly, it may be because the user has not done a mkiraf,
or because they have some construct in their loginuser.cl file which is
incompatible with V2.10 IRAF. The V2.10 login file is usable with V2.9
IRAF, however this is not recommended.
There have been many task parameter changes between V2.9 and V2.10. If
"parameter not found" messages are seen, most likely the user has an old
uparm directory, or has been switching back and forth between IRAF
versions. An unlearn or mkiraf should fix the problem.
The V2.10 IRAF networking system is not fully compatible with earlier
versions of IRAF. This can cause problems when, e.g., a newly installed
V2.10 system is used to communicate with an older version of IRAF on
another system. The best solution is to update to V2.10 on all
systems, but if this is not convenient it is possible to configure the
networking system to avoid the problems. See the discussion of the new
networking system given below.
Accessing a remote magtape device via IRAF networking will not work
between V2.10 and older versions of IRAF (the remote procedure calls
have changed). To remotely access magtape devices you will need to
install V2.10 IRAF on both the client and server nodes.
In most respects installing V2.10 IRAF will be very similar to
installing earlier versions of IRAF. The main difference is the
tapecap file required to use the new magtape system. The old
dev$devices file is no longer used. See the discussion of the new
magtape system given below for more details.
Due to name changes in certain low level system routines (made to avoid
name clashes when linking with host level libraries) the V2.10
libraries are incompatible with older versions of IRAF. Any IRAF
programs or external packages relinked under V2.10 will have to be
fully recompiled or the linker will complain about unresolved
externals. Note that so long as the old program is not relinked there
should be no problem, even if the program uses the IRAF shared image,
since the V2.9 shared image is included in V2.10 (this applies to
Sun/IRAF systems only).
Starting with V2.10, many IRAF applications now fully support
generalized world coordinates (WCS). While in principle this should
not pose any compatibility problems, the image headers do contain more
information in V2.10 than previously, and there can be problems if, for
example, an input image contains an illegal WCS. Previous versions of
IRAF would ignore this but in V2.10 such an image could result in an
error or warning message. If WCS related problems are encountered it
is probably best to contact the IRAF group for help.
2.2 CL Enhancements
2.2.1 Formatted scans and prints, scan from a pipe
New in the V2.10 CL (command language) are formatted scan and print
routines, and the ability to scan from a pipe or other form of
redirected input. These new facilities will prove most useful in CL
scripts.
The old unformatted scan and print routines are the following. These
are still available and are the simplest routines to use where they are
adequate.
scan (arglist) # scan standard input
fscan (list, arglist) # scan a list
print (expr, exprlist) # print to standard output
fprint (param, exprlist) # print to a string buffer
For example,
list = "filename"
while (fscan (list, x, y) != EOF)
print ("x=", x, "y=", y)
In the above, arglist is a comma delimited list of output arguments
(parameter or parameter field names) and exprlist is a comma delimited
list of expressions to be printed. list is the name of a
list-structured parameter to be scanned, and param is the name of a
parameter, the value field of which is to receive the output string.
The unformatted scan routines will automatically convert output data
values to match the types of the output arguments.
The new formatted routines are as follows. These take an extra format
argument which tells how to parse the input string in the case of the
scanf routines, or how to format the output in the case of the printf
routines.
scanf (format, arglist) # formatted scan from stdin
fscanf (list, format, arglist) # formatted scan from a list
printf (format, exprlist) # formatted print to standard output
Currently there is no fprintf routine. For the printf routine the
format argument is similar to that for the SPP/VOS printf (which is
similar to the C printf). The format argument for the scanf routines
is the same as the VOS LIBC scanf, which is patterned after the C scanf
(in fact the UNIX manual page for scanf can be used as a guide to the
CL scanf with only minor deviations).
The following examples illustrate the new routines.
cl> printf ("%d foo %7.3f\n", 5, 12.123) | scanf ("%d foo %g", i, x)
cl> printf ("new values are i=%d, x=%g\n", i, x)
new values are i=5, x=12.123
or,
while (fscanf (list, " %*d size=%d name=%s", i, s1) != EOF)
printf ("size=%05o, name=`%s'\n", i, s1)
Note in the first example the use of scanf to scan from a pipe. There
are actually two different versions of scan and scanf in V2.10 IRAF, an
intrinsic function version and a procedure version. When called as an
intrinsic function, a scan routine returns as its function value the
number of operands successfully scanned, or EOF. When called as a
procedure, the function value of a scan routine is discarded.
Here is another example illustrating scan from a pipe, in this case
using an unformatted scan since the hselect output is in a simple
tabular format (hselect prints selected fields of the image header).
cl> hselect dev$pix naxis,naxis1,naxis2 yes | scan (i, j, k)
cl> printf ("naxis=%d, axlen=[%d,%d]\n", i, j, k)
naxis=2, axlen=[512,512]
When using the formatted scan routines, care must be taken to ensure
that the data types implied by the format argument match the actual data
types of the output parameters. The scanf routines are implemented
using an internal call to the C (LIBC) scanf, with the output parameter
value fields referenced directly via a pointer. If the data type is
incorrect the output value may be meaningless.
2.2.2 Unlearning package parameters
The unlearn task now works for package parameters as well as task
parameters. In a command such as "unlearn pkgname" the package
parameters for the named package will be unlearned, as well as the
parameters for all the tasks in the package. This works whether or not
the package is loaded.
2.2.3 Loading packages at login time
A bug has been fixed which affected packages with package
parameters loaded at login time. It is now permissible to load any
package at login time regardless of whether it has package parameters
(V2.9 users will recognize this bug as one which prevented loading
CCDRED in the login script).
2.2.4 Environment variables
The environment variables imtype, cmbuflen, and min_lenuserarea are
now defined at login time. Previously, explicit values for these
variables were not defined, and the system would use the builtin
internal defaults. Explicit definitions were added so that the user
can query the current value, e.g.
cl> show cmbuflen
128000
A show or set with no arguments will print the full environment list.
New values for these and other common environment variables may be set
in the user login.cl file.
2.3 System Management Related Changes
2.3.1 Install script
The UNIX/IRAF install script underwent minor changes to make it more
robust. Problems are still possible if the IRAF root pathname is set to
different values in the various system dependent files modified by the
script. The system as shipped from NOAO has the same initial root
pathname set in all such files, but problems can occur if the files are
manually edited during or after installation. To avoid problems always
use the install script to make system changes such as installing at a
different root directory.
2.3.2 Caching of termcap entries
User caching of termcap or graphcap entries with the old mkttydata
task is no longer recommended. The most common entries (e.g. sun,
xterm, vt100) are already cached. Modern workstations are so fast that
there is no longer much point in caching termcap entries; it is
sufficient to merely place local additions near the top of the file.
Most programs that repeatedly access the terminal cache the entries
internally during execution. Custom caching of termcap or graphcap
device entries requires that the system be relinked, and the risk
inherent in relinking the system (hence giving up the prebuilt,
pretested binaries) is not worth the small performance gain achieved.
2.3.3 Sorting of UNIX directories
The UNIX-based versions of IRAF now sort UNIX directories whenever a
directory is accessed to expand a file or image template. This will
fix the problem sometimes seen in earlier versions of IRAF, in which an
image template could appear to be expanded in a seemingly random
fashion.
2.3.4 UMASK support
The UNIX-based versions of IRAF now support the host level umask
file creation mask correctly. If files or directories created by V2.10
IRAF do not have the desired permissions, it is because you do not have
umask set correctly at the UNIX level (most people set umask to 022).
2.4 Networking Enhancements
2.4.1 New networking driver
The UNIX/IRAF networking driver has been completely rewritten for
version 2.10 IRAF, with the goals of eliminating redundant password
prompts, improving efficiency, and enhancing system security. For the
most part the changes will be transparent to the user. Once the IRAF
system manager has configured the dev$hosts file for the local site the
networking system should function automatically; in the default
configuration a password prompt should be seen only when connecting to
a server for which rhosts ("trusted" hosts) permission is not granted.
The following information is provided mainly for IRAF system managers.
In normal use the user does not need to understand how the networking
system functions.
2.4.1.1 How it works
The IRAF networking system is an RPC (remote procedure call)
mechanism for the IRAF kernel; all kernel procedures may execute either
locally or remotely, and the client and server nodes do not even need
to run the same operating system. IRAF applications may be
distributed, and may access resources which reside anywhere on the
network. IRAF networking is layered upon standard low level networking
protocols such as TCP/IP and DECNET.
The IRAF networking system defines one or more connection protocols
which are used by a client to connect to the IRAF kernel server on a
remote machine. The old networking driver supported only one
connection protocol, the rexec protocol, which requires a login name
and password. The new driver adds support for an rsh based protocol.
This is the default connection protocol for V2.10 IRAF; automatic
fallback to the rexec protocol is provided in the event that the rsh
connect fails. The rsh connection protocol bootstraps off the
suid-root rsh command found in most BSD derived UNIX systems (most
System V systems provide the equivalent command remsh).
The connection protocol is used to start the in.irafksd IRAF networking
daemon on the remote server node. This daemon executes with the same
uid and permissions as the account which initiated the connection, and
there is one such daemon per user per server node. Once the daemon has
been started via the rsh or rexec connection protocol, new client
connections are made very quickly, by merely forking the daemon to
create the IRAF kernel server process, and setting up a direct socket
connection between the IRAF client process and the server. The daemon
process runs indefinitely, shutting down if idle for longer than a
certain interval (the current default is one hour). When connecting to
the daemon a client must supply an authentication key to gain access to
the daemon. If authentication fails the daemon shuts down and it is
necessary to reestablish the connection.
2.4.1.2 The .irafhosts file
The new networking driver retains the old irafhosts file, used to
store information telling how to connect to various IRAF hosts (the
irafhosts file is the file .irafhosts in the user's login directory).
The networking system will automatically create this file for the user
if the file is not found; if an old-style file is found, it will be
edited by the system to make it compatible with the new networking
system. While it is no longer necessary for the irafhosts file to
contain password information to avoid password prompts, the file is
used to store the user authentication key, hence the file should be
read protected. The networking system will automatically read protect
the file if it is not already protected.
To avoid authentication failures when clients on different nodes
attempt to connect to the same server, the same authentication code
should be used on all server nodes. Unfortunately there is no way that
the networking system can do this automatically (without going to some
much more complicated authentication scheme, such as a key server), so
users who make heavy use of the networking system should install a copy
of their irafhosts file in their login directory on all server nodes.
If this is not done the networking system will still work, but will be
less efficient than it could be, when simultaneously accessing the same
server from IRAF sessions running on multiple client nodes.
The truly paranoid may not be happy with even the unique user
authentication code used in the current networking system. If this is
the case the port parameter (see below) may be set to zero to force rsh
to be used for every connection (in effect the in.irafksd daemon has to
be restarted for every connection). This imposes an overhead of as
much as several seconds on every server connect. Alternatively, KSAUTH
can be defined in the user environment at login time, setting the value
string to some random integer value selected at login time. If defined
in the user environment, KSAUTH will override the value of auth given in
the irafhosts file. This approach would at least allow efficient
connects for a single login process tree.
The irafhosts file consists of two sections. The first section defines
several networking parameters: port, auth, hiport, and timeout. The
second section is a list of server nodes, with login and password
information describing how to connect to each node.
port = default
auth = 1234567890
hiport = default
timeout = default
ursa : <user> ?
* : <user> <user>
The example above illustrates a typical irafhosts file. Typically a
unique authentication code is allocated automatically by the system
when the file is first created, and the other parameters retain their
default values as shown (i.e., the string "default"). In the example
the host list consists of an entry for the node "ursa", and an entry
for everything else. The format of a host entry is "host-name :
login-name password". If login-name is the reserved string "<user>"
the user name on the client node is used for login authentication on
the remote node. Setting the password to "<user>" as well causes the
rsh connect protocol to be used; anything else causes the rexec
protocol to be used. If the rexec protocol is used the password field
may be set to the actual password or to the string "?", in which case
the system will prompt for the password whenever a connection attempt
is made. The "*" entry should always be the last entry in the list,
since it matches all nodes. The default host list contains only the
"*" entry.
Additional information on the irafhosts file is provided in the
comments in the file dev$irafhosts, and in the system notes file.
2.4.1.3 Compatibility
By default the new networking system will try to use the rsh
protocol to connect to the server node. If the server is running an
older version of IRAF the connection attempt will hang and eventually
time out. If this occurs the networking system will fall back on the
rexec protocol and issue a password prompt, but by then the user will
probably have interrupted the connect. The best way to avoid this
problem is to update the server node to V2.10, but if this is not
possible, an explicit entry for the node can be made in the irafhosts
file, setting the password field so that the rexec protocol will be
used.
An older, e.g. V2.9 client connecting to a V2.10 server should not be a
problem. In this case the V2.10 server will see an attempt to connect
with the rexec protocol, which should be processed as in the past.
Aside from the problem of making a connection the pre-V2.10 and V2.10
networking systems are compatible, except for the magtape i/o calls.
Since the magtape driver calling sequences were changed in V2.10, remote
magtape access between V2.10 and older systems is not supported.
2.4.2 The hosts file
The file dev$hosts is used to interface new host machines to the
IRAF networking system. This file defines, for each host, the primary
host name, any aliases, and the command to be executed remotely to start
up the IRAF kernel server on a remote node.
The format and role of the hosts file is unchanged in V2.10. Two
changes were made which affect the use of this file.
o A user can now have a private copy of the hosts file. To enable
this feature, the variable irafhnt (IRAF host name table) should be
defined in the user's IRAF or host level environment, with the
string value giving the file pathname of the user's private host
name table file.
o The maximum number of entries in the host name table has been
increased from 64 to 128. In the current implementation these
entries are statically buffered, so it is necessary to keep the
maximum number of entries to a reasonable value.
The hosts file must be configured to enable IRAF networking. IRAF
networking is disabled if there is no entry for the local host in
this file. The netstatus command may be used to examine the state
of the host name table after it has been loaded by the networking
system.
There is another file dev$uhosts which often confuses people. This
file is not used by UNIX/IRAF and can be ignored; it is there for
VMS/IRAF and other IRAF implementations which do not provide the
equivalent of the UNIX system routine gethostbyname. On host
machines which implement name server facilities IRAF will use the
name server, allowing any machine on the internet to be accessed
via IRAF networking, so long as there is an entry in the system
wide or user IRAF hosts file.
2.5 Magtape System Enhancements
The magtape subsystem underwent a major revision in V2.10. The VOS
level MTIO interface was extensively revised, and the host-level IRAF
magtape driver ZFIOMT is completely new. A new device configuration
facility called tapecap was introduced. Together with the new
"programmable" magtape driver, this makes it possible to interface
almost any removable media mass storage device to the magtape
subsystem. The DATAIO applications, in particular the FITS programs,
underwent minor, but important revisions.
A full specification of the new magtape subsystem, particularly the
tapecap facility, is beyond the scope of this document. Our intention
here is merely to introduce the new facilities. A reference paper is
planned, time permitting, which will fully document the new magtape
system and tapecap.
2.5.1 Basic usage
In most respects basic usage of the magtape system is unchanged from
previous releases. Many new capabilities have been added, but these are
upwards compatible with earlier releases.
2.5.1.1 Logical device names
Magtape devices are still referred to in IRAF applications much as
they have been in the past. Whether or not the logical device names
are the same before and after the V2.10 upgrade depends upon how the
IRAF system manager configures the tapecap file. The new magtape
system is much more flexible than previously regarding device names,
but to avoid user confusion it is recommended that the old names be
supported as aliases regardless of whatever the full device name may be.
As in earlier versions of IRAF, a simple magtape reference might be
cl> mtexamine mta
where "mta" is the device name. To examine only file 3 on the tape one
might enter
cl> mtex mta[3]
If the device is on the remote node "ursa" the command would be
cl> mtex ursa!mta[3]
If the device is a 9 track tape supporting multiple densities we might
specify the density explicitly, e.g.
cl> mtex mta1600[3]
Device names can be more complex. For example,
cl> mtex mtwd
might refer to a WangDAT drive, and
cl> mtex mtwdc
to the same drive, but with data compression enabled.
Devices can have multiple names, possibly with slightly different
behavior or characteristics. Device names such as "mta" are usually
only host specific aliases for the lower level, host independent device
names. The names "mta" and "mtb" might be aliases for the actual device
names "mthp0" and "mtxt1". This can be useful in networked systems
where IRAF and the tapecap file reside on a server node, but the user
is running IRAF on their workstation with a local tape drive attached.
In this case there may be no entry for the local drive in the installed
tapecap file, but the user can still access the local drive using the
lower level, host independent device entry (it is also possible to have
a private tapecap file as we shall see later).
To see what logical devices are known to the magtape system you can
enter the following command (the task gdevices was intended for
graphics devices but can be pressed into service to list a tapecap file
as well). Just because a device is listed does not mean that the
physical device actually exists on a particular host system.
cl> gdev devices='^mt' graphcap=dev$tapecap
In IRAF V2.10 it is possible to include tapecap parameters in the device
specification to do clever things, as in the following example.
cl> mtex "mta[:so=lepus:se:nb]"
This is discussed further below in the section describing the tapecap
facility.
2.5.1.2 Device allocation
Device allocation operates much the same in V2.10 as in earlier
versions of IRAF. The main change is that it is no longer necessary to
allocate a device in order to be able to access it. It is strongly
recommended however that you always allocate a device before accessing
it in IRAF. If the device is not allocated anyone can access the
drive, e.g., changing the tape position, and this can cause data to be
lost or improperly read back. The only reason to access the drive
without allocating it is if there is some problem with device
allocation on your system.
A device is allocated with the allocate command, e.g.
cl> alloc mta
A device is deallocated with deallocate. If the tape has already been
unmounted use the rewind=no option to avoid accessing the drive. By
default the tape will be rewound when it is deallocated. Deallocating
and reallocating a drive initializes the magtape system, i.e., all
cached knowledge of the status of the drive is discarded.
2.5.1.3 Device status
The device status can be examined at any time that the magtape
device is idle (not being actively accessed by an IRAF task) using the
devstatus task.
cl> devs mtc
# Magtape unit mtc status Thu 12:54:02 04-Jun-92 user v14
file = 4
record = 1
nfiles = 0
tapeused = 405
pflags = 0
Of particular interest are the tapeused and nfiles fields. nfiles
refers to the total number of files on the tape (if a file is appended
to the tape it will be file nfiles+1). If nfiles is given as zero that
means that the magtape system does not yet know how many files are on
the tape (it has not seen the end of tape).
tapeused reports the amount of tape used in units of kilobytes. This
is intended to refer to the amount of tape used up to and including the
end of tape (EOT). However, the magtape system only knows about data
that it has seen, i.e. physically read or written, so whether or not
tapeused is accurate depends upon how you have accessed the tape.
For example, if you started out with a fresh tape and appended a number
of files the number should be accurate. If you just completed a full
scan of the tape with mtexamine the number should be accurate, since
all the data was read. If you just put on a new tape and did a scan of
the FITS headers with "rfits make-", the number may or may not be
accurate, depending upon the device and how file skipping forward was
done. In most cases only the header area of each file will have been
read and tapeused will not reflect the full contents of the tape. If
however, "rfits make-" is done immediately after writing to a new tape,
the number will be accurate, since all the data was written before the
tape was rescanned to print the FITS headers.
Be advised that by default an explicit rewind will clear the nfiles and
tapeused fields, since by default rewind initializes the magtape
system. See the discussion of rewind below.
In summary tapeused can be useful for monitoring how much space is left
on a tape, but you have to know what you are doing. When writing to a
new tape the number will be accurate so long as you avoid doing an
explicit rewind. A simple procedure which will always work when
mounting a non-empty tape and appending files to it, is to do an
mtexamine of the tape and then append the new files. This can be time
consuming for large tapes but does provide a safe and
device-independent method for getting the maximum amount of data on a
tape.
2.5.1.4 File positioning
File positioning when accessing magtape files in IRAF is
straightforward in the sense that you can simply specify the file
number to read an existing file, or "append" (as in wfits new-) to
append a file to an existing tape. Most tasks accept range lists to
access existing files, e.g.
cl> mtexamine mta file="3,5,9-11"
It is even possible to position a tape to a specific record within a
file. Opening a tape with file zero (as in "mta[0]") disables file
positioning, allowing the tape to be positioned with host level
utilities to workaround media problems.
There can be problems with this simple scheme however, with modern tape
devices such as DAT and Exabyte which have capacities in the gigabyte
range and which may be used to store thousands of files. It is beyond
the scope of a revisions summary to go into this in detail here, but a
simple example will help illustrate the problem.
Assume we have a tape mounted on device "mtwd" containing 900 files. We
want to append image "pix" as a FITS file. The obvious thing to do is
enter the following command.
cl> wfits pix mtwd new-
This will certainly work. The only problem is that if the tape is
freshly mounted the magtape system will not know how many files there
are on the tape, and it will have to skip forward one file at a time to
count the files and determine where EOT is. In the worst case of a
tape containing several thousand files this could literally take hours.
If we know apriori the number of files on the tape, e.g., 900 in our
example, the following command would be much faster (something like
10-40 seconds on most DAT drives, assuming a decent host level driver).
cl> wfits pix mtwd[901]
Of course, if there were actually 930 files on the tape, the last 30
files would be overwritten.
There is a useful trick which works in some cases if we don't care
exactly how many files are on the tape (whether this works depends upon
the specific device and the host driver). Assume that we know there
are several hundred files on the tape, but less than 1000. We enter
the following command to append a file to the tape.
cl> wfits pix mtwd[1000]
If this works, after the operation the magtape system will think there
are 1000 files on the tape. A subsequent "wfits new-" would be very
fast regardless of the tape position, since so long as the magtape
system knows where the end of tape is relative to the current position,
it can get there very fast.
If the trick doesn't work for your particular device or driver you will
probably get a positioning error when attempting to position to a
nonexistent file beyond the EOT.
More automated techniques for rapid positioning with very high capacity
tapes are still a matter for study. One promising technique would be to
formalize the above approach by supporting EOT-relative positioning. A
tape catalog based approach is also possible. The best approach may
simply be to not write so many small files on large tapes, by grouping
images or other data system files into a single large tape file (as
with UNIX tar). None of these approaches are implemented in V2.10.
2.5.1.5 Rewind
In IRAF a magtape device is rewound with the rewind command, as in
the following example.
cl> rewind mta
By default this will not only rewind the tape but also initialize the
magtape system, in the sense that all cached information regarding the
named device is erased (e.g., nfiles and tapeused in the cached device
status). This is necessary when changing tapes without deallocating the
drive; always do an explicit rewind (or deallocate) of the device before
removing a tape or mounting a new one. Having rewind initialize things
is also useful if error recovery should fail following an interrupt or
program abort.
In some cases it may be useful to be able to do a rewind without
erasing the cached device status. This is done by specifying the init-
option on the command line.
2.5.2 New magtape driver
The IRAF magtape driver is new for V2.10. The chief feature of the
new driver is that it is programmable, via the tapecap device entry,
making it possible to interface new devices or host drivers without
having to make any binary code changes to IRAF. All one has to do is
add a device entry in the tapecap file.
2.5.2.1 Software structure
The IRAF magtape system has enough layers that it may be confusing
exactly what the magtape driver is and what role it plays. A brief
review of the software structure may help clarify things.
Starting at the top we have applications, such as the DATAIO and MTLOCAL
tasks, which can access magtape files. These use the IRAF/VOS
interfaces FIO (file i/o) and MTIO (magtape i/o) to do i/o to tape
devices. For the most part i/o is done with FIO and is device
independent, but a call to the magtape system is required to open a
magtape device. The tapecap file is read by the GTY interface, which
is called by MTIO. MTIO passes the tapecap device entry as a string to
ZFIOMT, the IRAF magtape device driver, when a tape file is opened.
All magtape positioning and i/o is done by ZFIOMT driver under the
control of the MTIO interface. Device allocation is done prior to
accessing the device by the CL and is handled by the allocation
routines in the ETC interface, with kernel support (this is largely
independent of the magtape system).
All of the code listed above is part of the portable IRAF system (i.e.,
is OS independent and shared by all host versions of IRAF) until we get
to the ZFIOMT driver. This is in the IRAF kernel and is host system
dependent. At present the only version of ZFIOMT is the UNIX version;
other versions, e.g., for VMS will follow as IRAF is updated to V2.10
on these systems. The UNIX version of ZFIOMT uses only generic UNIX
ioctls and should compile on most UNIX systems without change. All of
the system and device dependence has been concentrated in the tapecap
file. The ioctls used to communicate with a UNIX tape device are
fairly standard, but the semantics are often poorly defined and are
certainly not standardized.
Beneath the IRAF driver are the host level magtape device drivers. A
given host system may have more than one of these, typically one for
each class of magtape device. Some systems, notably Sun with their ST
(SCSI tape) driver, try to control more than one type of device with a
single host driver. The host driver may come with the OS or may be
supplied by a third party vendor.
Beneath the host driver is the actual tape device. Often these days
this is a SCSI tape device such as a DAT or Exabyte. These devices are
intelligent peripherals; control of the physical tape hardware resides
in the tape unit. There is a small computer in each tape drive which
communicates with the host computer via the SCSI interface, passing
commands and data back and forth. The drive will buffer 256K to 512K
of data passed in bursts over the SCSI bus, and then copied to or from
the physical media at a much slower rate. These drives emulate
variable size records by blocking and deblocking within the drive unit,
and writing fixed size blocks to the media. Features such as error
correction and data compression are also handled within the drive.
Although we usually speak of tape devices, the "magtape" device does not
have to be a tape device at all. The IRAF magtape system can also make
use of, e.g., a floppy disk, or anything that looks like a raw disk
partition. The problem with the latter devices is that they usually
don't support files and file positioning, hence can only be used to
store one "file".
2.5.2.2 Driver features
A detailed description of the magtape driver is beyond the scope of
this document. Briefly, the driver supports two main classes of
devices, variable record size devices and fixed block devices. All
file positioning is handled by the driver, and while the driver has the
device open it is responsible for keeping track of the device position
(the saved device position is passed in at open time and saved by the
high level code at close time). A couple of dozen tapecap parameters
are defined which describe the characteristics of each device, such as
whether it supports variable records, the maximum record size, whether
it can backspace, and so on. A socket or file based status logging
feature is provided which allows detailed monitoring of the tape status
during execution (see below).
In the simplest case the new magtape system, coupled with the UNIX
version of the magtape driver, will emulate simple UNIX commands such
as tar and mt insofar as the requests made to the host system and
magtape device are concerned. That is, if one writes a series of files
the only system requests made for each file will be open, write, and
close. When reading a series of files in sequence the only requests
made will be open, read, and close. Providing no file positioning is
attempted it is possible to get by with no file positioning requests
other than rewind. The driver provides extensive facilities for file
positioning, using tapecap parameters to work around any shortcomings
of the host system or device, but in case of trouble it is possible to
operate with only open, close, read, and write requests, which should
work for any device (unless it is truly broken).
2.5.3 Tapecap
The tapecap file, or magtape device capabilities file, defines the
magtape devices known to the system and how to access these devices.
While large portions of this file depend only upon the host operating
system and device type and hence are fairly site independent, it will
typically be necessary to customize the tapecap file to configure the
IRAF magtape system for a site. In normal use the tapecap file is
invisible to the user, but users with special problems may wish to
learn more about tapecap to gain more control over the behavior of the
magtape system.
2.5.3.1 Using tapecap
The standard tapecap file is the file dev$tapecap. A system
environment variable tapecap is defined which by default points to this
file.
Users can customize or manipulate tapecap entries in either of two ways.
The user can have their own private copy of the tapecap file (much as is
currently done with the termcap and graphcap files), by resetting the
value of the tapecap environment variable to point to their local copy
of the file. For example,
cl> reset tapecap = home$tapecap
This may be necessary to customize a device entry, or add support for a
local device not supported by the system wide tapecap file.
It is also possible to modify a tapecap device entry "on the fly", by
overriding the values of specific tapecap parameters on the command line
when the device is accessed. For example,
cl> mtex "mta[:so=/dev/tty]"
would override the default value of the tapecap parameter "so" for
device mta (in this case enabling status output logging and directing
this output to the user terminal).
Specifying tapecap parameters on the command line is useful for
experimentation but rapidly becomes tiresome if many commands are
entered. In this case it becomes simpler to redefine tapecap to include
the desired tapecap parameter overrides.
cl> reset tapecap = ":so=/dev/tty"
As we see, the tapecap environment variable can be used to either
specify the tapecap file name, or globally override specific tapecap
parameters (all device entries are affected). The full form of the
value string for tapecap is
cl> reset tapecap = [tapecap-file] [`:' capabilities-list]
Any number of colon-delimited tapecap capabilities or parameters may be
given.
It is beyond the scope of this document to detail all the tapecap
parameters here. A reference paper for the magtape system is planned.
For the present, there is a summary of the tapecap parameters in the
ZFIOMT driver source file, os$zfiomt.c. For examples of actual usage
refer to the tapecap file in the distributed system.
2.5.3.2 Configuring tapecap
The tapecap file uses the standard "termcap" file format,
originally developed for BSD UNIX and adopted long ago for IRAF. Any
UNIX system will probably have a manual page defining the termcap file
format, if not this can be obtained from the IRAF group.
The distributed tapecap file is divided into three sections: the host
machine specific device aliases (names like "mta" etc.), the site
logical devices, and the generic device entries. To customize the
tapecap file for a site you modify the first and second sections. To
configure the tapecap file for a particular host machine you uncomment
the entries for that host in the first section of the file. Sites
which don't have a large network of hosts may prefer to combine the
first two sections to simplify things. Site specific changes should
never be made to the bottom, or generic, part of the tapecap file, as
you will want to update this portion of the file when new versions of
IRAF are released.
Don't be intimidated by the apparent complexity of some of the tapecap
device entries. In most cases the generic device entry will already
exist for the device you need to interface, and all you need to do is
add a site entry which references the generic entry. In some cases the
generic entry itself may be sufficient (for example, in the distributed
SunOS version of tapecap, logical device "mtxb0" would provide access
to Exabyte unit 0 interfaced with the Sun ST driver, "mtxb1" would be
the same but unit 1, "mthp0" the HP 9 track tape on unit 0, and so on).
For example to interface Exabyte unit 2, using the Sun ST driver, as
local device "mta", the following entry would suffice.
mta| :tc=mtst2-exb:
If the generic device entry provided doesn't quite work and minor
modifications are needed, these should be made to the local entry and
not the standard generic entry. This is easily done with termcap
parameter redefinitions. For example, in SunOS 4.1.2 (but not earlier
versions of SunOS) there is a bug in the Sun ST driver which
necessitates rewinding the tape after a tape scan is made to position
to EOT (!). The capabilities ":se:nb" can be added to the tapecap
entry for the device to workaround this bug, as in the following
example.
mta| :se:nb:tc=mtst2-exb:
The parameters mean that the device spaces past EOT in a read (se) and
cannot backspace (nb). Neither of these things is actually true, but
the effect is that the tape is rewound and spaced forward to get back
to the desired file, working around the host level driver bug. Access
will be less efficient than it should be, but the interface will work
properly and the user does not have to be concerned with the problem.
As a final example, suppose we need to write a new tapecap entry from
scratch because there is no generic entry for the device in the
distributed tapecap file. To simplify things we assume that no special
tapecap parameters are needed, i.e., that the standard UNIX defaults
builtin to the driver will work for the device. We are upgrading from
an old version of IRAF so we already have an old dev$devices file to
work with. For the purposes of our example we use an old HP 88780 1/2
tape drive entry, but pretend that the device is something which is not
already supported.
The old devices file entry was as follows.
mta nrst20 nrst4 nrst12 nrst28 rst4 rst12 rst20 rst28
mta.6250 nrst20 nrst4 nrst12 nrst28 rst4 rst12 rst20 rst28
The minimal tapecap entry required to duplicate this is the following.
mta|mta6250|HP 88780 1/2 inch tape drive:\
:al=nrst4 nrst12 nrst20 nrst28 rst4 rst12 rst20 rst28:\
:dv=nrst20:lk=st4:tc=9tk-6250:
Here, the "al" parameter lists the device files to be allocated, the
"dv" parameter is the device file to be used for i/o at the desired
density (6250bpi in this case), and "lk" is the root file name to be
used for the ".lok", or device status file. The "tc=" picks up the
standard parameters for a 9 track 1/2 inch tape drive operating at 6250
bpi. Two device aliases are defined, "mta" and "mta6250".
2.5.3.3 Supported devices
The IRAF magtape system itself should support almost any magtape
device if properly configured. What devices are actually supported at
a site depends upon the tapecap file, and in particular upon the host
system (different host systems have different tapecap files).
Device Driver
QIC-11 cartridge tape Sun st
QIC-24 cartridge tape Sun st
QIC-150 cartridge tape Sun st
Pertec compatible 1/2 inch drives Xylogics
HP 88780 1/2 inch drive Sun st
WangDAT 1300, 2000 ApUNIX
HP DAT ApUNIX
Exabyte 8200, 8500 ApUNIX, Sun st, Ciprico
DC2000 cartridge tape A/UX tc
FDHD floppy disk A/UX fd
As an example, the tapecap file distributed in the V2.10.0 release of
Sun/IRAF supported the devices listed in the table above. New devices
are constantly being added. As V2.10 IRAF propagates to the various
platforms on which IRAF is supported, similar tapecap files will be
configured for those systems.
2.5.4 Status output
The new magtape driver has a facility known as status output
logging which can be used to monitor interactively lengthy tape jobs
while i/o is in progress. The status output facility can also be
useful for debugging magtape problems.
In its simplest form, the status output may be directed to a file,
e.g., an actual text file, or a terminal device. Status output is
enabled by setting the "so" option in tapecap. For example,
cl> mtex "mta[:so=/tmp/mta.out]"
would enable status output logging and direct the output text to the
file /tmp/mta.out. Likewise,
cl> mtex "mta[:so=/dev/ttyp7]"
would enable status output and direct the output to a pseudoterminal,
e.g., an xterm window. When this form of status output logging is used
one sees the raw, driver level status messages, as in the following
example.
cl> mtex "mta[:so=/dev/tty]"
open device tc2n\n
devtype = Apple Tape 40SC
tapetype = DC2000
tapesize = 38500
density = na
blksize = 8192
acmode = read
file = -1
record = -1
nfiles = 0
tapeused = 0
device tc2n opened on descriptor 4\n
rewinding...
done\n
position to file 1\n
file = 1
record = 1
reading...\n
recsize = 65536
record = 9
tapeused = 64
(etc.)
The UNIX version of the new magtape driver also has an option to direct
status output to a TCP/IP socket, which can be anywhere on the network.
For this option to be useful one must have a program which will listen
on the designated socket, accept the connection when the magtape driver
tries to open a connection, and then read and process the status
messages (which are still text, exactly as in the example above).
Status output to a socket is enabled by giving a host name instead of a
file name in the "so" directive:
cl> mtex "mta[3:so=lepus]"
to examine file 3, directing status messages to a socket on node
"lepus".
An X11 client application called xtapemon has been developed to listen
on a socket, read messages from the tape driver, and provide a real-time
display of the status of the tape device. While not included in the
V2.10 IRAF distribution, this utility will be available later in 1992
as part of the X11 support package currently under development.
2.5.5 Error recovery
Considerable effort went into "bullet proofing" the new magtape
system to make it recover gracefully from ctrl/c interrupts or other
program aborts. In practice it can be very difficult to reliably catch
and recover from interrupts in a multiprocess, distributed system such
as IRAF. No doubt there are still conditions under which an interrupt
will leave the magtape system in a bad state, but in most circumstances
the system should now be able to successfully recover gracefully from
an interrupt.
If it is necessary to interrupt a tape operation, it is important to
understand that the host system will not deliver the interrupt signal
to the IRAF process until any pending i/o operation or other driver
request completes. For example, in a read operation the interrupt will
not be acted upon until the read operation completes, or in a tape
positioning operation such as a rewind or file skip forward, the
interrupt will not be acted upon until the tape gets to the requested
position. For a device such as a disk you rarely notice the delay to
complete a driver request, but for a magtape device these operations
will take anywhere from a few seconds to a few tens of seconds to
complete. Type ctrl/c once, and wait until the operation completes and
the system responds.
If a magtape operation is interrupted, the IRAF error recovery code will
mark the tape position as undefined (devstatus will report a file
number of -1). This will automatically cause a rewind and space forward
the next time the tape is accessed. The rewind is necessary to return
the tape to a known position.
2.5.6 Device optimization
In addition to making it possible to characterize the behavior of a
magtape device to permit the device to be accessed reliably, the
tapecap facility is used to optimize i/o to the device. The parameter
"fb" may be specified for a device to define the "optimum" FITS
blocking factor for the device. Unless the user explicitly specifies
the blocking factor, this is the value that the V2.10 wfits task will
use when writing FITS files to a tape. Note that for cartridge devices
a FITS blocking factor of 22 is used for some devices; at first this
may seem non-standard FITS, but it is perfectly legal, since for a
fixed block size device the FITS blocking factor serves only to
determine how the program buffers the data (for a fixed block device
you get exactly the same tape regardless of the logical blocking
factor). For non-FITS device access the magtape system defines an
optimum record size which is used to do things like buffer data for
cartridge tape devices to allow streaming.
Some devices, i.e., some DAT and Exabyte devices, are slow to switch
between read and skip mode, and for files smaller than a certain size,
when skipping forward to the next file, it will be faster to read the
remainder of the file than to close the file and do a file skip
forward. The "fe" parameter is provided for such devices, to define
the "file equivalent" in kilobytes of file data, which can be read in
the time that it takes to complete a short file positioning operation
and resume reading. Use of this device parameter in a tape scanning
application such as rfits can make a factor of 5-10 difference in the
time required to execute a tape scan of a tape containing many small
files.
On most cartridge tape devices backspacing, if permitted at all, is very
slow. On such devices it may be best to set the "nf" parameter to tell
the driver to rewind and space forward when backspacing to a file.
2.5.7 MTIO interface changes
A number of new routines were added to the MTIO (magtape i/o)
programming interface. These are documented in the summary of
programming environment revisions given below. Existing magtape
applications may benefit from being modified to make use of these new
routines.
2.6 Graphics and Imaging Subsystem Enhancements
2.6.1 New graphics applications
New tasks for histogram plotting, radial profile plotting, and
producing lists of the available graphics devices have been added to
the PLOT package. All of the tasks in this package were revised to add
support for world coordinates. A related task for drawing world
coordinate grid overlays on images or plots was added to the IMAGES.TV
package. See the appropriate sections of IRAF and NOAO package
revisions below for further information on these changes.
2.6.2 Graphics system changes
2.6.2.1 Encapsulated postscript SGI kernel
A new encapsulated postscript SGI kernel has been added. Graphics
output may be directed to any of the logical graphics devices eps, epsl,
epshalf, etc. to render a plot in encapsulated postscript, e.g., for
inclusion as a figure in a document. For example,
cl> prow dev$pix 101 dev=eps; gflush
will leave a ".eps" file containing the plot in the current directory.
The command "gdev dev=eps" will produce a list of the available EPS
logical graphics devices.
2.6.2.2 Graphics output to the default printer
Graphics output (SGI postscript) can now be directed to the UNIX
default printer device by directing output to any of the logical
devices "lpr", "lp", or "lw".
cl> prow dev$pix 101 dev=lpr
Output will be sent to the default device for the UNIX lpr command (set
by defining "PRINTER" in your UNIX environment). This makes it
possible to make immediate use the distributed IRAF graphcap without
having to add entries for specific local devices (although you may
still wish to do so).
2.6.2.3 Tick spacing algorithm improved
The algorithm used to draw the minor ticks on IRAF plots was
replaced by an improved algorithm contributed by the STScI STSDAS group
(Jonathan Eisenhamer in particular). This was derived from similar
code in Mongo.
2.6.2.4 Graphics metacode buffer
The default maximum size of the graphics metacode buffer in the CL,
used to buffer graphics output for cursor mode interaction, was
increased from 64K to 128K graphics metacode words (256Kb). The
cmbuflen environment variable may be used to change this value.
2.6.2.5 IMTOOL changes
The imtool display server (SunView) was enhanced to add additional
builtin color tables, support for user color tables, and setup panel
control over the screen capture facilities. A version supporting
encapsulated postscript output is in testing. This will be the final
version of the SunView based version of imtool (the new display servers
are all X window system based).
2.6.2.6 IMTOOLRC changes
The imtoolrc file, used by display servers such as imtool and
saoimage to define the available image frame buffer configurations, has
been relocated to the DEV directory to make it easier for local site
managers to customize. The IRAF install script now uses a link to
point to this file, rather than copying it to /usr/local/lib. The
maximum number of frame buffer configurations was increased from 64 to
128.
2.6.2.7 X11 support
The released version of V2.10 does not contain any changes insofar
as X11 support is concerned. Since our X11 support code is host level
stuff, with minimal ties to IRAF per se, it is being developed
independently of the V2.10 release and will be distributed and
installed as a separate product.
2.7 Image Structures
2.7.1 Image I/O (IMIO)
The image i/o interface (IMIO) is that part of the IRAF system
responsible for imagefile access and management. The changes to IMIO
for V2.10 included the following.
2.7.1.1 Null images
Null images are now supported for image output, much like the null
files long supported by the file i/o system. For example,
cl> imcopy pix dev$null
would copy the image "pix" to the null image. This exercises the
software but produces no actual output image. Unlike null files, null
images do not work for input since images have some minimal required
structure, whereas files can be zero length.
2.7.1.2 Preallocating pixel file storage
In the UNIX versions of IRAF, when a new image is created storage
is not physically allocated for the output image until the data is
written. This is because most UNIX systems do not provide any means to
preallocate file system storage. The UNIX/IRAF file creation primitive
zfaloc, used by IMIO to create pixel files, now provides an option
which can be used to force storage to be physically allocated at file
creation time. This feature is enabled by defining the environment
variable ZFALOC in the UNIX environment. For example,
% setenv ZFALOC
can be entered before starting IRAF to cause space for all pixel files
to be preallocated as images are created. If it is not desired to
preallocate all image storage (there is a significant runtime overhead
associated with preallocating large images) then a value string can be
given to specify which types of images to preallocate storage for. For
example,
% setenv ZFALOC /scr5
would preallocate only those pixel files stored on the /scr5 disk, and
% setenv ZFALOC "/scr5,zero"
would preallocate only images on /scr5, or images containing the
substring "zero" in the image name. In general, the string value of
ZFALOC may be the null string, which matches all images, or a list of
simple pattern substrings identifying the images to be matched.
In most cases the default behavior of the system, which is to not
physically allocate storage until the data is written, is preferable
since it is more efficient. The preallocation option is provided for
users who, for example might have an application which spends a lot of
time computing an image, and they want to ensure that the disk space
will be available to finish writing out the image.
2.7.1.3 Image templates now sorted
As mentioned earlier in the System Management section, UNIX/IRAF now
sorts UNIX directories. One result of this is that the sections of
image templates which are expanded via pattern matching against a
directory will now be sorted, or at least logically ordered (the final
output list will not necessarily be fully sorted if string substitution
is being performed - this is the reason the output list itself is not
sorted).
2.7.1.4 The dev$pix test image
Minor changes were made to clean up the image header of the
standard test image dev$pix. A second test image dev$wpix has been
added. This is the same image, but with a different header containing a
test world coordinate system (actually the image is just a second image
header pointing to the dev$pix pixel file, now shared by both images).
2.7.2 Image kernels (IKI)
The IMIO image kernels are the data format specific part of the
IRAF image i/o subsystem. Each kernel supports a different image
format. Existing disk image formats range from the conventional image
raster formats (OIF and STF) to the photon image format (QPF and QPOE)
and the pixel mask image format (PLF and PMIO/PLIO). There also exist
special image kernels which allow IMIO to be used to access non-disk
storage devices such as image display frame buffers.
2.7.2.1 New PLF image kernel
A new image kernel, the PLF image kernel, has been added which
allows IRAF PMIO/PLIO pixel masks to be stored as images. This kernel
first appeared as a patch to version 2.9 IRAF but was actually
developed within V2.10.
Pixel mask images may be created, deleted, read, written, etc. like
other IRAF images, but the image is stored in a special compressed
format designed specially for image masks. An image mask is an
N-dimensional raster-like object wherein typically there are regions of
constant value but arbitrary shape. Masks are used by applications
involving image decomposition. The image is decomposed into regions of
different types, the type of region being very dependent upon the type
of image analysis being performed. A special type of mask image is the
bad pixel mask, used to flag the bad pixels in an image. Other
applications use masks for data quality, to identify the region or
regions to be used for analysis, and so on.
A PMIO image mask is a special case of a PLIO pixel list. Masks can
exist and be accessed independently of the image i/o system, but the
PLF image kernel allows a mask to be stored and accessed as an IRAF
image. Any IRAF application which operates upon images can operated
upon a mask image. For example, the imcalc (image calculator) program
in the SAO XRAY package can be used to combine images and masks, or
compute new masks by evaluating an algebraic expression over an image.
Mask images have the reserved image extension ".pl". Some of the
features of mask images are illustrated by the following example.
cl> imcopy dev$pix pix.pl
dev$pix -> pix.pl
cl> imstat dev$pix,pix.pl
# IMAGE NPIX MEAN STDDEV MIN MAX
dev$pix 262144 108.3 131.3 -1. 19936.
pix.pl 262144 108.3 131.3 6. 19936.
This is a sort of worst case test of the mask encoding algorithm, since
our test image is not a mask but a noisy image (and hence not very
compressible). Note that masks must be positive valued, hence the MIN
value is different for the two images. Storing dev$pix as a mask does
not result in any significant compression, but for a real mask very
high compression factors are possible. The compression algorithm used
in PLIO is simple and fast, combining 2D run length encoding and a
differencing technique in a data structure allowing random access of
multidimensional masks (masks are not limited to one or two dimensions).
For further information on pixel lists and pixel masks, see the document
plio$PLIO.hlp in the online system. This is also available as
plio.txt.Z in the IRAF network archive.
2.7.2.2 OIF image kernel changes
The OIF image kernel is the kernel for the old IRAF image format -
this is still the default IRAF image format. Revisions to the OIF
kernel for V2.10 included the following items.
o A valid image header is now written even if an application which
writes to a new image is aborted. Previously, the pixel file would
be written but the image header would be zero length until the
image was closed. If an image creation task aborts the image will
likely be incomplete in some way, e.g., part of the pixels may not
have been written to, or the header may be missing application
specific keywords. By valid image header we mean that the header
will be valid to IMIO, allowing the image to be accessed to try to
recover the data, or to delete the image.
o The image header file of a new image is now written to and closed
at image create time, then reopened at image close time to update
the header. This frees one file descriptor, an important
consideration for applications which for some reason need to write
to dozens of output images simultaneously.
o The OIF image kernel uses file protection to prevent inadvertent
deletion of the image header file. In UNIX/IRAF systems file
delete protection is simulated by creating a ".." prefixed hard
link to the file. This could cause problems with images if the
user deleted the image header file outside of IRAF, leaving the
".." prefixed link to the file behind. A subsequent image create
operation with the same image name would fail due to the existence
of the hidden ".." prefixed file. It is unlikely that such a file
(with a ".." prefix and a ".imh" extension) could ever be anything
but an old IRAF image header file, so the system will now silently
replace such files when creating a new image.
A couple of related system changes were made which, while not
implemented in the OIF kernel, do involve the OIF or ".imh" image
format. The default template login.cl now defines the environment
variable imtype and sets it to "imh". The environment variable
min_lenuserarea is also defined now at login time, with a default
value of 20000, allowing image headers with up to about 250 header
parameters.
2.7.2.3 STF image kernel changes
The STF image kernel is the kernel for the online HST (Hubble Space
Telescope) image format. This format allows multiple images to be
grouped together in a single "group format" image. There is a common
image header stored in a FITS-like format, as well as a small fixed
format binary header associated with each image in the group.
o A check was added for a group index out of range. This yields a
more meaningful error message about no such group, rather than
having IMIO complain about a reference out of bounds on the pixel
file.
o Support was added for non-group STF images (GROUPS=F in the header).
At first glance a non-group group format image might seem a little
silly, but it turns out that a non-group STF image with a zero
length group parameter block is very close to "FITS on disk", since
the header is FITS-like and the image matrix is simple. It is
still not true FITS since the header and pixels are stored in
separate files and the pixels are not encoded in a machine
independent form, but on UNIX hosts which are IEEE standard and not
byte swapped, it comes close enough to be useful for communicating
with external applications in some cases.
A true FITS image kernel is planned for IRAF. This will probably
appear in one of the V2.10 patches, sometime during 1992.
2.7.2.4 QPF image kernel changes
The QPF image kernel is the interface between the QPOE (position
ordered event file) interface and IMIO, allowing QPOE event files to be
accessed as images by general IRAF applications. Photon "images" are
unique in that the image raster is generated at runtime as the image is
accessed, optionally passing the photon list through event attribute
and spatial filters, and sampling the photons to produce a raster
image. For example,
cl> imcopy "snr[time=@snr.tf,bl=4]" snr.imh
would filter the event list stored in the file "snr.qp" by the time
filter stored in file "snr.tf", sample the image space blocking by a
factor of 4, and store the resultant image raster in the OIF image file
"snr.imh".
cl> display "snr[time=@snr.tf,bl=4]" 1
would display the same image raster without creating an intermediate
disk image.
The changes to the QPF interface for V2.10 included the following.
o A bug was fixed which would prevent very long filter expressions
from being correctly recorded in the IMIO image header. The
current version of IMIO stores applications header parameters in
FITS format, hence multiple FITS cards are required to store long
filter expressions. The bug would cause only one such card to be
output.
o A new facility was added which allows general expressions to be
computed for the input event list and stored as keywords in the
output image header. The header of the input QPOE file can contain
one or more parameters named defattr1, defattr2, and so on. If
present, the string value of each such parameter should be a
statement such as
exptime = integral time:d
which will cause a keyword named "exptime" to be added to the
output image header, the scalar value of the keyword being the
value of the expression on the right. Currently, only the integral
function is provided. This computes the included area of a range
list field of the event attribute expression, such as a time
filter. In the example, "time" is the name of the event attribute
to be integrated, and the ":d" means use a range list of type
double for the computation. The data types currently supported are
integer, real and double.
Other minor changes to QPF included improvements to the error
recovery, and other changes to support very large filters.
2.7.3 World coordinate system support (MWCS)
MWCS is the IRAF world coordinate system package, one of the major
new system interfaces developed for the new image structures project.
A full description of the MWCS interface is given in the file
mwcs$MWCS.hlp in the online system, and in the file mwcs.txt.Z in the
IRAF network archive.
2.7.3.1 Applications support
MWCS was first released in V2.9 IRAF and at the time the interface
was new and few applications were yet using it. In V2.10 IRAF most
(but not all) applications which deal with coordinates now use MWCS.
These include all of the system plotting tasks, and the spectral
reduction packages. Details of the MWCS support added to the system
and science applications in V2.10 are given in the IRAF and NOAO
package revisions section of this revisions summary.
2.7.3.2 New function drivers
Function drivers for the arc, sin, and gls nonlinear sky
projections were added, as well as a special function driver for the
multispec image format. The latter allows general programs which don't
know anything about spectra to access and display spectra in world
coordinates, e.g., for plotting.
2.7.3.3 WCS attributes
The maximum number of "attributes" per WCS was increased from 64 to
256, mainly to support the multispec function driver, which makes heavy
use of attributes. A limit on the size of a single attribute value
string was removed, allowing arbitrarily large attribute values to be
stored.
2.7.3.4 Axis mapping
Axis mapping is now fully implemented. Axis mapping is used when,
for example, you extract a 2 dimensional section from a 3 dimensional
image and write the result to a new image. Axis mapping allows the 2
dimensions of the new image to be mapped backed into the original 3
dimensional WCS, making it possible to get the full physical
coordinates (which are 3 dimensional) for any point in the extracted
image. A new header keyword WAXMAPnn was added to store the axis map
in image headers.
2.7.3.5 MWCS save format
The newer IRAF image formats such as QPOE are capable of storing
arbitrarily complex objects such as a MWCS in an image header keyword.
In the case of a stored MWCS object, the MWCS interface is responsible
for encoding the object in its external storage format, and restoring
it to a MWCS descriptor when the MWCS is accessed. The code which does
this was revised to add a new version number for the stored V2.10 MWCS,
to make it possible to differentiate between the MWCS save formats used
in V2.9 and V2.10 and allow recovery of MWCS objects from data files
written under V2.9.
2.7.3.6 Bug fixes
Since MWCS is a new interface and V2.10 saw its first real use in
applications, a number of interface bugs were discovered and fixed.
Most of the bugs turned out to be in the part of MWCS which converts
between the internal MWCS WCS representation, and the FITS WCS
representation, used for those image formats that still use FITS-like
headers. Bug fixes included a problem with the treatment of CROTA2, a
problem with the FITS CD matrix, an axis mapping problem that caused
"dimension mismatch" errors, and a problem with support for the
old-style CDELT/CROTA which could result in a singular matrix during
WCS inversion when compiling a transformation.
2.7.4 Event files (QPOE)
The QPOE interface is the interface responsible for creating and
accessing event files, the main data format produced by event counting
detectors. We summarize here the main enhancements to QPOE made during
the preparation of V2.10. Some of these features appeared earlier in
the patches made to V2.9 IRAF but these revisions have never been
formally documented so we summarize both the old and new revisions
here. See also the discussion of the QPF image kernel given earlier.
2.7.4.1 Blocking factors
The builtin default blocking factor, when sampling the event list
to make an image raster, is one. This may be changed on a per-datafile
basis by defining the parameter defblock in the QPOE file header.
2.7.4.2 Parameter defaults
Since parameters such as the blocking factor can be set at a number
of levels, a parameter defaulting scheme was defined to determine the
precedence of parameter overrides. The lowest level of precedence is
the builtin default. Next is any datafile specific value such as
defblock. A value such as blockfactor set in the QPDEFS file will
override the datafile default value if any. Specifying the parameter
value in a runtime filter expression or qpseti call is the highest
order of precedence and will override any other setting.
Another way to think of this is the more recently the parameter was
set, the higher the precedence. The oldest value is the default
builtin to the code. Next is the datafile value, usually set when the
datafile was created. Next is the QPDEFS macro file, usually set by
the user or for a site. Last (highest precedence) is the value set by
the user when the data is accessed.
2.7.4.3 Referencing the current datafile in macros
A special symbol $DFN is now recognized during macro expansion and
if present is replaced by the filename of the current QPOE file. This
allows macros to be written which automatically perform some operation
involving the datafile to which they applied, e.g., computing an
attribute list from aspect or data quality information stored in the
datafile header.
2.7.4.4 Bitmask expressions
Bitmask expressions may now be negated, e.g., "%3B" is the mask 03
octal, "!%3B" or "!(%3B)" is the complement of 03 octal.
2.7.4.5 Large time filters
A number of changes and bug fixes were made to QPOE for V2.10 to
support large time filters. These are lists of "good time" intervals
used to filter the event list. These large time filters may contain
several hundred double precision time intervals spanning the exposure
period. Often a large time filter is combined with a complex spatial
filter (PLIO mask) to filter an event list consisting of from several
hundred thousand to several million events. QPOE can handle this
efficiently regardless of whether the data is temporarily or spatially
sorted and regardless of the complexity of the temporal or spatial
filters.
A number of minor bugs were fixed caused by the large filter expressions
overflowing various buffers. The default sizes of the program and data
buffers used to compile filter expressions were increased to allow all
but very large filters to be compiled without having to change the
defaults. If the buffers overflow more space can be allocated by
setting the progbuflen and databuflen parameters in QPDEFS (these
buffers are not dynamically resized at runtime for efficiency reasons).
During testing with large time filters it was discovered that a routine
used to test floating point equality was being used inefficiently, and
compilation of large time filters was taking a surprisingly long time.
A change was made which reduced the time to compile large filters by a
factor of 10 to 15.
2.7.4.6 Default filters
QPOE now fully supports default event attribute and spatial
filtering of data at runtime. The idea is that the full data set (all
events) is stored in the QPOE file, along with default event attribute
and spatial filters. When the data is accessed these filters are, by
default, automatically applied. Any user specified event attribute or
spatial filters are "added" to the builtin default filters to produce
the combined filter used when the event list is accessed. The point of
all this is to make it easy for the user to modify or replace the
default filters and "reprocess" the raw event list. In effect the raw
and processed event list are available in the same file.
The default filter and mask, if any, are stored in the datafile header
parameters deffilt and defmask. If the datafile contains multiple
event lists a default filter or mask may be associated with each list
by adding a period and the name of the event list parameter, e.g.,
"deffilt.foo" would be the default filter for the event list "foo".
By default, if a default filter or mask exists for an event list it is
automatically applied when the event list is accessed. Use of the
default filter or mask can be disabled in QPDEFS with either of the
following statements:
set nodeffilt
set nodefmask
The default filter and mask can also be disabled in a filter expression
by adding a "!" before the expression, as in the following example.
display "snr[!time=@times.lst,pha=(3,8:11)]"
There are actually several variants on the "=" assignment operator used
in filter expressions such as that above. The operator "=" is
equivalent to "+=", meaning the expression term on the right adds to
any existing filter specified for the event attribute on the left. The
operator ":=" on the other hand, means replace any existing filter by
the new filter expression.
2.7.4.7 Alternate coordinate systems
The event structure used to represent each event in an event list
may contain multiple coordinate systems if desired, for example,
detector and sky coordinates. One coordinate system should be defined
as the default when the event list is created, and if the event list is
to be indexed it must be sorted using the coordinate system specified
as the default. Any coordinate system may be used when the data is
accessed however; this can result in quite different views of the same
data set. For example,
cl> display snr.qp 1
would display the QPOE file "snr.qp" in display frame 1, using all
defaults for the event list, blocking factor, filter, mask, and so on.
The default event coordinate system would probably be sky coordinates.
To display the same event list in detector coordinates, one could enter
the following command.
cl> display "snr.qp[key=(dx,dy)]" 1
This assumes that the event structure fields "dx" and "dy" are defined
somewhere, either in the datafile header or in QPDEFS (otherwise the
actual field datatype-offset codes must be given).
Currently if the QPOE datafile has a WCS associated with it this WCS can
only refer to one coordinate system, normally the default event
coordinate system. Additional WCS can be stored in the QPOE file,
either in the stored MWCS object or as separate MWCS objects, but at
present there is no mechanism for selecting which will be used (if the
parameter qpwcs exists in the QPOE file header, this is assumed to be a
stored MWCS object and this is the WCS which will be used).
2.8 System Specific Changes
2.8.1 Sun/IRAF
Since V2.10 has only just been completed and at this stage has only
been released on Sun platforms, there are few system specific revisions
to report. For SunOS there were several system specific revisions
worth noting here.
o The HSI binaries for the sparc architecture are now statically
linked. This makes them considerably larger, but avoids problems
with SunOS complaining about shared library version mismatches
(note that we refer here to to Sun shared libraries - this is not a
problem with the IRAF shared library facility since we control the
version numbers).
o The HSI binaries for the Motorola architectures (f68881 and ffpa)
are not statically linked since they cannot be without running into
linker problems warning about f68881_used etc. To avoid or at least
minimize warnings about SunOS shared library versions the system was
linked on 4.1.1 instead of 4.1.2. SunOS 4.0.3 versions of the
Motorola HSI binaries are also available upon request.
o The system as distributed was linked with the name server library,
-lresolv. This means that if the local host is configured to use
the name server, IRAF will be able to access almost any host on the
Internet without an entry in the /etc/hosts file on the local host.
Additional system specific changes will be reported in the
documentation distributed with each release, as V2.10 is moved to
the various platforms for which IRAF is supported.
3. IRAF and NOAO package revisions
The revisions for the various packages in the IRAF core system and
in the NOAO packages are summarized below. There have been many changes
with this release. Users are encouraged to refer to the help pages,
user's guides provided with some packages, revisions notes, and more
recent issues of IRAF Newsletters for more details. Comprehensive
notes on systems and applications modifications are in the files
pkgnotes.v210.Z and sysnotes.v210.Z in the directory iraf/v210 in the
network archive. This summary can be read online by typing news.
Revisions notes for the various packages can also be accessed online as
in the following example.
cl> phelp dataio.revisions opt=sys
One of the system changes that affects many tasks in the IMAGES, PLOT,
and LISTS packages as well as most tasks in the spectroscopic packages
in NOAO is the full implementation of the world coordinate system
(WCS), introduced in V2.9 but not fully integrated into the IRAF and
NOAO tasks at that time. There are currently three classes of
coordinate systems: the logical system is in pixel units of the current
image section; the physical system is in pixel units of the parent
image (for a WCS associated with a raster image); the world system can
be any system defined by the application, e.g. RA and DEC or
wavelength. In general, this should be transparent to the user since
most defaults have been chosen carefully so that tasks perform the same
as in V2.9. The NOAO spectroscopic tasks also use the WCS extensively,
but again, this should be transparent to the user, although it can
cause some problems with backward compatibility. Users will notice the
biggest difference in the image headers with additional keywords added
by the interface. Two tasks in the PROTO package may help the
interested user understand more about the WCS - see wcsedit and
wcsreset. Technical notes on the WCS are available in a paper in the
iraf$sys/mwcs directory. Type the following to read more about the
MWCS interface.
cl> phelp mwcs$MWCS.hlp fi+
3.1 Changes to the IRAF system packages
3.1.1 DATAIO package modifications
o The output defaults for wfits have been modified. If the pixel
type on disk is real or double precision the output will be IEEE
format (FITS bitpix = -32 or -64, respectively). If the pixel type
on disk is short integer then the output will be integer format
(bitpix = 16) as before. These defaults can of course be changed
by modifying the parameters for wfits.
o The wfits "blocking_factor" parameter can accept values up to and
including 10 for variable blocked devices, i.e. 1/2" tape drives,
Exabytes, and DATs. If the "blocking_factor" parameter is set to
"0", as in the default, the value is read from the "fb" parameter
in the tapecap file (usually 10 for variable blocked devices). For
fixed blocked devices such as cartridge tape drives the default
value for the "fb" parameter in the tapecap file is 22 (used to
determine a buffer size) and the block size of the device is given
by the "bs" parameter.
o All tasks were modified to accept tapecap parameters as part of the
magtape file name syntax. Tapecap parameters can now be appended
to the magtape file name to add to or override values in the
tapecap file. For example, "mta6250[:se]" is a valid magtape file
name (the "" are important when tapecap parameters are specified at
execution time). See the file os$zfiomt.c for more details about
the tapecap parameters.
o The rfits task has been modified to permit a short last record,
i.e. a last record that has not been padded out to 2880 bytes. No
warning messages are issued.
o rfits was modified to map ASCII control characters in the header to
blanks.
o The sequence number appended to disk file names by rfits and wfits
was modified to 4 digits, i.e. 0001 - 9999.
3.1.2 IMAGES package modifications
o WCS (world coordinate system) support was added to all tasks in the
IMAGES package that could introduce a coordinate transformation.
Some tasks had already been modified for the V2.9 release. These
tasks (blkavg, blkrep, geotran, imcopy, imlintran, imshift,
imslice, imstack, imtranspose, magnify, register, rotate, and
shiftlines), upon execution, will update the image header to
reflect any transformation.
o The listpixels task was modified to list the pixel coordinates in
either logical, physical, or world coordinates. A format parameter
was added to the task for formatting the output pixel coordinates
taking precedence over the WCS in the image header, if used.
o A new imcombine task was added to the package. This new task
supports image masks and offsets, and has an assortment of new
combining algorithms. See the help pages for complete details on
this powerful new task.
o The imhistogram task has a new "binwidth" parameter for setting
histogram resolution in intensity units.
o The default values for the parameters "xscale" and "yscale" in the
register and geotran tasks were changed from INDEF to 1.0, to
preserve the scale of the reference image.
3.1.3 IMAGES.TV package reorganization and modifications
o The TV package has been reorganized. The IIS dependent tasks have
been moved into a subpackage, TV.IIS. The imedit, imexamine, and
tvmark tasks that were previously in PROTO have been moved to TV.
o A new task wcslab developed at STScI by Jonathan Eisenhamer was
added to the package. wcslab overlays a labeled world coordinate
grid on an image using WCS information stored in the header of the
image or in parameters of the task.
o imexamine was modified to use WCS information in axis labels and
coordinate readback, if selected by the user. Two new parameters,
"xformat" and "yformat", were added to specify the format of the
WCS if it is not present in the header of the image.
o A new option was added to imexamine to allow for fitting 1D
gaussians to lines or columns.
o tvmark had two cursor key changes. The "d" keystroke command that
marked an object with a dot was changed to "."; the "u" keystroke
command that deleted a point was changed to "d".
3.1.4 LISTS package modifications
o Two new parameters were added to the rimcursor task, "wxformat" and
"wyformat". These parameters allow the user to specify the
coordinate formats for the output of the task and override any
values stored in the WCS in the image header.
3.1.5 OBSOLETE package added
o An new package called OBSOLETE was added. Tasks will be moved to
this package as they are replaced by newer tasks in the system.
OBSOLETE tasks will then be removed at the time of the next release.
o mkhistogram, imtitle, radplt, and the old imcombine task have been
moved to the OBSOLETE package. Users should become familiar with
phistogram, hedit, pradprof, and the new imcombine and use them
instead since mkhistogram, imtitle, radplt, and oimcombine will be
retired with the next release.
3.1.6 PLOT package modifications
o A new task called phistogram was added to the PLOT package. This
task takes input from an image or from a list and allows full
control over the plotting parameters. This task replaces the
obsolete.mkhistogram task.
o A new task pradprof was added to the PLOT package. This task plots
or lists the radial profile of a stellar object. This task
replaces the obsolete.radplt task.
o A new task called gdevices was added to the package. gdevices
prints available information known about a particular class of
device. The default parameters are set up to print a list of the
available stdimage devices in the standard graphcap file.
o WCS support was added to the tasks graph, pcol(s), and prow(s). A
new parameter called "wcs" was added to define the coordinate
system to be used on the axis, either logical, physical or world.
Two additional parameters, "xformat" and "yformat", were also added
to allow the user to set the format for axis labels overriding any
values in the image header. The "xlabel" parameter values were
extended to include the special word "wcslabel" to select the WCS
label for the x axis from the image header.
o WCS support was added to the task implot. A new parameter called
"wcs" was added to define the coordinate system for plotting,
either logical, physical, or world. Two new ":" commands were also
added: ":w" allows the user to set a new WCS type interactively;
":f" allows the user to change the axis format, for instance, ":f
%H" would change the axis to read h:m:s, if the world coordinate RA
had been defined in the image header. The "space" key now prints
out the coordinate and pixel value information.
o graph has a another new parameter "overplot" that allows the user
to overplot multiple plots with different axes.
o The default parameters for "floor" and "ceiling" in contour and
surface were changed to INDEF.
3.1.7 PROTO package reorganization and modifications
This package has been reorganized. The PROTO package now resides
in the core system. Tasks in the old PROTO package that were general
image processing tools were kept in this new PROTO package. Tasks that
were more data reduction specific were moved to the new NPROTO package
in the NOAO package. The current PROTO package now contains the tasks
binfil, bscale, epix, fields, fixpix, hfix, imalign, imcentroid,
incntr, imfunction, imreplace, imscale, interp, irafil, joinlines,
suntoiraf, wcsedit, and wcsreset.
o The new task hfix was added to the package. This task allows the
user to extract the image headers into a text file, view or modify
this file with a specified command, and then update the image
header with the modified file.
o A new task called suntoiraf was added to this package. This is a
host dependent task that will most likely be useful only on Sun's.
This task converts a Sun raster file into an IRAF image.
o Two new tasks dealing with the WCS were added to this package,
wcsreset and wcsedit. wcsreset resets the coordinate systems in
the image header to the logical coordinate system. wcsedit allows
the user to modify the existing WCS or to create a new WCS and then
update the image header.
o A new version of bscale was added to the package. The task now
takes a list of input images and output images.
o A new version of imfunction was added to the package. The task
supports many more functions, for example log10, alog10, ln, aln,
sqrt, square, cbrt, cube, abs, neg, cos, sin, tan, acos, asin,
atan, hcos, hsin, htan, and reciprocal.
o imreplace was modified to support the "ushort" pixel type.
o The task join has been renamed joinlines.
3.1.8 Additions to XTOOLS and MATH
Programmers may be interested in the following new additions to the
XTOOLS and MATH libraries.
o The interactive non-linear least squares fitting package INLFIT,
developed by Pedro Gigoux at CTIO, was added to XTOOLS.
o The 1D image interpolation routines in the MATH library were
modified to support sinc interpolation.
3.1.9 Glossary of new tasks in the IRAF core system
Task Package Description
gdevices plot List imaging or other graphics devices
hfix proto Fix image headers
imcombine images Combine images pixel-by-pixel
phistogram plot Plot or print the histogram of an image or list
pradprof plot Plot or list the radial profile of an object
suntoiraf proto Convert Sun rasters into IRAF images
wcsedit proto Edit the image coordinate system
wcslab tv Overlay an image with a world coordinate grid
wcsreset proto Reset the image coordinate system
3.2 Changes to the NOAO Packages
An updated version of the observatory task has been installed at the
NOAO level. The task sets observatory parameters from a database of
observatories or from the parameters in the task itself. Many packages
and tasks within the NOAO packages that need information about where
the data was observed use information supplied by the observatory task.
3.2.1 ARTDATA package modifications
A new version of the ARTDATA package was released with IRAF version
2.9.1. A summary of those changes plus modifications since that update
are listed below. A more comprehensive list of the changes included in
V2.9.1 are discussed in IRAF Newsletter Number 10.
o A new task mkechelle has been added that creates artificial 2D
echelle images.
o A new task mkexamples has been added. The task is intended to
generate a variety of artificial images to be used in testing or
demonstrations.
o A new task mkheader adds or modifies image headers using a header
keyword data file.
o The mk1dspec task was modified to create multispec and echelle
format images, line by line.
o A parameter "header" was added to mkobjects, mknoise, mk1dspec, and
mk2dspec so that a header data file could be added to the output
image. Other header parameters are also added to the image header
such as gain, rdnoise, and exptime.
o A "comments" parameter was added to various tasks to
include/exclude comments in the header of the output image that
describe the data parameters.
3.2.2 ASTUTIL package modifications
o A new task gratings has been added to the package. This task
computes grating parameters; five of the seven parameters must be
specified at one time.
o A new task setjd has been added to the package. setjd computes the
geocentric, heliocentric, and integer local Julian dates from
information given in the headers of the input list of images. This
information is then listed or added to the image headers.
o A few changes were made to setairmass. The default value for
"update" was changed to "yes"; an update field was added to the
"show" messages; a warning is printed if both "show" and "update"
are set to "no" indicating that nothing was done.
3.2.3 DIGIPHOT package modifications
A new version of the DIGIPHOT package was installed. Some changes
were made to the existing APPHOT package used for aperture photometry,
and those are mentioned below. But three new packages have also been
installed, DAOPHOT, PHOTCAL, and PTOOLS.
DAOPHOT has been available for the past two years as an add-on package
known as TESTPHOT. It is now part of the distributed system. The IRAF
version of DAOPHOT, used to do photometry in crowded fields, has been a
collaborative effort with Peter Stetson and Dennis Crabtree of the
DAO. For the convenience of the many TESTPHOT users, changes to the
package since the last release of TESTPHOT are summarized below.
PHOTCAL is the photometric calibration package for computing the
transformations of the instrumental magnitudes output from APPHOT and
DAOPHOT to the standard photometric system. This package has been a
collaborative effort with Pedro Gigoux at CTIO.
PTOOLS is a package containing an assortment of tools for manipulating
the data files produced from APPHOT and DAOPHOT. If users are using
STSDAS TABLES format data files then they must first install the TABLES
layered package. This package can be obtained from either STScI or
NOAO (see iraf/contrib in the IRAF network archive).
3.2.4 DIGIPHOT.APPHOT package modifications
o The apselect task was replaced with the functionally equivalent
txdump task. txdump allows the user to select fields of data from
the output data files produced from APPHOT tasks and either simply
list the extracted data or save it in another file.
o A new task called pexamine has been added to the package. pexamine
is a general purpose tool for interactively examining and editing
the data files produced from tasks in either APPHOT or DAOPHOT.
This task is intended to complement the batch oriented task txdump.
o All of the APPHOT tasks will now query to verify the "datamin" and
"datamax" values, if these values are used by the tasks.
o The time of the observation, i.e. UT, can now be carried over into
the output data files, if a keyword in the image header contains
this information.
o If there is bad data within the photometry aperture a value of
INDEF will be stored in the data file for that magnitude.
3.2.5 DIGIPHOT.DAOPHOT (TESTPHOT) package modifications
This is a new package but for the convenience of the many users of
the TESTPHOT add-on package, the changes to TESTPHOT between its last
release and its installation into the DIGIPHOT package as DAOPHOT are
summarized below.
o The append, convert, dump, renumber, select, and sort tasks were
renamed to pappend, pconvert, pdump, prenumber, pselect, and psort.
o The "text" parameter was moved from daopars to the DAOPHOT package
parameters.
o All the DAOPHOT routines were modified so that "psfrad", "fitrad",
and "matchrad" are defined in terms of scale.
o The tasks allstar, group, nstar, peak, psf, and substar were all
modified to include "datamin" and "datamax" in their verify
routines.
o Support was added for a time of observation parameter to all the
appropriate DAOPHOT tasks. If present, this time will be carried
over into the output data files.
o All the DAOPHOT tasks except psf have been modified to accept lists
of input and output files.
o The new pexamine task was added to this package. pexamine is a
general purpose tool for interactively examining and editing the
data files produced from tasks in either APPHOT or DAOPHOT. This
task is intended to complement the batch oriented task txdump.
o Several changes were made to psf: the default PSF image header will
now hold up to 66 stars (but it is still dependent on the
environment variable min_lenuserarea); a check was added so that
the same star can not be added to the PSF twice; potential PSF
stars are now rejected if their sky or magnitude values are INDEF;
a check was added so that stars with INDEF valued positions are
treated as stars that were not found.
o group was modified so that the groups are output in y order instead
of in order of the size of the group.
o Both group and peak were modified so that stars with INDEF centers
are not written to the output file.
3.2.6 IMRED package modifications
The spectroscopic packages within the IMRED package have undergone
quite a bit of work and reorganization. The spectroscopic packages are
now ARGUS, CTIOSLIT, ECHELLE, HYDRA, IIDS, IRS, KPNOCOUDE, KPNOSLIT,
and SPECRED. These packages are a collection of tasks from APEXTRACT
and ONEDSPEC that are appropriate for the specific applications. All
these packages use the new versions of the APEXTRACT and ONEDSPEC
packages; the changes for APEXTRACT and ONEDSPEC are summarized below.
Earlier versions of all these packages were released as the NEWIMRED
add-on package. The NEWIMRED package is now defunct and the
distributed system contains the latest versions of these packages.
The spectroscopic packages now contain "DO" tasks that are scripts that
streamline the reduction process for the user. The "DO" tasks have been
modified for each particular application.
The ARGUS package is for the reduction of data taken with the CTIO argus
instrument. Its corresponding script task is doargus.
The CTIOSLIT package is similar to the ARGUS package except that is a
collection of spectroscopic tasks used for general CTIO slit
reductions. The doslit task allows for streamlined reductions.
The ECHELLE package has the addition of the doecslit task for simplied
echelle reductions. The dofoe task has been added for the reduction of
Fiber Optic Echelle (FOE) spectra.
The HYDRA package is used for the reduction of multifiber data taken at
KPNO. The dohydra task has been customized for streamlining nessie and
hydra reductions.
The KPNOCOUDE package has been customized for the KPNO Coude. The
doslit task allows the user to go through the reduction process with as
few keystrokes as possible. The do3fiber task is specialized for the 3
fiber (two arc and one object) instrument.
The KPNOSLIT package can be used for general slit reductions at KPNO.
The doslit task in this package is similar to that in the other
packages.
The SPECRED package is the old MSRED package, used for general
multispectral reductions. This is a generic package that can be used
for slit and multifiber/ aperture data. General doslit and dofibers
tasks are available.
Several of the spectroscopic packages has a special task called msresp1d
for creating 1d aperture response corrections from flat and throughput
data. This task is specialized for multiaperture or multifiber data.
All of the "DO" tasks have reference manuals that are available in the
network archive in the iraf/docs directory.
3.2.7 IMRED.CCDRED package modifications
o A new task ccdinstrument was added to the package. The purpose of
this task is to help users create new CCD instrument translation
files to use with the package.
o The combine task (as well as darkcombine, flatcombine, and
zerocombine) is the same task as the new imcombine task in IMAGES.
A few parameters were added for compatibility with the CCDRED tasks.
o A new parameter was added to ccdproc called "minreplace". Pixel
values in flat fields below the value for "minreplace" are replaced
by the same value (after overscan, zero, and dark corrections).
This avoids dividing by zero problems.
o The default computation and output "pixeltype" in the package
parameter for CCDRED are both real. Note that both can now be
specified separately.
o The default parameters for darkcombine and flatcombine have been
changed.
3.2.8 NOBSOLETE package added
This new package has been defined but currently no tasks reside in
it. This package will be used as a repository for tasks that have been
replaced by newer tasks in the NOAO packages. The NOBSOLETE tasks will
then be removed at the time of the next release.
3.2.9 NPROTO package modifications
The old PROTO package was divided into two separate packages, one
called PROTO that now resides in the IRAF core system and the other
called NPROTO that resides in the NOAO package. The current NPROTO
tasks are binpairs, findgain, findthresh, iralign, irmatch1d,
irmatch2d, irmosaic, linpol, and slitpic. The imedit, imexamine, and
tvmark tasks were moved to the IMAGES.TV package. The ndprep task was
moved to ONEDSPEC. All other tasks were moved to the PROTO package at
the core level.
o Two new tasks called findgain and findthresh were added to this
package. findgain determines the gain and read noise of a CCD from
a pair of dome flats and from a pair of bias/zero frames using the
Janesick method. findthresh estimates the background noise level
of the CCD using a sky frame and the gain and readnoise of the CCD.
o A new task called linpol has been added to the PROTO package. This
task calculates the pixel-by-pixel fractional linear polarization
and polarization angle for three or four images. The polarizer
must be set at even multiples of a 45 degree position angle.
o The task ndprep used for CTIO 2DFRUTTI reductions was moved to the
ONEDSPEC package.
o The task toonedspec was removed since its function can now be
performed by the onedspec.scopy task.
3.2.10 ONEDSPEC package reductions
There have been significant changes to the ONEDSPEC package. A
more detailed revisions summary is available in the IRAF network
archive as file onedv210.ps.Z in the subdirectory iraf/docs.
The new package supports a variety of spectral image formats. The older
formats can still be read. In particular the one dimensional
"onedspec" and the two dimensional "multispec" format will still be
acceptable as input. Note that the image naming syntax for the
"onedspec" format using record number extensions is a separate issue
and is still provided but only in the IMRED.IIDS and IMRED.IRS
packages. Any new spectra created are either a one dimensional format
using relatively simple keywords and a two or three dimensional format
which treats each line of the image as a separate spectrum and uses a
more complex world coordinate system and keywords. For the sake of
discussion the two formats are still called "onedspec" and "multispec"
though they are not equivalent to the earlier formats.
In addition, the one dimensional spectral tasks may also operate on two
dimensional images. This is done by using the "dispaxis" keyword in the
image header or a package dispaxis parameter if the keyword is missing
to define the dispersion axis. In addition there is a summing
parameter in the package to allow summing a number of lines or
columns. If the spectra are wavelength calibrated long slit spectra,
the product of the longslit.transform task, the wavelength information
will also be properly handled. Thus, one may use splot or specplot for
plotting such data without having to extract them to another format.
If one wants to extract one dimensional spectra by summing columns or
lines, as opposed to using the more complex APEXTRACT package, then one
can simply use scopy (this effectively replaces proto.toonedspec)
Another major change to the package is that spectra no longer need to
be interpolated to a uniform sampling in wavelength. The dispersion
functions determined by identify/reidentify can now be carried along
with the spectra in the image headers and recognized throughout the
package and the IRAF core system. Thus the WCS has been fully
implemented in the ONEDSPEC tasks and although much is to be gained by
this it can cause problems with earlier IRAF systems as well as making
it difficult to import data into non-IRAF software. But it is now
possible to use imcopy with an image section on a spectrum, for
instance, and have the wavelength information retained correctly.
A sinc interpolator is now available for those tasks doing geometric
corrections or interpolations. This option can be selected by setting
the "interp" parameter in the ONEDSPEC package parameters.
Other changes are summarized:
o The new task deredden corrects input spectra for interstellar
extinction, or reddening.
o The new task dopcor corrects input spectra by removing a specified
doppler velocity. The opposite sign can be used to add a doppler
shift to a spectrum.
o The new task fitprofs fits 1D gaussian profiles to spectral lines or
features in an image line or column. This is done noninteractively
and driven by an input list of feature positions.
o The task ndprep was moved to this package from the PROTO package.
ndprep is used for CTIO 2DFRUTTI reductions.
o The new task sapertures adds or modifies beam numbers and aperture
titles for selected apertures based on an aperture identification
file.
o The new task sarith does spectral arithmetic and includes unary
operators. The arithmetic can be performed on separate input
spectra or on apertures within a multispec format image.
o The task scombine has been added to the package. This new task is
similar to the new imcombine task in the IMAGES package but is
specialized for spectral data.
o The new task scopy copies subsets of apertures and does format
conversions between the different spectrum formats.
o The new task sfit fits spectra and outputs the fits in various ways.
This includes a new feature to replace deviant points by the fit.
Apertures in multispec and echelle format are fit independently.
o The tasks addsets, batchred, bswitch, coincor, flatdiv, flatfit,
subsets and sums were moved to the IIDS and IRS packages.
o The task combine was removed with the all new scombine suggested in
its place.
o The tasks rebin, sflip and shedit were removed from the package.
Rebinning of spectra can now be done with scopy or dispcor. scopy
and imcopy can now be used in place of sflip. And hedit is an
alternative for shedit.
o bplot and slist have been modified to support the multispec format.
o The task continuum now does independent fits for multispec and
echelle format spectra.
o There is now only one dispcor task doing the work of the three
previous tasks dispcor, msdispcor and ecdispcor. Several of the
parameters for this task have been changed. The old "recformat"
and "records" parameters for supporting the old onedspec format
have been removed except in the IIDS/IRS packages. A "linearize"
parameter has been added for setting the interpolation to yes or no
on output. The "interpolation" parameter was removed since this
information is now read from the package parameter "interp". The
"ignoreaps" parameter has been changed to "samedisp".
o identify and reidentify now treat multispec format spectra and two
dimensional images as a unit allowing easy movement between
different image lines or columns. The database is only updated upon
exiting the image. The "j", "k", and "o" keys have been added to
the interactive sessions to allow for scrolling through the
different lines in a two dimensional image. The database format
now uses aperture numbers rather than image sections for multispec
format spectra.
o The task specplot has new parameters "apertures" and "bands" to
select spectra for plotting in the multispec format.
o splot now allows deblending of any number of components and allows
simultaneous fitting of a linear background. Several cursor keys
have been redefined and colon commands have been added to fully
support the new multispec format and to add even more flexibility
to the task than before! Please see the help pages for details.
o The reidentify task is essentially a new task. The important
changes are an interactive review option, the ability to add
features from a line list, using the same reference spectrum for
all other spectra in a 2D reference image rather than "tracing"
(using the results of the last reidentification as the initial
guess for the next one) across the image, and matching image
lines/columns/apertures from a 2D reference image to other 2D
images rather than "tracing" again.
3.2.11 RV package added
This is a new package installed with IRAF version 2.10. A version
of this package, RV0, has been available as an add-on for the past year
or so. The package contains one main task fxcor that performs a Fourier
cross-correlation on the input list of spectra. The package supports
the multispec format.
3.2.12 TWODSPEC package modifications
The old MULTISPEC package that has resided in the TWODSPEC package
for years has been disabled. The code is still there for browsing,
however, and the package can be resurrected easily if needed.
The observatory task was moved to the NOAO package itself. The
setairmass task was moved to LONGSLIT. And the setdisp task is no
longer needed; task or package parameters are used in its place.
Changes to the APEXTRACT and LONGSLIT packages are summarized below.
3.2.13 TWODSPEC.APEXTRACT package modifications
A new version, version 3, of the IRAF APEXTRACT package has been
completed. A detailed revisions summary is available in the IRAF
network archive (file apexv210.ps.Z in iraf/docs).
There were three goals for the new package: new and improved cleaning
and variance weighting (optimal extraction) algorithms, the addition of
recommended or desirable new tasks and algorithms (particularly to
support large numbers of spectra from fiber and aperture mask
instruments), and special support for the new image reduction scripts in
the various IMRED packages.
The multispec format, the default image format for all spectroscopic
packages except IIDS and IRS, is either 2D or 3D (the 3D format is new
with this release). The 3D format is produced when the parameter
"extras" is set to yes in the extraction tasks. The basic 2D format,
which applies to the first plane of the 3D format, has each 1D aperture
spectra extracted in a line. Thus, the first coordinate is the pixel
coordinate along the spectra. The second coordinate refers to the
separate apertures. The third coordinate contains extra information
associated with the spectrum in the first plane. The extra planes are:
1: Primary spectra which are variance and/or cosmic ray cleaned
2: Spectra which are not weighted or cleaned
3: Sky spectra when using background subtraction
4: Estimated sigma of the extracted spectra
If background subtraction is not done then 4 moves down to plane 3.
Thus:
output.ms[*,*,1] = all primary spectra
output.ms[*,5,1] = 5th aperture spectrum which is cleaned
output.ms[*,5,2] = raw/uncleaned 5th aperture spectrum
output.ms[*,5,3] = sky for 5th spectrum
output.ms[*,5,4] = sigma of 5th spectrum
The following summarizes the major new features and changes.
o New techniques for cleaning and variance weighting extracted
spectra have been implemented.
o Special features for automatically numbering and identifying large
numbers of apertures have been added.
o There is no longer a limit to the number of apertures that can be
extracted.
o The new task apall integrates all the parameters used for one
dimensional extraction of spectra.
o The new task apfit provides various types of fitting for two
dimensional multiobject spectra.
o The new task apflatten creates flat fields from fiber and slitlet
spectra.
o The new task apmask creates mask images from aperture definitions
using the new pixel list image format. No tasks have yet been
written, however, to use this new mask image format.
o The new tasks and algorithms, aprecenter and apresize, will
automatically recenter and resize aperture definitions.
o The task apio is defunct. Its functionality has been replaced by
the APEXTRACT package parameters.
o The task apstrip has been removed.
o Minor changes have been made to the old "AP" tasks to accommodate
these new changes in the package; in some cases new parameters have
been added to the task and in other cases new cursor options have
been implemented. See the help pages for details.
3.2.14 TWODSPEC.LONGSLIT package modifications
o The "dispaxis" parameter was added to the package parameters. This
value is used unless DISPAXIS is in the image header.
3.2.15 Glossary of new tasks in the NOAO packages
Task Package Description
addstar daophot Add artificial stars to an image
allstar daophot Group and fit psf to multiple stars
apall apextract Extract 1D spectra
apfit apextract Fit 2D spectra
apflatten apextract Remove shapes from flat fields
apmask apextract Create an IRAF pixel mask from apertures
aprecenter apextract Recenter apertures
apresize apextract Resize apertures
ccdinstrument ccdred Review instrument translation files
centerpars daophot Edit the centering algorithm parameters
chkconfig photcal Check the configuration file
continpars rv Edit continuum subtraction parameters
daofind daophot Find stars using the DAO algorithm
daopars daophot Edit the daophot algorithms parameter set
datapars daophot Edit the data dependent parameters
deredden onedspec Apply interstellar extinction correction
do3fiber kpnocoude Process KPNO coude three fiber spectra
doargus argus Process ARGUS spectra
doecslit echelle Process Echelle slit spectra
dofibers specred Process fiber spectra
dofoe echelle Process Fiber Optic Echelle (FOE) spectra
dohydra hydra Process HYDRA spectra
dopcor onedspec Apply doppler corrections
doslit ctioslit Process CTIO slit spectra
doslit kpnocoude Process KPNO coude slit spectra
doslit kpnoslit Process slit spectra
doslit specred Process slit spectra
evalfit photcal Compute standard indices
filtpars rv Edit the filter function parameters
findgain nproto Estimate the gain and readnoise of a CCD
findthresh nproto Estimate a CCD's sky noise
fitparams photcal Compute transformation equations
fitprofs onedspec Fit gaussian profiles
fitskypars daophot Edit the sky fitting parameters
fxcor rv Radial velocities via cross correlation
gratings astutil Compute and print grating parameters
group daophot Group stars
grpselect daophot Select groups from a daophot database
invertfit photcal Compute the standard indices by inversion
istable ptools Is a file a table or text database file?
linpol nproto Polarization and Stoke's parameters
keywpars rv Translate image header keywords
mkcatalog photcal Type in a standard star catalog
mkconfig photcal Prepare a configuration file
mkechelle artdata Make artificial 1D and 2D echelle spectra
mkexamples artdata Make artificial data examples
mkheader artdata Append/replace header parameters
mkimsets photcal Prepare an image set file for mkNobsfile
mk(n)obsfile photcal Make a starfield observations file
mkphotcors photcal Check photometric corrections files
msresp1d argus Create 1D response spectra
msresp1d hydra Create 1D response spectra
msresp1d kpnocoude Create fiber response spectra
msresp1d specred Create 1D response spectra
nstar daophot Fit the psf to groups of stars
obsfile photcal Make a starfield observations file
pappend daophot Concatenate daophot databases
pappend ptools Concatenate apphot/daophot databases
pconvert daophot Convert text database to tables database
pconvert ptools Convert text database to tables database
pdump daophot Print fields from daophot databases
pdump ptools Print columns of daophot/apphot databases
peak daophot Fit the psf to single stars
pexamine apphot Examine or edit an apphot output file
pexamine daophot Examine and edit a daophot database
pexamine ptools Examine and edit an apphot/daophot database
phot daophot Compute sky values and initial magnitudes
photpars daophot Edit the photometry parameters
prenumber daophot Renumber stars in a daophot database
prenumber ptools Renumber a list of apphot/daophot databases
pselect daophot Select records from daophot database
pselect ptools Select records from apphot/daophot databases
psf daophot Fit the point spread function
psort daophot Sort a daophot database
psort ptools Sort a list of apphot/daophot databases
sapertures onedspec Set or change aperture header information
sarith onedspec Spectrum arithmetic
scombine onedspec Combine spectra
scopy onedspec Select and copy apertures
seepsf daophot Compute an image of the PSF
setjd astutil Compute and set Julian dates in images
sfit onedspec Fit spectra
substar daophot Subtract the fitted stars
tbappend ptools Concatenate apphot/daophot tables databases
tbdump ptools Print columns of tables databases
tbrenumber ptools Renumber apphot/daophot tables databases
tbselect ptools Select records from apphot/daophot databases
tbsort ptools Sort apphot/daophot tables databases
txappend ptools Concatenate apphot/daophot text databases
txdump apphot Dump selected fields of apphot output file
txdump ptools Print columns of text databases
txrenumber ptools Renumber apphot/daophot text databases
txselect ptools Select records from text databases
txsort ptools Sort apphot/daophot text databases
4. Programming Environment Revisions
4.1 Compatibility Issues
V2.10 IRAF requires that any local IRAF programs external to the
V2.10 system (such as layered packages or locally written tasks) be
fully recompiled if they are relinked against V2.10. The problem
arises only if the programs are relinked; external programs will
continue to run after V2.10 is installed, but linker errors will be
seen if the programs are relinked without being fully recompiled. This
is because the internal names of some important system routines were
changed in V2.10 to avoid name clashes with host system routines. For
example, the SPP procedure "rename" is now translated to "xfrnam" when
the SPP code it appears in is compiled.
As always, actual interface changes affecting existing source code were
very few. The macro "E" in <math.h> was renamed to "BASE_E" to minimize
the chance of an accidental name collision. The calling sequence for
the onentry procedure (ETC) was changed, but since this is a little used
system procedure very few tasks should be affected. A number of new
procedures were added to MTIO and the syntax of a magtape device has
changed; old applications should be modified to use the MTIO procedures
to operate upon magtape device names.
These and other revisions affecting the programming environment are
discussed in more detail below.
4.2 Portability Issues
The V2.10 UNIX/IRAF kernel now includes "#ifdef SYSV" support for
System V UNIX, making it easier to port IRAF to SysV based systems.
The UNIX/IRAF HSI (host system interface) is still not as portable to
UNIX variants as it could be, but at this point it is easier for us to
make the minor revisions required for a new port than to further refine
the HSI. The disadvantage is that it is harder than it should be for
people in the community to do their own IRAF ports, due to the level of
IRAF expertise required to tune the code. Someday we plan to generate
a generic-UNIX HSI. Note that these comments pertain only to the few
thousand lines of code in the HSI - the bulk of the IRAF code is 100%
portable (identical on all IRAF systems) as it has always been.
The recent port of IRAF to A/UX (the Apple Macintosh UNIX) is
interesting from a portability standpoint because we used the
publically available Fortran to C translator f2c plus the GNU C
compiler gcc to compile all the SPP and Fortran code in IRAF. This was
remarkably successful and means that the IRAF code is now portable to
any system with a C compiler. In the process of performing these
compilations a few dozen minor bugs were found by the static compile
time checking performed by f2c and gcc. The IRAF C code was run
through gcc with ANSI mode enabled, and hence should now be ANSI-C
compatible. The GNU debugger gdb proved to be an effective tool for
debugging of IRAF code compiled with gcc.
4.3 Software Tools
4.3.1 LOGIPC feature
A new facility has been added to UNIX/IRAF (all systems) for
debugging interprocess communication (IPC). This feature will also be
useful for debugging tasks standalone, particularly in cases where a
bug seen when running a task from the CL is difficult to reproduce when
the task is run standalone. The new facility allows one to carry out a
normal IRAF session while transparently logging all interprocess
communications. Each process can then be rerun individually using the
logged IPC to exactly duplicate the functioning of the process to,
e.g., examine the program operation in detail under a debugger.
The facility is this: if LOGIPC is defined in the host environment when
an iraf process is started, the process will create two files pid.in
and pid.out, where pid is the process id. Everything read from the IPC
input file is copied to the .in file, and everything written to the IPC
output (i.e., sent back to the CL) is copied to the .out file. This is
done only for connected subprocesses. It will work for any connected
subprocess, e.g., normal cached processes and graphics subkernels in
both foreground and background CLs, but not the i/o of the CL itself
since the CL is not driven by IPC.
The IPC streams saved are an exact binary copy of whatever was sent via
IPC, including the binary IPC packet headers, and binary graphics data,
and so on. All the IPC communications used to start up the process,
run zero or more tasks, and close the process down will be logged.
Most IPC traffic is textual in nature so it will usually be possible to
examine the IPC files with a file pager, although the results may be
less than satisfactory if binary data such as a graphics metacode
stream is logged. It is not necessary to examine the IPC files to use
them for process execution or debugging.
A particularly interesting use of this feature is to allow a process to
be run under the CL in the normal fashion, then rerun under a host level
debugger using the saved IPC input to duplicate the input and actions
of the process when run under the CL. For example,
% setenv LOGIPC
% cl
cl> dir
cl> logout
% unsetenv LOGIPC
will run the CL, saving the IPC of all subprocesses, e.g. x_system.e.
We can then run the system process manually, using the saved IPC input:
% $iraf/bin/x_system.e -c < pid.in
To run the process under adb or dbx, using the saved input:
% adb $iraf/bin/x_system.e
:r <pid.in -c
or
% dbx $iraf/bin/x_system.e
dbx> r -c < pid.in
Note that the redirection has to be given first for adb, and last for
dbx. This also works for gdb using the run command. Some
implementations of adb are picky about syntax and will not allow a
space between the "<" and the process id in the :r command.
Running a task in this way is not identical to running the task
standalone, e.g. using a dparam file for parameter input, because the
full runtime context of the process as run under the CL is reproduced
by LOGIPC but not when the task is run standalone. The differences are
subtle but can be important when trying to reproduce a bug seen when
the process is run under the CL. It is often the case that a bug seen
when a task is run from the CL will disappear when the task is run
standalone, but in most cases LOGIPC will duplicate the bug.
4.3.2 XC changes
The xc program (IRAF compiler-linker) underwent the following
changes for V2.10, in addition to the usual host-system specific
changes required to support new host compiler versions.
Multiple "-p pkgname" arguments are now supported. These are used when
compiling a module which uses multiple package environments, e.g.,
cl> xc -p noao -p tables foo.x
Each package environment may define package environment variables,
directories to be searched for include files and libraries, and so on.
Package environments are used by the IRAF layered packages.
Host libraries named on the command line using the -l switch (e.g.,
"-lresolv") are now searched after the IRAF system libraries, rather
than before as in previous versions of xc. If the host library is
referenced by absolute pathname it is still searched before the IRAF
libraries, since the command line order determines the search order.
4.3.3 SPPLINT code checker
A static code checker utility spplint has been developed for
checking SPP programs. This uses the SPP translation utilities in IRAF
to convert SPP to Fortran, f2c to generate C code, and lint to check
the C code. The code is checked in various ways at all phases of
translation. Some of the most useful checking are performed by f2c,
which checks the number and type of arguments in all calls to IRAF VOS
library procedures. In other words, spplint will determine if an IRAF
library procedure is called incorrectly, as well as perform a variety
of other checks.
The spplint utility is not included in the main IRAF release, but is
available separately. Contact the IRAF project for information on the
availability of this and other optional code development utilities.
4.3.4 Multiple architecture support
Multiple architecture support is a way of using multiple sets of
compiled program binaries within a single copy of IRAF. Multiple sets
of binaries are used to support different machine architectures (e.g.
sparc and Motorola), different compiler options (floating point
options, vectorization options, profiling options), different
compilers, and so on.
The command for checking the architecture of the IRAF core system or a
layered package has been changed from showfloat to arch to reflect the
fact that multiple architecture support is no longer used merely to
select the floating point option.
cl> mkpkg arch
sparc
The default architecture of the distributed system is "generic", meaning
no specific architecture has been set (the source tree is generic, not
configured for any particular architecture).
It is suggested that developers of layered software for IRAF adopt this
same convention in their root mkpkg files.
4.3.5 Package environments
All the HSI software utilities now permit multiple "-p pkgname"
package environment references. The host PKGENV environment variable
now also permits multiple package environments to be referenced, e.g.
% setenv PKGENV "noao tables xray"
The package names should be whitespace delimited (PKGENV is used to
avoid having to give the "-p" flags on the mkpkg or xc command line).
To successfully load a package enironment, the package root directory
must be defined in hlib$extern.pkg or in the user's host environment.
A host environment definition will override the value given in
extern.pkg.
4.3.6 Finding module sources
IRAF V2.10 includes a "tags" file in the IRAF root directory to aid
software development. This file contains an index to all procedures in
the IRAF VOS and HSI and can be used with host editors such as vi to
rapidly find and display the source for IRAF system procedures. Note
that the names of the kernel procedures are given in upper case, e.g.,
"ZOPNBF", whereas the names of the VOS procedures are given in lower
case. To use the tags file with vi, start the editor at the IRAF root
directory and while in the editor, type a command such as ":ta foo" to
view the source for procedure foo.
4.4 Programming Interface Changes
4.4.1 IEEE floating support
Modifications were made to the IEEE floating conversion routines in
the OSB package to support NaN mapping. This is a low level package
used by, e.g., the MII package in ETC. The interface as it currently
stands is summarized below.
ieepak[rd] (datum) # pack scalar value
ieeupk[rd] (datum) # unpack scalar value
ieevpak[rd] (native, ieee, nelem) # pack vector
ieevupk[rd] (ieee, native, nelem) # unpack vector
iee[sg]nan[rd] (NaN) # set/get NaN value
ieemap[rd] (mapin, mapout) # enable/disable NaN mapping
ieestat[rd] (nin, nout) # get count of NaN values
ieezstat[rd] () # zero NaN counters
The new or modified routines are ieesnan, ieegnan, ieemap, ieestat, and
ieezstat. By NaN (not-a-number) we refer collectively to all IEEE
non-arithmetic values, not just IEEE NaN. The routines ieesnan and
ieegnan set or get the native floating value used to replace NaNs or
overflows occurring when converting IEEE to the native floating format
(any floating value will do, e.g., zero or INDEF). If NaN mapping is
enabled, the ieestat routines may be used to determine the number of
input or output NaN conversions occurring since the last call to
ieezstat. Both real and double versions of all routines are provided.
The NaN mapping enable switch and statistics counters are undefined at
process startup; programs which use the IEEE conversion package should
call ieemap to enable or disable NaN mapping, and ieezstat to
initialize the statistics counters.
4.4.2 MATH libraries
The following changes were made to the MATH libraries in the IRAF
core system. Refer to the online help pages of the affected routines
for detailed information.
o The one-dimensional image interpolation library iminterp was
modified to add support for sinc interpolation.
o A new library nlfit was added for non-linear function fitting. An
interactive graphics front end to this was also added in XTOOLS.
o The name of the symbol E in <math.h> was changed to BASE_E to
minimize the chance of name clashes.
4.4.3 CLIO interface
The CLIO (command language i/o) interface underwent the following
changes for version 2.10 IRAF.
o A README file was added to the source directory containing an up to
date interface summary.
o The routines clgpset and clppset (get/put string valued parameter)
were renamed clgpseta and clppseta. The old procedures were
retained for compatibility but are now obsolete and may at some
point in the future disappear or be reused for some other function.
o Two new routines cllpset and clepset were added for listing and
editing psets (parameter sets).
The calling sequences for the new pset routines are as follows.
cllpset (pp, fd, format) # list pset
clepset (pp) # edit pset
These new routines are still considered experimental and should be
avoided or used with caution (they could change).
Internal to the CLIO code, the CLIO parameter caching package
underwent minor changes to add a new clc_compress routine and
improve pset handling, as part of the minilanguage support effort.
4.4.4 ETC interface
The ETC interface contains miscellaneous system interface routines.
The ETC interface underwent the following changes for V2.10 IRAF.
4.4.4.1 Calling sequence for onentry changed
The calling sequence for the onentry routine was changed. The new
calling sequence is as follows.
action = onentry (prtype, bkgfile, cmd)
The onentry procedure is an optional procedure which is called when an
IRAF process starts up. Normally the onentry procedure is a no-op,
passing control to the IRAF main in-task interpreter. Custom IRAF
applications, e.g., the CL, have a special onentry procedure which
replaces the default in-task interpreter.
The change made to the onentry calling sequence was the addition of the
additional argument cmd. This argument receives the command line used
to invoke the IRAF process at the host level. The application can
parse this command line to extract arguments, allowing the IRAF process
to operate as a host program (it is already possible to call any IRAF
task from the host level, but use of an onentry procedure allows the
in-task interpreter to be bypassed and gives the application control
over parsing the command line).
See etc$onentry.x for additional information on how to use this
procedure.
4.4.4.2 New onerror and onexit procedures
Two new procedures onerror_remove and onexit_remove were added.
These have the following calling sequences.
onerror_remove (proc) # remove posted onerror procedure
onexit_remove (proc) # remove posted onexit procedure
The new routines are used to remove onerror or onexit procedures posted
by a task during task execution. Such user procedures are called if
the task aborts (onerror procedures) or during normal task exit (onexit
procedures). Formerly there was no way to "unpost" the procedures
other than by the normal cleanup occurring during task termination.
4.4.4.3 New gqsort routine
A new quick-sort routine gqsort (generalized quick sort) was added.
This has the following calling sequence.
gqsort (x, nelem, compare, client_data)
gqsort is identical to the old qsort routine except for the addition of
the fourth argument client_data. This is an integer value which is
passed on to the compare procedure during sorting to compare two data
values. The compare routine is called as follows.
result = compare (client_data, x1, x2)
The new routine eliminates the need to use a common area to pass
information to the compare routine, as was often necessary with qsort.
4.4.5 FIO interface
The FIO interface (file i/o) underwent minor changes to fix some
bugs affecting pushed back data. The F_UNREAD file status parameter
will now correctly report pushed back data as well as any buffered
input file data. The F_CANCEL file set option will now cancel any
pushed back data.
4.4.5.1 Nonblocking terminal i/o
A new nonblocking form of raw mode terminal input has been added.
This permits polling the terminal for input without blocking
(suspending process execution) if no input is available. In a
character read, EOF is returned if no input is available otherwise the
character value is returned. An example illustrating the use of
nonblocking terminal i/o follows.
include <fset.h>
task foo
procedure foo()
int fd, ch
int getci()
begin
fd = STDIN
call fseti (fd, F_IOMODE, IO_RAW+IO_NDELAY)
repeat {
if (getci(fd,ch) == EOF)
call printf ("no pending input\r\n")
else {
call printf ("ch = %03o\r\n")
call pargi (ch)
}
call sleep (1)
} until (ch == '\004' || ch == 'q')
call fseti (fd, F_IOMODE, IO_NORMAL)
end
This sample program sets the terminal i/o mode to nonblocking raw mode
and polls the terminal once per second, printing the character value in
octal if a character is typed on the terminal, exiting when ctrl/d or
'q' is typed. Note that in raw mode characters such as ctrl/d or
ctrl/c are passed through as data and do not get mapped to EOF,
generate an interrupt, and so on. Raw mode i/o such as this will work
both when running a task under the CL and standalone, and in
combination with IRAF networking (e.g. to access a remote device).
Nonblocking terminal input is used in applications which run
continuously but which we wish to be able to control interactively.
4.4.6 FMTIO interface
The FMTIO interface (formatted i/o) is used to format output text
or decode input text. The V2.10 release adds two new printf output
formats, %H and %M. These are used to print numbers in
hours-minutes-seconds or minutes-seconds format and are equivalent to
the older output formats %h and %m except that the number is first
divided by 15. This converts degrees to hours allowing values given in
units of degrees to be printed as hours with just a change of the output
format. In other words, given a number N in units of degrees, %H would
print the number in hours-minutes-seconds, i.e., "hh:mm:ss.ss", whereas
%h would print the same number as degrees-minutes-seconds,
"dd:mm:ss.ss". The %m formats are similar except that only two of the
three fields are printed.
4.4.7 GTY interface
The GTY interface is a generalized version of that portion of the
older TTY interface dealing with termcap format files. The TTY code
which accesses termcap format files has been extracted to form the new
GTY interface, allowing arbitrary termcap format files to be accessed
by filename, unlike TTY which returns TTY descriptors given the termcap
or graphcap device name. GTY was contributed by Skip Schaller of
Steward Observatory.
gty = gtyopen (termcap_file, device, ufields)
gtyclose (gty)
cp = gtycaps (gty)
pval = gtyget[bir] (gty, cap)
nchars = gtygets (gty, cap, outstr, maxch)
The gtyopen call returns the GTY descriptor for the named device from
the file termcap_file. The ufields string may be either NULL or a list
of colon-delimited device capabilities, which will override the
corresponding capabilities in the device entry given in the termcap
file. If termcap_file is the null string ufields is taken to be the
device entry for the named device. The gtycaps routine may be used to
get the entire device entry as a string, whereas the gtyget and gtygets
routines are used to get the values of individual capabilities or
parameters.
4.4.8 MTIO interface
MTIO is the magtape i/o interface. The magtape i/o subsystem was
extensively revised for V2.10 IRAF, as documented earlier in this
revisions summary. The VOS level MTIO interface itself was not changed
other than to add a few new routines. The tapecap facility is new in
V2.10. The revisions to the host level magtape driver ZFIOMT required
that the calling sequences of some of the interface routines be changed.
4.4.8.1 MTIO applications programming interface
The current MTIO applications programming interface is summarized
below. Most of these routines are new: the old routines are mtfile,
mtopen, mtrewind and mtposition.
yes|no = mtfile (fname)
yes|no = mtneedfileno (mtname)
gty = mtcap (mtname)
mtfname (mtname, file, outstr, maxch)
mtparse (mtname, device, fileno, recno, attrl, maxch)
mtencode (mtname, maxch, device, fileno, recno, attrl)
fd = mtopen (mtname, acmode, bufsize)
mtrewind (mtname, initcache)
mtposition (mtname, file, record)
The mtfile routine is used to test whether the given filename is a
magtape file or something else, i.e., a disk file. mtneedfileno tests
whether a file number has been specified in mtname (e.g., to determine
whether or not to query for a file number parameter). mtfname takes
mtname and a file number and constructs a new magtape device name in
the output string. mtparse parses a magtape device name into its
component parts, and mtencode does the reverse. mtcap returns the GTY
descriptor of the tapecap entry for the device. gtyclose should be
used to free this descriptor when it is no longer needed.
Some older magtape applications programs parse the magtape device name
directly, looking for characters like `['. These old programs are
making assumptions about the form of a magtape device name which are
probably no longer true. Such old applications should be rewritten to
use the new MTIO procedures for all magtape device name operations.
4.4.8.2 MTIO system procedures
The MTIO interface also includes a number of procedures intended
for use in systems code. These are summarized in the table below.
mtallocate (mtname)
mtdeallocate (mtname, rewind_tape)
mtstatus (out, mtname)
mtclean (level, stale, out)
The only new routine here is mtclean. This is called by the mtclean
task in the V2.10 SYSTEM package and is used to scan the magtape status
file storage area to delete old magtape position status files. Prior
to V2.10 these files were stored in the user's UPARM directory, but in
V2.10 the files are stored in /tmp so that multiple IRAF sessions will
share the same tape position information. A special task is needed to
delete old position files in order to protect against, e.g., one user
deleting another user's device position file while the device is
actively in use.
4.4.8.3 Magtape driver procedures
All access to the physical magtape device is via the host level
IRAF magtape device driver ZFIOMT. The driver procedures had to be
revised for V2.10 to add support for the tapecap facility and to
accommodate changes in the way the device position is maintained. The
new driver procedures are summarized below.
zzopmt (device, acmode, devcap, devpos, newfile, chan)
zzrdmt (chan, buf, maxbytes, offset)
zzwrmt (chan, buf, nbytes, offset)
zzwtmt (chan, devpos, status)
zzstmt (chan, param, value)
zzclmt (chan, devpos, status)
zzrwmt (device, devcap, status)
The corresponding KI (kernel interface) routines were also modified to
reflect the new calling sequences. A result of this is that IRAF
networking for magtape devices is incompatible between V2.10 and
earlier versions of IRAF.
The host level device driver procedures are not normally called directly
in applications code (applications use mtopen). Refer to the comments
in the source file os$zfiomt.c for additional details.
4.4.9 MWCS interface
The MWCS interface provides world coordinate system support for the
IRAF system and applications. The main changes in the MWCS interface
for V2.10 were bug fixes and semantic changes, e.g. various
restrictions having to do with WCS attributes were removed, new
function drivers were added, and so on. These changes are documented
elsewhere in this revisions summary.
The only interface change affecting MWCS was the addition of the new
MWCS procedure mw_show.
mw_show (mw, fd, what)
This is used to dump a MWCS descriptor to the given output file, and is
useful for examining MWCS descriptors while debugging applications.
5. Getting Copies of this Revisions Summary
Additional copies of this revisions summary are available via
anonymous ftp from node iraf.noao.edu in the directory iraf/v210, or via
email from iraf@noao.edu.
IRAF (Mar90) V2.9 Revisions Summary IRAF (Mar90)
IRAF Version 2.9 Revisions Summary
April 10, 1990
1. Introduction
This document summarizes the changes in IRAF version 2.9. This was
primarily a development release intended to support applications
software development, hence the major changes were in the programming
environment, although there are important new features of interest to
general users too. Since IRAF V2.9 is primarily a development release,
it is not being released on all platforms, and it is expected that many
sites will not need to upgrade until IRAF V2.10 is available. Sites
interested in obtaining IRAF V2.9 should contact the IRAF project to
determine if the release is available for a particular host system. At
the present time, the release is being made available for all Sun
systems, for VAX/VMS, and for the DECstation running Ultrix.
What follows is a brief description of some of the new features
available in IRAF Version 2.9. This is not intended to be an
exhaustive list, but rather a brief summary of the major changes since
the last release of IRAF, Version 2.8, released in July 1989. More
detailed revisions notes are available in the system notes file,
iraf$local/notes.v29, as well as in the online revisions notes for the
various packages.
Users looking for information on a particular new package should note
that if the package is not mentioned in these release notes and
therefore is not included in IRAF V2.9, that does not necessarily mean
that it is not available. Most major reduction and analysis packages
are now made available for testing as user installable layered packages
before they are included in the standard distribution. For information
on the available add-on packages, contact the IRAF group, or check the
latest IRAF Newsletter.
This revisions summary is organized as follows:
1. Introduction
2. IRAF System Revisions
3. IRAF Package Revisions
3.1. Changes to the System Packages
3.2. Glossary of New Tasks in the IRAF System Packages
3.3. Changes to the NOAO Packages
3.4. Modifications and Additions to Calibration Data
3.5. Glossary of New Tasks in the NOAO Packages
4. Programming Environment Revisions
4.1. Changes to the Programming Utilities
4.2. Programming Interface Changes
2. IRAF System Revisions
2.1 IEEE to native floating point conversions
Support has been added to the programming interfaces (section
4.2.3) for converting between the IEEE floating point and native
floating point data formats, including both single and double
precision. The FITS programs in DATAIO (section 3.1.1) make use of
this, allowing floating point data to be exchanged in FITS format
without having to convert to type integer.
2.2 World coordinate system support
A major new VOS interface MWCS has been added to support general
world coordinate systems (WCS) and transformations thereon (section
4.2.1). This includes support for linear, piecewise linear or sampled
WCS, and general nonlinear WCS such as the tangent plane or gnomonic
projection.
If a FITS image is read into the system which has WCS information in the
header, the WCS will be retained in the IRAF image header and can be
used in coordinate transformations. The IMAGES tasks which move pixels
around have been modified to edit the WCS to reflect the transformation
(section 3.1.2). The image i/o system will automatically propagate the
WCS of an image to a new copy of the image, and will edit the WCS as
necessary if an image section is copied (this applies to all IRAF tasks
which operate upon images). The task RIMCURSOR in the LISTS package
has been rewritten to add support for coordinate transformations
(section 3.1.3), and can be used, e.g., to read out the RA and DEC of
objects on the image display using the image cursor, if the image has
the necessary WCS information in the image header.
Full integration of the new world coordinate facilities into all the
IRAF applications, e.g., the graphics tasks and the spectral reduction
packages, will take a year or longer due to the amount of software
involved. In V2.9 the IRAF spectral packages have not yet been
converted to use MWCS, and if MWCS is enabled it could alter the normal
behavior of these packages. IRAF V2.9 is therefore shipped with MWCS
disabled. What "disabled" means is that WCS information in the image
headers is not edited to reflect operations involving image sections,
or geometric transformations of images. Tasks such as RIMCURSOR which
use an already existing WCS will still work whether or not header
editing is disabled. If the spectral tasks will not be used and it is
desired that world coordinates be propagated correctly in image
transformations, MWCS header editing can be enabled in either of the
following ways.
The MWCS transformations are disabled by defining the variable "nomwcs"
in the IRAF environment. To globally enable MWCS by default for
everyone using the system, edit the file "hlib$zzsetenv.def" and
comment out the following line as shown (you want to add the leading #,
which will be missing in the distributed version):
#set nomwcs = yes
To enable MWCS header editing temporarily, for the current IRAF run:
cl> reset nomwcs = no
Detailed information on the coordinate systems defined by MWCS can be
obtained in the online system with the command
cl> phelp mwcs$MWCS.hlp fi+
Additional information is also given in the help page for RIMCURSOR.
2.3 IMFORT changes
The IMFORT interface (host level Fortran or C interface to the IRAF
image format) has undergone the following bug fixes and enhancements:
o A couple of bugs associated with the IMDIR (image pixel-file
directory) feature introduced in IRAF V2.8 have been fixed.
o Image clobber checking has been added. By default, if you create a
new image and another image with the same name already exists, the
image create will now return an error code leaving the existing
image unchanged. To override clobber checking in IMFORT programs,
restoring the previous behavior of the interface, define "clobber"
in your host environment.
o IMFORT will now perform a limited filename translation service
using the IRAF VOS filename translation code. This should allow
most IRAF filenames to be used as input to host level IMFORT
programs. Full VOS filename mapping is not provided, but filenames
containing upper case characters and multiple "." delimited fields
should be translated as in IRAF programs.
o On systems with multiple architecture support (e.g., Sun, Convex)
the FC task, used to compile and link IMFORT programs from within
the IRAF environment, is now a script rather than a simple foreign
task front end to XC. The purpose of the script is to see that all
the necessary IRAF and host level command line switches and
environment definitions (IRAFARCH, FLOAT_OPTION, etc.) are used.
Previously, users had to make these environment definitions
manually, and if they forgot the IMFORT program could fail to link
or execute.
o On most UNIX/IRAF systems, the host library -lU77 is now searched
automatically by FC when an IMFORT program is linked. This library
is not used by any of the IRAF code, but is required to link some
Fortran programs that might want to use IMFORT.
Users are encouraged to use FC to link their IMFORT programs. It is
possible to manually link against the IRAF libraries if you know what
you are doing, but the location of the libraries and the required host
level command line switches vary for different systems and for
different architectures of a single system, and it is easy to make
mistakes.
2.4 MKIRAF now copies login.cl to login.cl.OLD
On UNIX/IRAF systems, the MKIRAF command will now copy any existing
login.cl file to login.cl.OLD, so that, for example, you can more
easily merge any custom changes back in after running MKIRAF. On
VMS/IRAF systems a new file version is created, as before.
2.5 Local additions to termcap/graphcap
The termcap and graphcap device capability files have been
reorganized with a section at the top for local additions. It is
recommended that any locally added entries be made in this area, to
simplify future system updates. The local additions can then be simply
transferred to the new version of the file when a new version of IRAF
is installed (any entries which are modified versions of standard
entries should always be checked to see if anything has changed in the
distributed version).
2.6 BIN directories now smaller
On systems with multiple architecture support, the architecture
save file OBJS.arc stored in the BIN directory for each architecture is
now maintained as a compressed file. In a typical case this reduces
the size of the file by about a factor of two, saving 1-2 Mb of disk
space in each BIN directory.
2.7 Various system buffers increased in size
The layered software support in IRAF V2.8 (extern.pkg and all that)
had a problem with very long helpdb environment strings, limiting the
number of external packages which could be defined. To fix this
problem, various buffers were increased in size all over the system.
The maximum length of an environment variable such as helpdb is now 960
characters (12 80 character lines of text). String parameters to tasks
can also be larger, and the system is more resistant to problems when
size limits are exceeded. Foreign task commands, OS escapes, etc., can
all be larger now. The current limit on such strings is about 1024
characters, and is defined at sysgen time by the new system parameter
SZ_COMMAND in hlib$iraf.h.
2.8 Shared library versions
The Sun/IRAF shared library mechanism was modified to add support
for shared library versions. The result is that when you install IRAF
V2.9, which has a different shared library than V2.8, any local
programs or other layered software linked under V2.8 will continue to
run, because both the old V2.8 shared library and the new V2.9 shared
library are included in V2.9 (with different version numbers).
Although old programs will continue to run with V2.9, it is recommended
that they be relinked eventually to take advantage of the many features
and bug fixes provided by V2.9. In the case of very large packages,
e.g., STSDAS 1.0, it may be wise to wait until the latest release can
be obtained and installed before relinking, as the old version will not
have been tested under IRAF V2.9 (which of course, didn't exist back
then).
2.9 File pager enhancements
The system file pager, used in the PAGE task, the new PHELP task,
and other places, has undergone the following enhancements.
o The N and P keys, used to move to the next or previous file when
paging a list of files, now have a dual meaning: when paging a
single file containing multiple formfeed delimited pages, the keys
will move to the next or previous page in the file. This feature
is used in the new PHELP task to page a large file containing,
e.g., all the HELP pages for a package.
o A limited upscrolling capability is now supported, e.g., if you hit
the 'k' key while in the pager, the screen will be scrolled up one
line in the file being paged. This feature may not be supported
for some terminals, in which case the entire screen is redrawn at
the new file location.
2.10 STF image kernel enhancements
Extensive work has been done on the STF image kernel in this
release (the STF kernel allows IRAF to access the Space Telescope image
format directly). The changes included the following.
o Header file caching. STF images often have quite large FITS
headers which can be time consuming to read. A header file caching
scheme is now used to optimize the kernel in cases where the same
imagefile is repeatedly accessed, e.g., when successively reading
each element of a large group format image. By default up to 3
header files will be cached; this default should be fine for most
applications. If necessary the number of cache slots can be
changed by defining the integer variable "stfcache" in the IRAF
environment (the builtin maximum is 5 cached headers per process).
o The semantics of the kernel regarding header updates have changed.
STF images differ from other IRAF images in that they may consist
of a group of images all in the same file, with each individual
image having its own header (the group header), plus a single
global FITS header shared by all images in the group. This is no
problem in a read operation, but in a write or update operation
there can be problems since parameters cannot be added to or
deleted from the individual group headers. The new semantics
regarding STF image header updates are as follows: 1) when updating
the header of a multigroup image (not recommended) only the group
header is updated, and attempts to add new parameters are ignored;
2) when updating the header of an image containing a single group,
both the group header and the FITS header are updated.
As a result of these changes, the behavior of a single group STF
image is now identical to that of a regular IRAF image. It is
recommended that multigroup STF images be treated as read only if
possible, creating only single group images during interactive
processing (except when running a program that is explicitly
designed to create multigroup images).
o The kernel was modified to work with the new MWCS (world coordinate
system) interface. The image section transformation is now
performed by MWCS rather than by the STF kernel.
o A number of minor changes were made to the way the group parameter
block (GPB) cards are maintained in the IRAF image descriptor. The
comments on GPB definition cards are now preserved. Restrictions
on the grouping of GPB cards in the header have been removed.
o A number of bugs were fixed and restrictions removed, e.g., the
size of a header is no longer limited to 32767 characters (404
lines).
The IRAF core system and NOAO science applications were extensively
tested with both single and multigroup STF images using the new
kernel, and we now feel that it is safe to use the STF image format
with these tasks, although the regular format is preferred if there
is no special reason to use the STF format (the regular format is
more efficient).
2.11 QPOE (event list image format) enhancements
The QPOE image kernel, used for event list data (photon counting
detectors, e.g., X-ray satellites such as ROSAT) underwent the
following changes.
o MWCS (world coordinate system) support has been added (section
4.2.2). This provides a consistent coordinate system despite,
e.g., the blocking factor, rect, or image section used to construct
an image matrix from an event list.
o When opening a QPOE file as an IRAF image, the runtime filter
expression used to create the image matrix is now saved in the
parameter QPFILTn in the image header (multiple cards are used for
long expressions).
o Region masks of arbitrary complexity and size can now be used to
mask the event list when reading time-ordered or unordered
(unindexed) event lists. This is done using the new PLRIO package
(section 4.2.5) which provides the capability to efficiently random
access large image masks of arbitrary complexity.
o Unmatched brackets, braces, or parentheses are now reported as an
error by the filter expression parser (this can occur even with a
valid expression, e.g., due to truncation of the expression
string). A reference to an undefined keyword, e.g., due to a
spelling error, is now detected and reported as an error. Any
errors occurring during expression parsing will now result in
termination of the calling task, unless caught in an error handler.
o A number of bugs were fixed.
2.12 Changes affecting image display in VMS/IRAF
A new version of Nigel Sharp's UISDISPLAY program, for image
display on VMS systems running UIS, has been installed in
"iraf$vms/uis". An executable for an early version of the SAOIMAGE
display program for the X window system, written by Mike VanHilst
(SAO), and ported to VMS by Jay Travisano (STScI) has been placed in
the directory "iraf$vms/x11". An executable for a VMS version of XTERM
(the X window terminal emulator, ported to VMS by Stephan Jansen), is
also in this directory. We wanted our VMS users to have access to
these programs, although more development work and testing is needed
before we can offer good support for X window based image display and
graphics on VMS. A more comprehensive package providing enhanced
capabilities should be available as an add-on later this year.
3. IRAF Package Revisions
The most notable changes to the tasks in the IRAF packages are
summarized below. Further information may be obtained by reading the
help page for each task, or by paging the revisions file for a
particular package. For example, to page the revisions for the DATAIO
package:
cl> phelp dataio.revisions op=sys
3.1 Changes to the System Packages
3.1.1 Modifications to tasks in the DATAIO package
o The RFITS and WFITS tasks have been modified to add support for the
IEEE floating point format. The "bitpix" parameter in WFITS can be
set to -32 or -64 to specify real or double precision IEEE floating
numbers on output. RFITS recognizes these same values in the
bitpix keyword in the FITS header on input and converts the data
accordingly. Note that this option must be selected by the user as
the defaults for writing a FITS tape have not changed. The user is
cautioned that support for the IEEE floating formats is a new
feature of FITS and may not be supported by all FITS readers.
o RFITS was modified so that the "iraf_file" parameter can be a list
of output images or a image root name.
3.1.2 Modifications to tasks in the IMAGES package
o MWCS (world coordinate system) support was added to those tasks in
the IMAGES package which change the geometry of an image, i.e.,
IMSHIFT, SHIFTLINES, MAGNIFY, IMTRANSPOSE, IMCOPY, BLKREP, BLKAVG,
ROTATE, IMLINTRAN, REGISTER, and GEOTRAN (REGISTER and GEOTRAN only
support simple linear transformations). If one of these tasks is
used to linearly transform an image, the world coordinate system
(WCS) in the image header will be updated to reflect the
transformation. Note that MWCS is disabled by default in IRAF
V2.9, and must be explicitly enabled to allow these tasks to edit
the image header to update the WCS (see section 2.2).
o The IMSTATISTICS task was modified. The "verbose" parameter was
renamed "format" with the default being set to "yes" (fixed format
with column labels). Otherwise the fields are printed in free
format with 2 blanks separating the fields. The name of the median
field has been changed to "midpt".
o The IMHISTOGRAM task has a new parameter called "hist_type" that
gives the user the option of plotting the integral, first
derivative, or second derivative of the histogram instead of the
normal histogram.
3.1.3 Modifications to tasks in the LISTS package
o The RIMCURSOR task in the LISTS package was completely rewritten to
add MWCS support, so that coordinates may be output in any user
specified coordinate system defined by the WCS information in the
image header of the reference image. For example, if an image with
a TAN projection WCS is loaded into the image display, RIMCURSOR
may be used to print the right ascension and declination at the
location defined by the image cursor. Refer to the help page for
details.
3.1.4 Modifications to tasks in the PLOT package
o A new graphics kernel task IMDKERN (written by Zolt Levay at STScI)
has been added to the PLOT package. The new graphics kernel allows
the graphics output of any task to be plotted as a graphics overlay
on the image display. As with the other graphics kernels, this may
be done by calling the IMDKERN task directly, but is more often
done by specifying the image display (e.g., device "imd") as the
output device when running a graphics task. Refer to the help page
for details.
o The CONTOUR task was modified so that it could be used with IMDKERN
to overlay contour plots on the image display. If the parameters
fill=yes and perimeter=no are set the contour plot is scaled to
fill the entire device viewport and all axis and plot labeling is
disabled. If the image being displayed also fills the entire
device viewport (display frame) then the contour plot will be drawn
to the same scale as the displayed image. Refer to the help page
for details.
o Several tasks in the PLOT package were modified to allow use with
image specifications containing brackets, e.g., group format
images, QPOE filter expressions, and image sections. The tasks
modified were PROW, PROWS, PCOL, PCOLS, SURFACE, and CONTOUR.
o An option was added to the PVECTOR task to output the vector (cut
through the image at an arbitrary angle and center) as a text file
or image, rather than plotting the vector.
3.1.5 Modifications to tasks in the SYSTEM package
o A new task PHELP (paged help) was added to the SYSTEM package.
PHELP is a script task front end to HELP which collects the output
of HELP in a scratch file and pages it with the system pager,
allowing one to randomly skip around to read the help text. Note
that paging of all the help pages in a package is supported, e.g.,
cl> phelp images.*
would page all the help files for the IMAGES package.
o The NEWS task was completely rewritten, and is now used to page the
revisions summary for the current and previous releases. In other
words, one can now type NEWS to find out what is new in the current
release.
o The GRIPES task was modified to send mail to iraf@noao.edu or
5355::iraf. The IRAF site administrator may want to check this
script for compatibility with the local mail system.
3.2 Glossary of New Tasks in the IRAF System Packages
Task Package Description
imdkern plot Image display device (IMD) graphics kernel
news system Summarize what is new in the current release
phelp system Paged HELP: collects and pages the output of HELP
rimcursor lists Read image cursor position in world coordinates
3.3 Changes to the NOAO Packages
3.3.1 New NOAO Packages
A new package ARTDATA, used to generate artificial data, has been
added to the NOAO packages. ARTDATA includes tasks for the generation
of star fields, optionally containing galaxies, and one and two
dimensional spectra as well as simple test pattern images. The tasks
GALLIST and STARLIST provide many options for producing lists of
galaxies or stars that can then be used by the task MKOBJECTS to produce
output images. The tasks MK1DSPEC and MK2DSPEC provide tools for making
artificial spectral data. The task MKNOISE allows the user to add
readout noise, poisson noise and/or cosmic ray events to new or already
existing images. The task MKPATTERN allows the user to make images
from a choice of patterns.
3.3.2 Modifications to Existing NOAO Packages
3.3.2.1 The ASTUTIL package
o The task SETAIRMASS in the ASTUTIL package was modified so that it
now precesses the coordinates to the epoch of the observation.
3.3.2.2 The DIGIPHOT.APPHOT package
o A new task APTEST was added to the DIGIPHOT.APPHOT package that
tests the execution of the package. Output files are generated that
the user can review.
o Two new parameters were added to DATAPARS, "datamin" and "datamax".
Pixels outside this range are rejected from the sky fitting
algorithms and from the non-linear least square fits in FITPSF and
RADPROF.
o An "update" parameter was added to all of the APPHOT tasks. If the
"verify" parameter is set to "yes" and the task is run in
noninteractive mode update=yes will update the critical parameters
in their respective parameter sets.
o Four new parameters, "airmass", "xairmass", "filter", and "ifilter",
were added to the DATAPARS task. These parameters provide the user
the option of having the filter and airmass quantities from the
image headers to be carried over into the APPHOT database files for
later transmission to calibration programs.
o A new algorithm "mean" was added to the sky fitting options.
o A setup menu mode was added to all the APPHOT tasks. When the user
types "i" in interactive mode a setup menu is presented rather than
a fixed set of predefined commands.
3.3.2.3 The IMRED.IRRED package
o The APSELECT task (from the APPHOT package) has been made visible.
o The image i/o for IRMOSAIC, IRALIGN, IRMATCH1D, and IRMATCH2D has
been optimized so things should run much faster. There is now an
option to trim each section before insertion into the output
image. The actions of these tasks can now optionally be output to
the terminal.
3.3.2.4 The IMRED.MSRED package
o A task called MSBPLOT was added to the IMRED.MSRED package. This
task allows the user to plot a range of lines in multispec images
in batch mode.
3.3.2.5 The ONEDSPEC package
o Several modifications were made to the ONEDSPEC package. These
changes affect all of the IMRED packages that include these tasks
as well.
o The equivalent width measurement using the "e" keystroke in SPLOT
is now computed using the ratio of the spectrum to the continuum.
The previous approximation is included in the logfile for
comparison.
o The DISPERSION task will now add CDi_j (CD matrix) keywords to the
image header as an alternative way of expressing the dispersion
function. If the keywords W0 and WPC or CRVALn and CDELTn are not
in the image header the tasks reading this information for setting
the wavelength (IDENTIFY, SENSFUNC, SPLOT, and SPECPLOT) will look
for the CDi_j keywords. This change should have no affect on the
NOAO applications but provides compatibility with STSDAS
applications using the new MWCS interface provided with IRAF
version 2.9.
o The call to the CALIBRATE task in the script task BATCHRED was
modified so that the "extinct" parameter is always set to "yes".
Since CALIBRATE checks to be sure the data has not been previously
extinction corrected this simple change provides more flexibility.
3.3.2.6 The PROTO package
o Two new tasks, IMALIGN and IMCENTROID, were added to the package.
IMCENTROID computes a set of relative shifts required to register a
set of images. The task IMALIGN both computes the shifts and
aligns the images.
o The JOIN task (previously a simple script) has been replaced by a
compiled version which removes many of the restrictions of the
previous version.
o The IR tasks have been modified as mentioned above under the
IMRED.IRRED section (section 3.3.2.3).
o The TVMARK task was modified to permit deletion (the "u" key) as
well as addition of objects to the coordinate file. Another cursor
keystroke, the "f" key, was added allowing the user to draw a
filled rectangle.
3.3.2.7 The TWODSPEC.LONGSLIT package
o Tasks in the TWODSPEC.LONGSLIT package that are used for setting
wavelength information (EXTINCTION, FLUXCALIB, and TRANSFORM) were
modified for the CDi_j keywords as outlined above for ONEDSPEC.
3.4 Modifications and Additions to Calibration Data
The calibration data used by some of the tasks in the TWODSPEC,
ONEDSPEC, and many of the IMRED packages are kept in a directory called
ONEDSTDS in noao$lib. The current contents of this directory are best
summarized by paging through its README file, e.g.,
cl> page noao$lib/onedstds/README
A new directory spec50redcal in "noao$lib/onedstds" has been added
containing flux information for standard stars. The data in this list
are from Massey and Gronwall, Ap. J., July 20, 1990.
3.5 Glossary of New Tasks in the NOAO Packages
Task Package Description
aptest apphot Run basic tests on the apphot package tasks
gallist artdata Make an artificial galaxies list
imalign proto Register and shift a list of images
imcentroid proto Compute relative shifts for a list of images
mk1dspec artdata Make/add artificial 1D spectra
mk2dspec artdata Make/add artificial 2D spectra
mknoise artdata Make/add noise and cosmic rays to 1D/2D images
mkobjects artdata Make/add artificial stars and galaxies to 2D images
mkpattern artdata Make/add patterns to images
msbplot msred Batch plots of multispec spectra using SPLOT
starlist artdata Make an artificial star list
4. Programming Environment Revisions
4.1 Changes to the Programming Utilities
4.1.1 MKPKG changes
The MKPKG utility can now substitute the contents of a file back
into the input stream, as a special case of the macro replacement
syntax. For example, the sequence
abc$(@file)def
would be translated as
abc10def
if the file "file" contained the string "10". The replacement is
performed by inserting the contents of the file back into the input
stream, replacing sequences of newlines, spaces, or tabs by a single
space, and omitting any trailing whitespace.
The "-p <pkg>" argument to MKPKG, XC, and so on loads the environment
of the named package pkg, to define the package environment variables,
load the mkpkg special file list, define the directories to be searched
for global include files and libraries, and so on. Multiple "-p"
arguments may be given to load multiple package environments. What is
new is that if pkglibs is defined in the environment of a package to
list the package library directories to be searched (the usual case),
and multiple package environments are loaded, successive redefinitions
of pkglibs will add to the list of directories to be searched, rather
than redefining the old list as each new package environment is
loaded. For example, if two package environments are loaded, and each
defines its own library, both libraries will be searched.
4.1.2 Generic preprocessor
A minor change was made to the generic preprocessor which affects
how strings such as "FOO_PIXEL" are translated. In the usual case, the
preprocessor replaces all occurrences of "PIXEL" by "int", "real", or
whatever the actual datatype is. The translation is now context
sensitive. Rather than translating "FOO_PIXEL" as "FOO_int" (e.g.,
"MII_PIXEL" -> "MII_int"), the type name will now be output in upper
case if the rest of the name in which it occurs is upper case. Hence,
a string such as "MII_PIXEL" will now be translated as "MII_INT". This
allows the use of generic constructs to symbolize SPP macros.
4.1.3 SPP changes
The language constant ARB, formerly defined as 32767, is now treated
differently depending upon how it is used. In a declaration of an array
argument, ARB is replaced in the output Fortran by a "*", e.g., "int
data[ARB]" becomes "INTEGER DATA(*)". In an executable statement, ARB
is replaced by a very large ("arbitrarily" large) integer value, e.g.,
to define a DO-loop which is to loop an arbitrary number of times. If
ARB is mistakenly used to dimension an array which is a local variable
rather than an argument, the SPP translator will now detect and report
the error.
4.1.4 Interactive development and the process cache
Whenever a CL task is run and the process containing the task is
already idling in the CL process cache, the CL will now check to see if
the modify date on the process executable has changed, and restart the
process if the executable has been modified. For example, when doing
software development from within the CL and a process is alternately
relinked and tested, the CL will now automatically detect that the
process has been relinked and will run the new process, without any
need to manually flush the process cache.
4.2 Programming Interface Changes
4.2.1 New MWCS interface (world coordinate system support)
A major new VOS interface MWCS, providing general facilities for
linear and nonlinear world coordinate systems, has been added to the
programming environment and is used in IRAF V2.9 in IMIO, IMAGES, and
other parts of the system. MWCS is intended for use in scientific
applications as well as in system code such as IMIO, hence is of
potential interest to anyone developing software within the IRAF
environment. The source directory is "mwcs" and the interface is
documented in the file "mwcs$MWCS.hlp". Users should be aware that,
although the new interface addresses the general WCS problem and has
been carefully designed, a second version of the interface is planned
and the current interface is not yet a "frozen" interface.
4.2.2 QPOE interface changes
The QPOE (event list image) interface has been extended to add
routines to store MWCS objects in the QPOE header. By default, there
is one MWCS per QPOE file, stored encoded in a machine independent
binary format in a variable length array qpwcs of type opaque. The new
routines are as follows:
mw = qp_loadwcs (qp)
qp_savewcs (qp, mw)
mw = qpio_loadwcs (io)
The routines qp_savewcs and qp_loadwcs merely save a MWCS in the QPOE
header, or load a previously saved one. The QPIO (event i/o) routine
qpio_loadwcs is like qp_loadwcs, except that it will also modify the
Lterm of the MWCS to reflect any blocking factor or "rect" specified in
the filtering expression when the event list was opened. The new
routine is called automatically by QPF and IMIO whenever a QPOE event
list is opened under image i/o, making the physical coordinate system
of the image matrix the same as physical event coordinates.
The calling sequences of the qp_add and qp_astr routines, used to
conditionally add or update header parameters, have been changed (so far
as we could determine very few programs exist yet which use these
routines, so we decided to risk an interface change). The change made
was to add a comment argument. This change was motivated by the
observation that people would not use the routines but would instead
use lower level routines, in order to be able to set the comment field
if the parameter has to be added to the header.
4.2.3 IEEE support routines added
Routines for IEEE floating to native floating conversions have been
added to the MII and OSB interfaces. The new MII routines are as
follows:
nelem = miiread[rd] (fd, spp, maxelem)
miiwrite[rd] (fd, spp, nelem)
miipak[rd] (spp, mii, nelems, spp_datatype)
miiupk[rd] (mii, spp, nelems, spp_datatype)
The miiread and miiwrite routines are like their FIO counterparts,
except that they are used only with data of the indicated type, and
perform the IEEE to native floating conversion (or vice versa) as part
of the i/o operation. The miipak and miiupk routines pack
(native->IEEE) and unpack (IEEE->native) arrays of the indicated type.
The lowest level conversion routines are the OSB routines, which are
what the MII routines use to perform the lowest level translation.
ieepak[rd] (datum)
ieeupk[rd] (datum)
ieevpak[rd] (native, ieee, nelem)
ieevupk[rd] (ieee, native, nelem)
iee[sg]nan[rd] (NaN)
The ieepak and ieeupk routines transform a single scalar value in
place, while the ieevpak and ieevupk routines transform vectors (note
that the package prefix is "iee", not "ieee"). In-place vector
conversions are permitted. Since IRAF does not support the IEEE
not-a-number formats, NaN, Inf etc. values are converted to a legal
native floating value on input. The native floating value to which
NaNs are mapped (default zero) may be globally set with ieesnan.
On some systems, e.g., the VAX, the low level conversion routines may be
written in assembler or machine dependent C. If so, the source file
actually used by the system will be found in the "host$as" directory.
4.2.4 New routine GETLLINE added to FIO
A new routine getlline (get long line) has been added to FIO. This
is similar to getline, except that it will reconstruct arbitrarily long
newline delimited lines of text, whereas getline returns at most
SZ_LINE characters.
nchars = getlline (fd, outstr, maxch)
The new routine should not be confused with the old routine getlongline,
a higher level routine which performs a similar function, but which
also ignores comment lines and help blocks, and maintains a line
counter.
4.2.5 Modifications to PLIO/PMIO
A new routine p[lm]_sectnotconst has been added to PLIO and PMIO
(the pixel list and image mask interfaces). As the name suggests, the
routine tests whether a given rectangular section of the mask is all at
the same value, and if so returns the mask value as an output argument.
bool = pl_sectnotconst (pl_src, v1, v2, ndim, mval)
A new subpackage PLRIO has been added. This is used to efficiently
random access any 2D plane of an existing pixel list or image mask.
plr = plr_open (pl, plane, buflimit)
plr_setrect (plr, x1,y1, x2,y2)
mval = plr_getpix (plr, x, y)
plr_getlut (plr, bufp, xsize,ysize, xblock,yblock)
plr_close (plr)
The mask is opened for random access on a special descriptor which
incorporates a scaled, active 2D lookup table. Most subsequent
plr_getpix calls will return the given mask value directly from the
table with very little overhead; only if the referenced pixel occurs in
a region too complex to be described by a single table entry is the
value computed by direct evaluation of the mask. A special 2D binary
recursive algorithm (using pl_sectnotconst above) with log2(N)
performance is used to calculate the scaled lookup table. These
algorithms provide efficient table generation and random mask pixel
access even for very large masks.
IRAF (Mar90) V2.8 Revisions IRAF (Mar90)
IRAF Version 2.8 Revisions Summary
June 30, 1989
1. Introduction
This revisions notice coincides with the release of version 2.8 of
IRAF. The V2.8 release is a general release for all supported IRAF
hosts.
The following is a brief description of some of the new features
available in IRAF Version 2.8. This is not intended to be an
exhaustive list, but rather a brief summary of the major changes since
the last major release of IRAF Version 2.5 in July 1987 and subsequent
intermediate releases primarily to support Sun/IRAF: IRAF Version 2.6
(February 1988), IRAF Version 2.6+ (March 1988), and IRAF Version 2.7
(December 1988).
More detailed revisions notes are available in the system notes files in
the iraf$doc and iraf$local directories, as well as in the online
revisions notes for the various packages.
2. IRAF System Revisions
This document highlights the most notable revisions made to the
IRAF core system software for Version 2.8. This is only a revisions
summary; no attempt is made to provide detailed technical documentation
for each revision, nor is there any attempt to exhaustively summarize
all revisions. A complete record of all core system revisions will be
found in the System Notes for V2.8. Additional information on some of
the topics covered below will be found in the various Installation
Guides and Site Manager's Guides, and in the IRAF User and Technical
Documentation manual sets.
2.1 Copyright notice
Subject to AURA and NSF approval, the IRAF software will be
copyrighted sometime during 1989. As a first step in this process, a
copyright notice has been added to all core system source files. The
notice reads as follows: "Copyright(c) 1986 Association of Universities
for Research in Astronomy Inc". We will also be adding a file called
COPYRIGHT to the distribution stating the terms of the copyright and
associated licensing agreement for the software.
The intent of this action is solely to protect the software from
unauthorized commercial exploitation, and the copyright grants, or will
grant, the right to copy, modify, and redistribute the IRAF software
provided the original copyright notice remains intact, the software is
made available in source form, and the rights we grant are passed on
with the software. We wish to prevent others, especially commercial
firms, from copyrighting IRAF software in their own name and possibly
taking away the rights we grant with the software. Granting the right
to modify and redistribute IRAF software does not mean we want to
encourage people to do so, we merely want them to have the legal right
to do so if they feel they need to.
2.2 Major system enhancements
The information in this section is provided primarily for the
benefit of IRAF site managers and programmers. The reader interested
primarily in science applications may wish to skip ahead. Some systems
level familiarity with the current IRAF system is assumed.
2.2.1 Layered software enhancements
A given IRAF installation consists of the core IRAF system, and any
number of layered software products or external packages. The goal of
the layered software enhancements introduced in V2.8 is to make layered
software products self contained and hence independent of the core
system and of other layered software. Examples of layered software
products are the NOAO packages, LOCAL, STSDAS, PROS, and so on.
The layered software enhancements make it possible to install or
deinstall a layered product by modifying only a single file in the core
IRAF system. The core system may be updated without affecting layered
software, and vice versa. Since layered products are independent and
are simple to install, IRAF can easily be configured with only those
packages needed at a particular site. Software developers benefit from
the layered software enhancements because the facilities provided for
development and maintenance of layered software are equivalent to those
provided for development of the core IRAF system and the NOAO
packages. User sites benefit because it is easy to extend the system
with LOCAL packages of their own making.
Each layered product (usually this refers to a tree of packages) is a
system in itself, similar in structure to the core IRAF system. Hence,
there is a LIB (global system library), one or more BINs (binary file
directories), a Help database, a set of global environment definitions,
and all the sources and runtime files, all contained within the same
directory tree. Layered software products, in their source only form,
are portable without change to any computer which runs IRAF.
2.2.1.1 The hlib$extern.pkg file
This is the file which is modified to install or deinstall layered
software products. To install a layered product, one creates a
directory to hold the software, restores the files to disk, and edits
the extern.pkg file to tell IRAF the name of the root package of the
layered product, and where the root directory is located. If the
layered software is distributed in source only form it will also be
necessary to recompile the software, but this is a completely automated
process.
2.2.1.2 NOAO and LOCAL packages reorganized
As part of the project to better support layered software, the NOAO
and LOCAL packages have been reorganized as layered products. These
packages are now structurally equivalent to third party (non-NOAO)
packages, except that the directory trees are rooted in IRAF. Both
packages are now self contained, with their own LIB, BINs, Help
database, etc., and with an entry in extern.pkg, like other layered
products. The NOAO package serves as a working example of how to
configure a layered package. The reorganization of these packages
should be transparent to anyone merely using the system.
2.2.1.3 The template LOCAL
The LOCAL package included with the distributed system has been
stripped of all NOAO site-local tasks and restructured as a layered
product, the template local. The template local contains only two
sample tasks and is not intended as an end-user package, but rather as
a template to be copied and modified by sites to construct their own
site dependent LOCAL package. The desire to be able to easily develop
and maintain locally added packages was one of the major motivations
for the layered software enhancements project, and we hope that sites
will realize the significance of this new capability and take advantage
of it.
2.2.1.4 CL now supports package level BIN directories
Rather than assuming a global BIN directory for all tasks and
packages, the CL now permits multiple BIN directories, each BIN
directory being associated with the package of definition and all
subpackages of that package (unless they have their own BIN). A new
BIN directory is declared with the optional argument bindir=path in the
package statement, e.g., in a package script task.
2.2.1.5 MKPKG support for package environments
Layered packages now have their own private LIB, including an
environment definitions file (zzsetenv.def), mkpkg global include file
(mkpkg.inc), and, optionally, a mkpkg special file list file for each
supported host system, listing files requiring special compilation to
work around host compiler bugs or whatever. The full mkpkg environment
is formed by reading the IRAF core system environment and mkpkg
definitions and include files, followed by the package definitions and
include files. Reading of the package environment occurs only if mkpkg
is called with the "-p" flag, or if the variable PKGENV is defined in
the user's environment.
Another way of expressing this is, when using mkpkg within a layered
package, one must now specify the name of the layered package in order
to pick up the package environment definitions. For example, to update
the MTLOCAL package in NOAO, one would type "mkpkg -p noao update" in
the mtlocal directory. If this is not done compilation errors may
result, or the exectable may not be successfully installed in the
package BIN directory.
2.2.2 Multiple architecture support
A single IRAF system (or layered package) can now simultaneously
support any number of machine architectures using multiple BIN
directories sharing a single machine independent copy of IRAF. Each
BIN directory contains all the object modules, object libraries, and
executables for a particular architecture. An architecture can
represent either a type of hardware, e.g., sparc, mc68020+f68881,
mc68020+ffpa, vax, etc., or a software distinction, e.g., systems
compiled with different sets of compiler flags, or different versions
of a system. Multiple architectures are now supported both for IRAF
execution, and for IRAF based software development, e.g., a single
version of IRAF can now be used to develop and run IMFORT programs on
both Sun-3 and Sun-4 nodes.
The only case where multiple architecture support is used at the present
time is in Sun/IRAF, which is often installed on a heterogeneous
network of workstations, e.g., Sun-3s with various hardware floating
point options, and Sun-4s. A single copy of IRAF will be configured
with several BIN directories, one for each supported architecture, and
NFS mounted on all the network nodes which will be using IRAF. There
is no reason that this feature need be restricted to use with Sun/IRAF,
however.
2.2.2.1 IRAFBIN and IRAFARCH
Starting with IRAF V2.8, the old environment variable IRAFBIN has
been obsoleted and replaced by IRAFARCH. On machines which support
multiple architectures, the latter defines the architecture to be used
for both IRAF execution and software development. If only IRAF
execution is needed the variable is optional, with the best
architecture being selected automatically when the CL is started. If
one will be doing software development (including IMFORT) it is best to
define the variable in the host environment before starting IRAF or
doing any host level software development. Typical values of IRAFARCH
for a Sun workstation are "sparc", "i386", "f68881", and "ffpa".
2.2.2.2 System libraries moved to the BIN directory
As part of the revisions required for multiple architecture support
for software development, all object libraries have been moved from the
global, architecture independent LIB to the architecture dependent BIN,
with the LIB entries being replaced by symbolic links (in the case of
Sun/IRAF). This should be transparent to both end users and
programmers.
2.2.2.3 New bin.generic architecture
On Sun/IRAF systems, which are distributed configured for multiple
architecture support, the system architecture is set to generic in the
distributed system. What this means is that all architecture dependent
files (objects and object libraries) have been removed from the system
directories and archived in the file OBJS.arc in the BIN directory for
each architecture. Rebuilding any of the packages in a system would
require restoring the binaries for a particular architecture, e.g.,
typing "mkpkg sparc" at the IRAF root would restore the sparc binaries
for the core system on a Sun/IRAF installation. Note that this only
affects software development for the system in question; software
development for external packages or private user software is not
affected.
2.2.3 Shared library facility
IRAF version 2.8 adds support for a general shared library facility
for UNIX based systems. Although currently only used with Sun/IRAF,
this facility is potentially useful for other UNIX based IRAF systems
as well (VMS/IRAF already has its own shared library facility).
What the shared library facility does is take most of the IRAF system
software (currently the contents of the ex, sys, vops, and os
libraries) and link it together into a special sharable image, the file
S.e in each core system BIN directory. This file is mapped into the
virtual memory of each IRAF process at process startup time. Since the
shared image is shared by all IRAF processes, each process uses less
physical memory, and the process pagein time is reduced, speeding
process execution. Likewise, since the subroutines forming the shared
image are no longer linked into each individual process executable,
substantial disk space is saved for the BIN directories. Link time is
correspondingly reduced, speeding software development.
With the introduction of the shared library facility, the disk space
required for Sun/IRAF is substantially reduced. Due to the increased
memory sharing and reduced process pagein times performance is
substantially improved, especially on systems like the Sun/386i which
has a relatively slow SCSI disk and often limited memory. The disk
size of small programs is reduced by up to a factor of ten in some
cases, e.g., an executable for a small program that was formerly 250 Kb
in size might be as small as 25 Kb if the shared library is used and
the shared image symbols are omitted at link time.
2.3 User interface changes
2.3.1 Calling IRAF tasks from the host environment
The IRAF main and zmain were modified to make it easier to call
IRAF tasks as host level tasks, i.e., without having to set up a
command file and run the process with the standard input redirected.
In the new scheme, any extra arguments given on the process command
line are passed into the IRAF main as a command buffer containing the
IRAF command or commands to be run. For example,
cl> x_system.e netstatus
would run the command netstatus in process x_system.e.
cl> x_system.e count "files=*.x"
would run the count task, counting all ".x" files in the current
directory.
cl> x_system.e count "files=*.x 4>_o"
would do the same, redirecting the output at the IRAF main level to the
file _o.
cl> x_system.e 'directory @pars $nargs=0'
would run the directory task with the given parameter set, with $nargs
set to 0. If any of the parameters to a task are omitted the task will
query the terminal for them in the usual way, so for example
cl> alias count "$iraf/bin/x_system.e count files="
would make the IRAF task count available in UNIX, allowing the IRAF
template specifying the files to be counted to be either given on the
UNIX command line, or prompted for if omitted. Given the above alias,
one could enter a UNIX command such as
cl> count 'cl$*.h'
This feature is available in all UNIX based versions of IRAF V2.8, but
did not make it into VMS/IRAF version 2.8.
2.3.2 Image packing density control (impkden)
Some users have complained about images taking up more disk space
than they have to, due to the IMIO feature which conditionally blocks
image lines to fill an integral number of disk blocks. This can result
in more efficient image i/o but can also make a significant difference
in the amount of disk space consumed by an image in some cases.
IMIO can actually support both block-aligned and fully packed images.
The decision is made at image creation time and is based on the image
packing density if image lines are block aligned. If the packing
density is too low for a block-aligned image, a fully packed image is
created to avoid wasting disk space. The default minimum packing
density is 0.6, i.e., up to 40% wasted space before IMIO switches to
full packing (no wasted space).
For finer control over the packing density, the user can now specify the
optional environment variable impkden, the numeric value being the
mininum packing density. For example,
cl> set impkden = 1.0
would completely disable block-alignment of image lines in IMIO.
2.3.3 User libraries (IRAFULIB)
It is now possible for the programmer (SPP or IMFORT) to specify a
private directory to be searched at compile or link time when
developing IRAF or IMFORT programs. This is done by defining the path
to the directory in the user environment as the variable IRAFULIB.
When locating a particular file, this directory will be searched before
the IRAF system libraries are searched, hence this feature may be used
to substitute custom versions of files in the IRAF system libraries,
e.g., for debugging purposes.
2.3.4 New logical printer device LPR
A new logical line printer or plotter device lpr is now supported on
all UNIX/IRAF systems. This treats the UNIX task lpr as a kind of
pseudo-device, leaving it up to UNIX to decide what physical device to
dispose of the output to. This default is system dependent, but on
some systems can be controlled by defining the variable PRINTER in the
user environment.
2.3.5 Machine independent help database
The IRAF help task uses a precompiled binary database to speed help
keyword searching. This file is now machine independent, allowing it
to be generated on one system and included in software distributions
without having to be recompiled. In addition, as part of the layered
software support, help now allows each external package to have its own
private help database. The first time help is run, all such databases
are read and linked to produce a database containing entries for all
help modules in the core system and all installed external packages.
The help database file is the file helpdb.mip in the LIB directory of
the core system and each external package.
2.3.6 Set terminal type will no longer hangup
On systems, e.g., workstations, which provide virtual terminal
windows which can change in size, IRAF may query the terminal at run
time to determine the screen size. This query is performed, for
example, at login time if the terminal type is set to gterm or sun.
Formerly this could cause the login process to hang indefinitely (i.e.,
until the user typed return or interrupt) if the terminal did not
respond to the size query, as would happen when the terminal type was
set improperly and the actual terminal ignored the query. Thanks to
the addition of non-blocking raw terminal i/o in V2.8 IRAF, the
terminal screen size query will now time out with a warning message to
reset the terminal type, if the terminal does not respond to the query
within several seconds.
2.3.7 Installing a new version of IRAF obsoletes old user parameter files
The problem of old, obsolete user (uparm) parameter files being
used with a newly installed version of IRAF, which could lead to
"parameter not found" error aborts, has been fixed. The CL now checks
the date of the file utime in HLIB, and refuses to use the user pfile
if it is older than either utime or the package pfile provided with the
new system. The contents of old user pfiles are merged into the new
system pfile, as before, preserving learned parameter values even when
the user pfile is obsolete.
2.3.8 @file list bug fixed
The problem of the "@file" (at-file-list) syntax not working when
the file in question was not in the current directory has been fixed.
2.4 Programming interface changes
2.4.1 IMFORT pixel directory control
IMFORT has been modified to permit specification of the pixel file
directory by the calling program. The modifications are completely
upwards compatible, i.e., existing programs linked with the new
interface will still create pixel files in the same directory as the
header file, with "HDR$" in the image header.
The Fortran programmer may set or query the pixel file directory using
the following routines:
imsdir (dir) # set pixel directory pathname
imgdir (dir) # get pixel directory pathname
where dir is a Fortran character variable. The value should be either
"HDR$" (the default) or a concatenatable host directory pathname (i.e.,
trailing / required for unix). Once set, the pixel directory will be
used for all subsequent image create or rename operations by the
calling process.
For example,
call imsdir ("/tmp3/pixels/")
call imcrea (image1, axlen, naxis, dtype, ier)
call imcrea (image2, axlen, naxis, dtype, ier)
If desired the default pixel directory may be specified in the host
environment as imdir or IMDIR before running the program. IMFORT will
check the host environment for this environment variable then use
"HDR$" as the default if no host definition is found.
Note that although this is similar to setting the value of imdir in the
IRAF environment, IMFORT programs are not part of the IRAF environment
and are not affected by changes to the IRAF imdir. Also, since IMFORT
is a host level facility and IRAF networking is not supported, the
network prefix (e.g., "node!") is omitted from the pixelfile pathname,
and since IMFORT programs are not necessarily used in conjunction with
IRAF, the ".." (hidden file protection) files are not used to protect
against image deletion.
2.4.2 Image display interface: IMD
A new interface IMD has been added to provide a rudimentary
facility for interactive image display device control. This is an
interim prototype interface which will be replaced by the new display
interfaces when the latter become available.
The IMD interface operates by mapping an image display device frame
buffer onto an IMIO image descriptor. The display frame buffer may
then be randomly edited by normal image i/o operations, e.g., to modify
subrasters of the displayed image, or overlay the image with color
graphics. The image pixel to display frame buffer coordinate
transformation is supported, allowing applications to work in image
pixel coordinates if desired. This interim interface is what is used
by the new display oriented tasks imexamine, imedit, and tvmark.
2.4.3 Image masks: PLIO, PMIO, MIO
The following new VOS interfaces have been added in V2.8 to provide
a general boolean or integer image mask facility.
PLIO pixel list i/o
PMIO pixel (image) mask i/o
MIO masked image i/o (image i/o through a mask)
PLIO is a general interface for storing and manipulating
multidimensional integer valued rasters containing regions of constant
value (i.e., masks). The masks are stored in a highly compressed form,
the size of the compressed mask being a function of the information
content of the mask. Both pixel array and range list i/o facilities
are provided, as well as a set of general boolean raster operators,
e.g., to extract or insert subrasters, AND or OR a source with a
destination, do the same through a stencil, draw regions of various
kinds (point, line, box, circle, polygon), and so on. See the PLIO.hlp
file in the PLIO source directory for further information.
An interactive debug program (plio$zzdebug.x) is provided for
experimenting with masks. Note that PLIO is a stand alone interface
and is not tied in any way to IMIO, even though the data structure
operated upon is similar to an image matrix.
PMIO is very similar to PLIO except that it is used to associate a
masks with an IMIO maintained reference image. Currently, the PMIO
mask must be the same resolution as the physical reference image. All
coordinates input to PMIO are in the image section coordinates of the
reference image. Hence, given a physical image and associated mask,
one can operate upon both through a user specified image section
transparently to the applications program. This includes all PLIO
style boolean rasterop operations, as well as mask pixel and range list
i/o. The PMIO interface is layered upon PLIO and IMIO, and the calling
sequences are identical with PLIO except for the package prefix, and
the addition of several new PMIO specific routines.
MIO is essentially an extension of image i/o for pixel i/o through a
mask. The central routines are the following:
mio_setrange (mp, vs, ve, ndim)
n|EOF = mio_[gp]lseg[silrdx] (mp, ptr, mval, v, npix)
One defines a rectangular region of the image with mio_setrange, and
then sequentially reads or writes line segments until all pixels
visible through the mask have been accessed. This type of i/o should
be ideal for most image processing applications which need to operate
upon only those pixels visible through a region mask (e.g., a surface
fitting task), upon all pixels except those on a bad pixel mask (e.g.,
any analysis program), and so on.
PLIO (or PMIO) masks may be stored in binary files on disk, the files
having the extension ".pl". The V2.8 version of IMIO has the
capability to treat such masks as if they were images, allowing masks
to be easily displayed, used in image expressions, converted to image
matrices and vice versa, etc. Applications may do either pixel or
range list i/o to a mask image via IMIO, if MIO is not suitable for
some reason.
2.4.4 Photon images: QPOE, QPIO, QPEX
A new set of VOS interfaces supporting photon or event list data
are now available. The QPOE interface implements the Position Ordered
Event list object, which consists of a general header mechanism plus an
event list, wherein the events are little data structures, e.g., the
attributes required to describe a photon detection (position, energy,
time, etc.). QPOE is designed to efficiently access very large event
lists, e.g., several hundred thousand or several million events in size.
Builtin event attribute filtering and region filtering capabilities are
provided for selecting photons from the event list. These filtering
capabilities may be combined with the sampling capability to produce
filtered, block averaged image matrices from event lists.
The QPOE interfaces are the following:
QPOE header and file access and management facilities
QPIO raw and filtered event i/o
QPEX event attribute filter mechanism
QPF IMIO/IKI kernel for image interface to QPOE files
QPOE and QPF add a new image type to the system, with .qp file
extension. Hence, event list data can be used as input to any of the
image processing tasks in standard IRAF, in addition to being analyzed
by tasks which deal with the individual photon events. A QPOE image is
contained in a single file. When a QPOE file is accessed as an image
the interface filters and samples the event list in real time, using a
user defined filter, block averaging factor, region mask, and so on,
producing the image matrix seen by applications at the IMIO level. The
QPOE object may be repeatedly examined with different event filters to
view the data in different ways.
The QPOE interface, in addition to providing an event list capability
for IRAF, serves as a prototype for the "flex-header" portion of the
new image structures project. Many of the capabilities to be provided
for image storage under the new image structures are already present in
QPOE.
Further information is given in the QPOE.hlp file in the QPOE source
directory.
2.4.5 File manager: FMIO
A new VOS library FMIO has been installed. FMIO is "File Manager
I/O", and is used to implement a simple binary file manager which
maintains the file data of so-called "lfiles" (lightweight files)
inside a single host binary file. The system overhead for accessing
lfiles is much less than that of host files, and many lfiles can be
used to store a complex data structure without cluttering a host
directory or incurring the inefficiency of accessing host files. FMIO
is part of the DFIO project and will serve as the lowest level
interface within DFIO; it is also used currently in the QPOE
interface. Additional information is given in the README file in the
source directory for the interface.
2.4.6 IMIO changes
IMIO is the image i/o interface, the standard IRAF VOS interface
for managing all varieties of image data.
2.4.6.1 Mask image support
IMIO now supports a new type of image, the mask image, stored as a
highly compressed binary (PLIO) file with the extension ".pl". Image
masks are most commonly used to store information describing selected
pixels in an associated data image. An image mask is logically a
boolean or integer image, up to 28 bits deep, containing information
only on selected pixels or regions of pixels. Masks are stored in
highly compressed format, e.g., a simple mask may be stored in only a
few hundred bytes of space. Mask images are readable, writable, and
randomly modifiable, like ordinary raster images. See \(sc2.4.3 for
more information.
2.4.6.2 Photon image support
Support has also been added to IMIO for event list images, stored
as position ordered event list datafiles using the QPOE interfaces.
This new image type has the extension ".qp". QPOE images are read-only
under IMIO. Subject to that restriction, they may be accessed like any
other image by any IRAF image analysis program. Accessing an event
list image as a raster image necessarily involves a runtime sampling
operation, wherein the events in the region of interest are accumulated
into an initially zero image matrix; in the process the event list may
optionally be filtered by event attribute or event position, e.g.,
cl> display "xray.qp[t=(30:40),pha=10,block=4]"
would display the QPOE image xray.qp with a blocking factor of 4,
selecting only those events with t (time) in the range 30 to 40 and for
which pha (energy) has the value 10. The event attributes and their
names are user definable and may vary for different types of data. See
\(sc2.4.4 for more information.
2.4.6.3 IMPUTH
A new procedure imputh has been added to the IMIO header access
library. The new procedure is used to append FITS like HISTORY or
COMMENT cards to the image header.
2.4.6.4 IMPARSE
The calling sequence of the internal IMIO procedure imparse has
changed. Although this procedure is internal to the IMIO interface and
is not supposed to be used within applications, there may be
applications which make use of this procedure. Any such applications
must be modified to reflect the new procedure calling sequence or
runtime problems are guaranteed.
2.4.7 Null string environment variables
The semantics of the VOS procedures envgets and envfind have
changed. This could affect existing programs and any programs which
use these functions should be checked to make certain they will still
work properly.
These procedures, used to fetch the string values of environment
variables, return the length of the output string as the function value.
Formerly, a value of zero would be returned both when the named
variable existed but had a null string value, and when the variable was
not found. This made it impossible to discriminate between the case of
a variable not being defined, and one which is defined but has a null
value. The routines have been changed to return the value ERR (a
negative integer) if the variable is not defined. Programs which do not
wish to make the distinction between undefined and null-valued should
check for a function value less than or equal to zero. Programs which
check for a function value equal to zero will fail if the named
variable is not defined.
2.4.8 Environment substitution in filenames
The VOS filename mapping code has been modified to add support a
powerful new environment substitution syntax. Previously the only
environment substitution mechanism available was the logical directory
facility, which could only be used to parameterize the directory
field. The new facility may be used to perform environment
substitution anywhere in a filename. This is used in IRAF version 2.8
to implement multiple architecture support, e.g.,
cl> set bin = "iraf$bin(arch)/"
is how the core system BIN is defined in V2.8 IRAF. The syntax
"(arch)" tells the filename mapping code to substitute the string value
of the environment variable arch, if defined. If the variable is not
defined the null string is substituted. Hence, if the host system does
not implement multiple architecture support and arch is not defined,
BIN is defined as "iraf$bin/", which is the backwards compatible
definition. If arch is defined as, e.g., ".vax", then BIN is defined
as "iraf$bin.vax/". The new feature allows use of a single environment
variable to define the architecture, not only to form filenames, but
for other purposes as well, e.g., to generate compiler switches or to
control library searching in mkpkg.
2.4.9 Nonblocking raw terminal i/o
The VOS file i/o interfaces have been modified to add support for
nonblocking terminal i/o. This facility makes it possible to, in
effect, "poll" the terminal to see if there is any input waiting to be
read, to allow interaction without having a program block if the user
has not typed anything.
The immediate application of this in version 2.8 was the modification
of the stty (set-terminal) facility to implement a time out for the
terminal size query. Formerly, stty would hang up indefinitely when
the terminal type was set to "gterm" but the actual terminal was
something different, causing the screen size query to be ignored.
In the more general case, nonblocking terminal i/o makes possible a new
class of user interface, which is not only interactive, but event
driven. Nonblocking i/o makes it possible for an application to be
continually processing, while checking the terminal occasionally to see
if the user has input any commands.
At present, nonblocking i/o is always used in conjunction with raw mode
input from a terminal. A new flag F_NDELAY, defined in <fset.h>, is
used to enable or disable nonblocking i/o. For example,
call fseti (fd, F_RAW, YES)
enables conventional blocking, single character raw mode reads, and
call fseti (fd, F_RAW, YES + F_NDELAY)
enables nonblocking raw mode input (YES, NO, and F_NDELAY are bit
flags). These modes are mutually exclusive, e.g., the first call may
be issued while nonblocking raw mode is in effect to make the reads
block, and vice versa. A call to fset(fd,F_RAW,NO) disables both raw
mode and nonblocking mode. Once nonblocking raw mode is in effect one
merely reads characters from the terminal in the usual way, using
getc. EOF is returned if a read is performed when no data is available
for input, otherwise the next character is returned from the input
queue. Further information on nonblocking i/o is given in the system
notes file.
2.4.10 Function call tables (ZFUNC)
IRAF has always had the ability to compute the integer valued
address of a procedure, store that address in a table, and later use
the address as an argument to one of the zcall kernel primitives to
call the addressed procedure. This facility has been extended by the
addition of a set of zfunc primitives, used to call integer valued
functions. Only integer valued functions are supported (in order to
simplify the kernel support required), but in the systems oriented
applications where procedure call tables are used, this is unlikely to
be a serious limitation.
2.5 Sun/IRAF specific revisions
2.5.1 IEEE exception handling
By default the IEEE hardware is now configured, on all Sun systems,
with the invalid, overflow, and divide by zero IEEE exceptions enabled,
and with the default rounding direction and precision modes (nearest,
extended) in effect. This configuration should ensure that all
questionable floating point operations are detected, and that no IEEE
"funny numbers" (NaN, Inf, etc.) get into the data. These values,
since they don't behave like ordinary numbers, can cause programs to
misbehave, e.g., go into an infinite loop. In Sun/IRAF V2.8, if a
computation results in an IEEE funny number being generated, an
exception abort will result. The most common example is divide by zero.
The IRAF/IEEE interface offers a special debug feature that may be of
interest to programmers developing numerically sensitive software. If
desired, one can change the default rounding direction and precision
(e.g., to test the numerical stability of applications) by using the
debugger to set a nonzero value of the variable debug_ieee before
running an executable. The procedure for doing this is documented in
the system notes file.
2.5.2 IMTOOL enhancements
A number of enhancements and bug fixes have been made for V2.8 to
the SunView based IMTOOL image display server. The most notable
changes are summarized here; refer to the IMTOOL manual page for a more
complete description of the new features.
2.5.2.1 Software ZOOM added
IMTOOL, which has had for some time the ability to pan about on a
large image, now has the ability to zoom as well. Both pan and zoom
are controlled very conveniently by the middle mouse button: place the
mouse on an object and tape the middle button once to pan the object to
the center of the display window; tap it again and the image will be
zoomed. Zoom, currently implemented by writing directly into the
hardware frame buffer, is very fast, almost as fast as a normal
unzoomed window refresh. The default set of zoom factors is 1,2,4,8
after which the sequence wraps around to 1. The zoom factors are user
configurable via the IMTOOL setup panel; very large zoom factors, e.g.,
x64, are possible. Dezoom (making a large image smaller) is not
currently supported.
2.5.2.2 WCSDIR eliminated
The host level WCSDIR environment variable, and the text file used
to communicate image coordinate (WCS) information between the display
task and the display server, have been eliminated. All WCS information
is now passed via the datastream used to pass commands and data between
the client and the display server. This eliminates the need for users
to have to remember to define WCSDIR in order to get coordinates in
image units, and some subtle process synchronization problems are
eliminated as well.
In a related change, the frame buffer configuration index is no longer
passed in during a frame erase, hence it is no longer necessary to
erase a frame before displaying an image to ensure that a frame buffer
configuration change is passed to the server. The configuration index
is now passed when the WCS information for a frame is set.
2.5.2.3 Graphics colors
IMTOOL now allocates a range of pixel values for use as graphics
overlay colors. Setting a frame buffer pixel to one of these values
causes it to always be displayed with the assigned color. The graphics
color values are not affected by windowing the display. The most
common use of graphics colors with V2.8 IRAF is for drawing graphics
into a displayed frame with the new tvmark task, available in PROTO.
See the IMTOOL manpage for a table listing the color index assignments.
2.5.2.4 New imtoolrc entries
Several new predefined frame buffer configurations have been added
to the default imtoolrc. These include an 128 pixel square frame
buffer (imt128), a 256 pixel square frame buffer (imt256), and a full
screen display with the same aspect ratio as a 35 mm slide (imtfs35).
2.5.2.5 System crash (FIFO) bug fixed
Versions of SunOS through at least 4.0.1 have a bug in the FIFO
driver code which can cause the internal kernel FIFO data buffer to be
deallocated while it is still in use. This will result in a bad kernel
which will eventually panic and reboot the system. This is the cause
of the IMTOOL crash problem which some sites may have experienced.
IMTOOL has been modified to avoid the circumstances (repeated 4096 byte
transfers) which cause the bug to surface. So far as we know, the real
bug (in SunOS) has not yet been fixed, but at least on the NOAO
systems, the frequency of occurrence of the system crashes is greatly
reduced with the new version of IMTOOL which incorporates the
workaround for the SunOS bug.
2.5.2.6 Cursor marking now disabled by default
When the interactive image cursor read facility was first added to
IMTOOL, the default response to each cursor read was to draw a small
white dot at the position of the cursor. This is convenient when
marking a series of objects to make a list, but with the increasing
number of IRAF programs making user of the interactive image cursor, it
has been necessary to change the default to disable automatic marking
of each cursor read. The cursor mark feature is still available as an
option and can be enabled via the setup panel.
2.5.2.7 Ctrl/b may be used for manual blinking
In addition to the list of blink frames and the timed blink feature
IMTOOL has provided for some time, it is now possible to manually cycle
through the blink frames with the <ctrl/b> key. Typing <ctrl/b> while
the mouse is in the image window will cause the display to display the
next blink frame in sequence.
2.5.2.8 F4 key will now toggle setup panel
The F4 function key on the Sun keyboard may now be used to toggle
whether or not the setup panel is displayed. This provides a single
keystroke alternative to calling up the setup panel with the frame menu.
2.6 VMS/IRAF specific revisions
2.6.1 NEWUISDISP added to VMS/IRAF
Nigel Sharp's NEWUISDISP display program, used for image display
under UIS on microvaxes with bitmapped displays, is now available in the
standard VMS/IRAF release, in the directory [IRAF.VMS.UIS].
2.6.2 New INSTALL.COM script
A new INSTALL.COM script (also written by Nigel Sharp) has been
added to VMS/IRAF. This script, run by the system manager to install
selected IRAF executable images, will now automatically check for and
deinstall any old versions of the executables before installing the new
ones.
2.6.3 VMS 4.7/5.0
Testing of the standard V2.8 VMS/IRAF release, which was prepared
on VMS 4.7, on a VMS 5.0 system has thus far not revealed any problems
(NOAO is still running VMS 4.7 as our standard system). Hence it
appears that the standard V2.8 VMS/IRAF will run under VMS 5. It is
likely, however, that any attempt to recompile VMS/IRAF under VMS 5
would cause problems, since we have not yet tried to rebuild IRAF under
VMS 5, and such a major operating system upgrade will often require
changes to the IRAF code. The system may be relinked under VMS 5 if
desired, and this does not appear to cause any problems, but neither
does there appear to be any benefit to be gained from doing so.
2.7 Summary of IRAF System Packages Revisions
o The tasks RFITS and WFITS in the DATAIO package now support the
reading and writing of arbitrary sized data blocks (IRAF version 2.7
and later).
o Several new tasks were added to the IMAGES package. IMCOMBINE (IRAF
version 2.6 and later) provides for the combining of images by a
number of algorithms. The new task CHPIXTYPE (IRAF version 2.7 and
later) changes the pixel types of a list of input images. The task
IMSLICE slices images into images of one less dimension (IRAF
version 2.8). The task IMSTACK has been moved into the IMAGES
package (although it still resides in PROTO as well).
The IMSTATISTICS task has been rewritten and now allows the user to
select which statistical parameters to compute and print (IRAF
version 2.8). The IMRENAME task has been modified to allow "in
place" image renames, used chiefly for moving the pixel files to a
new IMDIR.
Several other tasks in the IMAGES package were modified (IRAF
version 2.8). IMSHIFT was modified to accept a list of shifts from
a file. REGISTER and GEOTRAN were modified to accept a list of
transforms instead of only a single one. IMHISTOGRAM has undergone
extensive revision including support for "box" type plots, support
for linear or log scaling in the y coordinate, as well as support
for antialiasing of the histogram bins.
o All the tasks in the IMAGES.TV package were modified (IRAF version
2.8) so that if a task is used with an unsupported display device a
message is printed to that effect.
o The STTY task in the LANGUAGE package has been improved (IRAF
version 2.6 and later) to better facilitate its "playback" feature.
These changes have been documented in the online help for the
task. This feature is little used by external sites but can be a
very useful instructional aid if users are aware of its capability.
o A new task PVECTOR was added to the PLOT package that allows one to
plot an arbitrary vector in a two dimensional image (IRAF version
2.6 and later).
The task STDPLOT was modified (IRAF version 2.8) so that it uses
the more popular SGI kernel rather than the NSPP (NCAR) kernel
(STDPLOT is now equivalent to the SGIKERN task). A new task
NSPPKERN was added that uses the NSPP kernel.
o Two new tasks were added to the SYSTEM package (IRAF version 2.8).
The task DEVICES simply prints the dev$devices.hlp file as edited
by the site manager listing available devices on the local host or
network. The REFERENCES task is used to search the help database
for all tasks or other help modules pertaining to a given topic,
e.g., references vector will list all tasks that have the string
"vector" in their one line description.
2.8 Glossary of New Tasks in the IRAF System Packages
Task Package Description
chpixtype images Change the pixel type of a list of images
devices system Print information on the locally available devices
imcombine images Combine images pixel-by-pixel using various algorithms
imslice images Slice images into images of lower dimension
imstack images Stack images into a single image of higher dimension
nsppkern plot Plot metacode on a NSPP (NCAR) plotter device
pvector plot Plot an arbitrary vector in a 2D image
references system Find all help database references for a given topic
In addition, there are new image display oriented tasks imexamine,
imedit, and tvmark in the PROTO package in NOAO (used to interactively
examine and edit images, or draw graphics into image display frames).
These really belong in the core system but have been placed in
noao.proto since they are prototype tasks.
3. NOAO Package Revisions
Some of the major revisions to the NOAO packages are listed below.
3.1 Summary of NOAO Packages Revisions
3.1.1 New NOAO Packages
Several new packages have been added to the NOAO suite of packages.
o The APPHOT package is a set of tasks for performing aperture
photometry on uncrowded or moderately crowded stellar fields in
either interactive or batch mode. This package is now installed in
the DIGIPHOT package (IRAF version 2.7 and later). The APPHOT
package was available as an add-on package to IRAF version 2.5 and
later while it was undergoing alpha testing. Many new features
have been added to the package since it first became available
including the new task QPHOT (quick aperture photometry) and
interaction with the image display cursor for supported image
displays (Sun workstation, IIS model 70).
o The CCDRED package provides tools for the easy and efficient
reduction of CCD images. This package has been installed in the
IMRED package (IRAF version 2.6 and later). The CCDRED package was
also available as an add-on to IRAF version 2.5.
A short demonstration of many of the tasks in the CCDRED package is
provided with the DEMO task in the CCDRED.CCDTEST package.
o The IMRED.ECHELLE package has been replaced with a more
sophisticated collection of tasks for reducing echelle type data
(IRAF version 2.7 and later). The new ECHELLE package recognizes a
new image format in which each extracted echelle order becomes a
line in a two dimensional image rather than having a separate one
dimensional spectrum for each order, although this old output
format is still available as an option. Several new tasks exist
for computing and applying a wavelength calibration to the data
using the echelle relationship between the orders (ECIDENTIFY,
ECREIDENTIFY, and ECDISPCOR) as well as for manipulating the new
echelle format (ECSELECT, ECCONTINUUM, and ECBPLOT).
o The IRRED package has been added to the IMRED package. The IRRED
package collects together in one place those tasks used most
frequently by users reducing IR data such as that taken with the IR
imager at KPNO. The IRMOSAIC and IRALIGN tasks were available with
IRAF version 2.6 and later. IRMOSAIC takes an ordered list of
input images and places them on a grid in an output image. IRALIGN
uses this grid and a coordinate list of overlapping objects from
the individual subrasters to produce an aligned output image. The
tasks IRMATCH1D and IRMATCH2D were available with IRAF version 2.7
and later. These tasks are similar to IRALIGN expect that the
intensities of adjacent subrasters can be matched as well. A
script called MOSPROC (IRAF version 2.8) has also been added that
prepares a list of images for a quick look mosaic.
o The MSRED package has been added to the IMRED package. The MSRED
package is a collection of tasks used for reducing multispectral
types of data, e.g. fiber arrays, where the individual spectra are
for different objects. Like the ECHELLE package, it also has its
own multispectral image format (a two dimensional image in which
each line is an extracted spectrum). Several new tasks have been
added to the package for wavelength calibration of multispectral
data.
3.1.2 Modifications to Existing NOAO Packages
o The ASTUTIL package was reorganized (IRAF version 2.6 and later -
see IRAF Newsletter No. 3 for details) and several tasks were added
and/or modified. A new task ASTTIMES computes and prints
astronomical dates and times given a local date and time. A new
task RVCORRECT computes and prints radial velocity corrections for
an observation. The tasks PRECESS and GALACTIC were modified
slightly using different but more accurate algorithms.
The new task SETAIRMASS (IRAF version 2.8) computes the effective
airmass and middle UT of an exposure. This task was also made
available in the TWODSPEC and IMRED packages.
o The two tasks in the IMRED.BIAS package, COLBIAS and LINEBIAS, were
modified slightly (IRAF version 2.7 and later) so that the fitting
parameters for the overscan region can be set by the user as hidden
parameters to the tasks.
o The task COSMICRAYS (from the CCDRED package) was made available in
the IMRED.GENERIC package (IRAF version 2.6 and later).
o A new task called SYNDICO has been added to the IMRED.VTEL package
(IRAF version 2.6 and later). SYNDICO makes glossy prints on the
NOAO Dicomed printer of the synoptic, full disk, magnetograms and
spectroheliograms taken at the vacuum telescope at Kitt Peak.
o Modifications were made to the IMRED.DTOI package. These changes
have been documented in IRAF Newsletter No. 4.
o Three new tasks in the ONEDSPEC package, REFSPECTRA, SEXTRACT, and
SPECPLOT, were made available in the IMRED.COUDE, IMRED.IIDS,
IMRED.IRS, and IMRED.SPECPHOT packages.
o Many new tasks and features have been added to the ONEDSPEC package.
The SENSFUNC task was completely rewritten (IRAF version 2.6 and
later) to allow determination of extinction, display of flux
calibrated spectra, and many new features for displaying and
manipulating the data.
IDENTIFY, REIDENTIFY and DISPCOR were modified (IRAF version 2.6
and later) so that a dispersion solution from IDENTIFY could be
shifted without changing the original shape of the coordinate
function (see IRAF Newsletter No. 3 for details).
A new deblending algorithm was added to SPLOT (IRAF version 2.7 and
later). See the online help for SPLOT as well as the article in
IRAF Newsletter No. 4.
The tasks in the ONEDSPEC.ONEDUTIL package were absorbed into the
ONEDSPEC package (IRAF version 2.7 and later).
The EXTINCT task disappeared with its functionality being taken
over by a rewritten CALIBRATE (IRAF version 2.7 and later).
The COEFS task was moved to the IMRED.IIDS and IMRED.IRS packages
since this is a very instrument specific task (IRAF version 2.7 and
later).
Three new tasks were added to the package. SEXTRACT (IRAF version
2.6 and later) extracts subspectra from one dimensional input
spectra. REFSPECTRA (IRAF version 2.7 and later) takes over part
of the functionality of the old DISPCOR task and allows the user to
define which arc spectra are to be used in the calculation of the
dispersion solution of object spectra. SPECPLOT (IRAF version 2.8)
is a new plotting task that allows the compression of many spectra
to a page (see IRAF Newsletter No. 6).
o Several new tasks have been added to the PROTO package.
Four tasks were added to IRAF version 2.6 and later. BSCALE is a
task that can be used to linearly scale images by the mean,
average, or mode of the image. IRMOSAIC and IRALIGN can be used to
combine many frames into one large image. These three tasks are
also available in the IMRED.IRRED package. MKHISTOGRAM calculates
the histogram of the data in a text file.
Three new tasks were added to IRAF version 2.7 and later. IMSLICE
is a task that slices an image into images of lower dimension.
IRMATCH1D and IRMATCH2D are two tasks that allow combining of many
overlapping images while matching the background intensities in two
different ways.
Three new tasks have been added to IRAF version 2.8 that allow the
user to interact with the image display (for supported display
devices, ie Sun workstation, IIS model 70). IMEXAMINE allows the
user to interactively examine portions of the displayed image.
TVMARK allows the user to mark objects on the image display.
IMEDIT allows the user to interactively edit an image.
o The APEXTRACT package in the TWODSPEC package has ungone several
rounds of modifications, as discussed in the IRAF Newsletters, No.
3 and 4. These changes included improved techniques and additional
options for the extraction of data.
A new task, APSCATTER, has been added to the package (IRAF version
2.8). This task determines and subtracts scattered light from two
dimensional aperture or echelle spectra. The task was also made
available from within the ECHELLE package. This task was discussed
in IRAF Newsletter No. 6.
3.2 Modifications and Additions to Calibration Data
The calibration data used by some of the tasks in the TWODSPEC,
ONEDSPEC, and many of the IMRED packages are kept in a directory called
ONEDSTDS in noao$lib. The current contents of this directory are best
summarized by paging through its README file, e.g.,
cl> page noao$lib/onedstds/README
Two additional line lists (used by IDENTIFY) have been added to this
directory (IRAF version 2.8). These lists, vacidhenear.dat and
vacthorium.dat, are simply the standard .dat files in air wavelengths
converted to vacuum wavelengths. The equation used for the conversion
as well as the appropriate reference in the literature are contained in
the README file.
The thorium.dat file has been updated to contain thorium lines from
3180 Angstroms to 9540 Angstroms (IRAF version 2.6 and later). Please
see the README file for the source.
Two new directories have been added containing flux information for
standard stars (IRAF version 2.6 and later): SPECHAYESCAL and SPEC50CAL.
Both of these lists are from Massey et al., 1988, Ap. J., Vol. 328, p.
315.
3.3 Glossary of New Tasks in the NOAO Packages
Task Package Description
apscatter(1) apextract Fit and subtract scattered light
apselect apphot Extract select fields from apphot output files
asttimes astutil Compute UT, Julian day, epoch, and sidereal time
badpiximage ccdred Create a bad pixel mask image from a bad pixel file
bscale(3) proto Brightness scale images: new = (old-bzero) / bscale
ccdgeometry ccdred Discussion of CCD coordinate/geometry keywords
ccdgroups ccdred Group CCD images into image lists
ccdhedit ccdred CCD image header editor
ccdlist ccdred List CCD processing information
ccdproc ccdred Process CCD images
ccdred ccdred CCD image reduction package
ccdtypes ccdred Description of the CCD image types
center(3) apphot Compute accurate centers for a list of objects
centerpars(3) apphot Edit the centering parameters
combine ccdred Combine CCD images
cosmicrays(4) ccdred Detect and replace cosmic rays
daofind apphot Find stars in an image using the DAO algorithm
darkcombine ccdred Combine and process dark count images
datapars(3) apphot Edit the data dependent parameters
demo ccdtest Run a demonstration of the CCD reduction package
ecbplot echelle Batch plots of echelle spectra
eccontinuum echelle Fit the continuum of echelle spectra
ecdispcor echelle Dispersion correct spectra
ecidentify echelle Identify features in spectrum for dispersion solution
ecreidentify echelle Automatically reidentify features in spectra
ecselect echelle Select and extract apertures from echelle spectra
fitpsf apphot Model the stellar psf with an analytic function
fitsky apphot Compute sky values in a list of regions
fitskypars apphot Edit the sky fitting parameters
flatcombine ccdred Combine and process flat field images
flatfields ccdred Discussion of CCD flat field calibrations
guide ccdred Introductory guide to using the CCDRED package
imedit proto Examine and edit pixels in images
imexamine proto Examine images using image display, graphics, and text
imslice proto Slice images into images of lower dimension
instruments ccdred Instrument specific data files
iralign(3) proto Align the mosaiced image produced by irmosaic
irmatch1d(3) proto Align and intensity match image produced by irmosaic
irmatch2d(3) proto Align and intensity match image produced by irmosaic
irmosaic(3) proto Mosaic an ordered list of images onto a grid
mkfringecor ccdred Make fringe correction images from sky images
mkhistogram proto List or plot the histogram of a data stream
mkillumcor ccdred Make flat field illumination correction images
mkillumflat ccdred Make illumination corrected flat fields
mkimage ccdtest Make or modify an image with simple values
mkskycor ccdred Make sky illumination correction images
mkskyflat ccdred Make sky corrected flat field images
mosproc irred Prepare images for quick look mosaicing
msdispcor msred Dispersion correct spectra
msreidentify msred Reidentify features from/to a multispec image
msselect msred Select and extract apertures from spectra
observe ccdtest Create an artificial CCD observation
phot apphot Measure magnitudes for a list of stars
photpars apphot Edit the photometry parameters
polymark apphot Create polygon lists for polyphot
polypars apphot Edit the polyphot parameters
polyphot apphot Measure magnitudes inside a list of polygonal regions
qphot apphot Measure quick magnitudes for a list of stars
radprof apphot Compute the stellar radial profile of a list of stars
refspectra(5) onedspec Assign wavelength reference spectra to other spectra
rvcorrect astutil Compute radial velocity corrections
setairmass(6) astutil Compute effective airmass for an exposure
setinstrument ccdred Set instrument parameters
sextract(2) onedspec Extract subspectra from dispersion corrected spectra
specplot(5) onedspec Stack and plot multiple spectra
subsection ccdtest Create an artificial subsection CCD observation
subsets ccdred Description of CCD subsets
syndico vtel Make dicomed print of daily grams 18 cm across
tvmark proto Mark objects on the image display
wphot apphot Measure magnitudes for a list of stars with weighting
zerocombine ccdred Combine and process zero level images
Notes:
(1) Tasks also in echelle and msred packages.
(2) Tasks also in coude, iids, irs, and specphot packages.
(3) Tasks also in irred package.
(4) Tasks also in generic package.
(5) Tasks also in coude, echelle, iids, irs, msred, and specphot
packages.
(6) Tasks also in imred and twodspec packages.
NEWS (Jul86) Ancient News NEWS (Jul86)
30 July 86 IMIO Modifications
-------------------------------------------------------------------------------
The new IMIO interface, used by all IRAF tasks to access bulk image data
on disk, is now capabable of operating upon both the old IRAF format (OIF)
images as well as STScI SDAS/GEIS format images. The default image type is
the OIF format. Any existing OIF format images are readable by the new system
without change. Although IRAF can read either OIF or STF format images,
SDAS can read only STF format images, so serious SDAS users should configure
IRAF to work with STF format images as the default. All other users should
continue to use the OIF format images as image access is more efficient,
and the IRAF software has been extensively tested only for OIF format images.
Users of the OIF format should note that they can read a VMS BACKUP tape
(or UNIX TAR tape) containing STF format images directly to disk and immediately
access the images, without changing the default configuration of IRAF.
The image type is specified by a filename extension; extensions for the
OIF format images are new in this release of the system. The recognized
extensions are shown below.
image type header file extensions
OIF .imh
STF .??h (? stands for any character)
In most cases when operating upon an image with an IRAF task the extension
can be omitted. The most important exception occurs in image templates.
THE PATTERN GIVEN IN AN IMAGE TEMPLATE MUST FULLY MATCH THE IMAGE HEADER
FILE NAMES AS THEY APPEAR IN A DIRECTORY LISTING, i.e., the header filename
extension must be matched by the image template. The image type extension
must also be specified to access an image which is not of the default image
type (OIF or STF), or when changing the type of an image. For example,
cl> imcopy dev$pix.imh pix.hhh
will make an STF format copy in the current directory of the OIF format image
"dev$pix".
The default image type is controlled by the new environment variable IMTYPE.
The string value of IMTYPE is the desired image header file extension, e.g.,
"imh" (omit the dot) for an OIF format image. If IMTYPE is not defined the
default image type is "imh". For STF format images there are many possible
image header extensions, and IMTYPE specifies the default type IMIO should
look for when the extension is not explicitly given, or the default extension
to use when a completely new image is to be created. When making a new copy
of some existing image, IMIO will make a new image of the same type as the
existing input image unless an extension is given to force some other type
of image to be created.
environment variable description
IMTYPE the default image type (extension)
IMDIR pixel storage directory for OIF images
The IMDIR environment variable defines the directory in which the pixel file
is to be placed when creating a new OIF format image. In V2.2 and older
versions of IRAF, IMDIR could only be a logical or machine dependent directory
pathname. The new system also recognizes the special "builtin" logical
directory name "HDR$" (must be upper case). If the value of IMDIR is "HDR$",
IMIO will create the pixel file in the SAME directory as the header file,
rather than in some other directory. It is also possible to place the pixel
file in some subdirectory of the header directory, e.g., "HDR$pixels/" will
cause the pixel files to go into the subdirectory "pixels".
set imdir = "/tmp/user/" # pixel files -> specified directory
set imdir = "HDR$" # pixel files -> header file directory
set imdir = "HDR$pixels/" # pixel files -> subdirectory of HDR$
The root filename of OIF pixel files is now the same as that of the header
file, rather than a computer generated name. The filename extension of an
OIF pixel file is ".pix".
The STF format images support group format, a format very similar to that
used for group format FITS tapes. IRAF users accessing an STF group format
image can specify the `group' (subimage) to be accessed by appending a
subscript to the image name, e.g.,
cl> imstat pix.aah[3] # access group 3
or
cl> imstat pix.aah[3][*,100] # line 100 of group 3
A new group format image can be created in a similar fashion, specifing the
number of groups to preallocate space for, e.g.,
cl> imcopy dev$pix testimage[1/10]
would create a new group format image "testimage" with space for 10 groups
(subimages), and initialize group 1. The remaining groups would then be
initialized by specifying only the group subscript "[N]". Note that all
groups must be the same size, new groups cannot be allocated, old groups
cannot be deleted, the set of possible group parameters is fixed at creation
time, and all groups share the same FITS header.
15 June 86 System Tasks
-------------------------------------------------------------------------------
The DIRECTORY, HELP, and PAGE system tasks have all undergone important
revisions. The directory task has been completely rewritten and now handles
directory pathnames, etc., correctly, and in addition it has a more concise
syntax. The HELP and PAGE tasks have been modified to replace the old "more"
boolean query mechanism (used to pause between pages of output) with a nicer
keystroke driven mechanism which offers more options and is faster. Read the
manual pages for additional details. The old parameter files should be
unlearned.
28 April 86 Package Reorganization
-------------------------------------------------------------------------------
The basic package structure of IRAF has been modified to make a distinction
between the system packages and the NOAO optical astronomy packages. The basic
directory structure of the system was also changed to reflect the new package
organization, and the printed documentation will be changed as well when time
permits. These changes were necessary to better isolate science software such
as the NOAO and STScI/SDAS packages from the system software, for a more logical
package structure, and to make it easier to install and maintain the science
packages.
The new root menu of IRAF is as follows:
dataio images lists noao sdas system
dbms language local plot softools utilities
The NOAO menu is as follows:
artdata astutil focas mtlocal proto twodspec
astrometry digiphot imred onedspec surfphot
Three new packages were added and three old packages were extensively revised.
The NOAO mountain tape readers were moved from the DATAIO package into the new
MTLOCAL package. The astronomically oriented utility tasks were moved from
the UTILITIES package to the new ASTUTIL package. The old LOCAL package was
renamed PROTO, and a new LOCAL package was added in the directory
"iraf$local/tasks".
The concept of the new PROTO package is appropriate for what the old LOCAL
package was used for, i.e., prototype, temporary, or contributed tasks which
are part of the NOAO package and which are exported with the system, but which
are expected to eventually disappear or be replaced by planned system
facilities. The new LOCAL package is a place to put tasks of strictly local
interest, or tasks which are not portable, e.g., foreign tasks and the Peritek
package. The LOCAL package should be particularly useful for outside sites
as it gives them a place to put locally added tasks which will not be affected
by future updates of the system. Also, the framework (mkpkg, local.cl, etc.)
is all set up, making it easier for outside sites to add their own software
without having to figure out how to set up an IRAF package.
20 Feb 85 Recovery from Interrupts
-------------------------------------------------------------------------------
Tests today (unfortunately only shortly before the first IRAF release)
have shown that VMS/IRAF has higher failure rate than expected for recovery
from a ctrl/c interrupt of a running subprocess, especially if the process
is actively doing i/o. It is probably much safer to interrupt a compute
bound process than a process which is doing heavy i/o, e.g., reading a tape
or doing many file opens, or probably large numbers of any type of system
calls. The failure rate for i/o intensive processes was as high as 1 failure
to recover in 4 interrupts in some of the cases tested. Testing of UNIX/IRAF
turned up some of the same problems, but the failure rate was considerably
lower, probably because the kernel and i/o system are so much simpler.
Interrupting a process at the wrong time can cause many problems, e.g.,
[1] subprocess memory can be corrupted, resulting in unpredictable behaviour
and possibly deadlock when the CL later tries to talk the the process,
[2] a VMS forced exit of the process can occur, e.g., when trying to deliver
an AST to an invalid address, [3] corruption of the CL/subprocess communications
protocol can occur, resulting in deadlock or the loss of sync (the output of
a task will come out when the next command is entered), and various other
problems as well.
Tests on the old version of VMS/IRAF, which dates back to last fall, show
that the problem has existed all along. The fact that we did not fully
appreciate the problem until now indicates that the problem is not a serious
hindrance provided one is conservative about the use of interrupts. It also
appears that this will not be an easy problem to solve, hence it is likely
to be with us for a while. Probably nothing at all will be done about it
for some months since other projects are likely to have a higher priority
if this problem is understood and can be worked around.
In summary, try to minimize the use of interrupts, and in particular, avoid
interrupting processes which are doing heavy i/o. When in doubt, type
"flpr" after interrupting a process to force it to be restarted. If a
subprocess becomes hung it may be necessary to restart the CL itself.
20 Feb 85 Process connect failure during ":.snap" in cursor mode
----------------------------------------------------------------------------
We are still ocasionally having problems when trying to spawn a graphics
subkernel in response to a ":.snap" command in cursor mode. This happens
infrequently (which is why it is so hard to find the bug), and will usually
go away after exiting and reentering cursor mode and trying again. It might
also help to do a ":.gflush" while in cursor mode.
distrib
>
Mageia
>
6
>
armv5tl
>
media
>
core-release
>
by-pkgid
>
30819c093f498f9dfa6444d2407d0521
>
files
>
5348
