<!--Copyright (C) 1988-2005 by the Institute of Global Environment and Society (IGES). See file COPYRIGHT for more information.-->
<html>
<head>
<title>How to Generate BUFR Descriptor Files</title>
<link href="../../assets/NewIGES.css" rel="stylesheet" type="text/css">
</head>
<body text="#000000" bgcolor="e0f0ff">
<h2><b>How to Generate BUFR Descriptor Files</b></h2>
<p> <span class="plaintext">BUFR (Binary Universal Form for the Representation
of meteorological data) is a World Meteorological Organization (WMO) standard
for storing observational data (aka sequence or in-situ data). BUFR is self-describing
data format and can store a large amount of data and metadata in a small amount
of disk space by using look-up tables and bit-by-bit packing. </span>
<p class="plaintext">There is a GrADS interface for BUFR, which means that BUFR
data can be read directly in their native format and are handled as a GrADS
station data set with all the associated display an analysis capabilities. GrADS
requires a specially-formatted descriptor file to read BUFR data; the output
from <a href="gradutilbufrscan.html"><code>bufrscan</code></a>, an external
GrADS utility, is used to compose the descriptor file.
<p class="plaintext">Individual elements of a BUFR message are uniquely described
by three numbers: F, X, and Y. F is a type indicator and may be 0, 1, 2, or
3. X is a class or category indicator and varies between 0 and 63. Y indicates
an entry within an X class, and varies between 0 and 255. The F,X,Y trio provides
the required unique table reference, so that a value may be retrieved for the
BUFR element.
<p class="plaintext">To read BUFR with GrADS, the user needs to identify which
F,X,Y trios are in the BUFR file and then organize that information in a descriptor
file to give "shape" to the data by identifying the appropriate time
axis, vertical dimension, and number of variables. The GrADS-relevant data in
a BUFR message will always have an F value of 0. For this reason, it is only
necessary to put the X,Y pairs that are associated with the data or metadata
variables in the BUFR descriptor file.
<p class="plaintext">The GrADS station data interface requires a few pieces of
metadata for each report: the location of the station (lat/lon), a station ID
(a string no more than 8-characters long), a pressure level (if it is an upper
air variable), and a time stamp. The BUFR descriptor file must provide the X,Y
pairs for these metadata fields, plus a few other elements.
<p class="plaintext">Descriptor file entries used for BUFR files are:
<table width="600" cellpadding="2" cellspacing="5" class="plaintext">
<tr bgcolor="#b8d8dd">
<td>DSET</td>
<td>This entry points to the BUFR data file. It is not currently recommend
to use templating with BUFR data. (See <a href="#templating">Note on Templating</a>
below)</td>
</tr>
<tr bgcolor="#b8d8dd">
<td>TITLE</td>
<td>It is good general practice to include a descriptive title in every GrADS
descriptor file. </td>
</tr>
<tr bgcolor="#b8d8dd">
<td>UNDEF</td>
<td>This is required by GrADS, but not used for undef-testing in the BUFR
interface. Place an arbitrary number here that is unlikely to be confused
with a good data value.</td>
</tr>
<tr bgcolor="#b8d8dd">
<td width="76">DTYPE </td>
<td width="609">This entry should have the 'bufr' keyword. This data type
<em>must</em> be accompanied by the XVAR, YVAR, TVAR, and STID entries.
</td>
</tr>
<tr bgcolor="#b8d8dd">
<td> STID </td>
<td>This required entry provides the X,Y pair for the station ID. </td>
</tr>
<tr bgcolor="#b8d8dd">
<td> XVAR</td>
<td>This required entry provides the X,Y pair for the station's longitude.</td>
</tr>
<tr bgcolor="#b8d8dd">
<td>YVAR</td>
<td>This required entry provides the X,Y pair for the station's latitude.</td>
</tr>
<tr bgcolor="#b8d8dd">
<td> ZVAR</td>
<td>This optional entry provides the X,Y pair for the data's vertical coordinate
(usually pressure). This is only required if there are level-dependent variables
in the BUFR file. </td>
</tr>
<tr bgcolor="#b8d8dd">
<td> TVAR <br>
and <br>
TOFFVAR</td>
<td>
<p>The time for any individual BUFR station report is the base time plus
the offset time.The TVAR entry is required and provides the X,Y pairs
for the base time coordinate variables. The TOFFVAR entry provides the
X,Y pairs for the offset time. If the offset time is zero, the TOFFVAR
entry is not required. Each time coordinate variable (year=yr, month=mo,
day=dy, hour=hr, minute=mn, second=sc) is presented as a 2-letter abbreviation
followed by the X,Y pair that goes with that time unit. All six base/offset
time units are not required to appear in the TVAR/TOFFVAR record, only
those that are in the data file.</p></td>
</tr>
<tr bgcolor="#b8d8dd">
<td>TDEF</td>
<td bgcolor="#b8d8dd">For BUFR station data, the time axis defined by the
TDEF entry provides an evenly-spaced framework for the (sometimes) unevenly
spaced BUFR station reports to fit into. Choose a TDEF that spans the time
range of your BUFR data and has a time increment that matches the frequency
of the BUFR reports. (See <a href="#tdef">Note on TDEF</a> below)</td>
</tr>
<tr bgcolor="#b8d8dd">
<td> VARS<br>
through <br>
ENDVARS</td>
<td>The variable declarations in a BUFR descriptor file also have special
features. The <em>varname </em>field may be any 15-character alphanumeric
string that must start with an alphabetic character (a-z). It is not necessary
for the varname in the descriptor to match the varname in the BUFR file.
The <em>levs</em> field is 0 for surface variables, 1 for upper air variables.
Exception to this rule: replicated surface variables (i.e. variables for
which there may be more than one observation, such as present weather) are
given a<em> levs</em> value of 2. The <em>units</em> field contains the
X,Y pair for the named variable. </td>
</tr>
</table>
<p><br>
<span class="plaintext">The first step in generating a BUFR descriptor file
is figuring out the X,Y values for the data and metadata variables that GrADS
requires. Begin by perusing the header output from <a href="gradutilbufrscan.html"><code>bufrscan</code></a>
looking primarily at the numeric elements. Here is a link to some <a href="bufr.sample.headers">example
header output</a> -- a subset of this is given below. </span>
<p class="plaintext">These lines give the base time for the report, the station
identifier, the location and elevation of the station, plus some of the observed
variables:<br>
0 04 001 (numeric) YEAR YEAR <br>
0 04 002 (numeric) MNTH MONTH <br>
0 04 003 (numeric) DAYS DAY <br>
0 04 004 (numeric) HOUR HOUR <br>
0 04 005 (numeric) MINU MINUTES<br>
0 01 198 (text) RPID REPORT IDENTIFIER <br>
0 06 002 (numeric) CLON LONGITUDE (COARSE ACCURACY) <br>
0 05 002 (numeric) CLAT LATITUDE (COARSE ACCURACY) <br>
0 07 001 (numeric) SELV HEIGHT OF STATION <br>
0 11 001 (numeric) WDIR WIND DIRECTION <br>
0 11 002 (numeric) WSPD WIND SPEED <br>
0 12 101 (numeric) TMDB TEMPERATURE/DRY BULB TEMPERATURE <br>
0 12 103 (numeric) TMDP DEW POINT TEMPERATURE <br>
0 10 051 (numeric) PMSL PRESSURE REDUCED TO MSL<br>
0 10 061 (numeric) 3HPC 3 HOUR PRESSURE CHANGE
<p class="plaintext">The corresponding descriptor file entries would look like
this (N.B. This is not a complete descriptor file): <br>
TVAR yr 4,1 mo 4,2 dy 4,3 hr 4,4 mn 4,5 <br>
STID 1,198<br>
XVAR 6,2<br>
YVAR 5,2<br>
VARS 9<br>
slon 0 6,2 Station longitude<br>
slat 0 6,2 Station latitude<br>
selv 0 7,1 Station elevation<br>
wdir 0 11,001 Wind direction<br>
wspd 0 11,002 Wind speed<br>
temp 0 12,101 Temperature<br>
dewpt 0 12,103 Dew point temperature<br>
mslp 0 10,51 Mean sea level pressure<br>
dp 0 10,004 3-hour pressure change <br>
ENDVARS
<p class="plaintext"> The internal GrADS variables "lat", "lon",
and "lev" do not exist for station data, so it's a good idea to put
them in the variable list in case you need them for any calculations (BUFR variables
can be both coordinate and data variables at the same time). Just be careful
not to assign the names "lat", "lon" or "lev",
as this will confuse GrADS and you'll get the message that the predefined variable
is only for grid type files.
<p class="plaintext">The second step is to figure out what to use for a TDEF entry.
You may be aware ahead of time that your BUFR file contains hourly data covering
a known 6-hour period, in which case you are done (<code>TDEF 6 <em>start_of_period</em>
1hr</code>). But if you have no idea what's in your BUFR file, then you need
to examine the data output from <a href="gradutilbufrscan.html"><code>bufrscan</code></a>
looking for the F-X-Y triplets that appear in your TVAR entry. Here is a link
to some <a href="bufr.sample.data">example data output</a> -- a subset of this
is given below.
<p class="plaintext">7 (0) [-001] 0-04-001 2004<br>
7 (0) [-001] 0-04-002 4<br>
7 (0) [-001] 0-04-003 22<br>
7 (0) [-001] 0-04-004 15<br>
7 (0) [-001] 0-04-005 0<br>
<p class="plaintext">These lines indicate an appropriate TDEF might be:<br>
TDEF 1 linear 15z22apr2004 1hr
<p class="plaintext">If you found more occurrences of 0-04-004 with values other
than 15, change your TDEF:<br>
TDEF 24 linear 00z22apr2004 1hr
<p class="plaintext">If you found occurrences of 0-04-003 with values equal to
23 as well as 22, change your TDEF again:<br>
TDEF 48 linear 00z22apr2004 1hr
<p class="plaintext">
<p class="plaintext"><a name="tdef"></a><strong>Note on TDEF:</strong><br>
As mentioned above, the time axis you describe with TDEF provides an evenly-spaced
framework for the station reports to fit into. A display request at a specific
time will return all station reports whose time stamp is within a range of times
equal to the specific time plus or minus one half of the time axis interval.
Let's say you set TDEF to be hourly, set the time dimension to 12Z, and then
do a display request. GrADS will sift through every report in the BUFR file
and display only those which fall between 11:30Z and 12:30Z. If you change TDEF
to be six-hourly, set the time dimension to12Z, and then do a display request,
GrADS will show you all reports that fall between 09Z and 15Z. GrADS displays
the reports that fit between the grid points based on the time axis and time
dimension environment you describe. In BUFR, there's no reason to expect the
messages to be ordered in terms of time or place, and there's no index file
to help navigate the data file, so for each display request GrADS has to sift
through the whole file.
<p class="plaintext"><strong><a name="templating"></a>Note on Templating: <br>
</strong>In GrADS, file templating (aggregation) is only done in the the time
dimension. It's handled by matching substrings in the file names with time dimension
values. After a display request, GrADS determines which data files need to be
opened in order to get the data the user requested, based on the current dimension
environment settings. Templating in this manner should be avoided with BUFR.
Since there's no index file to help GrADS navigate the data file, a BUFR file
containing 10 reports is treated exactly the same way as a BUFR file with 10,000
-- the whole file is read into memory when the user submits the first display
request. For each display request, GrADS sifts through the entire collection
of reports to find the ones that match the current dimension environment. Since
the memory usage is so extreme, it's better to keep BUFR files small and only
access one at a time. When necessary and/or appropriate, it's possible to aggregate
BUFR files by simply concatenating them together.
</body>
</html>
