2010-04-06 Charlie Zender <zender@uci.edu> * NCO 4.0.1 release procedure: Changes since nco-4_0_0: fix_rec_dmn, mk_rec_dmn, .ncrename cd ~/nco;cvs tag -d nco-4_0_1;cvs tag -d nco-4_0_1-1 cd ~/nco;cvs commit -m "Preparing nco-4.0.1 release" cvs tag -c nco-4_0_1;cvs tag -c nco-4_0_1-1 * Revert change to libnetcdf-dev (>= 1:4.0.0) in control since Ubuntu does not yet support newer version. 2010-04-05 Charlie Zender <zender@uci.edu> 2010-04-04 Charlie Zender <zender@uci.edu> * Change Debian to libnetcdf4 with libnetcdf-dev (>= 1:4.0.0) in control 2010-03-29 Charlie Zender <zender@uci.edu> * Cleanup and document '--mk_rec_dmn' switch 2010-03-22 Henry Butowsky <henryb@hush.com> * Add '--mk_rec_dmn' switch to ncks: Allows user to (re-)define the record dimension. If the redefined record dimension is not the first dimension of a given variable, then ncks fails with an error message. 2010-03-15 Charlie Zender <zender@uci.edu> * Remove consti-fication of return values in global functions 2010-03-13 Charlie Zender <zender@uci.edu> * Comment-out "\input texinfo" from my-bib-macros.texi to prevent that string from appearing in printed output. 2010-03-11 Charlie Zender <zender@uci.edu> * Fix ncrename.c to not die after skipping non-existent targets (variables/dimensions/attributes) preceded by '.'. Longstanding bug! * Supercede NO_REC_DMN with FIX_REC_DMN * Various cleanups of ncoGrammer.g, nco_mss_val.c, ncks.c, nco_var_utl.c 2010-03-11 Henry Butowsky <henryb@hush.com> * Add '--no_rec_dmn' switch to ncks: causes record dimension in input to be converted to a regular dimension in output. 2010-03-01 Henry Butowsky <henryb@hush.com> * Added to manual chapter on Re-basing Time Coordinate 2010-02-16 Charlie Zender <zender@uci.edu> * Cleanup nco_lmt.c 2010-02-16 Henry Butowsky <henryb@hush.com> * Add ncrcat tests for modifications below 2010-02-15 Henry Butowsky <henryb@hush.com> * Hyperslab limits of type lmt_crd_val/lmt_udu_sng with stride > 1 now work with multifile operators, e.g., ncrcat -v time -d time,3.0,30.0,8 t1.nc t2.nc out.nc 2010-02-09 Charlie Zender <zender@uci.edu> * Remove MPI print line from non-MPI code in mpncra.c to fix build bug reported by Mark Van Moer 2010-02-05 Charlie Zender <zender@uci.edu> * Cleanup ncbo.c 2010-01-29 Henry Butowsky <henryb@hush.com> * Tidy-up value_list action in tree parser. Added value_list_string for NC_STRING * Added ncap_typ_hgh() to libnco 2010-01-26 Charlie Zender <zender@uci.edu> * As of 20091030 NCO_TYP_INT defaults to int rather than long Following changes bring nomenclature into alignment with this * Change NC_INT variable names from *_lng to *_ntg * Change ptr_typ union for NC_INT from lp to ip * Change val_typ union for NC_INT from l to i * Remove nco_gmm.[ch] to avoid Burkardt copyright Gamma functions now supplied by GSL. * Replace " != NULL" and " != NULL_CEWI" with "" This is just code cleanup which is always valid When pointer values are evaluated as boolean conditions, it is identical to comparing the pointer to NULL. Hence code now has, e.g., if(cmd_ln) ... instead of if(cmd_ln != NULL) Could/should also replace "foo == NULL" with !foo? 2010-01-19 Charlie Zender <zender@uci.edu> * Workaround ncoGrammer.g dependence on NC_MAX_INT which DNE in netcdf.h until ~2009 2010-01-19 Henry Butowsky <henryb@hush.com> * Add chapter on "GSL Least-Squares Fitting" to ncap2 manual 2010-01-18 Charlie Zender <zender@uci.edu> * Fix out of range index in fmc_gsl_cls.c 2010-01-14 Charlie Zender <zender@uci.edu> * Patch configure.in for broken GSL installations like bluefire's, where GSL exists though gsl-config does not * Only attempt duplicating netCDF4 settings when output is netCDF4 Do not rely on no-ops from netCDF3 and netCDF4-classic files 2010-01-08 Charlie Zender <zender@uci.edu> * Patch --enable-netcdf-4 implementation 2010-01-05 Charlie Zender <zender@uci.edu> * configure: make --enable-netcdf-4 synonym for --enable-netcdf4 * Add ${misc:Depends} to Depends rule in debian/control * NCO 4.0.0 release procedure: Changes since nco-3_9_9: bilinear, fitting, chunking cd ~/nco;cvs tag -d nco-4_0_0;cvs tag -d nco-4_0_0-1 cd ~/nco;cvs commit -m "Preparing nco-4.0.0 release" cvs tag -c nco-4_0_0;cvs tag -c nco-4_0_0-1 * Change HAVE_NEW_CHUNKING_API to NC_HAVE_NEW_CHUNKING_API to leverage information now in netcdf.h about correct API. Required for successful .deb builds with netCDF3. 2009-12-10 Henry Butowsky <henryb@hush.com> * Added initialization to NULL of chunk sizes prs_arg.cnk_sz in ncap2.cc. This was causing a bug on CentOS builds. 2009-11-20 Henry Butowsky <henryb@hush.com> * Added to manual chapter on bilinear_interp_wrap() 2009-11-16 Henry Butowsky <henryb@hush.com> * Tidy ups in main() ncpdq/ncecat/ncflint/ncra/ncbo 2009-11-09 Henry Butowsky <henryb@hush.com> * Modify 'dsort(var,&var_mp)' if output file is netcdf4 then the created map is of type NC_UINT64 else it is of type NC_INT * Refactor all gsl_stats code -hnd_fnc_stat? So that the 'size_t n' argument is user input to functions 2009-11-07 Charlie Zender <zender@uci.edu> * Verify nco_is_spc_in_crd_att() implementation on NARCCAP file * Add nco_is_spc_in_crd_att() to test whether variable is specified in "coordinates" attribute * Change sprintf() and scanf() formatting for nco_int from long to int * Implement empirical list of N-D coordinate "fixed" variables: lat, lon, lev, longxy, latixy * Change fixed variable exclusion condition from is_ncbo to is_rnk_prs_rth_opr() 2009-10-30 Charlie Zender <zender@uci.edu> * #define NCO_INT NCO_TYP_INT instead of NCO_TYP_LONG Longs are 8-bytes on 64-bit systems, and NCO_INTs are always 4 bytes. Stuffing the 4 bytes into 8 bytes of RAM is harmless locally, and ensures the NCO code is robust in other ways. However, it also seems to confuse/break DAP-handling of ints. 2009-10-29 Charlie Zender <zender@uci.edu> * Add short option -6, and better document -6, --64, --64bit options 2009-10-21 Henry Butowsky <henryb@hush.com> * Ensure ncap2 compiles with NCO_INT==NCO_TYP_INT. Affects fmc_all_cls.cc msk_cls srt_cls fmc_gsl_cls.cc, map_srt_tmp.hh, ncoGrammar.g 2009-10-15 Henry Butowsky <henryb@hush.com> * Tidy-up sort methods srt_cls 2009-10-13 Henry Butowsky <henryb@hush.com> * Add secret flag tst_udunits option to ncks. Usage: --tst_udunits,'units_in','units_out','calendar type' 2009-09-30 Charlie Zender <zender@uci.edu> * Remove attempt to set chunking in ncap2 writes 2009-09-26 Charlie Zender <zender@uci.edu> * Add CNK option to Makefile to set HAVE_NEW_CHUNKING_API to handle netCDF 4.0.x vs. netCDF 4.1+ chunking API differences * Fix library flags for parallel HDF/netCDF4 2009-09-24 Henry Butowsky <henryb@hush.com> * Add manual sub-section on calendar CF support in UDUnits node 2009-09-21 Charlie Zender <zender@uci.edu> * Implement Intel mpicc/mpiCC compilers on greenplanet with lustre 2009-09-11 Henry Butowsky <henryb@hush.com> * Committed first cut of nco_cal.{c,h} * These are calendar functions for hyperslabbing coordinate variables with CF attribute "calendar". Currently supported are "360_day","no_leap","365_day". Other values for date evaluation default to "standard", the UDUnits1/2 calendar functionality. NB: regardless of the chosen calendar type all date strings are parsed by UDUnits1/2 2009-08-28 Charlie Zender <zender@uci.edu> * Add spiral logo by Andrea Cimatoribus to bakeoff 2009-08-26 Henry Butowsky <henryb@hush.com> * Change nco_lmt_udu_cnv() UDUnits1 code branch Parse bare date with utScan() instead of manually 2009-08-25 Charlie Zender <zender@uci.edu> * Add numeric cog logo by Rob Hetland to the bakeoff 2009-08-21 Henry Butowsky <henryb@hush.com> * Tidy-up of srt_cls---functions for sorting vars 2009-08-21 Henry Butowsky <henryb@hush.com> * Fix spline interpolation with gsl-1.12 * Tidy-up gsl_spl_cls() 2009-08-19 Henry Butowsky <henryb@hush.com> * Add first cut of GSL least squares fitting to fmc_gsl_cls 2009-08-17 Henry Butowsky <henryb@hush.com> * Modify nco_lmt_typ() to recognize plain dates of form "%d-%d-%d" as valid UDUnits strings 2009-08-10 Charlie Zender <zender@neige> * Define chunking compatibility tokens in nco_netcdf.c * Verify WIN32 build on neige 2009-08-07 Charlie Zender <zender@uci.edu> * Chunking passes all regression tests and makes sense at last * Ensure that record, compressed, and checksummed variables are chunked, stay chunked, and are not subject to unchunking attempts. Passes regression tests. * Only fill in shuffle, deflate, dfl_lvl when pointers are non-NULL in netCDF3 nco_inq_var_deflate() * Add nco_inq_var_fletcher32() 2009-07-26 Charlie Zender <zender@uci.edu> * More man page tweaks for ellipses 2009-07-24 Charlie Zender <zender@uci.edu> * Fix small memory leak in nco_vrs_att_cat() 2009-07-23 Charlie Zender <zender@uci.edu> * Remove NULL-dereferencing, segfault-causing problem in nco_netcdf.c 2009-07-21 Henry Butowsky <henryb@hush.com> * Add first cut of bil_cls - functions for bilinear interpolation of gridded 2D data 2009-07-16 Charlie Zender <zender@uci.edu> * Replace ellipsis ("\.\.\.") to better ROFF ellipsis ("\ .\|.\|.\ ") Found this hack by examining /usr/share/man/man1/nroff.1.gz * Convert ROFF hyphens ("-") to ROFF minuses ("\-") * Examined Debian build logs and cleaned up minor problems * Correct ROFF hypens to minuses, and format ellipsis in man pages * NCO is once again C++-compliant, compiles/runs/tests fine with g++ * Explicitly set ddra_info.flg_ddra=False in g++-compiled code since C++ does not allow structure-member initialization * Satisfy C++ by explicitly casting (void *) to (lmt_sct *) and to (char *) (rather than relying on implicit conversion) 2009-07-15 Charlie Zender <zender@uci.edu> * Thank all packagers in nco.texi * Barry deFreese uploaded 3.9.9 to Debian Sid. Hallelujah! * NCO 3.9.9 release procedure: Changes since nco-3_9_8: chunking, NC_STRING, GSL stats/interp, rebasing cd ~/nco;cvs tag -d nco-3_9_9;cvs tag -d nco-3_9_9-1 cd ~/nco;cvs commit -m "Preparing nco-3.9.9 release" cvs tag -c nco-3_9_9;cvs tag -c nco-3_9_9-1 * Cleaned up and annotated copyright history in copyright file * Added Benjamin Sittler and John Burkhardt to copyright file * Received copyright assignment and removed Henry Butowsky from copyright headers in nco_msa.c, nco_msa.h * Received copyright assignment and removed Karen Schuchardt from copyright headers in nco_aux.c 2009-07-14 Charlie Zender <zender@uci.edu> * Henry fixed UDUnits1 code to pass all UDUnits2 regression tests 2009-07-13 Charlie Zender <zender@uci.edu> * Cleanup and annotate nc_def_var_chunking() API problems * Fix HAVE_NETCDF4_H test when enable_netcdf4=yes * Change Debian tgz file suffix from erroneous '-1.tar.gz' to '.orig.tar.gz' * Upgrade debhelper compatibility by changing debian/compat from 4 to 5 * Fix configure.in bug in HAVE_NETCDF4_H test 2009-07-12 Charlie Zender <zender@uci.edu> * 20090712: For unknown reason autobuild sees netCDF4 netcdf.h header when it should not. Moreover, autobuild thinks "const nco_string *sng" conflicts with "const char **sng" in all nc_put_var?_string() and nc_put_att_string() functions. Even though nco_string = char * = should be synonymous Implement kludge workaround to latter until former is solved * Fix prototype error for NCO_GET_VAR[1ASM]_USHORT, NCO_GET_ATT_USHORT Buffer had erroneously been set to nco_ubyte in these functions * Introduce HAVE_NETCDF4_H to indicated presence of netCDF4 headers but not necessarily netCDF4 libraries. Use HAVE_NETCDF4_H instead of ENABLE_NETCDF4 in some checks ENABLE_NETCDF4 now refers to presence of netCDF4-enabled library Thus ENABLE_NETCDF4 implies HAVE_NETCDF4_H The converse is not true * Clean-up, de-space NCO_rgr.pm 2009-07-07 Charlie Zender <zender@uci.edu> * Add Rich Signell's "cog" logos to homepage and doc directory 2009-06-26 Henry Butowsky <henryb@hush.com> * New GSL class gsl_spl_cls provides GSL spline interpolation wrappers * LHS hyperslabs of multidimensional variables with a single index three_dmn_var_dbl(1,1,2)=10.0 is equivalent to three_dmn_var_dbl(14)=10.0 2009-06-15 Charlie Zender <zender@uci.edu> * Restrict gsl_stats_correlation() to NCO_GSL_MINOR_VERSION >= 10 2009-06-14 Charlie Zender <zender@uci.edu> * Only unchunk chunked variables 2009-06-11 Charlie Zender <zender@uci.edu> * Clean-up re-basing code 2009-06-09 Charlie Zender <zender@uci.edu> * Chunking works: plc=all for map=rd1,dmn Chunking fails : plc=g?d for map=rd1,dmn Chunking fails : plc=uck * Allow cnk_map=dmn to work with record hyperslabs, e.g., ncks -O -4 -D 4 --cnk_map=dmn -d time,0,3 ~/nco/data/in.nc ~/foo.nc * Improve chunking corner-case logic * Restore precedence of explicitly specified chunking for record dimension over that implicitly specified by chunking map rd1 * Discriminate between BASIC_DMN and non BASIC_DMN * Fix chunking to look at dmn_sz_org for record variables 2009-06-05 Henry Butowsky <henryb@hush.com> * nco_lmt_udu_cnv() supports UDUnits2 for parsing timestamps in user specified co-ordinate limits * Added hnd_fnc_stat4() method to gsl_cls to handle gsl_stats_covariance/gsl_stats_correlation gsl_stats_pvariance/gsl_stats_ttest 2009-06-02 Charlie Zender <zender@uci.edu> * Add CUDA options to Makefile 2009-05-29 Charlie Zender <zender@uci.edu> * Improve chunking diagnostics when copying files with ncks * ncks and ncecat chunksize implementations seem to perform identically * Implement new chunksize API into ncecat.c for testing * Remove old chunksize API from non-ncks operators 2009-05-29 Henry Butowsky <henryb@hush.com> * Tidy up nco_lmt_udu_cnv() into three functions 1) for no udunits 2) for udunits1 3) for udunits2 2009-05-26 Charlie Zender <zender@uci.edu> * Change rcd_one to rd1 * Add chunksize usage to ncks in nco_ctl.c, nco.texi, ncks.1 * Improve cnk_map/cnk_plc defaults, minimize their main.c footprints * ncks copies original deflation and chunking parameters by default 2009-05-25 Charlie Zender <zender@uci.edu> * Per-dimension chunksizes in nco_prn() * nco_cnk_sz_set() beta works well, still need uck * Draft nco_cnk_sz_set() ready for testing * Add nco_vrs_att_cat() for later use * Harmless nco_cnk_sz_get() implemented for ncks 2009-05-22 Charlie Zender <zender@uci.edu> * Comment lmt_all_lst handling * Adapt chunksize infrastructure from limit parsing * bin_cnt.nco documentation * nco_typ_sgn() infrastructure 2009-05-17 Charlie Zender <zender@uci.edu> * Add nco_cnk.[ch] for chunking functions 2009-05-19 Henry Butowsky <henryb@hush.com> * Committed first cut of GSL statistical functions 2009-05-08 Charlie Zender <zender@uci.edu> * Begin chunking documentation in nco.texi 2009-05-07 Charlie Zender <zender@uci.edu> * Add autogen.sh from Magnus Hagdorn * Patch configure.in for antlr detection and netcdf.h (Magnus Hagdorn) * Clean up NC_STRING handling and add printed ncks info. 2009-05-05 Charlie Zender <zender@uci.edu> * Need test for whether installed netCDF is version 3 or 4 * Re-do netCDF4 configure: Remove redundants tests, new features: enable-netcdf4 is default true disable-netcdf4 should work when only netCDF4 installed? * Configure allows --enable-shared with netCDF4-enabled NCO 2009-05-05 Henry Butowsky <henryb@hush.com> * Tidy up user error messages in fmc_all_cls.cc 2009-05-04 Charlie Zender <zender@uci.edu> * Create/diagnose more complete netcdf4.nc files from netcdf4.nco * Verify that netCDF4 problems in netcdf4.nco as of 20080512 have been fixed (by updating to snapshot netCDF4) as of 20090504. One remaining problem is that NCO silently succeeds on overflows. Should it? * Improve logic/warnings/info for DAP-wget fall-throughs 2009-05-03 Charlie Zender <zender@uci.edu> * Some chunking parameters work, some cause "Bad chunk sizes" errors * libnco builds cleanly on gcc and icc with netCDF3/4 * Cast NC_TYPE naked tokens to nc_type to finesse icc compiler warnings * Link to libnsl.a only for Solaris and HPUX * Tweak NCO wrappers to isolate all weak prototype compiler warnings (caused by netCDF4 prototypes) in nco_netcdf.[ch] 2009-05-02 Charlie Zender <zender@uci.edu> * Verify cnk_sz stubs pass all regression tests * Allow DAP_NETCDF with ENABLE_NETCDF4 in Makefile's * Add cnk_sz to var_sct, dmn_sct, var_dfl_set(), nco_var_free(), and nco_var_dpl() * Add chunking section to manual * Change cnk_sz from int * to size_t * * Include newer nco++ files in "make tags" * Improve in_4.cdl availability, conversion, documentation * Print chunking parameters, when used, with ncks -m 2009-05-01 Charlie Zender <zender@uci.edu> * Enable command-line chunking arguments to ncks only Generic chunking requires dimension/variable-specific routines similar to dimension limits for hyperslabbing, something like rcd=nco_cnk_sz_set(cnk_sz_lst,cnk_nbr) nco_cnk_sz_get(dmn_lst,chk_sz) * Add NCO-wide chunking infrastructure 2009-05-01 Henry Butowsky <henryb@hush.com> * Rewrite RHS hyperslabbing routine var_lmt() in grammer file * More hyperslabbing tests in ncap2_tst.nco 2009-04-30 Charlie Zender <zender@uci.edu> * Output GCC compiler version from nco_cmp_get() 2009-04-29 Charlie Zender <zender@uci.edu> * Changes since nco-3_9_7: DAP-netCDF support, sort(), GSL distributions, chunksize cd ~/nco;cvs tag -d nco-3_9_8;cvs tag -d nco-3_9_8-1 cd ~/nco;cvs commit -m "Preparing nco-3.9.8 release" cvs tag -c nco-3_9_8;cvs tag -c nco-3_9_8-1 * man pages tweaks for nroff * UDUnits2 build tweaks for Debian * Change debian files per Barry deFreese's instructions 2009-04-29 Henry Butowsky <henryb@hush.com> * Add array function usage: var_out=array(srt,ncr,$dim); * Add more GSL handler functions for _ran and _pdf 2009-04-23 Henry Butowsky <henryb@hush.com> * Add total() regression test for missing_values to ncap2_tst.nco 2009-04-22 Henry Butowsky <henryb@hush.com> * Added unary functions abs(), sqr(), fabs(): abs()=absolute value for any type; fabs()=absolute value for float/double types; sqr()=square of number for any data type 2009-04-21 Charlie Zender <zender@uci.edu> * Add COPYING and INSTALL to top-level directory to make autoreconf, and FSF, happy 2009-04-20 Charlie Zender <zender@uci.edu> * ncks --secret lists hidden/unsupported functions * UDUnits2 supported by configure --enable_udunits2 2009-04-18 Charlie Zender <zender@uci.edu> * NCO_GSL_MINOR_VERSION >= 8 for select GSL *_cdf_* and *_ran_* functions * Add abs() as synonym for fabs() in ncap2 2009-04-16 Charlie Zender <zender@uci.edu> * Add nco_def_var_chunking() and stubs for chunk-size patch 2009-03-27 Charlie Zender <zender@uci.edu> * Add ncap2 threading to all docs: .texi, .man, ncap2 -r * Restrict other operators to maximum four threads by default * Turn-off ncap2 threading (until ANTLR threading issues resolved?) * Clean-up ncap2 2009-03-26 Charlie Zender <zender@uci.edu> * Place new NCO logo candidate, logo_pch, by Tony Freeman, on homepage 2009-03-25 Henry Butowsky <henryb@hush.com> * Change assign action to redefine non-scalar variable with RHS scalar: e.g., n1=three_dmn_var_dbl;n1=20; fills 3-D array n1 with 20 and retains missing value, if any 2009-03-21 Charlie Zender <zender@uci.edu> * More attempts to configure DAP-netCDF 2009-03-20 Charlie Zender <zender@uci.edu> * Introduce full DAP-netCDF block to configure 2009-03-19 Charlie Zender <zender@uci.edu> * Changes since nco-3_9_6: improve GSL-builds, ncap2 threading cd ~/nco;cvs tag -d nco-3_9_7;cvs tag -d nco-3_9_7-1 cd ~/nco;cvs commit -m "Preparing nco-3.9.7 release" cvs tag -c nco-3_9_7;cvs tag -c nco-3_9_7-1 * Remove archaic TEST_DAP * Finally get gsl-config to work in automake * Cleanup compiler warnings in fmc_gsl_cls.cc 2009-03-18 Charlie Zender <zender@uci.edu> * Dummy stub function so fmc_gsl_cls.o not empty when GSL unavailable 2009-03-17 Charlie Zender <zender@uci.edu> * Convert DAP to DAP_OPENDAP, add stub for DAP_NETCDF in configure.in * Eliminate obsolete ${DAP_ROOT} lines from configure.in * Rename DAP option to DAP_OPENDAP in preparation DAP_NETCDF option * More GSL tidying 2009-03-06 Charlie Zender <zender@uci.edu> * Yet another GSL configure patch 2009-03-05 Charlie Zender <zender@uci.edu> * GSL configure patch 2009-02-27 Henry Butowsky <henryb@hush.com> * Add srt_cls to fmc_all_cls.cc Only one method at the moment var_expr.sort() 2009-02-26 Charlie Zender <zender@uci.edu> * GSL build tweaks for NCO_GSL_MINOR_VERSION * Adding texinfo bibliography to nco.texi 2009-02-25 Charlie Zender <zender@uci.edu> * Edit ncap2 GSL documentation 2009-02-24 Henry Butowsky <henryb@hush.com> * Add basic macro to the spreadsheet doc/nco_gsl.ods Macro outputs selection in texi @multitable format 2009-02-24 Charlie Zender <zender@uci.edu> * Add doc/.htaccess file to copy to sourceforge to re-enable directory index display 2009-02-24 Henry Butowsky <henryb@hush.com> * Add chapter on ncap GSL functions 2009-02-23 Charlie Zender <zender@uci.edu> * Use native float prototypes for math functions if __INTEL_COMPILER 2009-02-13 Henry Butowsky <henryb@hush.com> * Global ncap_gsl_mode_prec initialized from environment var. GSL_PREC_MODE This sets the precision of the Airy & elliptical functions: GSL_PREC_DOUBLE=0 GSL_PREC_SINGLE=1 GSL_PREC_APPROX=2 2009-02-12 Henry Butowsky <henryb@hush.com> * Made gsl_cls aware of GSL library version Add pre-processor token NCO_GSL_MINOR_VERSION Token is initialized in nco++/Makefile.old 2009-02-09 Charlie Zender <zender@uci.edu> * Added sym_cls.[cc,hh] and fmc_all_cls.[cc,hh] to configure 2009-02-03 Charlie Zender <zender@uci.edu> * Tweak ./configure for GSL 2009-02-02 Charlie Zender <zender@uci.edu> * Add variable wildcarding in ncatted example * Use nco_lst_prs_sgl_2D() not nco_lst_prs_2D() to avert TODO nco944 in nco_var_lst_crd_add(). Fixes crash on 'ncks -c ~/nco/data/in.nc' separately listed as TODO nco963. * ncks --no_dmn_var_nm implements naked printing requested in TODO nco962 2009-01-28 Charlie Zender <zender@uci.edu> * Specify Szip library in bld/Makefile[.old] with SZ and SZ_LIB * automake now builds src before doc in case makeinfo not present * Allow specification of Szip library in bld/Makefile with SZ and SZ_LIB * Allow configure to work when GSL_ROOT specified and when gsl-config is not necessarily on path * Eliminate GSL references from libnco++ when GSL not present * Tweak configure.in for GSL 2009-01-27 Charlie Zender <zender@uci.edu> * Prefix GSL header file search path with 'gsl/' 2009-01-22 Charlie Zender <zender@uci.edu> * cd ~/nco;cvs tag -d nco-3_9_6;cvs tag -d nco-3_9_6-1 cd ~/nco;cvs commit -m "Preparing nco-3.9.6 release" cvs tag -c nco-3_9_6;cvs tag -c nco-3_9_6-1 Changes since nco-3_9_5: fix MSA bug, add GSL * Added m4 directory * Updated m4, libtool, autoconf macros with autoreconf -f -i 2009-01-20 Charlie Zender <zender@uci.edu> * Separate HDF5_ROOT from NETCDF4_ROOT in bld/Makefile * Update copyright year to 2009 2009-01-20 Henry Butowsky <henryb@ntlworld.com> * Removed gmm_inc_cls() and associated code from ncap2_utl.cc. Superseded by GSL library gamma functions. 2009-01-16 Charlie Zender <zender@uci.edu> * Add warning about MSA threading bug 2009-01-13 Charlie Zender <zender@uci.edu> * Tweaked bld/Makefile to incorporate Russ Rew's modifications 2009-01-08 Henry Butowsky <henryb@ntlworld.com> * Added gsl wrapper functions fmc_gsl_cls.cc, fmc_gsl_cls.hh. Added test script data/gsl_sf.in 2008-12-10 Charlie Zender <zender@uci.edu> * Check for external (rather than internal) netCDF4 function nc4_enddef_netcdf4_file() in libnetcdf.a 2008-12-03 Charlie Zender <zender@uci.edu> * Implement --enable-gsl in configure * Add ENABLE_GSL functionality to configure and to most source code Still needs correct propagation into fmc_cls.cc, maybe ncoGrammar.g 2008-11-04 Charlie Zender <zender@uci.edu> * Document new ncatted rx capabilities for var_nm 2008-11-04 Henry Butowsky <henryb@ntlworld.com> * Added variable wildcarding to ncatted -a arg -a att_nm,var_nm,mode,type,type,val var_nm can now be a regular expression 2008-10-30 Charlie Zender <zender@uci.edu> * Add GSL functionality to nco++/Makefile.old 2008-10-29 Charlie Zender <zender@uci.edu> * Update to build on NCAR bluefire 2008-10-17 Charlie Zender <zender@uci.edu> * Add ncdump -k method to learn file format. Revise NCO method. * Implement new SF shell access through web.sf.net 2008-10-08 Henry Butowsky <henryb@ntlworld.com> * Added indexing to ncap2 section 2008-10-05 Henry Butowsky <henryb@ntlworld.com> * Added to manual chapter on ID Quoting, include files & general tidy up 2008-10-02 Henry Butowsky <henryb@ntlworld.com> * Reversed out new var/att/dim quoting * Tidy up of fmc_cls.cc 2008-09-29 Charlie Zender <zender@uci.edu> * Clean-up ncap_var_var_gmmi() */ 2008-09-27 Charlie Zender <zender@uci.edu> * Prototype ncap2 supports for incomplete gamma functions: gamma_inc_regular(a,x), gamma_inc_complement(a,x), gamma_inc_upper(a,x), gamma_inc_lower(a,x), 2008-09-25 Charlie Zender <zender@uci.edu> * Added nco_gmm.[ch] for incomplete gamma function. Source from http://people.scs.fsu.edu/~burkardt/cpp_src/asa032/asa032.html 2008-09-24 Charlie Zender <zender@uci.edu> * Added Invoke.hh 2008-09-24 Henry Butowsky <henryb@ntlworld.com> * ncap2 supports #include files, syntax is #include "pi.nco" 2008-09-22 Henry Butowsky <henryb@ntlworld.com> * Added get_miss() method to utility class returns the missing value of a variable in Input/Oytput 2008-09-18 Charlie Zender <zender@uci.edu> * make lib_cln now removes NCO C++ libraries libnco++.a and libnco_c++.a 2008-09-17 Charlie Zender <zender@uci.edu> * --version ported to rest of operators * --version only calls nco_vrs_prn() in ncks, rest to follow 2008-09-15 Henry Butowsky <henryb@ntlworld.com> * New rules for variable/attribute/dimension quoting * Can now do 'var_nm'@att_nm or 'var_nm'@'att_nm' or var_nm@'att_nm' * Dimension quoting now $'dim_nm' * Quoting supports the following characters: 'a..z'|'A..Z'|0..9|'_'|'-'|'+'|'.'|'('|')'|':'|'@' 2008-09-12 Henry Butowsky <henryb@ntlworld.com> * Added convert(nctype) method 2008-09-02 Charlie Zender <zender@uci.edu> * Add ncks --library to print library version and exit * Split nco_cnf_prn() out of nco_lbr_vrs_prn() 2008-09-02 Henry Butowsky <henryb@ntlworld.com> * Re-enabled -f switch in ncap2. Prints methods available to user. 2008-08-18 Charlie Zender <zender@uci.edu> * Fix/cleanup nco_att_cpy() when _FillValue is global attribute 2008-08-14 Henry Butowsky <henryb@ntlworld.com> * Deleted from parser nonsense NCL-style operators "<<=" ">>=" 2008-08-05 Charlie Zender <zender@uci.edu> * More nco_aed_prc() cleanup 2008-07-31 Charlie Zender <zender@uci.edu> * nco_aed_prc() cleanup 2008-07-28 Henry Butowsky <henryb@ntlworld.com> * Permanent fix for TODO nco944: New parsing function nco_lst_prs_sgl_2D() Returns only non-zero length strings unlike nco_lst_prs_2D() 2008-07-25 Charlie Zender <zender@uci.edu> * Applied 20080718 patch to nco_cnv_csm.c and deferred permanent fix to TODO nco944: handle redundant whitespace "coordinates" attributes before calling nco_lst_prs_2D() 2008-07-23 Charlie Zender <zender@uci.edu> * Notes and cleanup on TODO nco941 kludge 2008-07-22 Charlie Zender <zender@uci.edu> * "ill-defined" warning in nco_var_lst_dvd() uses correct var_typ_fnk 2008-07-18 Charlie Zender <zender@uci.edu> * Allow spaces at the end of CF-convention coordinate attribute strings in nco_var_lst_crd_ass_add() call to nco_lst_prs_2d(). 2008-07-07 Charlie Zender <zender@uci.edu> * Verify bug TODO nco941 where ncra tries to average nc_char variables Reported by Alberto Corbi Bello 2008-06-17 Charlie Zender <zender@uci.edu> * Limit OpenMP thr_nbr_rqs=1 for netCDF4-enabled operators. TODO nco939. 2008-06-02 Charlie Zender <zender@uci.edu> * Patch ./configure for Solaris gcc 2008-05-30 Charlie Zender <zender@uci.edu> * Add Scott's example ncap2 script for 500mb heights * Remove _POSIX_SOURCE from ./configure on Solaris 2008-05-26 Henry Butowsky <henryb@ntlworld.com> * mods to ncatted so it works with _FillValue in NETCDF4 2008-05-26 Charlie Zender <zender@uci.edu> * Change test version to 3.9.6 * Make NCO_NETCDF4_AND_FILLVALUE available in all source files 2008-05-12 Charlie Zender <zender@uci.edu> * cd ~/nco;cvs tag -d nco-3_9_5;cvs tag -d nco-3_9_5-1 cd ~/nco;cvs commit -m "Preparing nco-3.9.5 release" cvs tag -c nco-3_9_5;cvs tag -c nco-3_9_5-1 Changes since nco-3_9_4: MSA everywhere * Correct -X documentation * Clean-up nco_aux.c * Add -X to ncecat, ncflint, ncpdq 2008-05-11 Charlie Zender <zender@uci.edu> * Add -X to ncbo * Add -X to more man pages * Improve -X and ncap2 documentation 2008-05-02 Charlie Zender <zender@uci.edu> * Cleanup ncbo, ncecat, ncflint, ncks, ncpdq, ncra *.c files 2008-04-23 Charlie Zender <zender@uci.edu> * Replace H5LT_open_id with H5LT_find_attribute in configure.in Fixes cygwin build problem with netCDF4. Thanks to Mark Hadfield. 2008-04-21 Charlie Zender <zender@uci.edu> * Merge recent 3.9.0-1.1 Debian NMU patches into upstream * Un-DOS-ify files 2008-04-10 Henry Butowsky <henryb@ntlworld.com> * Added multi-hyperslabbing code to ncra 2008-04-04 Henry Butowsky <henryb@ntlworld.com> * Added multi-hyperslabbing code to ncecat 2008-04-02 Charlie Zender <zender@uci.edu> * Prevent copying global metadata with ncecat -M * Print netCDF4-dependent switches only if netCDF4 is enabled 2008-04-01 Charlie Zender <zender@uci.edu> * Add rule to post-process ANTLR-files with broken tokens: nco_antlr_pst_prc.pl < ncoParserTokenTypes.hpp > ncoEnumTokenTypes.hpp This should fix build problems on Solaris 2008-03-13 Charlie Zender <zender@uci.edu> * Improve error messages 2008-03-06 Charlie Zender <zender@uci.edu> * Add new author's note 2008-03-04 Charlie Zender <zender@uci.edu> * Add man pages to tags * cd ~/nco;cvs tag -d nco-3_9_4;cvs tag -d nco-3_9_4-1 cd ~/nco;cvs commit -m "Preparing nco-3.9.4 release" cvs tag -c nco-3_9_4;cvs tag -c nco-3_9_4-1 Changes since nco-3_9_3: -3, -X, --msa_usr_rdr, ncap2 mask, WIN32 * Make netCDF4 changes to libnco_c++ back-compatible with netcdf 3.6.1 still widely used including by FC DAP RPMs. * Add netCDF4 put_varx_string() functions to nco_c++ 2008-03-02 Charlie Zender <zender@uci.edu> * Compiler warning message cleanup for ncks.c, nco_msa.c, nco_aux.c * Change malingering GPL2 files to GPL3 2008-02-22 Charlie Zender <zender@uci.edu> * Add auxiliary coordinate test variables to in.cdl * Document lon_min,lon_max,lat_min,lat_max ordering * Plug auxiliary coordinate memory leaks (aux=malloc(),aux_arg[]=strdup()) * -X now implies --msa_usr_rdr 2008-02-21 Charlie Zender <zender@uci.edu> * Use 64-bit fill values for uint and long long types * Make compatible with hdf-1.8.0 final and netcdf-4.0-snapshot2008022009 * Transplant chunks of nco_typ.h into nco_utl.hh to make libnco_c++ forward-backward compatible with netCDF3/netCDF4 * Rely on netcdf4 netcdf.h for NCO_FILL_STRING 2008-02-20 Charlie Zender <zender@uci.edu> * Preliminary -X documentation 2008-02-19 Charlie Zender <zender@uci.edu> * Add flg_ddd stub to ncwa to delete degenerate dimensions * MSA miscellania 2008-02-18 Charlie Zender <zender@uci.edu> * ncks --msa_usr_rdr causes MSA to leave hyperslabs in user order 2008-02-13 Henry Butowsky <henryb@ntlworld.com> * First cut of NCL-style operators "<<", "<<=", ">>", ">>=" 2008-02-06 Henry Butowsky <henryb@ntlworld.com> * Added functions "mask" & "mask_clip" 2008-01-22 Henry Butowsky <henryb@ntlworld.com> * Work-arounds related NCO_NETCDF4_AND_FILL_VALUE changes to ncap_var_write_omp(). Added NCAP4_FILL property to prs_cls. 2008-01-17 Charlie Zender <zender@uci.edu> * Re-define NCO_NETCDF4_AND_FILLVALUE to workaround CPP problems 2008-01-16 Charlie Zender <zender@uci.edu> * nco.h now #defines NCO_NETCDF4_AND_FILLVALUE when NCO is netCDF4-enabled and nco_mss_val_sng_get() returns "_FillValue" * Finish -3 option implementation, add --3 too 2008-01-15 Charlie Zender <zender@uci.edu> * Add -3 option to produce CLASSIC files for symmetry with -4 option that produces NETCDF4 files 2008-01-11 Charlie Zender <zender@uci.edu> * Add nco_create_mode_prs() functionality to libnco_c++ 2008-01-07 Charlie Zender <zender@uci.edu> * Fix OpenMP builds of ncap2 with pgCC 2008-01-06 Charlie Zender <zender@uci.edu> * xlC needs to search /usr/vacpp/include to find atan2() * Autoconf builds with AIX fail because unable to find atan2() * Use nco_inq_attlen() to avoid pointer warning in nco_aux.c * Update copyright year to 2008 2008-01-05 Charlie Zender <zender@uci.edu> * Mark TODO nco852 as done (Henry fixed some time ago) 2007-12-29 Charlie Zender <zender@neige> * Put nco_sng_utl.h in headers that may reference strdup() compatibility function * Add math.h definitions to nco_aux.c to resolve M_PI on WIN32 * Fix strdup()-compatibility function prototype * Cleanup some WIN32 warnings 2007-12-22 Henry Butowsky <henryb@ntlworld.com> * Work-arounds for NCO_NETCDF4_AND_FILL_VALUE changes to ncap_var_write_omp(). Added NCAP4_FILL property to prs_cls. 2007-12-20 Charlie Zender <zender@uci.edu> * Use NCO_GET_ATT_CHAR() instead of nc_get_att_text() * Change lenp from size_t to long in nco_aux.c 2007-12-19 Charlie Zender <zender@uci.edu> * Verified no new regressions * Merged nco_aux patch by Karen Schuchardt * Tagged nco-20071219_pre_nco_aux_patch prior to adding nco_aux.c * Fix mpncrcat to work with netCDF4 (thanks to Takeshi Enomoto) 2007-12-16 Charlie Zender <zender@uci.edu> * Add ncecat -u mods to mpncecat TODO nco918 2007-12-14 Charlie Zender <zender@uci.edu> * Add ncecat -u example to manual 2007-12-14 Henry Butowsky <henryb@ntlworld.com> * Added new functions atan2(y,x) and pow() 2007-12-13 Charlie Zender <zender@uci.edu> * Add hint for NC_ENOTNC errors * Make nco_inq_var_deflate() wrapper support netCDF3 and netCDF4 2007-12-12 Charlie Zender <zender@uci.edu> * Fix output/input file consanguinity (TODO nco836) 2007-12-11 Charlie Zender <zender@uci.edu> * ncks prints netCDF4 LZ compression information to stdout TODO910 * ncap2 calls nco_thr_att_cat() to archive nco_openmp_thread_number 2007-12-11 Henry Butowsky <henryb@ntlworld.com> * Added print attributes function e.g., print(global@history) 2007-12-10 Charlie Zender <zender@uci.edu> * Fix ncks.1 typos reported by Rich Signell * CVS rm ssdap 2007-12-09 Henry Butowsky <henryb@ntlworld.com> * ncap2 now uses ncap_lmt_evl() instead of nco_lmt_evl() This fixes problems with record dimension in limits 2007-12-08 Charlie Zender <zender@uci.edu> * Add hooks to nco_prn_var_dfn() to print compression state * cd ~/nco;cvs tag -d nco-3_9_3;cvs tag -d nco-3_9_3-1 cd ~/nco;cvs commit -m "Preparing nco-3.9.3 release" cvs tag -c nco-3_9_3;cvs tag -c nco-3_9_3-1 Changes since nco-3_9_2: SWAMP, ncecat -u, wget, ncap2 OpenMP, GCC 4.2 * Update documentation for SWAMP, AGU, and IJHPCA 2007-11-29 Henry Butowsky <henryb@ntlworld.com> * First cut of OpenMP for ncap2 * ncap2 processes ncap2.in in parallel on LINUX & AIX platforms 2007-11-27 Henry Butowsky <henryb@ntlworld.com> * Added "hyperslab normalization" to the parser 2007-11-16 Charlie Zender <zender@uci.edu> * Add NcapVar.cc to distribution to fix native AIX ncap2 bug 2007-11-13 Charlie Zender <zender@uci.edu> * Propagate Makefile OPTS argument to children * Allow specification of record dimension name with ncecat -u dmn_nm 2007-11-11 Charlie Zender <zender@uci.edu> * Add packing caveat to ncrcat and ncecat documentation 2007-10-31 Charlie Zender <zender@uci.edu> * Update man page ncwa.1 * Add -t thr_nbr to applicable man pages * Remove (temporarily?) deprecated ncwa -n and -W options from nco.texi 2007-10-24 Charlie Zender <zender@uci.edu> * OpenMP-enabled, gcc-compiled NCO passes regression test * Enable OpenMP with GCC 4.2 gcc, g++, gfortran * Change MUST_CONFORM from private() to shared() in ncflint 2007-10-22 Charlie Zender <zender@uci.edu> * Add Makefile support for GCC OpenMP 2007-10-21 Charlie Zender <zender@uci.edu> * Tweak nco_cmp_get() to work with GCC on AIX 2007-10-02 Charlie Zender <zender@uci.edu> * Periods in attribute names choke OPeNDAP Temporarily removed att_nm.dot from var_nm.dot in in.nc 2007-09-18 Charlie Zender <zender@uci.edu> * xlC and SGI CC no longer search for <stdint.h> in nco_var.hh 2007-09-17 Charlie Zender <zender@uci.edu> * Add big.nc to CLEANFILES to (hopefully) fix Debian FTBFS problem with two consecutive builds 2007-09-12 Charlie Zender <zender@uci.edu> * Add helpful hint for DAP -l mis-users 2007-09-03 Charlie Zender <zender@uci.edu> * Finish implementing and testing wget capability. Looks good. * Use wget -r: re-download file (clobber existing file of same name, if any) * Remove leading slash from derived local path for HTTP_URL retrievals 2007-09-02 Charlie Zender <zender@uci.edu> * wget retrieval works iff -l used, regardless of -p use * More exhaustive debugging and local file searches prior to wget attempt * Implement nco_close() to pair with nco_open() in DAP probe. Not doing this previously probably means that NCO via DAP left lots of open files dangling on the DAP server. This may have caused noticeable drag on large jobs. * Finish wget draft code, introduce nco_open_flg() for DAP attempts * Add stubs to retrieve non-DAP HTTP files with wget 2007-09-01 Charlie Zender <zender@uci.edu> * Fix #ifdef NEED_NC_INQ_FORMAT to have else so NCO/netCDF3 build with OPeNDAP DAP=Y option so long as NEED_NCO 2007-08-31 Charlie Zender <zender@uci.edu> * cd ~/nco;cvs tag -d nco-3_9_2;cvs tag -d nco-3_9_2-1 cd ~/nco;cvs commit -m "Preparing nco-3.9.2 release" cvs tag -c nco-3_9_2;cvs tag -c nco-3_9_2-1 Changes since nco-3_9_1: GPL3, UDUnits2, _FillValue, ncap2 printing * Support nco_put/get_att_string/ubyte() now in netCDF4 * Update to netCDF4 daily snapshot 20070822 * Abandon TKN2SNG(x) method for nco_mss_val_sng Use direct global variable to get rid of superfluous double quotes This fixes all regressions with _FillValue<->missing_value switch 2007-08-29 Charlie Zender <zender@uci.edu> * Regressions with _FillValue<->missing_value switch * Supplant missing_value with _FillValue in documentation 2007-08-29 Henry Butowsky <henryb@ntlworld.com> * First cut of user functions permute() & reverse() see examples in ncap2.in. 2007-08-23 Charlie Zender <zender@uci.edu> * Implemented Henry's new ncap2 files, smaller ncoGrammer.g 2007-08-22 Charlie Zender <zender@uci.edu> * Quiet UDUnits "override" messages to stderr unless dbg_lvl >= 9 2007-08-21 Charlie Zender <zender@uci.edu> * UDUnits2 finds default database, obeys UDUNITS2_XML_PATH 2007-08-07 Charlie Zender <zender@uci.edu> * Simple UDUnits2 test works with many TODOs for me, Unidata * Complete compilable UDUnits2 implementation for all units except (I think) time = time since ... units * Splice udunits2 functionality stubs inside HAVE_UDUNITS2 tokens 2007-07-24 Charlie Zender <zender@uci.edu> * pbuilder successfully built an NCO deb in a sid chroot! 2007-07-23 Charlie Zender <zender@uci.edu> * Add pbuilder-sid to build NCO in a chroot environment 2007-07-22 Charlie Zender <zender@uci.edu> * Add license text to nco++ source files * Change licenses from GPL2 and FDL1.1 to GPL3 and FDL1.2 for code and documentation, respectively * Try to prevent debian rules configure/libtool from invoking RPATH 2007-07-19 Charlie Zender <zender@uci.edu> * Add ncap2_tst.nco, Henry's ncap2 test script, to distribution 2007-07-18 Henry Butowsky <henryb@ntlworld.com> * Turned prs_sct into class. It has four methods: contructor, ncap_var_init(), ncap_var_write(), ncap_def_ntl_scn() 2007-07-14 Charlie Zender <zender@uci.edu> * Add failsafe NC_FORMAT_CLASSIC definition so tst.cc builds with OPeNDAP netcdf.h, which lacks pre-netCDF 3.6.1 tokens 2007-07-09 Henry Butowsky <henryb@ntlworld.com> * First cut of print function can do the following print(var_id),print(var_id,"format_string") 2007-07-04 Charlie Zender <zender@uci.edu> * Warn when nbr_xtr_1 < nbr_xtr_2 in ncbo * Note ncap2 type-conversion support 2007-06-30 Charlie Zender <zender@uci.edu> * Add -4 and -L dfl_lvl to man pages 2007-06-29 Charlie Zender <zender@uci.edu> * cvs tag -c nco-3_9_1-2 * Define NC_64BIT_OFFSET no-matter-what so FC6 RPMs build with OPeNDAP * cd ~/nco;cvs tag -d nco-3_9_1;cvs tag -d nco-3_9_1-1 cd ~/nco;cvs commit -m "Preparing nco-3.9.1 release" cvs tag -c nco-3_9_1;cvs tag -c nco-3_9_1-1 Changes since nco-3_9_0: netCDF4 deflation support, ncap2 loops, build improvements on many platforms * Finish deflation in ncap and ncap2 2007-06-28 Charlie Zender <zender@uci.edu> * Beef up deflation documentation * Add deflation to low-level ncap, ncap2 routines * Put deflation in ARM convention routine * Implement deflate command-line switches in all applicable (i.e., variable-defining) operators * Allow ncbo to subtract files with more processed variables in file_2 This is done by excluding file_2 variables that are not in file_1 2007-06-27 Charlie Zender <zender@uci.edu> * Polish deflate functionality and documentation 2007-06-26 Charlie Zender <zender@uci.edu> * Describe deflation in nco.texi * Request variable deflation iff dmn_nbr > 0 This works around netCDF4/HDF5 limitation where deflating a scalar crashes 2007-06-19 Charlie Zender <zender@uci.edu> * nco.spec patch by Daniel to always put netcdf-3 on path and, for x86_64, add -L/usr/lib64, when present, to correctly resolve 32/64-bit libraries * Define some netCDF tokens for really old versions of libnetcdf.a still in use on enterprise distributions like RHEL/CentOS. These tokens need are defined both in nco.h and nco_netcdf.c to enforce code firewall to keep netCDF wrappers separate from NCO. 2007-06-18 Charlie Zender <zender@uci.edu> * Remove configure.in rule to add -DNC_64BIT_OFFSET even when DAP version > 3.6.1 * Add SWAMP test template ~/nco/data/swamp.sh * Place DAP netcdf.h path ahead of system netcdf.h path in configure.in 2007-06-14 Charlie Zender <zender@uci.edu> * Fix flukey ncks %s printing * Alter bin_cnt.nco to use loops 2007-06-14 Henry Butowsky <henryb@ntlworld.com> * Commited the first cut of while loops and for loops Can use "break" & "continue" in both. Can have any level of nesting. These are vanilla unoptimized loops. Optimization to come!! 2007-06-07 Charlie Zender <zender@uci.edu> * Add [put/get]_string functions now in netcdf4 daily snapshots In particular, synchronize with snapshot2007060701 2007-06-05 Charlie Zender <zender@uci.edu> * Remove stdint.h from xlC and SGI builds of ncap2 stdint.h "helps" g++ builds (by defining [U]LLONG_MAX) stdint.h is required by icpc builds (by defining int64_t) 2007-06-03 Charlie Zender <zender@uci.edu> * Switch to daily snapshot 20070603 for development 2007-06-01 Charlie Zender <zender@uci.edu> * Unable to get compression working. Generates HDF5 errors. * Implement --dfl_lvl to set HDF Lempel-Ziv deflation level Always turn shuffle on, since it is inexpensive and improves compression * Allow user-specified down-conversion of netCDF4->netCDF3 * Include stdint.h where "long long" types are used This should fix build problems with icpc * Add nco_var_def_deflate() and stub to libnco 2007-05-31 Charlie Zender <zender@uci.edu> * Fix typos spotted by donor Kevin Raeder 2007-05-25 Charlie Zender <zender@uci.edu> * NC_64BIT_OFFSET patch in configure.in causes beaucoups de RPM warnings * Fix version in nco.spec * cd ~/nco;cvs tag -d nco-3_9_0;cvs tag -d nco-3_9_0-1 cd ~/nco;cvs commit -m "Preparing nco-3.9.0 release" cvs tag -c nco-3_9_0;cvs tag -c nco-3_9_0-1 Changes since nco-3_2_0: netCDF4 atomic type support, ncks attributes * Fix nco_prn_att() to handle netCDF4 atomic types * Use type-appropriate conversion to read ncatted attribute values * Document netcdf4 ncatted features * Add netcdf4 atomic type functionality to ncatted nco837 2007-05-24 Charlie Zender <zender@uci.edu> * Include NC_UBYTE and NC_STRING as funky variable types in var_lst_dvd() * Add better ncdiff regression for packed subtraction * Change put_var(var_prc_out[idx]->type) to var_prc_out[idx]->typ_upk in arithmetic operators to output inflated variables to disk. This should fix TODO nco862, a regression introduced sometime between 2.7.1 and 3.2.0 and noticed by Remik Ziemlinski. * Postpone 3.9.0 release until ncdiff/packing bug TODO nco862 is fixed 2007-05-23 Charlie Zender <zender@uci.edu> * Add nco_inq_format() compatibility stubs for pre-netCDF3.6.1 libraries 2007-05-22 Charlie Zender <zender@uci.edu> * Added netCDF4 support section: http://nco.sf.net/nco.html#nco4 * Decided to release as 3.9.0 rather than 3.2.1 * cvs tag -c nco-3_2_1_beta2 Changes since last tag: NCO builds and tests fine from both netCDF3 and netCDF4! Candidate is releasable. ncap2 NC_UBYTE problem is not solved. * Testing with ~/c++/ccc.cc shows that NC_UBYTE support is non-trivial No need to hold up release to finish this * antlr 2.7.6 fails on mpc64.nacs.uci.edu with pgCC This does not seem to be an NCO problem * NCO builds and tests fine from both netCDF3 and netCDF4! * Verified sng2nbr() handles cross-platform strtoll(), strtoull() issues with AIX xlC and PGI pgCC compilers. * Typecast CEWI variables in nco.h for safety * Added sng2nbr() from ~/c++/utl.hh to sdo_utl.hh 2007-05-20 Charlie Zender <zender@uci.edu> * AIX xlC has ncap2 build problem: "std::strtoll" is not declared. * Cast all naked nc_type tokens being thrown around in ncap2 code In addition to being bad programming style, these naked tokens trigger the netCDF3/netCDF4 nc_type incompatibility bug. The bug is that the nc_type NC_TYPE tokens are enumerated (enums) in netCDF3 and are simple pre-processor tokens in netCDF4. * cvs tag -c nco-3_2_1_beta1 Changes since last tag: netCDF3 build works again and tests fine. Huzzah. Candidate is releasable except for ncap2 NC_UBYTE problem * Add empty netCDF3 forward compatibility stubs for all netCDF4 new atomic type I/O functions * cvs tag -c nco-3_2_1_alpha4 Changes since last tag: netCDF4 works fine with all operators and regression tests. Exceptions are that ncap2 does not work with ubyte and that netCDF3 does not build because ncoGrammer.g lacks netcdf4 #ifdef's * Add netcdf4.nco test script for netCDF4-enabled ncap2 * Add netCDF4 casting functions ushort(), int64(), etc. to ncap_lex.l * Verified ncap ll,LL,ull,ULL,us,ui, etc. parsing works * Finished netCDF4'ing ncap_lex.l, ncap_yacc.y, ncap_utl.c 2007-05-19 Charlie Zender <zender@uci.edu> * Verified that ddra.nco works with int64 types so storing large counts as floats no longer necessary! * cvs tag -c nco-3_2_1_alpha3 Changes since last tag: After netCDF4'ing most files. netCDF4 references in ncoGrammer.g break netCDF3 ncap2 * Success at creating all netCDF4 atomic types with ncap2 except NC_UBYTE * Add netCDF4 atomic types (except NC_STRING) ncoGrammer.g 2007-05-18 Charlie Zender <zender@uci.edu> * Add netCDF4 atomic types (except NC_STRING) ncap2_utl.cc, ncap_yacc.y * Replace nc_type with nco_rth_prc_rnk() in ncap2.cc * Fix type spaghetti in nco_lst_utl.c * Create nco_rth_prc_rnk(nc_type) to determine relative precision * Add netCDF4 atomic types (except NC_STRING) to ncap_lex.l * Add netCDF4 atomic types to nco_rec_var.c, nco_rth_utl.c * Commit working build with netCDF4 atomic types in previously listed files * Add netCDF4 atomic types to mpncwa.c, nco_lmt.c, nco_lst_utl.c, nco_pck.c * Define fl_in_fmt before using in ncap.c, ncap2.cc * Working build with netCDF4 atomic types in nco_var_rth.c, nco_var_scv.c * Working build with netCDF4 atomic types in nco_var_avg.c * Use -Wno-switch to quiet GCC enum warnings on netCDF4 case labels * Introduce nco_short in nco_var_scv.c, nco_var_rth.c, nco_var_avg.c * cvs tag -c nco-3_2_1_alpha2 Changes since last tag: Prior to adding netCDF4 atomic types to: nco_var_scv.c, nco_var_rth.c, nco_var_avg.c * Resolve forward-compatibility issues and stub for nc_inq_format() 2007-05-15 Charlie Zender <zender@uci.edu> * Add netCDF4 atomic types to nco_scl_utl.c * Add netCDF4 atomic types to nco_mss_val.c * Add netCDF4 atomic types to nco_att_utl.c * Add netCDF4 atomic types to nco_cnf_typ.c * cvs tag -c nco-3_2_1_alpha1 Changes since last tag: Working subset of netCDF4 atomic type features with no regressions * Regression tests all fine except for known netCDF4/HDF defects * Make fl_out_fmt>=fl_in_fmt ("consanguinity") NCO-wide TODO nco836 * This seemed to introduce ncap2 regression in both netCDF3/4 * Use ISO standard strto(l,d,ul) not ato(f,i) to parse ncap2 atomic types 2007-05-14 Charlie Zender <zender@uci.edu> * Support NC_STRING in ncks functions, nco_prn.c, nco_msa.c, libnco_c++ * Temporarily make fl_out_fmt>=fl_in_fmt ("consanguinity") in ncks * Fix fl_out_fmt bug for NC_FORMAT_CLASSIC * Add all new netcdf4 atomic data types (except string) to libnco_c++ * Implement NC_STRING support stubs in nco_typ.h * Support NC_INT64 in nco_c++ * Complete netCDF4 type support (except NC_STRING) in nco_prn.c, nco_msa.c ncks now prints netCDF4 atomic types correctly 2007-05-13 Charlie Zender <zender@uci.edu> * Support netCDF4 types in nco_typ_fmt_sng() * Make tst.cc produce netCDF4 files when compiled with netCDF4 * Printing changes fix many netcdf4 regressions * Commit major changes to ncks printing 2007-05-12 Charlie Zender <zender@uci.edu> * ncks does not copy scale_factor and add_offset to output files! * ncks reports input file type, e.g., NC_FORMAT_CLASSIC, NC_FORMAT_NETCDF4 2007-05-11 Charlie Zender <zender@uci.edu> * Explain ncwa memory usage in nco.texi 2007-05-09 Charlie Zender <zender@uci.edu> * Comment out unused variables. These caused noisy compiles. Retain some for future use in cases where they might help in debugging or where their presence enhances code symmetry or would be non-trivial to re-introduce. Mark such variables as CEWI RUVICFFU = Retain Unused Variables In Comments For Future Use * Re-format error messages to print error code if available, and to always print msg, if any * Changed prototype and disambiguated ncap_sclr_var_mk() This allows g++ 4.1.2 to compile ncap2 again. 2007-05-08 Charlie Zender <zender@uci.edu> * Remove OpenMP and thr_nbr references from ncrename.c * Fix regressions stemming from passing --fl_fmt to ncatted * Put ${NETCDF_INC} before ${ANTLR_INC} in CPPFLAGS in nco++ Makefile This ensures netCDF4 in /usr/local/netcdf4 is searched before ANTLR in /usr/local (which often contains netCDF3 as well). * Add "copy global attributes" example to ncks documentation * Add "growing dimensions" example to ncap2 documentation * Use ncks -G to avoid copying global metadata during appends 2007-05-05 Charlie Zender <zender@uci.edu> * Working on 2D grid descriptions and ncap2 documentation 2007-04-24 Charlie Zender <zender@uci.edu> * Add "build-from-source" requirements to homepage * Backport settings (e.g., -c99) for SGI IRIX 6.5 compilers 2007-04-19 Charlie Zender <zender@uci.edu> * cd ~/nco;cvs tag -d nco-3_2_0;cvs tag -d nco-3_2_0-1 cd ~/nco;cvs commit -m "Preparing nco-3.2.0 release" cvs tag -c nco-3_2_0;cvs tag -c nco-3_2_0-1 Changes since nco-3_1_9: rmssdn normalization bugfix, exclusion list handling more flexible, RAM variables. 2007-04-18 Charlie Zender <zender@uci.edu> * Remove obsolete NR indexx*() routines * nco_var_nrm_sdn() returns mss_val when N=0 or 1 * Allow absent variables with EXCLUDE_INPUT_LIST in nco_var_lst_mk() 2007-04-18 Henry Butowsky <henryb@hush.com> * Tested nco_put_var_mem() & nco_get_var_mem() Can now hyperslab RAM variables on the RHS RAM hyperslabs on the LHS to follow shortly 2007-04-17 Charlie Zender <zender@uci.edu> * Remove obsolete -d hyperslab options from ncap, ncap2 code and docs * Insert computation of anomaly in ncap2 standard deviation examples 2007-03-31 Charlie Zender <zender@uci.edu> * Change -s "" to -s '' in most all examples to prevent unintended ncap2 behavior 2007-03-30 Charlie Zender <zender@uci.edu> * Implement all GCC custom switches in nco++, nco_c++ directories nco directory and configure.in have all cust switches except -Werror -Werror still triggers problems with ncap_lex.c * Add wildcards to end of all compiler tests in configure.in so that, e.g., CC='gcc' tests same as CC='gcc -std=c99' 2007-03-29 Charlie Zender <zender@uci.edu> * Fix ncap2 comparison between signed/unsigned integer expression warnings * Fold in Daniel's nco.spec change required for 64-bit builds on hybrid 32/64-bit systems, e.g., RHEL, CentOS 2007-03-29 Henry Butowsky <henryb@hush.com> * first cut of RAM variables new syntax works with assign operators (=,-=,+=,/=, *=) and post increment & post decrement operators 2007-03-24 Charlie Zender <zender@uci.edu> * Update configure.in for Mac OS X on Intel 2007-03-10 Charlie Zender <zender@uci.edu> * cd ~/nco;cvs tag -d nco-3_1_9;cvs tag -d nco-3_1_9-1 cd ~/nco;cvs commit -m "Preparing nco-3.1.9 release" cvs tag -c nco-3_1_9;cvs tag -c nco-3_1_9-1 Changes since nco-3_1_8: nc[wre]a normalization, ncap2 qutoes variables, quieter output, RPMs support ncap2, ncecat does per-file hyperslab expansion * NCO 3.1.8 RPM builds/installs/removes properly with FC6 Resulting executables work fine on local files DAP access is enabled, but broken. Unsure why. * enable-optimize-custom and -Wall are close to working on ncap2 Should be possible to get this working again by 3.2.0 * Cryptographically sign RPMs with GPG during BFS * Tweak nco.spec * Change to CFLAGS to -Wno-shadow so ncoLexer.cpp compiles 2007-03-09 Charlie Zender <zender@uci.edu> * Un-shadow variables in ncoGrammar.g 2007-03-08 Charlie Zender <zender@uci.edu> * Avoid FC6 redefinition of yy_flex_realloc() in ncap_lex.l Note sure if this will bite me on another platform 2007-03-06 Charlie Zender <zender@uci.edu> * Fix [mp]ncwa so normalization occurs and mean coordinates are returned when totals with weighting are requested. Regression test caught this error I re-introduced 20070223. 2007-03-03 Charlie Zender <zender@uci.edu> * Add ANTLR_ROOT to simultaneously set ANTLR_BIN, ANTLR_INC, ANTLR_LIB 2007-03-01 Charlie Zender <zender@uci.edu> * Document new ncea/ncecat per-file record dimension hyperslabs * Add #include <functional> to NcapVector.hh to provide STL binary_function routines * Add C_INC option to configure.in to disambiguate C-library headers from antlr headers and thus fix AIX/ESMF ncap2 build * Start new dedication 2007-02-28 Charlie Zender <zender@uci.edu> * Link with -lC on all xlc_* compiles * Re-work configure.in to accept ANTLR_INC, be friendlier on AIX 2007-02-27 Charlie Zender <zender@uci.edu> * Change configure.in to check for ASTFactory() in libantlr.a 2007-02-24 Charlie Zender <zender@uci.edu> * Finished (I think) quieting all non-fatal INFO/DEBUG/WARNINGS in main() routines, and converted from naked numbers to consistent and descriptive enums 2007-02-23 Charlie Zender <zender@uci.edu> * Add sys target to nco++ Makefile.old for ncap2 * Do not normalize coordinates by tallys/weights when no tallys exist (i.e., minimization, maximization, totals). Fixes TODO nco810. * Add nco_dbg_typ_enm to nco.h and implement in ncwa.c * Add ncwa coordinate-minimization bug to regression test 2007-02-15 Henry Butowsky <henryb@ntlworld.com> * Changed prefix of parser generated vars from '_' to '~'. Attribute propagation & inheritance should now work correctly with vars prefixed by '_'. Can now quote vars and atts '--u@..-v' 2007-02-14 Charlie Zender <zender@uci.edu> * Declare <algorithm> in ncap2_utl.cc and std::-qualify functions 2007-01-30 Daniel L. Wang <wangd@uci.edu> * Changed names in ssdap/ to reflect name change from SSDAP to SWAMP 2007-01-24 Charlie Zender <zender@uci.edu> * Committing nco-3_1_8-2 for upload to Debian unstable 2007-01-21 Charlie Zender <zender@uci.edu> * Change copyright year to 2007 * Built and tested with libdap-3.7.3 and libnc-dap-3.7.0 2007-01-12 Charlie Zender <zender@uci.edu> * Add variables and attributes with dashes and dots to in.cdl 2007-01-05 Charlie Zender <zender@uci.edu> * Add missing #xmp_ncecat link to nco.texi 2006-12-08 Charlie Zender <zender@uci.edu> * nco-3_1_8-1 is in Debian experimental 2006-12-06 Charlie Zender <zender@uci.edu> * cvs tag -c nco-3_1_8 Changes since nco-3_1_7: ncap2 methods work, nascent _FillValue support 2006-12-03 Henry Butowsky <henryb@ntlworld.com> * Commited second cut of variable methods & var/att properties 2006-12-02 Charlie Zender <zender@uci.edu> * Noticed that configure;make;make test breaks because ncap2 is not available in ../src/nco like all other executables * Link nco_c++/in.nc to ../../data/in.nc * Fix all rule in data so *.nc are always built 2006-11-29 Charlie Zender <zender@uci.edu> * NB: psd.nco uncovered a bizarre scanner error with ncap2 comment fields * Add equal-V/S approximation for Tallandier depth hoar to psd.nco * Print _FillValue WARNING only when dbg_lvl > 0 2006-11-27 Charlie Zender <zender@uci.edu> * Re-format long description in debian/control * Remove unused $(MAKE) -f debian/rules DH_OPTIONS=-i binary-common * Change Debian section from "math" to "science" in control and doc-base * Remove pgiCC OpenMP threading on ncap2 (compiler limitation) 2006-11-24 Charlie Zender <zender@uci.edu> * Change 'redhat' to 'rpm' and 'rpm -ba' to 'rpmbuild -ba' in bld/Makefile * Merge Fedora nco.spec UDUnits patch into configure.in * Update bld/nco.spec to Fedora Core 6 spec for nco-3.1.5 * Change ORO values in in.cdl * Change ncap to ncap2 in nco.texi * Remove *lex.c, *yacc.c, .cvsignore, and debian from _all_ tarballs 2006-11-23 Charlie Zender <zender@uci.edu> * Fix typo now use _FillValue not _Fillvalue * Fix nco_bm.pl success test for ncap2 reading ncap2.in * Regression/benchmarks now use ncap2 everywhere 2006-11-22 Charlie Zender <zender@uci.edu> * Exclude all .cvsignore's from tarballs * Replace 'unstable' with 'UNRELEASED' from 3.0.0-3.1.7 * Change debian/nco.doc-base to debian/doc-base * Update bld/Makefiles for DAP build for ncap2 * More precise language for interactive o/a/e command * Created non-native debian packages 2006-11-21 Charlie Zender <zender@uci.edu> * Add @dircategory netCDF to nco.texi * Patch ncap2 with ncap_sclr_var_mk() prototypes to re-enable builds * More package fixes suggested by Daniel Baumann: * Removed debian/docs, alphabetize control, less precise debhelper version, refine copyright history, remove debian/doc, de-comment debian/rules, * Remove debian/[postinst,prerm, postrm]. These only contained instructions for handling *.info files. Rely on dh_installinit to automagically create these instructions. 2006-11-20 Charlie Zender <zender@uci.edu> * Alter documentation rules to not-split * Include ncap2 in debian distribution * Remove compiler-preference list arguments to AC_PROG_CC and AC_PROG_CXX in configure.in. Presence was breaking builds on virga where broken icpc compiler was automatically selected. Henceforth non-default compilers must be specified as environment variables as in configure.eg 2006-11-19 Charlie Zender <zender@uci.edu> * Create ncap2 manpage for debian compatibility * Update debian build-dependencies to include antlr * Make missing_value implementation generic and allow compile-time override with Makefile variable cd ~/nco/bld;make USR_TKN='-DNCO_MSS_VAL_SNG=_FillValue';cd - 2006-11-18 Charlie Zender <zender@uci.edu> * Warn when _FillValue defined and missing_value not. Fixes TODO nco682 2006-11-16 Charlie Zender <zender@uci.edu> * Added new demonstration ncap2 script data/psd_wrf.nco 2006-11-12 Charlie Zender <zender@uci.edu> * Update copyright year in man pages * Do not create mpncdiff,mpncea,mpncrcat links when enable_mpi=false 2006-11-10 Charlie Zender <zender@uci.edu> * Tweak configure.in to work with pure Debian assumptions * Fix --disable-netcdf4 handling in configure.in * Merged Debian nco_2.9.9-3.diff into upstream sources * Removed UDUnits support from Debian build * Update debian/convert, copyright, prerm, rules, nco.doc-base, postinst, * Added udunits.dat for eventual Debian compatibility (a mistake?) 2006-10-24 Charlie Zender <zender@uci.edu> * Replace ncap by ncap2 in all benchmarks and in regression tests ncap1 is still used in non-ncap regression tests 2006-10-21 Charlie Zender <zender@uci.edu> * Include srd argument in all hyperslab documentation * Fixed parenthetical notation for hyperslab arguments in nco.texi 2006-10-20 Charlie Zender <zender@uci.edu> * Die gracefully with HINT rather than segfault with hyperslab limits of form -d dimension, (i.e., dangling comma with no minimum value) 2006-10-19 Charlie Zender <zender@uci.edu> * Add hdr_pad documentation to nco_ctl.c, nco.texi as per TODO nco650 2006-10-16 Charlie Zender <zender@uci.edu> * Test for find_nc4_file() not nc_check_for_hdf5() in configure.in as per patch from Simon Wood 2006-09-18 Charlie Zender <zender@uci.edu> * Confirmed psd.nco works correctly * Add psd.nco, an ncap2 script for particle size distributions * Make ncap2 read 3.14L as float not erroneous int as per C++ 2006-09-15 Charlie Zender <zender@uci.edu> * Change ncbo type difference from warning to level 1 info message 2006-09-14 Charlie Zender <zender@uci.edu> * Fix ncecat/mpncecat regressions caused by new stride capability 2006-09-12 Charlie Zender <zender@uci.edu> * Remove trunc() symbol (which AIX lacks) from ncap2 to simplify AIX builds * Revise stride documentation in nco.texi and usage strings * Stride now works on non-ncks non-record dimensions, fixes TODO nco773 * Set stride = 1 in nco_var_srt_srd_set(), formerly nco_var_srt_zero() * Implement stride in nco_var_val_cpy() 2006-08-31 Charlie Zender <zender@uci.edu> * Try not to invoke mpncpdq in regression tests on pbs * Add link to mpirun to $MY_BIN_DIR when building MPI executables with bld/Makefile 2006-08-30 Daniel Wang <wangd@uci.edu> * Added --mpi_upx <user-specified mpi command> option to nco_bm.pl to allow job-scheduler mpi directives like -machinefile 2006-08-26 Charlie Zender <zender@uci.edu> * icc/icpc autoconf build/link succeeds but C++ executables segfault 2006-08-25 Charlie Zender <zender@uci.edu> * icc improvements to get icpc working * Direct autoconf to build nco_c++ prior to nco++ * pathcc and pathCC now work with autotools 2006-08-24 Charlie Zender <zender@uci.edu> * Add pgcc, pathcc support to autotools builds * Only add GCC flags when sure compilers are gcc and g++ * Re-arrange compiler ordering to AC_PROC_CC * Prototype nco_var_lst_crd_xcl() in nco_var_lst.h * Prototype rnd_nbr() in nco_rth_utl.h * Fix cast-qual warnings in newer qsort()-compatible comparison routines nco_cmp_chr() and nco_cmp_int() in nco_lst_utl.c * Eliminate un-utilized variable warnings in nco_ddra(), cnv_ccsm* * Correct OPeNDAP version test in configure.in * Utilize ncap2 incremental operators in ddra.nco * Quiet ncap2 informational messages * Add sdo_utl.cc sdo_utl.hh to libnco++ 2006-08-23 Charlie Zender <zender@uci.edu> * Change default to CCACHE=N because ccache interacts so poorly with OpenMP switches during builds * Re-fix NCO_rgr.pm to directly access main:: variables fl_fmt, omp_flg, nco_D_flg, fl_out * Kludge for pgcc compiler to workaround C99 designated initializers Timers should be OK but DDRA answers for ncbo, ncwa should not be trusted with pgcc (or any C++ compiler). 2006-08-22 Charlie Zender <zender@uci.edu> * Verified that pathcc OpenMP problems have disappeared and were likely due to ccache inadvertently mixing OpenMP-enabled and dis-abled object code. Must be more careful about that in future. This resolves TODO nco765 and 766 * Add more informative OpenMP diagnostics * Fix OpenMP flag-passing in Makefile for Intel icc compilers 2006-08-14 Charlie Zender <zender@uci.edu> * More benchmark cleanup * Fix specialized nco_bm.pl file processing for tms_lng.nc This resolves TODO nco767. 2006-08-10 Charlie Zender <zender@uci.edu> * pgcc -c9x currently fails to handle C99 designated initializers * Update PGI compiler options in Makefile 2006-08-01 Charlie Zender <zender@uci.edu> * Add stubs for znetcdf support 2006-07-19 Henry Butowsky <henryb@ntlworld.com> * First cut of C like it iteration operators * these are +=,-=,*=,/= --,++ (pre & post) * New code checked out OK with valgrind * Need to add code to check that first operand is valid Lvalue -also need some new regression tests to fully test this new feature 2006-07-12 Charlie Zender <zender@uci.edu> * Improve, add, clarify, rx examples in nco.texi, in.cdl 2006-07-10 Charlie Zender <zender@uci.edu> * Add pathcc/pathCC to MPI handling in bld/Makefile 2006-07-03 Charlie Zender <zender@uci.edu> * Clean-up ncra.c file INFO messages 2006-06-30 Charlie Zender <zender@uci.edu> * OpenMP with pathcc segfaults with ncra fxm TODO nco764 * Regressions tests pass as expected with native pathcc on ipcc * Change to gettimeofday() to avoid strict Perl errors in NCO_bm.pl * Tweak bld/Makefile to work with pathcc compiler on ipcc.ess.uci.edu 2006-06-26 Charlie Zender <zender@uci.edu> * Replace some WARNINGs with less-paranoid INFOs when -A appending 2006-06-22 Charlie Zender <zender@uci.edu> * Support AIX runantlr=antlr in Makefile.old for ncap2 on AIX 2006-06-17 Charlie Zender <zender@uci.edu> * Add data/ddra.nco to compute DDRA statistics * Add nco++/*.[ch]pp files to repository to bring ncap2 one step closer to users. Still requires many antlr-distribution *.hpp files. * Update MANIFEST, add nco++ entries * Clean up nco_bm file create routine * Rename IPCC_dly_T85 to gcm_T85, sml_stl to stl_sml, skn_lgs to tms_lng 2006-06-12 Charlie Zender <zender@uci.edu> * Write initial and final timers to stdout not stderr 2006-06-08 Charlie Zender <zender@uci.edu> * Re-release this as 3.1.4 * Always initialize flg_ddra=False in ddra_info Fixes spurious TIMER info on compilers (like xlc) which do not zero data * Re-release this as 3.1.4 * Take mtd timer out of loop in ncra.c and ncecat.c 2006-06-07 Charlie Zender <zender@uci.edu> * cvs tag -c nco-3_1_4 Changes since nco-3_1_4: timers, clean/dirty switches * Add timer start and end calls to all operators * Make --clean memory exits default in regression tests * Make --dirty memory exits default in operators * Add cln,drt,mmr_cln,mmr_drt,dirty flags to all operators * Quiet MRV verbosity, stylize TIMER output * Make ddra_info firstprivate() * Add flg_ddra to ddra_info * Change timers to enumerated type * Add metadata portion timer call 2006-06-02 Charlie Zender <zender@uci.edu> * Put --disable-shared everywhere for AIX * Add operator-specific speeds to nco_ddra() * Add fractional time diagnostics to nco_ddra() 2006-05-30 Charlie Zender <zender@uci.edu> * cvs tag -c nco-3_1_3 Changes since nco-3_1_2: ncap2 attribute propagation, CF coordinates, ncwa -b -B, ncra 0 records * Daniel fixed nco_bm.pl * Document retain-degenerate-dimensions option in nco.texi This finishes TODO nco739 and allows nco738 to be possible 2006-05-29 Charlie Zender <zender@uci.edu> * Implement mask_condition as -B and document in nco.texi * retain-degenerate-dimensions implementation works in [mp]ncwa * Update opendap.sh to DAP 3.6 download/build procedures 2006-05-27 Charlie Zender <zender@uci.edu> * Improve rx examples in nco.texi * Add flg_rdd hooks to retain degenerate dimensions in ncwa TODO nco739 * Add Q00--Q100 to in.cdl to text regex functionality 2006-05-26 Charlie Zender <zender@uci.edu> * Correct regular expressions in nco_bm_hyp.sh * All previous ncwa benchmarks which used variable subsetting on the IPCC dataset were incorrect due to significantly wrong regular expressions in nco_bm_hyp.sh. Incorrect regular expressions did not affect satellite dataset because there were fewer than ten variables. * Correct max variable name for var_4d in nco_bm_hyp.sh * Give wgt_brd_flg correct sign so not miscounted by nco_ddra() 2006-05-24 Charlie Zender <zender@uci.edu> * Correct lon attributes in in.cdl 2006-05-22 Charlie Zender <zender@uci.edu> * Avoid designated initializers to build ncbo and ncwa in C++ * Add nco.1 man page which Sourceforge caused me to lose 2006-05-20 Charlie Zender <zender@uci.edu> * Improve help message in nco_usg_prn() 2006-05-19 Charlie Zender <zender@uci.edu> * Return 1 early from nco_omp_ini() on compilers without OpenMP * Put CNV_CCM_CCSM_CF back in nco_var_lst_crd_ass_add(), and add it into nco_var_lst_crd_add(). CF coordinates now handled correctly with -C/-c switches. * Move some nm_id_sct var_lst routines from nco_var_utl.c to nco_var_lst.c * Split CF coordinate convention nco_cnv_cf_crd_add() out of nco_var_lst_ass_crd_add() so it can be called based on other command line switches. * Update documentation to new CVS and ViewCV URLs 2006-05-18 Charlie Zender <zender@uci.edu> * Old-style Makefiles now assume ccache is installed on LINUX* builds to speed up compilations. Disable with CCACHE=N 2006-05-16 Charlie Zender <zender@uci.edu> * Found bug in libnco_c++: library does incorrectly initializes function parameters which default to CPP macros, e.g., NC_NOERR. These parameters cannot be over-ridden by non-default values. Same problems occurs whether passing rcd_opt as reference or not. Attempting to fix bug by creating named constant with global library visibility. 2006-05-14 Charlie Zender <zender@uci.edu> * Master code repository is now on nco.cvs.sf.net Old code at nco.sf.net is a hodgepodge of erratic commits over past week. * Finished hand-merging one week's patches from development on multiple machines due to sourceforge.net outage. Master code is now on ashes. Everything from about 20060506--20060512 was originally "out-of-tree" during outage. 2006-05-13 Charlie Zender <zender@uci.edu> * Out-of-sync commits from ashes to new sourceforge repository 006-05-11 Charlie Zender <zender@uci.edu> * Fix syntax error in data/Makefile.am so .nc's build correctly * Fix syntax errors in configure.in 2006-05-09 Charlie Zender <zender@uci.edu> * Add CF coordinate convention to documentation * Add regression test for CF "coordinate" convention * CF "coordinates" convention is fully implemented * Add C++ directory contents to TAGS source list * CF "coordinates" convention is semi-working 2006-05-08 Charlie Zender <zender@uci.edu> * Change in.cdl Conventions from NCAR-CSM to CF-1.0 * Implement nco_cnv_ name convention in ARM routines * Support OPeNDAP on AIX in bld/Makefile 2006-05-07 Charlie Zender <zender@uci.edu> * Change from --dbg_lvl=73 to --ddra or --mdl_cmp for computational model = DDRA diagnostic output * Add nco_ddra() to ncbo * Create stand-alone NCO(1) man page * Allow zero-record record variables in nco_var_val_cpy() This should allow copying metadata in most NCOs * Make zero-record record variables always fixed in arithmetic operators This ensures airthmetic loops to not encounter empty variables 2006-05-06 Charlie Zender <zender@uci.edu> * Improve warning messages in ncra for size 0 records * Build in_rec_zero.nc from in_rec_zero.cdl in Makefiles 2006-05-04 Charlie Zender <zender@uci.edu> * Fix ncks to work with record dimensions of size zero * Added data/in_rec_zero.cdl for torture testing This file contains a record coordinate of length zero 2006-05-02 Charlie Zender <zender@uci.edu> * Remove obsolete -Z option from help strings * Get MRV_flg, dmn_avg_nbr, lmn_nbr_avg from nco_var_avg(), use in nco_ddra() 2006-05-01 Charlie Zender <zender@uci.edu> * Update configure.in for Cray compatibility 2006-04-30 Charlie Zender <zender@uci.edu> * Some cleanup of nco_bm suite * Add missing NCO_bm:: scope to functions in nco_bm.pl * Change from time_t=time() to clock_t=clock() timers * Add observed time to diagnostics * Filter noisy INFO messages in main ncwa loop * Modify DDRA diagnostics to work with long long's = int64 sizes * Add MRV_flg, rnk_avg, and lmn_nbr_avg outputs to nco_var_avg() * Hooked-in call to nco_ddra() with estimates for some inputs Calculations should be more accurate than ddra scripts. 2006-04-29 Charlie Zender <zender@uci.edu> * Added nco_ddra() to estimate computation time. Not invoked yet. Still need to functionalize rnk_avg computations. 2006-04-28 Charlie Zender <zender@uci.edu> * Change $sz_wgt_blk to $sz_avg_blk in ddra.theory_model.pl * Autoconf-conditions for getpagesize(), getrusage() in nco_mmr.c This should cure build failures on Cray platforms, and make nco_mmr.c bulletproof (though still not perfect) for other esoteric platforms. 2006-04-27 Charlie Zender <zender@uci.edu> * Remove superfluous semi-colons from some ncbo/ncflint comments This should fix Solaris builds 2006-04-26 Charlie Zender <zender@uci.edu> * Re-order PVM_ARCH compatibility portion of configure.in so it follows not precedes $host definition. This should fix cygwin (and other) build problems introduced in NCO version 3.1.2. 2006-04-25 Charlie Zender <zender@uci.edu> * Note in nco_var_cnf_dmn() how to turn off weight re-use 2006-04-21 Charlie Zender <zender@uci.edu> * cvs tag -c nco-3_1_2 Changes since nco-3_1_1: packed data fixed, ncap2 in configure, ncra/ncea/ncwa behavior * Change documentation to reflect new ncra/ncea/ncwa behavior * Add new ncra/ncea/ncwa regression tests to verify this * Change ncra/ncea/ncwa to always average coordinate variables 2006-04-14 Charlie Zender <zender@uci.edu> * Constify srd in nc_put/get_varm() calls and prototypes * Correct type of srd in nc_put/get_vars() calls and prototypes * Add put_vars() routines to nco_netcdf.[c/h] for completeness 2006-04-13 Henry Butowsky <henryb@ntlworld.com> * The first iteration of hyperslabing in ncap2 * Can hyperslab on LHS with scalar on RHS eg time(5:9)=.4; or var on RHS. * Can also hyperslab on RHS eg three_dmn_var(1,,)=three_dmn_var(9,,); 2006-04-06 Charlie Zender <zender@uci.edu> * Change bool's to nco_bool's in nco.h 2006-03-17 Charlie Zender <zender@uci.edu> * Un-botch ncatted regression test, fixing TODO nco691 * NECSX does not support getpagesize() in nco_mmr.c 2006-03-17 Harry Mangalam <hjm@tacgi.com> * spiffin' config/make to bypass nco++ politely if antlr not available also now forces regeneration of hpp files - 1st .cc file triggers rebuild. Builds (or doesn't) on a clean checkout. * 'make install' installs ncap2 to bindir if it's built. * mod configure to skip nco++ if trying to build for MPI (nco++ fails on the MPI build - a concern at this point?) * autoconf/automake patches to allow nco++ and MPI to co-exist and mods to allow MPI regressions and benchmarks to (mostly) work. 2006-03-15 Henry Butowsky <henryb@ntlworld.com> * Fix NUMBER lexer. Can now lex number like 23. or 111. etc * Fix NcapVar.hh -- Code was crashing when var->nm was set equal to s_va_nm * Start code for slab indices added lmt_peek() & lmt_init() to ncoTree.hpp 2006-03-14 Harry Mangalam <hjm@tacgi.com> * nco++ dir now works with autoconf/automake, altho only tested with simple cases (only gcc/g++ compilers, etc). Original Makefile renamed to Makefile.old. Tested under --enable-nco++=yes (default) & --enable-nco++=no. Added more logic as per CZ request on 3.15 - now checks for runantlr, libantlr, antlr headers, 2006-03-13 Harry Mangalam <hjm@tacgi.com> * all but 1 benchmark work server-side. When trying to execute the ncrcat bench, the serverside fails when doing the ncwa command. It uses a very large amount of mem (~1.3G when it dies - de to an apache limitation?) Daniel and I are giving up for now - marked as TODO 692. * failing regressions tested and some resolved to bad handling by benchmark script. Remaining ones are marked in the failure message. Only new one is TODO691 - looks like a problem with ncatted returning mysteriously null values. Total of 3 regressions failing - 2 ncap, 1 ncatted. 2006-03-09 Harry Mangalam <hjm@tacgi.com> * namespace cleanup on benchmark code and added rc capability to the code, first capability just records whether users said yes to send UDP data and then uses that answer to bypass the question from then on. Other info can be added in NAME=VALUE pairs and the detection routine reads them all into a global hash %NCO_RC that can be used to set various things. 2006-03-09 Charlie Zender <zender@uci.edu> * Added regression test to make sure nco686 stays fixed. * Re-work nco_mss_val_cnf() to handle all cases of var1 and var2 having/not having missing values. Fixes TODO nco686. Probably fixed analogous latent bug in ncflint. * Remove bogus const prototype on return value of nco_typ_fmt_sng() Apparently gcc 4.0.3 actually pays attention to intended const-ness of return values in these situations * Fix TODO nco687 so unequal missing values are formatted correctly in warning messages. This should help debug nco686. 2006-03-09 Harry Mangalam <hjm@tacgi.com> * nco_bm_benchmarks.pl converted to NCO_benchmarks.pm with a good bit of namespace cleanup. There are still some inexplicable explicit declarations that are needed but only about 2-3. * both benchmark and regression code support both serverside and clientside execution. There are still some remaining conflicts, but most benchmarks and most regressions run identically. * the go() routine was rewritten to be a bit more coherent. Again, there is a bit more cleanup to do, but it's time to get it into cvs * Daniel's ss code changes have tracked the benchmarking code, so both sides work with each other. There are some unfortuneately primitive hacks to get them working together, but while it's ugly (perl writes data to file and executes python to pick it up, it's simpler than bringing swig into the picture. 2006-02-25 Charlie Zender <zender@uci.edu> * Hunt for double-free() in ncap, cleanup var_ycc-related operations 2006-02-20 Charlie Zender <zender@uci.edu> * Change more internal bool's to nco_bool's so g++ builds libnco.a (again) 2006-02-19 Charlie Zender <zender@uci.edu> * Re-arrange packing/unpacking documentation to describe default handling by arithmetic operators. 2006-02-18 Charlie Zender <zender@uci.edu> * Add tst target to ncap2 Makefile * Verified regression test results unaltered * Change argument bool's to nco_bool's throughout libnco and frontends NB: I may have missed non-front-end bools in libnco. * ncap2 now compiles, links, and runs when libnco built with g++ * Change ncap2 argument bool's to nco_bool's which are typedef's to int's 2006-02-17 Charlie Zender <zender@uci.edu> * ncap2 compiles, links, and fails when libnco built with gcc * ncap2 compiles, links, and runs when libnco built with g++ * Change to int yywrap() from bool yywrap() to work with g++ * Begin to clean up libnco++ usage/headers * Add libnco++.hh to store all libnco++ headers * Add ncap2 target to bld/Makefile for convenience * Re-make libnco (when necessary) before linking ncap2 * Add ncap2 to prg_prs() * Add doc/ncap2.in to test ncap2 * ncap2 builds within tree now! * Add udunits libraries, remove dat target 2006-02-16 Harry Mangalam <hjm@tacgi.com> * Corrected BM code to account for error strings being eval'ed as 0 when compared as numeric and a number of other error-checking improvements. 2006-02-16 Charlie Zender <zender@uci.edu> * This fixes TODOs nco672, nco673, nco402 * Modify regression test to expect packed fixed data * Check is_var_fix flags in nco_var_dfn() to handle packed fixed data * Add is_var_fix to var_sct so nco_var_dfn() can use it to identify variables to pass through packed straight to output 2006-02-15 Charlie Zender <zender@uci.edu> * Same problem exists with all other non-processed data. TODO nco673. * Add test whether non-record data is unpacked in ncra output file. ncra currently fails this test. Reported by Martin Dix. TODO nco672. 2006-02-14 Charlie Zender <zender@uci.edu> * Temporarily add ncap2 object files directly to link line 2006-02-07 Charlie Zender <zender@uci.edu> * Remove redundant antlr actions. Add -lnco to LDFLAGS. 2006-02-06 Charlie Zender <zender@uci.edu> * Fix nco++ library definition 2006-02-03 Charlie Zender <zender@uci.edu> * Move Henry's Makefile to Makefile.orig and Makefile.old to Makefile * nco++ build mechanism appears to work. Dies in VarOp.hh due to syntax error. Not understood yet. 2006-02-02 Charlie Zender <zender@uci.edu> * Progress on build mechanism but nothing working * Avoid name clashes between nco++ and nco code by changing file names from ncap* to ncap2* 2006-01-31 Charlie Zender <zender@uci.edu> * Created nco++/Makefile.old from nco_c++/Makefile.old Made progress toward in-tree nco++ build 2006-01-30 Charlie Zender <zender@uci.edu> * cvs tag -c nco-3_1_1 Changes since nco-3_1_0: C++ builds work again, ssdap, nco++, 4.0-alpha11-compatible * Changed copyright to 2006 * Daniel added server side extensions to ssdap * nco++ still has Henry's hardcoded paths---will fix in nco-3_1_1 * Committed Henry's C++ ncap2 antlr-based code to nco++ 2006-01-20 Charlie Zender <zender@uci.edu> * Verified regular NCO builds with gcc-4.0.3 on dapper drake dirt * netcdf 4.0-alpha11 (still) fails regression tests as per TODO nco658 * Make nco_lbr_vrs_prn() work with netcdf 4.0-alpha11-style values * Added integration example to ncwa in manual 2006-01-04 Harry Mangalam <hjm@tacgi.com> * Fixes to the BM code to force the retrieval of DAP-requested files and a variable ($prsrv_fl) to allow a regression test to maintain files if a previously downloaded file is needed for a successive test. * noted in the TODO as well, but there is a difference between the opendap code running on sand (which passes all server-side regression tests) and the code running on hiperstore, which fails a number of them, many with OpenDAP errors emitted from the lib. Still figuring this out. * mods to nco_fl_utl.c: in nco_fl_mk_lcl() to make ncatted and ncrename fail gracefully when asked to DAP-retrieve a file on which to operate. 2005-12-29 Charlie Zender <zender@uci.edu> * Add xbitmap icon to homepage 2005-12-19 Harry Mangalam <hjm@tacgi.com> * mod bm code to make a new dir for dap data and explicitly delete everything in it for each new DAP test to force the network transfer. 2005-12-13 Charlie Zender <zender@uci.edu> * Modify `make test` to automatically run regression test * Remove old nco_tst.pl references from Makefile.am `make test` * Fix yyunput argument type in ncap_lex.l * Reference prs_sct in yyerror to fix CEWI with g++ -Werror 2005-12-11 Charlie Zender <zender@uci.edu> * C++-compatibility tweaks to nco_lmt.c, nco_omp.c, and configure.in Current code causes g++ ICE TODO nco662 2005-12-09 Charlie Zender <zender@uci.edu> * Strip -std=c99 from CC and CFLAGS if CC is gcc* in order to support to support using CC=g++ within configure. Ditto for -Wmissing-prototypes, and analogous procedures for xlC* and icpc*. 2005-12-08 Harry Mangalam <hjm@tacgi.com> * fixed passive/active ftp problem exhibited on soot by making 'ftp' explicitly '/usr/bin/ftp' (it was a kerberos'ed ftp on soot for some reason - /usr/kerberos/bin/ftp). For linux and esmf, the system ftp supports passive ftp (-p flag). The kerberos ftp supports it as well but not as a flag which casued problems in the 'here' script. 2005-12-05 Charlie Zender <zender@uci.edu> * Today's modifications allow G++ to build NCO limited by make non_ncap UDUNITS=N This NCO passes all regression tests ncap and UDUNITS linkage remain C++ problems and prevent closing TODO nco656 * Hacked nco_malloc_*() to remove errno references with G++ Presence triggers G++ error: undefined reference to `__errno_location()' This is TODO nco659 * Set #define YYDEBUG 0 in ncap_yacc.y so C++ compiles ncap_yacc.c * Fix type conversions in nco_mss_val_cnf() This fixed a latent bug that was probably never expressed * Explicitly coerce pointers in implicit conversions to make C++ happy in nco_cmp_chr(), nco_cmp_int(), ncpdq.c * Correct const-ness of string list arguments with prototypes in nco_prs_rnm_lst(), nco_fl_nm_prs() * Add doc/netcdf4.sh to install latest netcdf4 and HDF libraries for NCO 2005-12-02 Charlie Zender <zender@uci.edu> * cvs tag -c nco-3_1_0 Changes since nco-3_0_3: MPI ZKB, configure netcdf4/mpi, ncwa MRV, ncap rnd_nbr(), netcdf4-alpha10 * Preparing for NCO 3.1.0 release 2005-11-30 Harry Mangalam <hjm@tacgi.com> * Changed ncap.c to allow MACOSX to use erf, erfc, gamma * changed configure.in to accept a more flexible OS/arch string to be able to set AMD64 (reason for previous failing the erf() tests in ncap). * changed the ncap benchmark scripts to correct syntax errors and change dimensions to correspond to the casting dimension order. 2005-11-29 Charlie Zender <zender@uci.edu> * bld/Makefile accepts FL_FMT for netCDF4 testing 2005-11-29 Harry Mangalam <hjm@tacgi.com> * Changed configure.in to use NETCDF4_ROOT rather than renaming to libnetcdf4. 2005-11-28 Charlie Zender <zender@uci.edu> * Remove nco_open_par() again. Seems to require parallel mode in netCDF4 configure to appear in library, possible hardware dependency 2005-11-28 Harry Mangalam <hjm@tacgi.com> * Spiffing configure/makefiles to make sure that netcdf4 works as expected; adding dependencies into configure rather than requiring the full set to be run at the commandline. Still a question about the specification tho (--fl_fmt vs --netcdf4). 2005-11-26 Charlie Zender <zender@uci.edu> * Test for averaging dimensions being MRV dimensions 2005-11-25 Charlie Zender <zender@uci.edu> * Change "long missing_value" to "nco_int missing_value" 2005-11-23 Charlie Zender <zender@uci.edu> * Place operation count stanzas in expensive routines for paper: nco_var_cnf_dmn(), nco_var_avg() 2005-11-19 Charlie Zender <zender@uci.edu> * Synchronize with netCDF4 alpha10, verified passes regression tests * Expose new types in attribute function calls * Change _int64/_uint64 to _longlong/_ulonglong in function names * Change NC_STRICT_NC3 to NC_CLASSIC_MODEL 2005-11-15 Charlie Zender <zender@uci.edu> * Added cheesy random number generator function rnd_nbr() to ncap ncap -O -v -s 'x=rnd_nbr(lat)' ~/nco/data/in.nc ~/foo.nc;ncks ~/foo.nc for generating fake data. Search source for TODO nco652 and improve! 2005-11-14 Harry Mangalam <hjm@tacgi.com> * retired following TODOs bld68. hjm Verify configure with minimal options works with MACOSX bld53. hjm Add --fl_fmt option to nco_bm.pl (see ncks.c for fl_fmt values) bld62. hjm Make regressions fail when input file not created but re-used from previous test etc bld71. hjm Mac OS X: remove false positive regressions in ncwa (cut?) bld72. hjm nco_bm: ensure "foo" (intermediate files) accessed locally not over DAP bld46. Is DAP_ROOT (or DODS_ROOT) an obsolete variable in DAP 3.5+? if so obsolete it 2005-11-14 Harry Mangalam <hjm@tacgi.com> * corrected Benchmark code for opendap access. 2005-11-11 Harry Mangalam <hjm@tacgi.com> * fixed net tests to test only returned value, not entire msg * Added UDP reporting query to regression and benchmark runs if users explicitly agree, to capture failures in regressiions automatically for better profiling. Queries users if they will allow data to be returned UNLESS the --udpreport is already set * Changes to Makefile.am in src/nco to enable MPI binaries to be built. * Notes about problems with old aclocal, automake posted have to assure that new versions are manually updated in *ubuntu anyway. * back on track with updating Makefiles the right way(c) * updated MANIFEST to include new bm scripts and the breakup of nco_bm.pl. 2005-11-09 Charlie Zender <zender@uci.edu> * Omit -lstdc++ from AIX xl* DAP builds * Add CVS Header and -*-makefile-*- to all Makefile.am's This automatically invokes Emacs Makefile-mode syntax/highlighting 2005-10-31 Harry Mangalam <hjm@tacgi.com> * Changes to mk_bm_plots to scriptify the last of CZ's requests * Changes to configure/make to straighten out build protocols for netcdf4 (NETCDF4_ROOT to be defined, needs to be named libnetcdf4, not libnetcdf). Automatic paths for -I, -L based on the NETCDF4_ROOT) 2005-10-28 Harry Mangalam <hjm@tacgi.com> * Changes to the configure/make chain to start to resolve autobuild issues with MPI / netcdf4 / DAP / etc * now builds serial NCOs cleanly on Linux (x86,AMD64) and also MPI on Linux (x86,AMD64). Suports Rich Lowry's mpich2 debs (no additional flags) and MPICH2 from src if given MPICH_ROOT. * added checks for -lf95, more error checking 2005-10-27 Charlie Zender <zender@uci.edu> * Tried ncwa memory free()'ing technique to avoid freeing dangling pointers in mpncra/mpncea. Does not fix hanging processes. 2005-10-26 Charlie Zender <zender@uci.edu> * More unused prototyping of MPI-IO access code. Implementing calls in mpncecat.c for testing, and wrappers in nco_netcdf.c 2005-10-25 Charlie Zender <zender@uci.edu> * Support pnetCDF in bld/Makefile * Deprecate nco_fl_typ (never-used) from nco_netcdf.c wrappers * Add ncompi_open(), first pnetCDF interface wrapper 2005-10-25 Harry Mangalam <hjm@tacgi.com> * Changes to support MacOSX better: - configure detects Darwin and inserts -fno-common - bits added to configure.eg to describe other dependencies of this flag for libnetcdf (3.x) - nco_bm.pl detects Darwin and tests for bad cut. 2005-10-24 Daniel Wang <wangd@uci.edu> * Implement barrier in mpncra.c, which should fix TODO nco593. 2005-10-24 Charlie Zender <zender@uci.edu> * Initialize var->nc_id in ncpdq. Fixes TODO nco638. * Initialize dangling in_id in ncra, ncwa. Fixes TODO nco637. * Support netCDF4 NC_UBYTE,NC_USHORT,NC_UINT,NC_INT64,NC_UINT64 in nco_netcdf.c wrapper functions for libnetcdf.a library * Support netCDF4 NC_UBYTE,NC_USHORT,NC_UINT,NC_INT64,NC_UINT64 in metadata and some structures. Still need to add read/write and conversion/promotion functions. 2005-10-22 Charlie Zender <zender@uci.edu> * Eliminate small [mp]ncra leak with -Y pseudonym * Fixed all known bugs with parallelization over in_id MPI (except for known problems) and MPI_FAKE regression tests all passed This resolves TODO nco611 and is opportune time to re-benchmark SMP to check for any performance boost due to parallelization over in_id * #ifdef ENABLE_MPI spinlock -S switch so MPI_FAKE executables build * Finished re-parallelization of front-ends over in_id SMP operators appear to work, pass all tests on soot and ESMF 2005-10-21 Harry Mangalam <hjm@tacgi.com> * configure[.in], Makefile[.in], src/nco/Makefile[.in] in good enough shape to commit. Still some shakiness, but time to check clean checkouts on other platforms. Now supports --enable-mpi, --enable-netcdf4 * configure supports MacOSX build (detects OSX and deletes POSIX flag). 2005-10-21 Charlie Zender <zender@uci.edu> * Clean up old aborted attempt to parallelize ncecat over files * Verfied that parallelizing use of in_id in SMP code allows parallel read access to file data with nco_get_var*() in ncra. Hence, removed last critical read region from libnco.a SMP on other (non-ncra) operators temporarily broken until I parallelize their front end use of in_id. * Re-wrote nco_omp_ini() logic to use stubs outside of #ifdefs Now UP and SMP execute virtually same code in nco_omp_ini() * Prototype and declare OpenMP stub functions in nco_omp.c/h 2005-10-20 Charlie Zender <zender@uci.edu> * Replaced many instances of var->nc_id with function parameter nc_id to possible parallelization over nc_id in calling routine. Most code was already written this way. * Realized that TODO nco28 is bogus and deleted it: "Use var.nc_id as file ID from now on and rewrite old routines to do so" Bad idea because it would prevent parallelizing over nc_id in SMP code * Improve warning message for non-monotonicity in rec_crd_chk() to better distinguish inter-file from intra-file non-monotonicity. 2005-10-19 Charlie Zender <zender@uci.edu> * Mark all routines containing nco_put_var*() calls as OpenMP critical regions. Most nco_put_var*() calls occur in top level routines and were already enclosed in critical regions. Routines newly marked tended to be "under the radar" exceptions present only in programs which were not parallelized (e.g., ncks). These include: nco_cpy_var_val_mlt_lmt(), However, at least one routine, arm_time_install(), contained a write statement which I had forgotten all about. * Mark all routines containing nco_get_var*() calls as potential OpenMP critical regions (they are critical when called with identical nc_id's). These include: arm_base_time_get(), nco_cnv_ccm_ccsm_cf_date(), nco_prn_var_val_lmt(), nco_msa_rec_calc(), nco_var_get() * Mark nco_var_mtd_refresh() as non-OpenMP Critical Based on discussions with Unidata, metadata access (as opposed to data access) is thread-safe because all metadata is cached when file is opened. 2005-10-17 Harry Mangalam <hjm@tacgi.com> * Updated index.shtml to include Benchmark info 2005-10-16 Charlie Zender <zender@uci.edu> * Verify netcdf4 features work on Ubuntu AMD64 (sand) and Libranet x86 (elnino) 2005-10-14 Charlie Zender <zender@uci.edu> * Alphabetize lists in Makefile.am * Add nco_mpi.h to HEADER_SOURCE in Makefile.am 2005-10-13 Charlie Zender <zender@uci.edu> * Re-working SMP code to open files in parallel mode * Update MANIFEST * Attempt to add -U_POSIX_SOURCE token to MACOSX in configure build * Change MACOSX to include sys/time.h rather than time.h so that struct timeval is defined before use in struct rusage 2005-10-12 Charlie Zender <zender@uci.edu> * Insert stub for MPI documentation * Finish netCDF4 documentation in manual * Commit netCDF4 output support for all operators New options -4,--4,--64bit,--fl_fmt,--netcdf4 control file format * ncks: Eliminate --fmt,--format synonyms to avoid netCDF4 confusion Change --sng to --sng_fmt 2005-10-12 Harry Mangalam <hjm@tacgi.com> * added --enable-netcdf4 to configure, configure.in to find the bits necessary (libhdf5, libhdf_hl) to build the netcdf4 version of NCOs (TODO bld52) * mods to mk_bm_plot.pl to zero time axis (TODO bld56), compute regression and place on plots (TODO bld57), 2005-10-08 Charlie Zender <zender@uci.edu> * netCDF4 causes a regression in ncpdq packing/unpacking TODO nco615 2005-10-07 Charlie Zender <zender@uci.edu> * Remove USE_FORTRAN_ARITHMETIC from bld/Makefile * Support netCDF4 in bld/Makefile * nco_create_mode_prs() interprets user-specified file format * nco_create_mode_mrg() merges clobber mode with user-specified file format 2005-10-06 Charlie Zender <zender@uci.edu> * Add netCDF4 file writing capability to ncks Compatibility tokens are in nco.h so netCDF3 libraries do not complain * Change NETCDF4 to ENABLE_NETCDF4 2005-10-03 Charlie Zender <zender@uci.edu> * Add hooks to move ncra.c threading to embrace file opens 2005-09-30 Daniel Wang <wangd@uci.edu> * Branch mpncra.c to reorganize and understand nco593 2005-09-29 Charlie Zender <zender@uci.edu> * Clean up mpncra.c by removing barriers, adding comments * Remove superfluous close/open fl_in for MPI mpncra/mpncrcat/mpncea 2005-09-28 Harry Mangalam <hjm@tacgi.com> * Added --dust_user to allow setting a user that can log into dust for testing (hmangalm@esmf -> hjm@dust) * Fixed file tests broken in nco_bm.pl re-org. * Fixed udp reporting as per ditto. 2005-09-27 Charlie Zender <zender@uci.edu> * Implement static executable option to bld/Makefile with STC=Y 2005-09-25 Charlie Zender <zender@uci.edu> * Move final worker open/close calls to with ncra/ncea block * Change NC_WRITE to NC_WRITE|NC_SHARE in MPI codes 2005-09-24 Charlie Zender <zender@uci.edu> * Include Daniel's signal suspend code (-S switch) in mpncra * Use nco_set_fill() prior to each worker write 2005-09-23 Charlie Zender <zender@uci.edu> * Remove nco_tst.pl from distribution 2005-09-23 Harry Mangalam <hjm@tacgi.com> * Check for IP#/hostname conflicts, LAM/lamd, and MPICH/mpd in nco_bm.pl to help debug MPI errors. 2005-09-22 Charlie Zender <zender@uci.edu> * Change remaining NCAR_CCSM references to CCM_CCSM_CF 2005-09-21 Charlie Zender <zender@uci.edu> * Filter garbage files from bld/Makefile tags target * Tokenize/variable-ize various static MPI-related constants This is intended to improved readability and literate programming-ness Replaced naked constants with following compile-time tokens/enums: mgr_id,WORK_ALLOC,WORK_REQUEST,TOKEN_ALLOC,TOKEN_REQUEST,TOKEN_RESULT, NO_MORE_WORK,TOKEN_WAIT,TOKEN_ALLOC * Create new header nco_mpi.h for all MPI-specific header information Currently this header is only accessed by main() MPI programs Hence it is not (yet) part of libnco.a Header assumes its inclusion is already conditional on ENABLE_MPI This is currently a header mostly for pre-processor tokens with few odd file scope constants needed by MPI routines There are no functions yet Adding functions would require creating a corresponding *.c file Adding *.c file requires including it in libnco.a or separately linking MPI executables to it, i.e., slightly complicating the build procedure. I'm not yet sure I want _any_ MPI link dependencies in libnco.a Cross-platform link weirdness (e.g., AIX) makes this something to avoid as long as possible 2005-09-20 Harry Mangalam <hjm@tacgi.com> * added --xdata to nco_bm.pl to allow commandline data path to be entered (to allow benchmarking on different filesystems without resetting DATA env var). 2005-09-18 Charlie Zender <zender@uci.edu> * Tweak ncatted expected answer to pass hdr_pad test with new nco_bm.pl * Hack nco_bm* to pass thr_nbr,omp_flg,nco_D_flg * Re-remove prohibition on mpi_prc and thr_nbr simultaneously defined * Add mpi_fake_cp target to overwrite nc* with mpnc* (for AIX tests) * SMP mpncwa fails regression test 4 when thr_nbr > 1. TODO nco604. * All SMP mpnc*.c codes now compile cleanly * Fix SMP mpncpdq TODO nco598 by calling nco_close() once not twice 2005-09-17 Charlie Zender <zender@uci.edu> * SMP mpncpdq fails ncpdq regression test 4. This is TODO nco598. * Add mpi_cln target to bld/Makefile to remove MPI executables This makes it simple to build/erase all MPI_FAKE executables * Yesterday's MPI_FAKE mpncea fix broke MPI mpncea regression test 1 Today's fix works with SMP and MPI mpncea/mpncra/mpncrcat Testing shows this fix does nothing about ncrcat hanging TODO nco593 * Add $HOME/nco/bm to Perl @INC so nco_bm.pl runs from directories (e.g., bld) from which bm is ../bm * Verify icc 9.0-compiled NCO works in UP/SMP/MPI modes * Add '/' to $prefix so nco_bm.pl works when executables in $MY_BIN_DIR NB: nco_bm.pl does not run for me unless $MY_BIN_DIR is defined Probably same for other bld/Makefile users. 2005-09-16 Harry Mangalam <hjm@tacgi.com> * added --mpi_fake (aka -mpi_fke aka --fake_mpi) to allow running of the mpi executables as single processes. Since my laptop cannnot yet generate valid MPI apps, I can't tell what the real reulst of running them is but the commandlines look right. 2005-09-16 Charlie Zender <zender@uci.edu> * Move mpncra.c nco_var_srt_zero() from MPI to common code so UP/SMP executables zero start vectors. Fixes mpncea bug TODO nco594. 2005-09-16 Harry Mangalam <hjm@tacgi.com> * Re-org of regression and benchmarking script. nco_bm.pl has been broken into 5 files: - nco_bm.pl - now does initialization and calls control scripts for benchmarking and regression - NCO_bm.pm - a Perl module that contains most of the functions except the regression tests. - NCO_rgr.pm - a Perl Module that contains only the regression tests to isolate them and make it easier to find the right one. - nco_bm_benchmarks.pl - the code chunk which is do-included that contains all the benchmark code. - nco_bm_md5wc_tbl.pl - the code chunk that contains all the md5 & wc checksums. (currently NOT working - see below) * the error/debug messages have been cleaned up considerably and now are pumped out (mostly) at the end of each test if asked for. * a LOT of cruft was stripped out. There was quite a bit of duplicate code, commented out code, code that didn't seem to be perl code. It's not all gone, but a lot of it is. * the only thing that definitely does not work at this point is the MD5 checking due to some particulary hairy namespace problems. * the MPI versions of the checks have not been tested yet either, altho the commandlines it generates are correct. * I belive it is up to date with the latest regression tests and benchmarks. The relevant bits were diffed against the current code and they appear to be in sync. 2005-09-15 Charlie Zender <zender@uci.edu> * Port ncwa 20050608 bugfix for DO_CONFORM_[MSK,WGT] to mpncwa * mpncea hangs on ncea regression test 4 (on sand with LAM) TODO nco597 * Use fake end scopes to correct Emacs indentation in mpncra.c, mpncpdq.c * Test all FAKE_MPI mpnc* operators in place of nc* counterparts FAKE_MPI mpncea fails one test (TODO nco594) (regardless of thr_nbr) * Implement/fix OpenMP in MPI_FAKE portions of all mpnc*.c codes * Use icc ncecat bld/Makefile compilation fix for mpicc too * Use nco_mpi_get() to return MPI implementation information Invoke with undocumented ncks --mpi switch This (unwisely?) introduces mpi.h into libnco.a via nco_ctl.h Formerly, mpi.h only appeared in front-end codes Still no MPI _calls_ are in libnco.a, just header information * Allow MPI and OpenMP to both be true Now MPI_FAKE tests OpenMP so that MPI_FAKE operators are plug-compatible replacements for UP/SMP operators. No code actually takes advantage hybrid MPI/OMP since the MPI and OpenMP implementations are currently orthogonal in mpnc*.c level. This may change if benchmarks uncover good reasons to hybridize * Accomodate LAM MPI environment No difference between regressions with LAM and with MPICH2 2005-09-14 Charlie Zender <zender@uci.edu> * Remaining MPI bugs do not appear to be NCO-caused memory problems * valgrind.txt suppressions are complete for default MPI configurations on ashes (mpich2, valgrind 2.2, gcc 4.0.0) and elnino (mpich2, valgrind 3.0.1, icc 8.1) * Make LAST_RECORD shared in OpenMP portion of mpncra.c * ncea regression test 3 is bizarre: ncea and MPI mpncea pass it, MPI_FAKE mpncea fails it. This is TODO nco594. * Verify UP and SMP mpncra execute successfully while MPI mpncra still hangs just before writing output file * Clean up mpncra.c to build cleanly with MPI_FAKE * Fix mpncecat (same ole' strlen(fl_out_tmp) bug) * Change all valgrind suppressions examples to use ${HOME}/nco/doc/valgrind.txt, a direct copy of ${HOME}/c++/valgrind.txt * New MPICC, MPICXX logic in Makefile simplifies OS-specific sections AIX MPI builds must be done from scratch, i.e., make cln;make MPI=Y AIX MPI builds break non-MPI operators Hence a complete set of AIX executables requires two makes * Add valgrind suppressions for MPICH2 to ~/c++/valgrind.txt With these suppressed, valgrind finds no NCO-caused leaks in mpncbo MPID_* appears to cause a few harmless leaks * Move nco_free(cmd_ln) from master processor code to NCO-generic memory clean-up to avoid small leak in MPI'd operators * Regression test shows mpncbo commutivity modifications work! Same two-file framework should be implemented in ncflint/mpncflint * Only allow master MPI process to strlen(fl_out_tmp) This fixes nasty regressions which I introduced into Gayathri's codes Still need to understand why it caused the bug in the first place 2005-09-13 Charlie Zender <zender@uci.edu> * Move TOKEN_FREE=True to after nco_close() after nco_var_val_cpy() * Declare all MPI-specific variables in one place in mpncbo.c This removes all warnings when spoofing MPI with MPI_FAKE * Clean up bld/Makefile header * Rename FAKE_MPI switch to MPI_FAKE in bld/Makefile 2005-09-12 Charlie Zender <zender@uci.edu> * Change $caseid and $xpt_dsc to traditional meanings in nco_bm.pl * Add MPI CMD_LN's to nco_bm.sh for MPI regressions/bm's in loadleveler * Change pvmgetarch location from ~/bin/sh to ~/sh everywhere 2005-09-11 Charlie Zender <zender@uci.edu> * Update bld/Makefile to work with MPICH2 rather than MPICH * Add doc/mpi.txt to document MPI environments for NCO 2005-09-10 Charlie Zender <zender@uci.edu> * Pruned chapter/section/node titles * Found that many/most MPI tests complete successfully on soot (fedora) MPI failures on debian may be temporary/unreliable Documented different test results in MPI section of TODO * Committed recent mpncbo.c commutivity changes 2005-09-09 Charlie Zender <zender@uci.edu> * Complete and clean-up NCO MPI synonyms in prg_prs() 2005-09-08 Charlie Zender <zender@uci.edu> * Change CPP consistently with CC for Linux MPI compiles This ensures CPP finds mpi.h's hidden in strange directories * Add FAKE_MPI switch to compile MPI executables without MPI * Fix sundry compiler warnings with MPI front end codes * Port ncbo commutivity changes to mpncbo.c Resulting SPMD executable mpncbo hangs where previous mpncbo works Refrain from committing new mpncbo.c until it works 2005-09-07 Charlie Zender <zender@uci.edu> * Add ncrcat to regression test. Test chosen hangs with MPI. * Make mpncra regression first ncra test for faster failure * Change sftp regression test to succeed by breaking into two steps so that automatic sftp "Fetching..." string does not confuse tester * Allow mpncbo regression test to fail TODO 579 2005-09-06 Charlie Zender <zender@uci.edu> * mpncra.c: Correct printf arguments for type * Change bld/Makefile MPI handling so CFLAGS gets used 2005-08-22 Charlie Zender <zender@uci.edu> * cvs tag -c nco-3_0_2 Changes since nco-3_0_1: icc, OPeNDAP, ftp, sftp, .netrc, hdr_pad, cf, bm for MPI + OPeNDAP * Plug two small leaks in ncks when printing and fl_out=NULL * Plug double-free() error when fl_out=NULL in ncatted Double-free() errors cause operator failure on Fedora, which has a security-hardened glibc. Hence this was probably an unreported headache for many Fedora NCO users since 3.0.0 or 3.0.1. Recommend they upgrade and this spurs release of 3.0.2. * Add regression tests for ncatted mss_val and hdr_pad features * Remove duplicate nc_enddef() from ncatted and ncrename 2005-08-19 Charlie Zender <zender@uci.edu> * ncbo status: Works better than ever as far as I can tell. Now it: 1. Correctly handles cases where input variable orders are different 2. Correctly diagnoses (and attempts to broadcast) mis-matched dimensions 3. Can serve as template for similar [mp]ncflint re-write (TODO 578) 4. Improved logic needs to be merged into mpncbo.c so mpncbo passes (currently deprecated) regression test ncbo #8 (TODO 579). 5. Works well when nbr_var_fix_1>nbr_var_fix_2 so long as discrepant variables are type fix not type prc but needs list merge to work when nbr_var_fix_1<nbr_var_fix_2 (TODO 581). 6. Works for rank(var_1) > rank(var_2), not rank(var_2) > rank(var_1) Pre-compute broadcast size before var_def(out_id,var_out) (TODO 552) * Completely fixes TODO 550. Serial NCO regression test is spotless again. * Plug memory leak caused by ncbo two list re-write 2005-08-19 Harry Mangalam <hjm@tacgi.com> * Mods to nco_bm.pl to include but skip flaky mpi versions of ncpdq ncra * added alias line to accept mpncra 2005-08-19 Charlie Zender <zender@uci.edu> * Invert order of operands given to nco_var_sbt() by ncbo This fixes numerics of ncbo regression TODO 550 However, operand order to nco_var_[sbt,dvd,...] is non-intuitive and should be reversed NCO-wide in the future (now TODO 576) 2005-08-18 Charlie Zender <zender@uci.edu> * Add SFTP and .netrc-based password-protected FTP (to ESG) regression tests * Add stub for wget-based regression test * Fix ncbo regression to fail for right reason * Re-work nco_bm.pl net tests to all (potentially) pass for anyone 2005-08-17 Harry Mangalam <hjm@tacgi.com> * Add MPI bits to nco_bm.pl to test the MPI'ed operators. 2005-08-17 Charlie Zender <zender@uci.edu> * Add MPI_PRC option for bld/Makefile to pass through to nco_bm.pl * Add --hdr_pad/--header_pad options to ncatted and ncrename * Document --hdr/--header options 2005-08-14 Charlie Zender <zender@uci.edu> * Implement --hdr/--header options to ncks with nco__enddef() * Add hooks for output file-mode specification to ncks.c This may break compatibility with netCDF nersion 3.5.x and hence OPeNDAP Currently shrouded by #ifdef NETCDF4 tokens 2005-08-12 Harry Mangalam <hjm@tacgi.com> * small mods to nco_bm.pl to track diff in wall vs (sys+user) to get idea of how much time is lost to I/O (theoretically) 2005-08-13 Charlie Zender <zender@uci.edu> * Finished ncbo heart transplant: Symmetric subtraction still not working Regression tests appears to be no worse than before (ncbo causes 2 fails) Valgrind indicates new code is leak-proof except for one double-free() caused by ncap_var_cnf_dmn() Code appears to be commit-able since it's cleaner and extensible * Extended dual list method into heart of ncbo Saved last known working version as ~/ncbo.last_known_working.c * Change NCAR_CCSM_FORMAT to CNV_CCM_CCSM_CF, ARM_FORMAT to CNV_ARM 2005-08-11 Charlie Zender <zender@uci.edu> * Implement -Z/--64-bit-offset switch in ncatted and ncrename * Deprecate ctime_r() code from ncatted, put it (deprecated) into ncra * Add -4, --hdf, --netcdf4 switches to request HDF storage format 2005-08-08 Harry Mangalam <hjm@tacgi.com> * change nco_fl_utl.c to use passive ftp rather than active (many ftp servers no longer work with active requests) * change nco_bm.pl to do net tests (except for those requiring a specific login or location). Tests ftp (see 1st change) and DODS if compiled with DODS/DAP. 2005-08-08 Harry Mangalam <hjm@tacgi.com> * final updates to nco_bm.pl for testing DAP (reduced number of files requested under DAP (also becasue of bug in DAP that refuses > 30 files at once in some cases. * changed Makefile.in to work with 'make test' & 'make devtest' 2005-08-05 Harry Mangalam <hjm@tacgi.com> * cleaned out refs to '-n' in ncflint.c (doesn't support it) * major mods to nco_bm.pl to add DODS support to testing * may be a bug in DODS server code that doesn't allow >30 files in one session - posted. 2005-08-03 Charlie Zender <zender@uci.edu> * Added msk_* to ncbo's do not touch list for CF 2005-08-02 Charlie Zender <zender@uci.edu> * Added area, lat_bnds, lon_bnds to ncbo's do not touch list for CF * Re-wrote Conventions section in terms of CF rather than CCSM 2005-07-25 Charlie Zender <zender@uci.edu> * Re-coded so NCO uses .netrc, if present, only when .netrc is read-able and contains remote host name. Anonymous FTP is automatically invoked if .netrc does exist or does not contain remote host name. Henceforth NCO access to password protected FTP sites (like ESG) is scriptable! * FTP uses .netrc, if present, in all cases instead of anonymous FTP This essentially breaks anonymous FTP 2005-07-22 Charlie Zender <zender@uci.edu> * Finished SFTP support in nco_fl_mk_lcl() Syntax is sftp://hostname:/filename Updated manual accordingly * Add Gayathri's latest mpncra.c to distribution and Makefile 2005-07-21 Charlie Zender <zender@uci.edu> * Change test1.sh to use CWD ncks so cygwin works We should deprecate the FTP and DODS tests and move to nco_bm.pl 2005-07-18 Charlie Zender <zender@uci.edu> * Add .netrc information to FTP description in manual * Add hooks for sftp support in nco_fl_utl() * Improve OPeNDAP examples in manual 2005-07-15 Harry Mangalam <hjm@tacgi.com> * Change UDP log format from nco_bm.pl to make for better parsing * Add new script to bm dir (mk_bm_plots.pl) to convert UDP data to postscript plots for each operator (currently wall, real, sys, user time) but can add output of rusage if it looks useful as well. 2005-07-14 Charlie Zender <zender@uci.edu> * MPI NCO with icc 9.0 appears to work on dual Opterons! * OpenMP NCO with icc 9.0 passes regression tests on dual Opterons! * Change nco_mmr_typ to enumerated type to fix icc warning on Opteron * Put more icc flags in configure.in * MPI NCO with icc 8.1 successfully compiles and execute on Fedora It failed to execute with message "lamd not running" until "lamboot" command was executed first to setup LAM/MPI environment. Debian machines need no such initial setup. After logging out and in once, MPI NCO ran fine without having to run lamboot first. Apparently it's just a one-time setup (per machine? or per user?) * Verified that MPI NCO works with GCC on dual opteron machines * Add -D_POSIX_SOURCE to default gcc and icc flags to fix compiler warnings in ncap_lex.l * Add -D_BSD_SOURCE to default icc flags to fix MPI with icc 2005-07-10 Charlie Zender <zender@uci.edu> * Verified icc passes regressions tests and icpc-compiled libnco_c++.a works as intended. Have not tried with autotools yet. Have not yet tested MPI programs with icc. * -axW flag causes icc 8.1 to die compiling ncecat This appears to be an icc bug Implement target-specific workaround in bld/Makefile * Implement Intel 8.1 compiler compatibility 2005-07-07 Charlie Zender <zender@uci.edu> * Fix small leak in ncap_var_stretch() TODO ncap74 ncap now appears to be leak-proofed! * Added lots or mirror fl_1=fl_2 code to ncbo in preparation for attempted solution to subtraction problems related to asymmetry. In plain English: copied lots of code, didn't solve any problems yet. * Add td,tx,ty,tz,txyz variables to in.cdl * Match limit with dimension based on name not ID so nco_dmn_lmt_mrg() works with single limit list on any input file 2005-07-03 Charlie Zender <zender@uci.edu> * Realized that ncbo bug is longstanding/systemic and may require re-writing approach to determining confmormality of var_2. Current code is expected to break when second file has same name but different-sized dimensions, or has same-sized dimensions in different order, i.e., all sort of corner-cases. 2005-07-02 Charlie Zender <zender@uci.edu> * Remove thr_nbr argument from ncatted in nco_bm.pl * DAP 3.5.1 triggers reproducible double-free() memory error in ncbo step of ncra regression test 16. Problem occurs regardless of OpenMP (soot,elnino) and compiler (gcc,icc) ncra -O -C -v rec_var_flt_mss_val_dbl ~/nco/data/in.nc ~/foo_avg.nc ncbo -O -C -v rec_var_flt_mss_val_dbl ~/nco/data/in.nc ~/foo_avg.nc ~/foo.nc free(): invalid pointer 0x80e4a98! * Verified DAP 3.5.1 works with both build mechanisms with gcc NCO/gcc DAP (Debian elnino), and with icc NCO/gcc DAP * Report NC_64BIT_OFFSET rather than old-fashioned LARGEFILE test with -r option in nco_lbr_vrs_prn() * Change 'DODS' to 'DAP' or 'OPeNDAP' where possible in codbase, e.g., 'enable-dap', make DAP=Y, ENABLE_DAP * Support OPeNDAP 3.5.x in bld/Makefile * Re-write dods.sh for OPeNDAP 3.5.x terminology and save in new file opendap.sh 2005-06-30 Charlie Zender <zender@uci.edu> * Critical region on OpenMP reads nco_get_vara() are still necessary Want to establish if this is NCO or netCDF problem Testing for reproducibility, following regressions on soot, esmf: Run ncra ncwa soot 1 01 01,13 soot 2 02,06,10,11 none soot 3 01,04 01 esmf 1 01,02,11,12 none esmf 2 04,05,06,07,10,11,12,16 none esmf 3 02,03,04,09,15,16 13 Regressions are non-reproducible, characteristics of threads over-writing eachother due to random timing conflicts. Hard to debug because of irreproducibility Q: NCO calls nco_get_vara() in many routines---why is the call in nco_var_get() the only one that needs to be critical? A: Probably because it is that most important, done on every variable Options: 1. Ask Unidata whether they have tested SMP robustness of netCDF reads under demanding battery of conditions? 2. Scrutinize uninitialized data reported by icc/valgrind? 3. Write own SMP on read test? 4. ??? * Finish making opaque types nco_byte, nco_char, nco_int self-consistent 2005-06-30 Charlie Zender <zender@soot.ess.uci.edu> * Change nco_long to nco_int to allow for future 64-bit type called nco_long * Abstract opaque types into new header nco_typ.h 2005-06-29 Charlie Zender <zender@uci.edu> * Explicitly cast NC_CHAR type as "char" (rather than "unsigned char") in nco_netcdf.c and nco.h. Previously, there were a mixture of "char" and "unsigned char" occurances. Not sure if this helps anything except consistency * Removed two of three critical regions in nco_var_utl.c Removing these causes no regressions with SMP on icc/x86 or xlc/Power4 Final critical region is, well, critical Region contains the nco_get_vara()/nco_get_var1() calls themselves 2005-06-28 Charlie Zender <zender@uci.edu> * Conditionally free both $1 and $3 in var,var addition in ncap_yacc.y * Try and fail to free input variables if they were promoted in ncap_var_cnf_dmn(). Not sure what prevents this from working. 2005-06-27 Charlie Zender <zender@uci.edu> * Merge CCSM data processing example into nco.texi 2005-06-25 Charlie Zender <zender@uci.edu> * Un-constify cvs_Name to workaround gcc-4.0+ warnings 2005-06-22 Charlie Zender <zender@uci.edu> * Fix bld/Makefile indent that caused make to treat ncap: as command not rule 2005-06-21 Charlie Zender <zender@uci.edu> * Make LINUX_CC more MPI-friendly for both gcc and icc 'make MPI=Y' just works on my Debian machines now when MPICH_CC=gcc and fails when MPICH_CC=icc * Set OMP=N when MPI=Y in bld/Makefile * Add ${MDL_BIN_TRG} to .PHONY in bld/Makefile to solve problems with building MPI executables * Change configure.in 'case' statement from sparc-sun-solaris2 to sparc-sun-solaris2* to handle newer solaris triplets 2005-06-21 Harry Mangalam <hjm@tacgi.com> * fixed Makefile.in and nco_bm.pl to work with: make test does regression test with no debug info make devtest does regression test with debug info and udpreporting make benchmark runs benchmark with udpreporting make files creates the benchmark files * nco_bm.pl now defaults to <nco_root>/src/nco as MY_BIN_DIR (if no environment var set) instead of going thru the dialog. 2005-06-20 Charlie Zender <zender@uci.edu> * Add sys target to bld/Makefile. 'make sys' installs executables in $MY_BIN_DIR to /usr/local/bin 2005-06-17 Charlie Zender <zender@uci.edu> * Verified that nco_var_refresh() change does not introduce any memory errors on any operator. Marking TODO nco543 as closed. * Re-name nco_var_refresh() to nco_var_mtd_refresh() to better describe functionality (metadata, not data, is refreshed). * Refresh variable type in nco_var_refresh() Previously, only ID, nbr_dims, and mss_val were refreshed Now type is also refreshed. Was not sure full implications of this. It appears to work and causes no regressions Hence back out previous fix for nco543 and handle this way This is cleaner and more robust to varying variable types with each input file * Revert ncra input variable structure to unpacked type at end of record loop Reverting input values is not necessary since those are free()d next However, type member of structure must be unpacked value so that cast of new missing_value in nco_mss_val_get() in next nco_var_refresh() call (i.e., next file) This solves TODO nco543 but will break when variables are different unpacked types between files and both files contain missing values. Better solution is to refresh variable type in nco_var_refresh()? Will commit this first then try that next Best solution is to carry missing_value type in variable structure? 2005-06-17 Harry Mangalam <hjm@tacgi.com> * Tested NCO with new (3.5.1) version of DAP/NC-DAP, with netcdf 3.6. Seems to work for small transfers but hangs with large transfers from sand's DODS server. Still testing. Brought configure and Makefile and .in's up to date with new DAP. May have to drop back to netcdf 3.5 tho. 2005-06-17 Gayathri Venkitachalam <gvenkita@uci.edu> * Cleaned up mpncflint 2005-06-17 Charlie Zender <zender@uci.edu> * Convert var_prc[idx]->mss_val back to on-disk type after every record _except_ the last record in ncra. This fixes valgrind memory error associated with TODO nco543 Regression test still not passed, though no new regressions introduced 2005-06-16 Charlie Zender <zender@uci.edu> * Added ncea regression test similar to current failing ncra TODO nco543 * Remove call to nco_typ_cnv_rth() for var_prc[idx] in ncra and ncea Call appears redundant since variable is converted to output type with nco_var_cnf_typ() two lines later anyway. Removing nco_typ_cnv_rth() for var_prc[idx] causes no new failures in regression tests. * Set default packing policy for all potential MPI ncpdq operators 2005-06-16 Gayathri Venkitachalam <gvenkita@uci.edu> * Added new operators mpncwa, mpncpdq, mpncecat * Modified mpncbo so that all processes do the final clean-up * Modified nco_ctl.c and nco_pck.c to accommodate new MPI operators 2005-06-15 Gayathri Venkitachalam <gvenkita@uci.edu> * Added a couple of tests for ncra (average (i)rec_var_int_mss_val_int (ii)rec_var_int_mss_val_flt across 2 identical input files) in nco_bm.pl which seem to fail 2005-06-15 Harry Mangalam <hjm@tacgi.com> * MD5 version of nco_bm.pl checked in - works on SV (Single Values) on x86-32, x86-64, esmf. Passes benchmarks on all these platforms. on x86-32, passes all regressions except for funny-wrap-on-Intel regression, passes all MD5 & WC tests for regression (except noted above) passes all benchmarks with MD5, WC, SV on x86-64, passes all regressions except ncap funny business (fails 3) passes all but 4 MD5 checks (makes sense in terms of what fails) passes all benchmarks on SV but fails all MD5 checks on AIX-64 passes all regressions but 4 of the ncaps tests passes all MD5 & WC tests for regression (except noted above) passes all benchmarks on SV but fails all MD5 checks * pick up DATA and MY_BIN_DIR correctly * reduce debugging cruft, tho in debug mode, it still burps text in ugly ways 2005-06-15 Charlie Zender <zender@uci.edu> * Add MD5 and SHA1 digests of nco.tar.gz to index.shtml 2005-06-13 Charlie Zender <zender@uci.edu> * Improve threading code intention comments in nco_omp.c * Still need to tweak run and data directories in nco_bm.pl and nco_bm.sh * nco_bm.pl and nco_bm.sh appear to work with loadleveler Solution is not general yet because unable to pass environment variables (except DATA and HOME) through scripts for now * nco_bm.sh: Never pass strings with whitespace through Perl Getopt * nco_bm.pl: Print many arguments when dbg_lvl=1 2005-06-12 Charlie Zender <zender@uci.edu> * Noticed that regression tests in nco_bm.pl report some apparent successes as failures on SUNMP, e.g., Result: [1e+36] != Expected: [1e+36] always recorded as failure. Seems like any "Expected's" containing "+" string may fail. Perl problem? * Ran benchmarks on laptop (elnino), desktop (soot), supercomputer (esmf). Noted that file creation is sickeningly slow on laptop and desktop, takes thousands of seconds. Fast on ESMF. Same on benchmarks: very slow on laptop and desktop, fast on ESMF interactive node. * Add -t thr_nbr to nco_usg_prn() and nco.texi for newly threaded operators * Do not call ncap with threads in nco_bm.pl fl_cr8() * Allow nco_bm.pl to run locally from bld/Makefile 2005-06-10 Charlie Zender <zender@uci.edu> * cvs tag -c nco-3_0_1 Changes since nco-3_0_0: 1. ncap exponentiation 2. ncwa threading 3. intel icc compatibility 4. Improved benchmarking * Simplify treatment of math symbol table C99 mandates support for erf(), erfc(),tgamma(), acosh(), asinh(), atanh(), cosh(), sinh(), tanh() Some OS's, like Linux, Solaris, provide these without C99. Many others (AIX, SGI*, WIN32) do not. Complicating things, many C99 implementations, e.g., AIX xlc, do not yet support these functions. Hence, ncap only supports them on Linux machines. 2005-06-09 Charlie Zender <zender@uci.edu> * Change default AMD64 compiler to gcc-4.0 * Revert patches to known working state (i.e., make reads critical) * Temporarily un-revert critical region patches in nco_var_get.c With icc and xlc_r, this causes variable numbers of ncra and ncwa regressions. Usually many more ncra than ncwa. icc/valgrind combination gives capability to narrow down causes. 2005-06-08 Charlie Zender <zender@uci.edu> * Change thr_nbr_max_fsh from 1 to 2 for ncrcat * Change thr_nbr_max_fsh from unlimited to 2 for ncecat * Change thr_nbr_max_fsh from 1 to unlimited for ncwa * Change thr_nbr_max_fsh from 4 to unlimited for ncbo * Marked TODO nco325, nco345, nco523, nco189, nco214 as resolved * Re-ran all valgrind tests on soot with two threads enabled All operator appear memory clean in NCO portions except ncwa ncwa may have a few unfree()d buffers but hard to tell with valgrind version 2.2 which does not have good tracebacks. Threading library has plenty of unfree'd memory which apparently is harmless and there's nothing to be done about that anyway. * ncbo memory problems disappeared * Add calls to nco_omp_ini() and nco_thr_att_cat() in ncecat This fixes problem where thr_nbr was being set but not used * Use icc warnings to improve printf() formatting in nco_prn.c * Optimize bld/Makefile icc switches for OPTS=O,R,D,X * Do not pass --thr_nbr to ncatted in nco_bm.pl * Change DO_CONFORM_MSK,DO_CONFORM_WGT from private to firstprivate and make sure both always have default values This fixed all ncwa regressions with threads! * icc-compiled version passes regression test perfectly on soot only when threading is disabled at compile time, e.g., OMP=N. With threading enabled, code fails many tests (1 ncra and 22 ncwa) ....regardless of thread number! Not requesting any threads, (i.e., -t 0 or no -t specification), requesting one thread (-t 1), or requesting more threads breaks regression tests. Interesting that AIX version passes all, and icc fails many, regressions when threading enabled with one thread. icc fails _most_ ncwa calls. Averaging a single variable fails! Lots of juiciness in error messages! icc will help to isolate any problems because it is so much more sensitive than xlc_r. Moreover, valgrind appears to work fine with icc-compiled OpenMP code on Xeon. * Code compiles as clean as possible with icc 8.1 and gcc 4.0 * Prohibit double definition of cmp_nm with icc * Updated bld/Makefile and configure.in to work better for icc by making -std=c99 standard switch * Changed default compiler on Ubuntu to GCC-4.0 Fixed new CEWI warnings resulting from change Testing shows no new regressions * Reverting the 20050519 critical region patches fixes ncra regressions on AIX OpenMP. Hypothesis that libnetcdf.a is not thread-safe on reads (or writes) gains considerable weight, since all the patches do, I think, is make reads critical. * Reverted 20050519 critical region patches * About to revert patches of 20050519 which removed critical regions from NCO read routines * Added nco.texi example on changing non-record to record dimensions 2005-06-07 Charlie Zender <zender@uci.edu> * Noticed that many ncra regression tests fail on AIX When did this begin? What causes it? Results from date-stamped-version tests 20050605: Fail; 20050524: Fail; 20050521: Fail; 20050520: Fail; 20050519: Succeed; 20050517: Succeed; 20050510: Succeed; 2005-06-06 Charlie Zender <zender@uci.edu> * Add THR_NBR option to bld/Makefile to pass to nco_bm.pl * Learned that Linux gamma() function is just plain broken and obsoleted, at least in C99, by tgamma() function. Linking gamma() symbol to tgamma() intrinsic solves Linux problem Note that tgamma() is a C99 feature Referencing tgamma() in the code is the second C99 feature, after the restrict type keyword. 2005-06-05 Charlie Zender <zender@uci.edu> * gamma() function appears broken in ncap regression test 07 * Added ncap regression tests for modulo, exponentiation, sin, cos, atan operations 2005-06-02 Charlie Zender <zender@uci.edu> * Henry implemented ncap exponentiation bugfix. Versions < 3.0.1 incorrectly handle exponentiation of a variable to a variable power (V^V). 2005-05-29 Charlie Zender <zender@uci.edu> * Account for typesize in allocation of ncap_ln_nbr_crr and ncap_fl_spt_glb in glb_init_free(). Fixes memory read/write error. * Began purifying ncap memory * ncks appears to be memory-pure! * Leak-proofed nco_msa.c 2005-05-26 Hary Mangalam <hjm@tacgi.com> * finally brought in ncecat in OMP form * changed configure.in configure scripts to reflect sync with bld/Makefile flags; (again) noted problem with static/shared and DODS in configure.eg * accepted CZ's peculiar variable naming in bm script. 2005-05-26 Charlie Zender <zender@uci.edu> * More fully initialize lmt_all_crr->lmt_dmn[0] to enable subsequent free()ing with nco_lmt_free() * Tidy up nco_cpy_var_dfn_lmt() prototype and use lmt_all rather than lmt to designate lmt_all rather than lmt structures 2005-05-26 Henry Butowsky <henryb@ntlworld> * Changed in ncks.c lmt_all_lst from a 1D to a 2D variable also changed associated functions in nco_msa.[ch] 2005-05-24 Charlie Zender <zender@uci.edu> * Changed from ctime() to re-entrant ctime_r() with ncatted Does not help reduce unfree()'d system memory according to valgrind 3.0 development branch. Does not hurt, either. How portable is ctime_r()? SUS, yes, but POSIX? 2005-05-23 Charlie Zender <zender@uci.edu> * Touch parameter rusage_who for CEWI in nco_mmr_rusage_prn() * Fix double-free() problem in ncrename 2005-05-22 Charlie Zender <zender@uci.edu> * Add memory cleanup to sng_lst_cat() * Nearly purified ncatted memory * Purified ncrename memory * Began ncks memory cleanup * Fix union type-casting in nco_var_cnf_typ() 2005-05-21 Charlie Zender <zender@uci.edu> * Debug statements for TODO nco523, vexing SMP OpenMP thread bug * Execute free(var_prc[idx]) sooner in ncwa * Re-tweak print formatting in nco_bm.pl 2005-05-19 Charlie Zender <zender@uci.edu> * Change nco_get_var*() routines in nco_var_get() from SMP critical to non-critical. Not sure why they were originally critical. * Change nco_var_upk*() routine in nco_var_get() from SMP critical to non-critical. Not sure why it was originally critical. * Make nco_var_refresh() routine from SMP non-critical. Not sure why it was originally critical. 2005-05-18 Charlie Zender <zender@uci.edu> * All arithmetic operators except ncap now have no known memory leaks or errors * Finished memory cleanup on ncwa 2005-05-17 Charlie Zender <zender@uci.edu> * Appear to have fixed all leaks in nco_var_avg() * Fix problem with dangling pointers in var_out caused by destruction/creation of var_prc_out in nco_var_avg(). Solution adopted is somewhat hackish: rearranging nco_var_lst_free()s Perhaps better solution would be to copy new var_prc_out pointers into var_out, the free() memory normally. * Plug leak when nco_var_avg() called with variables without averaging dimensions (i.e., fixed variables). When this occurs, return copy of original input instead of NULL (old method). * Add cleanup_and_exit branch to nco_var_avg() * Never allocate tally array for var_prc[idx] in ncwa, since it is never used and caused double free() problems when it share the var_prc_out[idx] tally buffer 2005-05-16 Charlie Zender <zender@uci.edu> * Stop cross-referencing fix with var->xrf->xrf=fix in nco_var_avg() * Make nco_bm.pl output format for regression test look like configure 2005-05-07 Charlie Zender <zender@uci.edu> * Reduced default verbosity of nco_bm.pl * Fixed ncap_var_init() prototype and nco_cnf_typ() to compile cleanly 2005-05-06 Henry Butowsky <henryb@ntlworld> * Cleaned up double parsing in ncap.c,ncap_*.[hc] now passes basic tests 2005-05-05 Charlie Zender <zender@uci.edu> * Comment out three_double_dmn from in.cdl to avoid triggering errors in DODS server and/or netCDF client library 2005-05-03 zender <zender@uci.edu> * Threaded ncflint. Not tested yet. 2005-04-29 Gayathri Venkitachalam <gvenkita@uci.edu> * Cleaning up mpncbo.c prior to replacing ncbo.c 2005-04-27 Charlie Zender <zender@uci.edu> * Cleaning up mpncbo.c prior to replacing ncbo.c 2005-04-25 Gayathri Venkitachalam <gvenkita@uci.edu> * Updated mpncbo to compile with and without MPI. Compiling without MPI=Y will result in identical executables mpncbo and ncbo * Updated Makefile to reflect the same * Added mpncflint, the MPI version of ncflint 2005-04-23 Charlie Zender <zender@uci.edu> * Clean up main.c headers a little 2005-04-20 Charlie Zender <zender@uci.edu> * cd to bm for all bm targets in bld/Makefile 2005-04-19 Charlie Zender <zender@uci.edu> * Changed/added 'make tst' targets in bld/Makefile to use nco_bm.pl instead of nco_tst.pl * Finished ncecat cleanup 2005-04-19 Gayathri Venkitachalam <gvenkita@uci.edu> * (Apparently) fixed the bug in mpncbo while hyperslabbing 2005-04-19 Charlie Zender <zender@soot.ess.uci.edu> * Initial ncecat memory cleanup---still one problem with dimension list 2005-04-19 Harry Mangalam <hjm@tacgi.com> * Large cleanup of benchmark code integrated file build, benchmark, and regression testing udp reporting only by request * added --queue to denote submission to queue (loadleveller, etc) which kills off script if the proper env vars aren't set. If --queue isn't set, and env vars aren't set, script will prompt for correct paths added Failure column to summary output to make it easier to see which ones failed. * added nco commandline to UDP report so can verify timing exactly. * added --thr_nbr to option list to set # of threads to use (not implemented internally yet tho) * finished adding those benchmarks that CZ wants added: 1 - ncwa averaging all variables to scalars 2 - ncra time-averaging 2^5 (i.e., one month) files (using symlinks) 3 - ncea averaging 2^5 files 4 - ncbo differencing two files (using '-' as operator) 5 - DEFERRED (Martin Schultz multi-slabbing test) 6 - ncap long algebraic operation 7 - ncpdq packing the file 8 - ncpdq dimension-order reversal the file * above benchmarks complete cleanly on sand and on those that can be done on bodi 2005-04-18 Charlie Zender <zender@uci.edu> * With following six modifications, mpncbo compiles, runs and gives apparently correct results on my laptop with default run. However, mpncbo exits with errors when hyperslabs are specified, e.g., mpncbo -d lon,1 ... * Synchronize mpncbo memory clean-up with latest ncbo * Fix lmt handling in mpncbo as per 20050416 changes * Add prototype for nco_mpi_att_cat() to nco_att_utl.h * Add mpncbo synonym for ncbo in prg_prs() and nco_op_typ_get() * Remove #ifdefs around nco_mpi_att_cat() which requires no additional MPI-specific headers so is always safe to compile * Change MPI attribute to nco_mpi_task_number 2005-04-18 Gayathri Venkitachalam <gvenkita@uci.edu> * Checked in mpncbo.c & nco_att_utl.c after updating them to compile with MPI=Y. Compiling with MPI=N still behaves the way it used to. 2005-04-18 Charlie Zender <zender@uci.edu> * Memory status: The only commonly used code which has leaks/errors in the arithmetic operators (excluding ncap) appears to be the dimension conformance code, i.e., broadcasting. ncwa highlights these problems. ncbo, ncea, ncflint, ncpdq, ncra, ncrcat are either memory-clean or causes of problems are known and await resolution. ncecat has yet to be examined but is expected to respond well to same treatment given to ncea/ncra/ncrcat. Problem #1: Promotion of char->double (and byte->double?) uses uninitialized memory. This may be a C-language or libc problem, ultimately, which we can do nothing about. Not sure. It is not serious because promotion of char->double for arithmetic is intrinsically ill-defined and rarely used. It would be helpful to document this problem in the manual, and possibly give users some control over code paths, e.g., --disable_promote_char_double. Problem #2: nco_var_cnf_dmn(), in its various incarnations and uses, leaks worse than Harry's roof. The upside is that it is memory-efficient and fast. This problem is serious and needs careful fixes. Problem #3: ncap. I have not examined ncap since fixing the huge ncap_var_stretch() leak. There seem to be many tiny leaks in the flex and bison-generated code, which surprises me. If these really are leaks, not sure what we can do beyond creating a valgrind suppression file for them. 2005-04-17 Charlie Zender <zender@uci.edu> * Clean up leaks in file name code when abbreviation lists used Solving leak in nco_fl_nm_prs() was rather contorted but appears to work * Fix off-by-two error in memmove() in ncwa 2005-04-16 Charlie Zender <zender@uci.edu> * Initial memory work on ncflint * Currently, memory in ncpdq and ncbo appears faultless memory in ncra has one problem when up-converting char to double for averaging * Tighten up memory in ncwa by free()ing dimension lists * Changed limit (lmt) memory paradigm from 1D to 2D structure list in accord with all string, variable, and dimension lists. This makes handling limits easier, at the expense of more indirect referencing in metadata routines. Replace lmt[idx]. with lmt[idx]-> All lmt-related memory leaks appear to be gone 2005-04-15 Charlie Zender <zender@uci.edu> * Gayathri noted problem with missing_value inconsistency between var_prc and var_prc_out for identical input files on both packed and unpacked variables. Still investigating. * Case where var2 did and var1 did not have missing_value was treated incorrectly in ncbo.c and inconsistently in nco_mss_val_cnf(). Fixed. 2005-04-14 Charlie Zender <zender@uci.edu> * Noticed that Fedora core 2 glibc automatically warns about double free()s etc and that regression test doesn't look so hot when all these are printed. More reasons to fix the remaining memory problems... * Fixed AIX regression for printing bytes by moving from non-standard %hhi format (which printed all bytes as "hi" on AIX) to standard %c format, which is lame and does not print signed integer values for bytes. Now it should not trigger regressions. * Added mpncbo.c as MPI-enable ncbo alternative Builds only with bld/Makefile using make MPI=Y switch and requires MPI packages (e.g., mpich) and access to mpi.h Has not been verified not to break autotools builds yet 2005-04-13 Charlie Zender <zender@uci.edu> * Fixed all memory leaks in simplest ncbo calls 2005-04-12 Charlie Zender <zender@uci.edu> * Free() dmn_lst in all applicable operators * Free() xtr_lst in all applicable operators * Free() cmd_ln in all operators * NULL'ify var_prc_out[idx]->tally when free'ing var_prc[idx]->tally in ncra. This fixes double free() condition caused because these both point to same buffer. 2005-04-10 Charlie Zender <zender@uci.edu> * Implement preliminary memory free routines in ncra Fairly straightforward---best place to start now that ncpdq is done * Implement preliminary memory free routines in ncwa Clear that limit structures will need some deep work Genuine memory errors do occur and give hope that ultimately ridding all ncwa problems may be possible by fixing leaks * Implement preliminary memory free routines in ncap and immediately give up due to size and scope of job * Move list free functions to respective utility routine files * Changed names of ncap routines that maniputlated nm_id structures These routines used the *_var_* nomenclature, e.g., nco_var_lst_free(), even though they manipulated lists of nm_id_sct structures. Changed *_var_* to *_nm_id_* in these functions so that routines are named after structure types which they manipulate, rather than what their contents are * Created new routines to free lists of lmt, dmn, var structures * ncpdq is officially leak-free! The first leak-free operator * var_nm is finally free'd, takes care of TODO nco224 * Fixed bug in nco_var_cnf_dmn() and nco_var_stretch() where var->nm pointer was being copied instead of deep-copying This appears to have fixed ncwa regression TODO nco490 * Verified that var->nm freeing is last obstacle to perfect memory releases in ncpdq 2005-04-09 Charlie Zender <zender@uci.edu> * Simplify nco_lmt_udu_cnv() because of new lst_prs_2D() list processing * Transition all of NCO from lst_prs_1D() to lst_prs_2D() All operators still pass regression test * Use nco_sng_lst_free() to free fl_lst_in in ncpdq * Transition ncpdq from lst_prs_1D() to lst_prs_2D() All operators still pass regression test 2005-04-08 Charlie Zender <zender@uci.edu> * Patch nco_mmr.h to include time.h. Fixes Mac OS X build for struct rusage * Change default printing behavior of ncks again using suggestion of Takeshi Enomoto. ncks in.nc : Prints all data and metadata ncks -M in.nc : Prints only global metadata ncks -m in.nc : Prints only variable metadata ncks -H in.nc : Prints only variable data 2005-03-27 Charlie Zender <zender@uci.edu> * Create lst_1D_to_2D() to replace lst_prs(), which is responsible for much craziness and was renamed lst_prs_old(). Unfortuneately, immediately switching showed many regressions lst_1D_to_2D() needs more extensive testing * Implement nco_sng_lst_free() to free string lists * More fixes to ncpdq memory management...only one leak remains! 2005-03-26 charlie <zender@uci.edu> * Introduced unknown regression into ncwa nco_var_cnf_dmn() * Pared ncpdq unfree'd memory down to three, final, hard-to-understand, problem malloc's totaling ~150 B. * Free dmn_rdr_lst_in() in ncpdq * Fix inherent leak in prg_prs() * Deep-copy nm in nco_var_dpl(); free nm in nco_var_free() Ditto with dmn->nm, nco_dmn_dpl(), and nco_dmn_free() * Implement nco_dmn_free() to free dimensions * Guaranteed system memory (optarg) reaches no further than command-line parsing in all operators. This has side effect of temporarily increasing un-free'd memory in programs, because, formerly, valgrind would not count that memory against NCO because it was system memory (optarg). Now NCO owns it and needs to free it. * Planning to eliminate all remaining memory leaks from NCO Un-free'd strings cause the most memory leaks (I think) When NCO originated in 1993-1994 I was zealous about memory usage. I chose to use the system memory allocated to optarg rather than duplicate that memory in NCO. As NCO grew and became more object-oriented, being able to create structures (e.g., variables) dynamically but not able to free all their elements (e.g., names), some of which referenced system memory (from optarg), lead to today. Here are four examples (from ncks.c) of memory that NCO cannot free(): fl_pth_lcl=optarg; fl_pth=optarg; dlm_sng=optarg; var_lst_in=lst_prs(optarg,",",&nbr_xtr); The first two or three lines do no cause much problem because those strings are not duplicated much or contained in structures and so never lead to situations that valgrind count against NCO Using system memory for var_lst_in, however, causes problems Strategy: 1. Replace all such fragments in NCO by strdup()'d memory 2. Follow their code paths and free() these strings when possible 3. Add block to free all pointers at end of code (already in ncpdq) 4. Modify destructors (e.g., nco_var_free()) to free strings Initial work to free memory will be conducted in ncpdq Implement same methods in other operators once ncpdq is leak-free... 2005-03-25 charlie <zender@uci.edu> * Documented and fixed some small ncap memory leaks with valgrind 2005-03-24 Harry Mangalam <hjm@tacgi.com> * added bm dir for benchmarking; initial files are: gne_exp.cdl template -> ~54MB file of gene expression data ipcc_dly_T85.cdl template -> ~1GB file with 4D,3D,2D,1D,scalar variables ipcc_dly_T85.sh shell file that creates & populates the above file nco_bm.pl the perl benchmarking script (derived from bld/nco_tst.pl) sml_fls.sh script to generate both small files: gne_exp.cdl & sml_stl.cdl sml_stl.cdl template for a small sat data file stl_5km.cdl template for ~300MB sat data file stl_5km.sh script to generate and ncap the above file 2005-03-24 Charlie Zender <zender@uci.edu> * cvs tag -c nco-3_0_0 Changes since nco-2_9_9: 1. ncap bugfix 2. Add benchmarking 3. libnco_c++ implements nco_var_puta() 2005-03-23 Charlie Zender <zender@uci.edu> * Fix gaping memory leak in ncap_var_stretch() 2005-03-22 Charlie Zender <zender@uci.edu> * Implement nco_mmr_rusage_prn() for memory diagnostics 2005-03-21 charlie <zender@uci.edu> * Implemented Harry's suggestion to use memset(vp,0,size) to initialize float and integer buffers to zero. Previously, this was done in a loop over each index value Presumably this old method was slower because of pointer de-referencing New algorithm implemented in nco_var_zero() and nco_zero_long() * Set NC_NOFILL in ncap_var_write() * The call to nco_aed_prc() (a metadata writing call) was un-necessarily late in ncap.c. I moved it to just after nco_var_dfn(). This takes advantage of the netCDF feature that metadata (not data) can be manipulated before it is flushed to disk (with nc_enddef()). You can learn/use a variable's ID before populating the variable's data section. Now ncap ends after writing in data mode and avoids the heavy disk penalty of moving variables to accomodate the increased metadata from the nco_aed_prc() call. * Turns out the only NC_NOFILL in ncap.c was done in data mode after the variables were defined. Not sure why this did not cause an error. ncap was probably doing redundant data filling for the non-processed variables defined. Fixed that. 2005-03-20 Charlie Zender <zender@uci.edu> * Apply same optimization to nco_var_cnf_dmn() * Handle expansion of scalar variable as special case in ncap_var_stretch() to speed up important cases of variable synthesis. This speeds up user-time ipcc_dly_T85 benchmark by ~300%. 2005-03-14 Charlie Zender <zender@uci.edu> * Modified big.cdl to facilitate benchmarks 2005-03-04 Harry Mangalam <hjm@tacgi.com> * RE-Fixed configure, configure.in to find test for and add ssl to LIBS 2005-03-02 Harry Mangalam <hjm@tacgi.com> * RE-Fixed configure, configure.in to find nc-dods lib AND also to treat UDUnits correctly if it's built and used from a local directory. 2005-02-24 Harry Mangalam <hjm@tacgi.com> * Fixed configure, configure.in to find nc-dods lib 2005-02-24 Charlie Zender <zender@uci.edu> * Long options --prn --print now point to -P instead of -H * Introduced -P as abbreviation for -C -H -M -m -u * Restored -H -m -M functionality to old behavior mostly * Change -u to toggle switch 2005-02-13 Harry Mangalam <hjm@tacgi.com> * used cz's ncks --compiler flag to grab compiler info * hires timer & udp reporting work on aix with perl >= 5.6.1 2005-02-13 Harry Mangalam <hjm@tacgi.com> * added hires timing to test script bld/nco_tst.pl * using udpserver2 collector modified from tacg project * added udp reporting " * more debugging for " * cleaned up and strictified code 2005-02-10 Harry Mangalam <hjm@tacgi.com> * added the getopt & usage section to test script bld/nco_tst.pl * added the logging and more debug 2005-02-13 Charlie Zender <zender@uci.edu> * Added --cmp, --compiler switch to ncks to report compiler name with new function nco_cmp_get() 2005-02-04 Charlie Zender <zender@uci.edu> * Polished up ncap intrinsic math function definition examples 2005-01-13 Charlie Zender <zender@uci.edu> * Added -DABORT_ON_ERROR to Makefile.am for libnco_c++ only 2005-01-12 Charlie Zender <zender@uci.edu> * ncks prints units, when present, by default (-u now turns this off) 2005-01-09 Charlie Zender <zender@uci.edu> * Make ncks -q switch a true quiet switch that stops all printing to screen. Make new -Q switch handle old -q job. Make ncks -R automatically invoke -q so that remote files may be downloaded without attempting to print them. * Remove -Wmissing-prototypes from g++ options * Fixed g++-3.4.2 prototype warnings with nco_c++ * Added return to about five void functions to squelch erroneous gcc-3.4.2 warnings (and hopefully allow enable-custom-* to work) 2004-11-30 Charlie Zender <zender@uci.edu> * Updated config.guess to timestamp 20040611 * Updated config.sub to timestamp 20040312 2004-11-05 Charlie Zender <zender@dust.ps.uci.edu> * Use S_IFLNK instead of S_ISLNK for NEC SX in nco_fl_utl.c * Change test -e to more restrictive test -f in configure.in. Makes sense and allows NEC SX to build. * Add docs on how to build NCO >= 3.0.0 with netCDF < 3.6 to homepage * Change so ncks in.nc out.nc does not print global metadata 2004-10-16 Charlie Zender <zender@uci.edu> * Add -Z, --64-bit-offset switch for FORCE_64BIT_FST mode * Prevent printing global metadata on ncks file copies 2004-10-13 Charlie Zender <zender@uci.edu> * Put in commented version of netCDF 64-bit file format support * Added 32-bit/64-bit distinction discussion to LFS support 2004-09-23 Charlie Zender <zender@uci.edu> * Remove extra carriage returns from ncks global metadata printing in nco_prn_att() * Declare nco_pck_map as shared in OpenMP loop 2004-09-12 Charlie Zender <zender@uci.edu> * ncks: Print data to screen when only input file is specified * Add NC_BYTE to types of variables with known ndrv in nco_var_pck() 2004-09-08 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_9_9 Changes since nco-2_9_8: 1. ncbo threading 2. ncpdq -P packing * Fix problems with building --enable-*-custom * Fully merge packing policy/map decisions into nco_var_lst_dvd() This should mean that fix/prc distinction is fully integrated 2004-09-07 Charlie Zender <zender@uci.edu> * Finish packing map documentation. Ready to release. * Cause ncpdq in.nc out.nc to default to pck_plc=all_new rather than aborting * Re-arrange debugging info verbosity levels to be sane * Fix erroneous triggering of PURE_MSS_VAL_FLD in nco_var_pck() 2004-09-06 Charlie Zender <zender@uci.edu> * Verified that all packing except NC_XXX->NC_CHAR works pretty well * Add new nco_pck_map types hgh_byt and flt_byt to nco_pck_plc_typ_get() * Showed packing three_dmn_var_dbl to NC_BYTE works better than packing to NC_CHAR * Verified packing/unpacking for: NC_INT->NC_SHORT, NC_DOUBLE->NC_INT, NC_DOUBLE->NC_BYTE, NC_INT->NC_BYTE, NC_SHORT->NC_BYTE, * Allow add_offset, scale_factor to be any type but NC_BYTE and NC_CHAR in nco_pck_dsk_inq(). This fixes NC_INT->NC_SHORT unpacking. * Learned that NC_INT->NC_SHORT does not work. Presumably all non-floating point packing is suspect until verified * Verified each pack map works without memory problems for all_new policy * Allow packing to NC_INT in nco_var_pck() * Successfully segregated var_fix from var_prc * Change nco_pck_typ to nco_pck_plc for "Packing Policy" * Incorporated packing policy decisions in nco_var_lst_dvd() to segregate var_fix from var_prc. Still not working but infrastructure in place. * Deprecated nco_is_packable() in favor of more flexible nco_pck_plc_typ_get() which accounts for policy desired by nco_pck_map * Eliminated all memory problems in ncpdq Verified no excess large blocks remain at program end Valgrind rules! * Update missing value in nco_var_upk_swp() * Handle all nco_pck_typ cases in nco_var_dfn() and end clause of ncpdq * Remove dangling pointer from ncpdq all_xst clause * Update ncpdq man page. Document -M most everywhere. 2004-09-05 Charlie Zender <zender@uci.edu> * Create nco_pck_map_get() to parse nco_pck_map arguments to ncpdq -M option * Introduce nco_pck_map to nco_typ_pck_get() Only default type nco_pck_map_hgh_sht currently works * Move high level packing routines from ncpdq.c to nco_pck.c * Finished initial ncpdq packing documentation * Verified unpacking complete missing values works, with both scalars and arrays * Fixed packing pure missing values fields in nco_var_pck() * Replaced duplicated noisy text branches in packing routines with ugly goto's and single blocks of code * Add ncpdq packing/unpacking test to nco_tst.pl * Fix unpacking variables with only add_offset in nco_pck_dsk_inq() 2004-09-04 Charlie Zender <zender@uci.edu> * Do not attempt unpacking unless variable already packed * Replace USE_EXISTING_PCK in nco_var_pck() with PCK_VAR_WITH_NEW_PCK_ATT * Implement synonyms ncpack, ncunpack, and option -U and xst_new * More robust handling of cases when only one packing attribute defined * Cast missing_value to double before double comparisons in nco_var_pck() 2004-09-03 Charlie Zender <zender@uci.edu> * NULLify dangling pointer to Fix double-free()'d value buffer problem 2004-09-02 Charlie Zender <zender@uci.edu> * Finished implementing packing types: all_new, all_xst, upk 2004-09-01 Charlie Zender <zender@uci.edu> * Worked on re-packing already packed variables. Almost working. 2004-08-25 Charlie Zender <zender@uci.edu> * Added nco.css contributed by Alper Ersoy of the TeXInfo project Does not quite work yet, but points the right direction for XML compatibility 2004-08-22 Charlie Zender <zender@uci.edu> * Include sys/types.h prior to regex.h on Mac OS X 2004-08-15 Charlie Zender <zender@uci.edu> * ncpdq all_new packing seems to work. Next need to get unpacking working. * Alter nco_var_dfn() to define dummy scale_factor, add_offset for ncpdq 2004-08-14 Charlie Zender <zender@uci.edu> * Cleanly segregate pack from re-order functionality in ncpdq Assume both are not performed in same ncpdq invocation * Pass nco_pck_typ from all operators to nco_var_lst_dvd() 2004-08-11 Charlie Zender <zender@uci.edu> * Create nco_pck_mtd() to off-load metadata manipulation 2004-08-10 Charlie Zender <zender@uci.edu> * Copy packing infrastructure from ncra to ncpdq * Finished thr_nbr documentation * Threaded main variable loop of ncbo 2004-08-08 Charlie Zender <zender@uci.edu> * Changed some formats from %e to %g in nco_tst.pl so that Solaris cc executables will pass tests 2004-08-07 Charlie Zender <zender@uci.edu> * Added first ncatted test to nco_tst.pl * Added lots more yummy ncpdq documentation * cvs tag -c nco-2_9_8 Changes since nco-2_9_7: 1. ncpdq, 2. ncwa --msk_sng consolidated mask switch, and 3. C99 * Fixed bug where nco_var_dmn_rdr_mtd() returned before reversal flag list created for 1-D arrays. This cause reversal of 1-D arrays to fail (do nothing). * Made valiant stab at ncpdq documentation. Ran out of steam. 2004-08-06 Charlie Zender <zender@uci.edu> * split ncpdq var list into var_fix and var_prc like in ncwa All tests appear to work * Fixed last known ncpdq problem caused by indexing var_prc_out list with var_out index * Added duplicate dimension flag has_dpl_dmn to var_sct * Handling situations where re-order list caused non-record variables to become record variables was completely broken. Inserted mucho logic which implements new policies: 1. First record variable change request is honored 2. All subsequent requests are ignored and produce warnings 2004-08-04 Charlie Zender <zender@uci.edu> * Fix ncpdq bug where record coordinate matching was done with strstr() instead of strcmp() * Fix ncpdq dimension reversal list processing * Make all error exits go through nco_err_exit() so that error exits uniformly call exit(EXIT_FAILURE) or abort() * Fixed diagnostic bug in nco_var_dfn() which would dump core when dbg_lvl=3 and prg=ncwa * Fix ncwa bug introduced in nco_var_dmn() last week. nco_tst.pl completes successfully again. * Improved/updated man_end.txt and catenated onto end of all man pages * Wrote ncap man page ncap.1 * Added ncpdq.1 man page View with nroff -man ~/nco/man/ncpdq.1 2004-08-02 Charlie Zender <zender@uci.edu> * Implemented dimension reversal with negative signs in ncpdq * Changed dimension re-order switch from -z to -a like ncwa * Fixed NCO-wide bug where I used strstr(sng1,sng2) as boolean instead of !strcmp(sng1,sng2) This bug occurred in: nco_var_cnf_dmn() three times nco_var_stretch() three times ncbo.c() once time When used as a boolean strstr(sng1,sng2) is approximately interchangeable with !strcmp(sng1,sng2). Exceptions occur where sng1 is a substring of sng2, in which case the dimensions would be different but strstr() would return a match. In all locations, the bug appears to have caused extra work, or to have incorrectly allowed the above exception where one dimension is a substring of another, but in no location does the bug appear to have caused wrong answers. 2004-07-29 Charlie Zender <zender@uci.edu> * Eliminate ostensibly wrong memmove() in ncwa/ncpdq avg/rdr list processing * Rewrote appropriate ncpdq routines to handles record dimension re-definitions correctly 2004-07-28 Charlie Zender <zender@uci.edu> * Broke nco_var_dmn_rdr() into two sequential functions nco_var_dmn_rdr_mtd() for metadata and nco_var_dmn_rdr_val() for values. Calling routine (ncpdq) now holds required intermediate information in dmn_idx_out_in. * Implement -DNCO_ABORT_ON_ERROR in nco_err_exit() so developers can perform easy tracebacks * Add AX_CFLAGS_GCC_OPTION(-D_BSD_SOURCE) so default build works * Added ncpdq to autoconf build machinery * Finish first theoretically working version of ncpdq. Of course, it crashes. 2004-07-27 Charlie Zender <zender@uci.edu> * Merge ncwa dmn_avg list handling into ncpdq for dmn_rdr * Removed un-used dimension nbr_dmn_avg == 0 branch from ncwa 2004-07-26 Charlie Zender <zender@uci.edu> * ncpdq now copies files perfectly, just like ncks No re-order routines have been turned on---that is next Tagged this ncpdq version (1.1) with ncpdq_perfect_do_nothing_template User ncpdq.c version 1.1 as template for future new operators * Require default clauses for switch(prg_id) * Created ncpdq.c from stripped version of ncwa.c 2004-07-25 Charlie Zender <zender@uci.edu> * Installed hooks to call nco_var_dmn_rdr() in ncks May need to create new operator to use this capability * Added skeleton nco_prs_rdr_lst() * First complete draft of nco_var_dmn_rdr() compiles warning-free 2004-07-21 Charlie Zender <zender@uci.edu> * Implement infrastructure for dimension re-ordering routine nco_var_dmn_rdr() 2004-07-19 Charlie Zender <zender@uci.edu> * Create nco_bnch.sh to benchmark big file operations * Implement preprocessor method in nco.h to disable restrict * big.nc "restrict" test with yields incorrect answers with AIX xlc * Implement restrict in nco_var_cnf_dmn() * Change default xlc from -qlanglvl=extended to -qlanglvl=extc99 to support C99 features like restrict. This does not seem to bother xlC, which appears to ignore the -qlanglvl=extc99 switch. * Implement restricted pointers throughout arithemetic routines * Introduced restrict type qualifier into pointer unions * cvs tag -c nco-2_9_7_pre_restrict Last tag prior to implementing C99 features * const'ify arithmetic counters used only for normalization * Standardized arithmetic prototypes in preparation for implementing restricted pointers 2004-07-08 Charlie Zender <zender@uci.edu> * nco.texi: Correct mix-up of -T and -t options in documentation 2004-07-06 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_9_7 Changes since nco-2_9_6: Read MFO input files from stdin, add more attribute functionality to nco_c++, turn threading on in ncra * Updated Makefile to use sophisticated make depends rules that handle plusses and slashes in filenames better * Added libtest, libtst targets to Makefile * Change from C++ to CXX in Makefile * Finished replacing %zu's with %lu's with casts to (unsigned long) * Provide definitions of fabsf(), sqrtf() in nco_var_rth.c from nco_flt_rth.h rather than from special #ifdefs 2004-07-04 Charlie Zender <zender@uci.edu> * Changed some %zu's back to %lu's with (unsigned long) casting Will I have to do this everywhere since %zu is non-ISO? 2004-07-03 Charlie Zender <zender@uci.edu> * Add undocumented -Y, --pseudonym, or --prg_nm switch to ncra to force it to behave as ncea or ncrcat 2004-07-02 Charlie Zender <zender@uci.edu> * Change default from OMP=N to OMP=Y in old Makefile 2004-07-01 Charlie Zender <zender@uci.edu> * Changed configure.eg to enable OpenMP on AIX. 'make' and 'make install' now work but 'make test' fails with zillions of linker errors: rtld: 0712-001 Symbol omp_get_num_threads was referenced from module /u/zender/nco/src/nco/.libs/libnco.so(), but a runtime definition of the symbol was not found. * AIX xlc does not activate OpenMP unless -qsmp=omp argument used * Fixed AIX xlc_r compiler pragma warnings by removing /usr/vacpp/include from CPPPATH in configure.eg. I think nco_c++ used to need this to find <valarray>, but that no longer seems to be the case. Simpler is better. * Improve LFS test in big.cdl. Created version that creates large files (> 4 GB) on small-RAM (< 4 GB) systems, and NCO seem OK to deal with resulting files. * Added $dbg_lvl variable to print nco_tst.pl tests during execution * Fixed nco_tst.pl test hanging while waiting for interactive response to overwrite query * Fix brokenness in nco_fl_utl:nco_fl_lst_mk() which triggered segfaults in non-multi-file-operators when no input filename was offerred, i.e., when there were no positional arguments * Fix brokenness in nco_tst.pl caused by renaming -o to -T * Changed old (only since 2.9.6) ncwa -t to ncwa -T so can use -t for threads consistently everywhere * Pass thr_nbr on command line * Added LFS stub to nco.texi, pass LFS info from configure.in * Added large file support LFS to custom Makefiles. Rorik added it to autoconf. 2004-06-30 Charlie Zender <zender@uci.edu> * Add nco_openmp_thread_number global attribute to threaded programs if HISTORY_APPEND is true * nco_openmp_ini() is now mandatory for any program using OpenMP * Fixed "variable "_iob" has undefined data scope" compiler warnings by passing stdout/stderr filehandles as local variables scoped shared in OpenMP parallel clauses 2004-06-29 Charlie Zender <zender@uci.edu> * Update OpenMP directives in ncra in preparation for adding robust OpenMP diagnostics 2004-06-28 Charlie Zender <zender@uci.edu> * Remove configure.in AC_CHECK_LIB check for cosf() in libC.a. NCO used to get the float functions from this library but it turns out that all of these system-supplied float functions are broken on AIX. NCO now uses it's own version of these functions. Fixes TODO ncap57. 2004-06-25 Charlie Zender <zender@uci.edu> * Added ln() as synonym for log() in ncap * Fix size_t format printing in memory allocation routines 2004-06-23 Charlie Zender <zender@uci.edu> * Never solved problem with reading o/a/e response after using stdin for filenames. 2004-06-19 Charlie Zender <zender@uci.edu> * Reworked stdin to avoid mixing string and character input. Interactive responses to "overwrite (o/a/e)?" questions must now be terminated with carriage return (and are case-insensitive) * Add nco_put_att<int,short,long> functions * Add nco_fl_info_get() to nco_fl_mk_lcl() * Add readfile, realpath, canonicalize_file_name to AC_CHECK_FUNCS 2004-06-18 Charlie Zender <zender@uci.edu> * Added new -H section to documentation * Change FORTRAN_STYLE to FORTRAN_IDX_CNV 2004-06-17 Charlie Zender <zender@uci.edu> * Added nco_fl_lst_att_cat() to add input file list global attribute * nco_fl_ls_mk() reads input file list from stdin when appropriate for multi-file operators. For some reason it no longer pauses for manual stdin, however. 2004-06-16 Charlie Zender <zender@uci.edu> * Fixed numerous errors in xargs example * Change nco_lib_tst.c to libnco_tst.c and add libnco_c++_tst.cc 2004-06-15 Charlie Zender <zender@uci.edu> * Check for strdup() and strcasecmp() with NCO_CHECK_FUNCS rather than AC_CHECK_FUNCS so they are protected by NEED_* rather than HAVE_* tokens. This improves portability of libnco by reducing number of tokens non-NCO programs need to define in order to link to libnco.a. This make nco_lib_tst simpler. * Add bld/nco_lib_tst.c to test libnco offline * cvs tag -c nco-2_9_6 Changes since nco-2_9_5: Add -o fl_out option, change ncwa -o meaning, AMD64 build tweaks 2004-06-14 Charlie Zender <zender@uci.edu> * Move old ncwa -o switch to -t, add synonyms --truth_condition, --msk_cmp_typ, delete old synonyms --op_cmp, --compare, --cmp. * Change warning size for large filenames from 80 to 255 characters * Modify nco_fl_lst_mk() so operators accept output filename specified with -o (or --fl_out or --output), or as last positional argument (traditional behavior). Finished TODO#58. 2004-06-13 Charlie Zender <zender@uci.edu> * Added demonstration script to handle large numbers of input files to nco.texi 2004-06-06 Charlie Zender <zender@uci.edu> * Implement -m64 option for ABI=64 on LINUXAMD64 2004-06-02 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_9_5 Changes since nco-2_9_4: ncflint interpolation is commutative, libnco_c++ *.hh headers installed * Install libnco_c++ *.hh headers in include directory. * Explicitly restrict license of libnco_c++.h to GPL version 2 * Change behavior of ncflint and nco_var_add_tll_ncflint() to return missing_value when either input is missing 2004-06-01 Charlie Zender <zender@uci.edu> * nco.texi: Documented funny ncflint behavior when one variable is missing * Implement nco_mss_val_cnf() in ncflint to handle corner cases where same variable has different missing values in each file 2004-05-22 Charlie Zender <zender@uci.edu> * Print shared/static configuration with -r * Support newer AC_INIT syntax * Default configure.in AIX GCC ABI is 64-bit with -maix64 Fixes TODO bld31. 2004-05-21 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_9_4 Changes since nco-2_9_3: ncecat works on pure scalar files * Allow AIX builds with GCC in bld/Makefile * Rearranged nco_tst.pl, added ncecat test, activated ncbo, ncea, ncflint tests. Fixes TODO nco332. 2004-05-20 Charlie Zender <zender@uci.edu> * Patched ncecat to address scalar file concatenation problem Fixes TODO nco331. * Set rec_dmn->is_rec_dmn=True; in ncecat.c This appears to have been a bug that never triggered any failures 2004-05-13 Charlie Zender <zender@uci.edu> * Move blurb from nmn_get to last info message 2004-05-05 Charlie Zender <zender@uci.edu> * Implement AMD64 pgcc/pgCC support in build mechanism 2004-05-04 Charlie Zender <zender@uci.edu> * Change cvs.nco.sourceforge.net to cvs.sf.net in accord with new SourceForge BIND 9-induced policy 2004-04-19 Charlie Zender <zender@uci.edu> * Build NCO on sand.ess.uci.edu, SuSE Linux 9.0 dual opteron with GCC 3.1.1 for x86-64 * Put CEWI's in nco_lmt.c to allow builds with GCC and --enable-*-custom when UDUnits not installed 2004-04-13 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_9_3 Changes since nco-2_9_2: ncra/ncea arithmetic works when missing_value equals zero or any intermediate sum * Closed bugs nco121 and nco325 * Segregate nco_var_add_tll() into nco_var_add_ncra() (which checks only current record against missing_value) and nco_var_add_ncflint() (which checks both arguments against missing_value). This fixes bug where record averages were zero when missing_value=0.0. * Switch argument order in nco_opr_drv() for clarity/consistency 2004-04-06 Charlie Zender <zender@uci.edu> * Updated News & Announcements section 2004-03-23 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_9_2 Changes since nco-2_9_1: Fix S/V, S%V, S^V, workaround broken UDUnits installations * Correctly implement non-commutative ncap S%V modulo * Correctly implement non-commutative ncap S^V empowerment * Document promotion for empowerment (sounds more political than it is) in nco.texi 2004-03-17 Charlie Zender <zender@uci.edu> * Change nco_lmt.h from HAVE_UDUNITS_H to ENABLE_UDUNITS to make bld/Makefile builds work again when udunits is not installed. 2004-03-16 Charlie Zender <zender@uci.edu> * Correctly implement non-commutative ncap S/V division 2004-03-12 Charlie Zender <zender@uci.edu> * Verify Solaris does autobuild correctly with up-to-date bison and when UDUnits workaround is installed. 2004-03-11 Charlie Zender <zender@uci.edu> * Re-do UDUnits logic in configure.in to ignore broken UDUnits installations and build successfully (with UDUnits, of course). * Print variable names when writes and reads fail in nco_c++ 2004-03-10 Charlie Zender <zender@uci.edu> * Handle corner case in nco_lmt.c where HAVE_UDUNITS_LIB is defined but HAVE_UDUNITS_H is not 2004-03-07 Charlie Zender <zender@uci.edu> * Tweaked DODS library logic in configure.in to support AIX Link to -lC on AIX instead of -lstdc++ as on Linux 2004-03-04 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_9_1 Changes since nco-2_9_0: prp_sei, fix packing bug with ncwa 2004-02-29 Charlie Zender <zender@uci.edu> * Described pending NCO proposals in NUG * Described how to donate in NCO User's Guide (NUG) * Added/explained OPeNDAP vs. DODS in documentation * Rorik modified nco_cnf_typ.c: nco_typ_cnv_rth() to fix bug with arithmetic on packed data in ncwa. Probably fixed same bug with other artithmetic operators (besides ncap) as well. 2004-02-25 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_9_0 Changes since nco-2_8_8: CF convention, ncap corner cases, Deactivate non-ANSI __GNUC__ extensions, make check nco_c++ * Implement CEWI workarounds so tst.cc builds warning-free and make check works again with --enable-*-custom * Deactivate __GNUC__ extensions for rank reduction in nco_var_avg.c Replace #ifndef __GNUC__ with #define FXM_NCO315 1; #ifdef FXM_NCO315 Will change back to original code only when stack memory problem is fully understood/solved/benchmarked to improve on ANSI code branch 2004-02-10 Charlie Zender <zender@uci.edu> * Make Convention CF-1.0 behave same as existing CSM convention 2004-02-08 Charlie Zender <zender@uci.edu> * Make TODO list easier to understand to facilitate contributions * Systematically "symmetrize" all ncap arithmetic operations so var OP var, var OP scv, and scv OP var are handled for all operations. * ncap handles var % scv, scv % var * ncap handles scv / var * Add nco_var_var_pwr() and nco_var_pwr() functions to allow empowerment of two variables = var_1^var_2 and scv^var * Change CSM to CCSM most everywhere 2004-01-30 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_8_8 Changes since nco-2_8_7: --enable-[optimize/debug]-custom works again * Remove -fshort-enums from GCC switches as per Rorik. It breaks ABI and caused mysterious failures because libnetcdf.a must be compiled with same size enums as NCO apparently. 2004-01-20 Charlie Zender <zender@uci.edu> * Implement DODS 3.4 switches as default in bld/Makefile 2004-01-19 Charlie Zender <zender@uci.edu> * Use valgrind 2.1 to identify and fix small leaks: Free prg_nm in nco_exit_gracefully() 2004-01-16 Charlie Zender <zender@uci.edu> * nco_netcdf.c: Do not initialize rcd unless not doing so might cause CEWI 2004-01-15 Charlie Zender <zender@uci.edu> * Change dods.sh from RC6 to 3.4 final, and pre-clean * Correct documentation of covariance procedure 2004-01-13 Charlie Zender <zender@uci.edu> * Change AIX xlC enable-optimized-debug from -O5 to -O3 so shared libraries work again * cvs tag -c nco-2_8_7 Changes since nco-2_8_6: Default GCC builds work again, ncap packing works again * Fix GCC compiles when enable-*-custom is not specified. This should fix reported Cygwin build problem. * nco_pck() uses tally-free arithmetic. Fixes ncap packing bug introduced in 2.8.4. 2004-01-12 Charlie Zender <zender@uci.edu> * Only manually prototype lexer function in ncap_lex.l if enable-*-custom build option specified with GCC. Fixes build problems with cygwin * Document make install-strip in doc/README as per Rorik * cvs tag -c nco-2_8_6 Changes since nco-2_8_5: Add -Werror to GCC compiles, fix AIX and SGI builds * Wrap new ncap_lex.l content in ifdef __GNUC__ since breaks on AIX, SGI. AIX and SGI now build again. 2004-01-11 Charlie Zender <zender@uci.edu> * NCO compiles warning free again so add -Werror to GCC custom flags * Add and elaborately comment on multiple yylex() prototypes required to build ncap warning-free with pedantic debugging flags. * Use correct token, NCO_MSA_H, instead of NCO_VAR_UTL_H, in nco_msa.h This fixes all remaining implicit function declaration warnings Until now, no routines in nco_msa.c had prototypes known outside of that file unit. Unlikely that this caused problems since no known casts were employed in prototypes. This demonstrates the usefulness of -Werror. Without it, this bug would still be there. 2004-01-10 Charlie Zender <zender@uci.edu> * Conjoin setting GCC flags in configure.in, avoid duplication * Systematize header file names * Improved dods.sh documentation 2004-01-09 Charlie Zender <zender@uci.edu> * Unconditionally include nco_lst_utl.h and nco_sng_utl.h in nco_att_utl.h (now that nco_prs_aed_lst() is there). * Moved system headers from nco_lmt.c to nco_lmt.h 2004-01-08 Charlie Zender <zender@uci.edu> * Give DODS example in -R documentation * Add -R to correct -R examples in documentation 2004-01-07 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_8_5 Changes since nco-2_8_4: DODS support, configure.in rewrite, ncap attribute assignment fix * Remove -Dinline= so gcc CFLAGS may be reused for g++ This fixes make install problem with --enable-optimize-custom 2004-01-06 Charlie Zender <zender@uci.edu> * Improved type conversion documentation 2004-01-04 Charlie Zender <zender@uci.edu> * Qualify GPL license as Version 2 everywhere * Prototype main.c functions prior, rather than post, definition in nco.h * prefix ncap global and file scope variables with ncap_ to reduce namespace pollution * De-linted nco_c++ library by adding rcd+=0 CEWI statements 2004-01-03 Charlie Zender <zender@uci.edu> * Fix bug where nco_put_varm() called nc_get_varm_float() instead of nc_put_varm_float() * Replace nco_netcdf.h prototypes with type-safe const'ified prototypes 2004-01-02 Charlie Zender <zender@uci.edu> * Use sqrtf() where appropriate * Preserve const'ness of string literal assignments and returns * Use trigraph return in qsort() comparison functions nco_cmp_*() to completely avoid arithmetic and overflows 2004-01-01 Charlie Zender <zender@uci.edu> * Improve qsort() comparison functions nco_cmp_*() based on responses to comp.lang.c thread 20040101: 1. Avoid overflow conditions by not performing subtraction 2. Preserve const-ness by using comparison rather than using casting and subtraction 2003-12-29 Charlie Zender <zender@uci.edu> * De-lint code with --enable-debug-custom Fixed sloppy casts, signed/unsigned comparisons, missing const's in nearly all source code. * make dst_cln cleans volatile autotools build files in root directory * Add top-level file descriptions to MANIFEST * Expand GNU triplet names to specific architectures tested (i.e., use more restrictive triplets) * Add correct compiler and OS criteria for custom compiler switches * Make --enable-debug-custom set --enable-debug-symbols * Replaced deprecated AC_HELP_STRING with AS_HELP_STRING 2003-12-28 Charlie Zender <zender@uci.edu> * Update copyright years to 2004, tweak GPL license message 2003-12-27 Charlie Zender <zender@uci.edu> * Perform extensive self-diagnostic check of compile-time pre-processor configuration settings in nco_ctl.c:nco_lbr_vrs_prn(). Check results displayed with -r, e.g., ncks -r 2003-12-26 Charlie Zender <zender@uci.edu> * Add new tokens to config.h: ENABLE_[OPTIMIZE,DEBUG,EXTREME] * Clean up and comment configure.in 2003-12-23 Charlie Zender <zender@uci.edu> * Modified Rorik's script and added to distribution as doc/dods.sh * Updated/improved DODS section in manual 2003-12-01 Charlie Zender <zender@uci.edu> * Document hyperslab bug in 2.7.3--2.8.3 on homepage * Document NCAR SCD TCG support on homepage 2003-11-26 Charlie Zender <zender@uci.edu> * Rorik fixed install to remove links before install. Fixes install on Solaris. 2003-11-25 Charlie Zender <zender@uci.edu> * Squelch "declared but never referenced" warnings on IRIX in nco_utl.cc * cvs tag -c nco-2_8_4 Changes since nco-2_8_3: Fix index-based hyperslab bug, reduce ncap memory usage to 2*sizeof(var) for binary operations * Sourceforge made debian/rules executable fixes TODO #280 * Rorik fixed index-based hyperslabbing bug introduced in 2.7.3 Fixes TODO #286, #289 * Specify success correctly for ncap T42 test 2003-11-24 Charlie Zender <zender@uci.edu> * Add ncks test 14 to check for hyperslabbing problems 2003-11-23 Charlie Zender <zender@uci.edu> * Change cerf to erfc in documentation. Dyslexia strikes again. * Add example documentation of intrinsic ncap math functions 2003-11-21 Charlie Zender <zender@uci.edu> * Henry reduced ncap memory usage in binary operations to 2*sizeof(var). Also removed superfluous (for now) tally allocation in ncap_var_init(). * Beefed up memory usage section in nco.texi 2003-11-20 Charlie Zender <zender@uci.edu> * Implement nco_malloc_err_hnt_prn() * Implemented nco_malloc_dbg() in ncap_utl * Created nco_malloc_dbg() for alternative to nco_malloc_flg(). This simplifies calling routines by moving most diagnostic handling to nco_malloc_dbg(). * Remove two more overly scary packing warnings 2003-11-19 Charlie Zender <zender@uci.edu> * Rorik replaced nco_tst.sh with nco_tst.pl, completing TODO #202 2003-11-16 Charlie Zender <zender@uci.edu> * Add LINUXAMD64 to support AMD Opteron * Update Sourceforge documentation 2003-11-14 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_8_3 Changes since nco-2_8_3: memory leaks, regressions fixed * Fixed ncwa, ncbo regressions in nco_tst.sh * Rorik found and I fixed little nco_var_lst() memory leak * Rorik and Henry found and fixed huge ncap memory leak 2003-11-11 Charlie Zender <zender@uci.edu> * Add nco_mmr_stt() hook call to all memory routines 2003-11-10 Charlie Zender <zender@uci.edu> * Add debian docs to make tags target * Remove qualifying warning about packed variables from nco_pck.c 2003-11-09 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_8_2 Changes since nco-2_8_1: missing_values handled better * Applied nco_mss_val_cnf() to ncbo to fix TODO #274 * Wrote nco_mss_val_cnf() to change missing_value of var2, if any, to missing_value of var2, if any, before arithmetic operation. 2003-11-05 Charlie Zender <zender@uci.edu> * Remove ncap_lex.c, ncap_yacc.[ch] from .tar.gz file created by nco_dst.pl 2003-11-04 Charlie Zender <zender@uci.edu> * Moved DODS problem note to homepage 2003-10-11 Charlie Zender <zender@uci.edu> * Add section on union of two files * Complete named HTML sections for each node in nco.texi 2003-09-16 Charlie Zender <zender@uci.edu> * Add URLs to named sections in HTML comments in nco.texi * Add verbose errror message and HINT for NC_ERANGE errors 2003-09-03 Charlie Zender <zender@uci.edu> * Makefile (ABI): Change default ABI to 64 2003-09-02 Charlie Zender <zender@uci.edu> * Change default HTML production from texi2html to makeinfo 2003-09-01 Charlie Zender <zender@uci.edu> * Clean up nco.texi * Expand regex documentation 2003-08-31 Charlie Zender <zender@uci.edu> * Correct some regex example errors in nco.texi 2003-08-29 Charlie Zender <zender@uci.edu> * nco.texi: Added #sections in English to nco.html for easy direct referencing of man pages 2003-08-28 Charlie Zender <zender@uci.edu> * Add section on ncbo portability 2003-08-27 Charlie Zender <zender@uci.edu> * Improve ncbo documentation * cvs tag -c nco-2_8_1 Changes since nco-2_8_0: variable wildcarding 2003-08-21 Charlie Zender <zender@uci.edu> * Quote backslash character with backslash in nco_var_lst.c * Add skeleton ncap.1 man page to satisfy autotools 'make install' * Bring bld/Makefile up-to-date with wildcarding, getopt changes 2003-08-20 Henry Butowsky <henryb@ntlworld> * Rolled out variable wildcarding to all operators. Configure script checks for presence of regular expressions lib regex.h 2003-08-16 Charlie Zender <zender@uci.edu> * Verified autotools builds fine again on SGI, AIX * Straighten out getopt_long() preprocessor logic to work with architectures with getopt.h but no getopt_long() (SGI, AIX) 2003-08-15 Charlie Zender <zender@uci.edu> * Rorik synchronized shared library numbering with NCO version number * Document wildcarding in nco.texi 2003-08-14 Henry Butowsky <henryb@ntlworld.com> * Added wildcarding of variables to ncks 2003-08-13 Charlie Zender <zender@uci.edu> * Make nco_malloc_flg() return NULL when malloc() returns ENOMEM * cvs tag -c nco-2_8_0 Changes since nco-2_7_4: ncbo, Debian integration * Add Debian package uploads to nco_dst.pl * Add Debian packages homepage * Add developer autoconf-automake-configure-make example to configure.eg 2003-08-03 Charlie Zender <zender@uci.edu> * Finish first cut at ncbo documentation in nco.texi 2003-08-02 Charlie Zender <zender@uci.edu> * Rename val_conform_type() to val_cnf_typ() * All regressions due to introduction of ncbnr appear to be fixed * Correct nco_tst.sh procedure for ncap * Restore nco_op_avg inadvertently removed from nco_op_typ_get() 2003-08-01 Charlie Zender <zender@uci.edu> * Fix ncbnr case statement and verifiy ncbnr works for +-*/ 2003-07-29 Charlie Zender <zender@uci.edu> * Verify ncdiff and ncbnr -y sbt work as expected but problems exist with add, mult, and, presumably, divide. * Make ncdiff symbolic link to ncbnr for backward compatibility 2003-07-27 Charlie Zender <zender@uci.edu> * Create ncbnr operator * Rename nco_var_add_no_tally() to nco_var_add() * Rename nco_var_add() to nco_var_add_tll() 2003-07-20 Charlie Zender <zender@uci.edu> * Changed math table printing from -r to -f in ncap so that -r is consistent among all operators. Documented this in nco.texi. 2003-07-06 Charlie Zender <zender@uci.edu> * Removed FTP requirement from make tst. Create nco_tst.sh creates foo.nc from scratch without needing to FTP nco_tst.nc. Test involving FTP is now called make tst_ftp. 2003-07-04 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_7_4 Changes since nco-2_7_3: Better AIX autotools support, UDUnits fix 2003-07-01 Charlie Zender <zender@uci.edu> * Add --Mtd switch for global metadata, update nco.texi accordingly 2003-06-24 Charlie Zender <zender@uci.edu> * Added PayPal donation button, updated homepage 2003-06-23 Charlie Zender <zender@uci.edu> * Rorik fixed UDUnits handling of hyperslab specifications in floating point format, e.g., 1.23e-7. 2003-06-15 Charlie Zender <zender@uci.edu> * Update copyright headers * Change GPL version 2 or later license to version 2 only. * Clean up some MSA namespace * Document supported non-ANSI functions (e.g., acosh(), trunc()) in nco.texi * Changed --nocoords to --no-coord or --no-crd. Decided all disabling actions will have same keyword as enabling action except switch will start with --no-XXX 2003-05-30 Charlie Zender <zender@uci.edu> * Add CPP_TKN_OS += -I/usr/vacpp/include for nco_c++ on AIX * AIX autotools modifications by Jim * Jim Edwards completed Autoconf TODO 17. Fix CXX valarray to work on AIX 2003-05-21 Charlie Zender <zender@uci.edu> * Replaced malloc() with nco_malloc_flg() in about 10 locations where malloc() ENOMEM errors are handled directly by the calling code for additional diagnostics. 2003-05-06 Charlie Zender <zender@uci.edu> * Move RPM-wishlist off high-priority list, add restrictive MSA parsing to list as per Shultz request 2003-04-06 Charlie Zender <zender@uci.edu> * Added numerous long option synonyms in CSZ abbreviation format documented in abb.sty 2003-03-30 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_7_3 Changes since nco-2_7_2: UDUnits support for "units" attributes * Clean up nco_lmt * Documented UDUnits feature in User's guide 2003-03-27 Charlie Zender <zender@uci.edu> * Add UDUNITS_LIB, UDUNITS_INC CLI args to bld/Makefile 2003-03-26 Charlie Zender <zender@uci.edu> * Remove TODO 95. Satisfactorily addressed by Rorik's UDUnits work. 2003-03-25 Charlie Zender <zender@uci.edu> * Made --op_rlt, --cmp, --op_cmp, --compare synonymous with -o for ncwa * Re-arrange output from --usage, --help options 2003-03-24 Charlie Zender <zender@uci.edu> * Put UDUnits stuff on homepage * Added bld/Makefile support for UDUnits when UDUNITS=Y * Rorik committed UDUnits support for time coordinate limits in hyperslabs 2003-03-19 Charlie Zender <zender@uci.edu> * Remove TODOs 138, 139, 140, 141, 142---all solved or made obsolete by Henry's MSA printing algorithm. 2003-03-18 Charlie Zender <zender@uci.edu> * General honing of nco.texi * Implemented TeX and TeXInfo macros to keep math variables consistent with LaTeX sources. Mostly successful, except Info does not like macros inside or macros, e.g., @var{@wndznl{}} * cvs tag -c nco-2_7_2 Changes since nco-2_7_1: disk-based, MSA printing indices work 2003-03-17 Charlie Zender <zender@uci.edu> * Replaced examples in nco.texi with output from MSA printing 2003-03-09 Charlie Zender <zender@uci.edu> * Modifying debian/* files to build with 2.7.x as zender 2003-03-07 Charlie Zender <zender@uci.edu> * Remove -ansi flag from gcc builds because it is (apparently) equivalent to -std=c89 and thus conflicts with -std=c99. 2003-03-02 Charlie Zender <zender@uci.edu> * Began Debian-ization of upstream source by populating debian directory with nco-2.2.0 Debian package files by Brian Mays NB: Files in debian/ are for 2.2.0 (i.e., way out of date) We are trying to Debian-ize upstream source for NCO ~2.8.0 Expect some hiccups when that finally happens * Insert LINUXARM in Linux portion of bld/Makefile * Synchronize Debian nco/doc/man with CVS nco/man * Add Debian files by Brian Mays to CVS repository: debian: changelog control convert copyright postinst prerm rules bld: linux1 linux2 (appear obsolete and may soon be removed) 2003-02-25 Charlie Zender <zender@uci.edu> * Builds cleanly on LINUX, IRIX, AIX * Remove unused variables from ncap.[cl], nco_msa.c * Clean up nco_att_cpy() 2003-02-05 Charlie Zender <zender@uci.edu> * Change from WARNING to level 1 INFO when wrapped limits processed in nco_lmt_evl() 2003-01-29 Charlie Zender <zender@uci.edu> * Documented procedure for computing covariance of two variables using http://jisao.washington.edu/data/nco/ as template 2003-01-20 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_7_1 Changes since nco-2_7_0: long options on all platforms, packing fix for ncap * Initiate HPPA support in bld/Makefile 2003-01-19 Charlie Zender <zender@uci.edu> * Fix inconsistent storing of fixed variables that are packed in ncap Packed values were being saved in unpacked type without packing attributes 2003-01-18 Charlie Zender <zender@uci.edu> * Clean up some ncap_utl prototypes 2003-01-14 Charlie Zender <zender@uci.edu> * Updated nco_c++ build instructions and documentation to refer to Makefile.old * Changed copyright end year to 2003 2003-01-08 Charlie Zender <zender@uci.edu> * Rorik added conditional compilation of Sittler's getopt_long() where necessary, removed *_GETOPT_LONG tokens from all source code. 2003-01-03 Charlie Zender <zender@uci.edu> * Successfully built NCO 2.7.0 on Redhat 8.0 at HP Testdrive site spe141.testdrive.compaq.com. The HP Testdrive site has virtually every imaginable combination of hardware and OS, so now NCO may be tested on all of these machines. My first test with a new architecture, HP-UX (HPPA), failed because of lack of C++ compiler. Thus Testdrive computers do not appear to be fully loaded with reliable compilers. 2003-01-02 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_7_0 Changes since nco-2_6_6: long options and multislabbing 2002-12-29 Charlie Zender <zender@uci.edu> * Add rudimentary tests of multislabbing to nco_tst.sh * Switch from ~ to @w{} for unbreakable spaces in nco.texi 2002-12-28 Charlie Zender <zender@uci.edu> * Document multislabbing in user's guide * Document long options in user's guide * Implement more long options in TLA format * Henry changed arithmetic in nco_msa to char * arithmetic to remove warnings * Clean up ncap_utl.c some 2002-12-27 Charlie Zender <zender@uci.edu> * Clean up nco_msa.[ch]. Still need to fix void * arithmetic 2002-12-19 Charlie Zender <zender@uci.edu> * Moved multi-slabbing algorithm to nco_msa.[ch] 2002-12-19 Henry Butowsky <henryb@ntlworld.com> * First version of multi-hyperslabbing along a single dimension for ncks 2002-12-14 Charlie Zender <zender@uci.edu> * Took advantage of NEED_FNC to remove MACOSX kludge in nco_rth_flt.h * Merged Rorik's long options mods 2002-11-04 Charlie Zender <zender@uci.edu> * Restore ./configure file from 2.6.5 and re-tag as 2.6.6 2002-10-25 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_6_6 Changes since nco-2_6_5: ncrename fixes, ncap type conversion * Use '.' consistently in .var@.att renaming * Changed variable-attribute delimiter in ncrename from ":" to "@" to be consistent with ncap * Fixed ncrename to not fail when . specified and variable does not contain attribute 2002-09-24 Henry Butowsky <henryb@ntlworld.com> * Added type conversion functions for attributes and variables byte, char, short, int, float, and double 2002-09-22 Charlie Zender <zender@uci.edu> * info-friendly changes to nco.texi 2002-09-20 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_6_5 Changes since nco-2_6_4: CRAY and SUN builds * Fix SunOS builds in bld/Makefile 2002-09-19 Charlie Zender <zender@uci.edu> * Patches from Len Makin for CRAY 1. Cray patch for ncap symbol table 2. Makefile fixes for Cray 3. nco_tst.sh tweaks Applied the Makefile flags to configure.in as well 2002-09-17 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_6_4 Changes since nco-2_6_3: MACOSX builds, SGI, NEC tweaks * Match SX-* with NECSX architecture in pvmgetarch * Correct some dependencies for string compatibility functions * Adapt for missing <cstdlib> with SGI CC 2002-09-14 Charlie Zender <zender@uci.edu> * Added esoteric math functions to MACOSX using float<-->double coercion method * Implement compatibility strdup() and strcasecmp() Comeau should now work but have not tested it * cvs tag -c nco-2_6_3 Changes since nco-2_6_2: Numerous build improvments and removal of PVM_ARCH ifdefs MACOSX builds with bld/Makefile, documentation fixes * MACOSX builds appear to work * Add ranlib to bld/Makefile libnco.a target for MACOSX builds * Get rid of inline qualifier on nco_rth_flt function definitions * Move function definitions from nco_rth_flt.h to nco_rth_flt.c * Verified that bison 1.29 on MACOSX screws up ncap_yacc.y translation. Installed bison 1.35 on compile farm MACOSX system, and builds work fine. * Remove ncap_utl.o from libnco.a * #include <math.h> in nco_rth_flt.h * Do not look for <getopt.h> in MACOSX 2002-09-09 Charlie Zender <zender@uci.edu> * Added preliminary MACOSX support to bld/Makefile Everything appears to work but ncap, which has numerous problems on MACOSX because of bison, float functions, and ranlib. * Code compiles with gcc -ansi -pedantic -D_BSD_SOURCE Switches -std=c99 and -std=gnu99 both work too -D_BSD_SOURCE tells GCC nameser.h is ANSI-compliant * Moved float math function definitions to nco_rth_flt.h, created declarations for MACOSX in nco_rth_flt.c 2002-09-08 Charlie Zender <zender@uci.edu> * Implemented pre-processor macros CST_X_PTR_CST_PTR_CST_Y, X_CST_PTR_CST_PTR_Y to be as const-correct as possible for the language the source code is treated as. If treated as C++, full const-correct prototypes are implemented, else if treated as C, a const is dropped here or there to make things compile warning-free. Fixes TODO #213. 2002-09-06 Charlie Zender <zender@uci.edu> * Clean up ncap section of nco.texi, polish documentation builds, add XML, txt formats from makeinfo * Fixed nco.texi problems, generating docs at compile farm 2002-09-03 unknown <zender@uci.edu> * Typecast return value of strdup() to char * 2002-09-02 Charlie Zender <zender@uci.edu> * #include <config.h> in all headers which reference strdup() * Crude but workable HAVE_STRCASECMP support, include config.h headers to allow future HAVE_STRDUP handling * Implement some Comeau support, but presence of non-ANSI functions strdup(), strcasecmp() makes full Comeau support unlikely anytime soon 2002-08-27 Charlie Zender <zender@uci.edu> * Tweak CC list in configure.in, remove c89, xlc_r * cvs tag -c nco-2_6_2 Changes since nco-2_6_1: Improvements to autotools builds Initial ncap handling of user-defined dimensions * Replace PVM_ARCH token with HAVE_GETOPT_H for getopt.h handling * Remove Fortran files nco_cal_utl.F, nco_fortran.F, nco_fortran.h since it is unlikely anyone will step forward to maintain fortran support. Of course these can be resurrected if necessary. * Add instructions for using compile farm, modify AIX, MACOSX builds 2002-08-22 Charlie Zender <zender@uci.edu> * Define var_in_typ before using in nco_cnv_mss_val_typ() * Began using HAVE_CONFIG_H, config.h in source files (nco_scm.[ch],tst.cc) 2002-08-21 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_6_1 Changes since nco-2_6_0: Unpacking is now supported for all arithmetic operators * Improved regression testing in configure.eg * Eliminate unnecessary type conversion to typ_dsk in ncra, ncea, ncwa This could reduce precision of answers Thanks to Martin Dix for pointing this out 2002-08-20 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_6_0 Changes since nco-2_5_6: Unpacking support is now beta for all arithmetic operators * Print attributes with same formatting as variables * Added hooks for packing support in arithmetic operators: Code was changed in locations marked with "pck_dbg" 1. Change type of output in nco_def_var() from var->type to var->typ_upk when is_rth_opr() is true 2. Unpack variable in nco_var_get() when is_rth_opr() is true 3. nco_att_cpy() does not copy attributes "scale_factor", "add_offset" for unpacked output variables 4. nco_att_cpy() ensures "missing_value" is converted to unpacked type before copying 5. nco_var_get() refreshes var->pck_ram on each read 6. Use nco_cnv_mss_val_typ() to keep type of missing_value in sync Unpacking appears to be working for all arithmetic operators! 2002-08-19 Charlie Zender <zender@uci.edu> * Added .cvsignore to dirs affected by autoconf * Added autoconf-specific list to doc/TODO * Added configure.eg to keep track of autoconf problems * Changed bld/Makefile to work with ncap_lex.l and ncap_yacc.y Removed ncap.l and ncap.y. 2002-08-18 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_5_6 Changes since nco-2_5_5: fixed processing of missing_values for integer variables in ncra, initial auto build tools * Merged Rorik's autobuild infrastructure. New (autobuild) and old (bld/Makefile) systems both appear to work but there may be some interference on ncap since ncap.l had to be renamed ncap_lex.c and ncap.y is ncap_yacc.c. * Altered pck_dsk_inq() to set var->typ_upk=var->type by default so that nco_cnv_mss_val_typ_upk() has something to work with * Fixed problem in nco_cnf_typ.c which caused failure for ncra when variable had missing_value attribute and needed to be promoted before arithmetic (e.g., int to float). Freeing variable each record but leaving mss_val the promoted type resulted in mss_val being overwritten (usually to value of 0.0) in all records but first for each file. Problem was ncra-specific and is described in more detail in header of nco_cnv_mss_val_typ_upk() routine. Better fix would be to carry type of missing_value as separate element of var structure. 2002-08-14 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_5_5 Changes since nco-2_5_4: Global "history" attribute fix * Fix bug in nco_hst_att_cat() introduced in 2.4.2 which left att_nm undefined when global history attribute did not exist 2002-08-13 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_5_4 Changes since nco-2_5_3: NECSX support * Extensive support for NECSX architecture contributed by Len Makin <len at hpc.csiro.au> 2002-08-10 Charlie Zender <zender@uci.edu> * Make Rorik Peterson a developer. Add sourceforge logo to homepage. 2002-07-28 <zender@uci.edu> * Updated to cygwin-gcc-3.1.1-4 and build problems appear to have gone away, but syslimits.h is still not found so GSL cannot build 2002-07-08 Charlie Zender <zender@uci.edu> * Correct ncap symbol table to build on SGI6, SGI64 platforms * Add ncap to nco_tst.sh test suite 2002-07-08 <zender@LANINA> * All operators build with cygwin gcc-3.1.1 as long as -I/usr/include is given first (fixes weird syslimits.h problem), non-kludge fix may require updated gcc and mingw packages. * Modify Makefile for cygwin-1.3.x compatibility. Everything appears to build except ncap.l/ncap_lex.c which needs syslimits.h which current cygwin gcc-3.1.1 does not appear to support 2002-07-03 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_5_2 * conditionally added acosh, asinh, atanh, cosh, sinh, tanh to ncap * unconditionally added floor(), ceil() to ncap * Print double precision numbers with 12 digits of precision * conditionally added erf(), erfc() to ncap * Document ncks -b, -B switches in manual * Fix size of extracted dimension list for wgt and var calls to nco_var_fll(). Print error and hint when encountering TODO #111 problems rather than dumping core. 2002-07-01 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_5_1 * Modified ncatted to delete all attributes for specified variable when no specific attribute name is given for that variable. 2002-06-16 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_5_0 * Added man pages for NCO operators to man directory. Source of man pages is Debian distribution by Brian Mays, which automatically generates man pages from texinfo source. Thus man pages are moderately out of date, currently for version 2.2.0. 2002-06-15 Charlie Zender <zender@uci.edu> * Make namespace safe by prefixing all functions with nco_ or ncap_ 2002-06-09 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_4_6 * Changes since nco-2_4_5: Sorting routines * Replace indexx* routines with system qsort()-based routines 2002-06-08 Charlie Zender <zender@uci.edu> * Lexer/parser changes to implement conditional, clean up rx's 2002-06-07 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_4_5 * Changes since nco-2_4_4: Arithmetic routines use const. More warning messages during compilation due to size_t printf * Fixed bug where mss_val_sht was dereferenced into long in one arithmetic routine. * Made as many parts of arithmetic routines const as possible and implemented const instantiaion outside of loops to reduce dereferencing wherever possible. If NCO suddenly seems faster, this is why. 2002-06-06 Charlie Zender <zender@uci.edu> * Noticed that GCC gives warnings when printf'ing size_t with %lu format flag since type of size_t is non-portable. GNU offers non-portable %z workaround. Recommended solution is to typecast all size_t's to unsigned longs before printf'ing. * Worked on 64-bit clean issues identified by AIX -qwarn64 flag The only possible non-functioning code found was date conversion code which could break on YYYYMMDD dates. Converted int's to longs where necessary to solve this. Used size_t instead of int or long where possible. Used ptrdiff_t instead of int where warranted. Most of remaining warnings could be solved by using size_t instead of int or long for counters, but mismatches are unavoidable at some point since type nco_long variables (i.e., netCDF data) needs to interact with size_t variables on occasion. * cvs tag -c nco-2_4_4 * Changes since nco-2_4_3: Build system 2002-06-05 Charlie Zender <zender@uci.edu> * Allow 64 bit mode ABI (addresses are all 64 bits) on AIX when compiled with ABI=64. This should allow NCO to work on files larger than 2--4 GB The default for SGI was already 64 bits. 2002-06-04 Charlie Zender <zender@uci.edu> * Make flex/bison default on AIX 2002-05-31 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_4_3 Changes since nco-2_4_2: History attribute handling * Added target non_ncap to Makefile in attempt to prevent NCAR AIX breakage * Fixed hst_att_cat() to handle non-standard "History" attribute same as standard "history" attribute 2002-05-23 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_4_2 Changes since nco-2_4_1: #include file mechanism works, AT&T lex-compatibility broken by using flex <EOF> rules * Documented ncap file insertion * Fixed #including file mechanism, generalized line counting and file name storage 2002-05-22 Charlie Zender <zender@uci.edu> * Realized that AIX lex does not support <EOF> like flex * Ported to NPACI environment 2002-05-20 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_4_1 Changes since nco-2_4_0: minor bugfix which should fix use of operators in scripts where return code is checked 2002-05-19 Charlie Zender <zender@uci.edu> * More ncap work to prepare for dimension list structures which are going to be implemented correctly by tokenizing each element of dimension list and assembling full list in parser and letting scanner pass in each token. 2002-05-18 Charlie Zender <zender@uci.edu> * Added nco_op_typ element to prs_sct for handling dimension reduction functions 2002-05-15 Charlie Zender <zender@uci.edu> * Fixed bug where nco_exit always returned EXIT_FAILURE 2002-05-14 Charlie Zender <zender@uci.edu> * Added skeleton code to read #include'd files. Not yet working. 2002-05-13 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_4_0 Changes since nco-2_3_1: Builds out of box on all Unices * Surround fabsf and fmodf prototypes with ifndef __GNUG__ since g++ linking fails when prototypes explicitly specified in header _and_ in function. This is a kludge to a kludge to enable full g++ compilation. * Version now builds on AIX (even ncap.y with pure_parser) as long as cc compiler, not xlc or xlC, is used. Prototype warnings similar to gcc prototype warnings can be ignored. All major platforms are once again working out of the box. 2002-05-11 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_3_1 Changes since nco-2_3_0: New design, same functionality Appears to work on Linux, IRIX but compile problems on AIX * gcc compiles with spurious errors but links and tests fine * g++ compiles fine but unable to link with fabsf fmodf * Code builds and test correctly when compiled with icc * New code compiles completely warning-free with icc 6.0 2002-05-08 Charlie Zender <zender@uci.edu> * Added ability to write unformatted binary output to ncks using -B switch. Seems to work 2002-05-07 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_3_0 Changes since nco-2_2_4: Completely unstable version with new modular design. Library does build cleanly. * Redesigned libnco.a as modular library Split into ~20 smaller source files each with private header Used const wherever possible 2002-04-27 Charlie Zender <zender@uci.edu> * Finished name cleanup in ncap. Still need to break large files into smaller granules and const'ify prototypes * cvs tag -c nco-2_2_4 Changes since nco-2_2_3: Packing and unpacking functions now both work in ncap * Reducing ndrv number of discretely representable values by 1 appears to fix the rounding problems. Will probably have to reduce it by one more to make room for a missing value. 2002-04-26 Charlie Zender <zender@uci.edu> * Packing appears to be working, but not recursively, and there are still issues of speed and rounding problems * Automatically call var_upk() in var_get() in ncap only * Improved pck/upk values in in.cdl * Make var_pck() alter typ_pck, typ_upk elements when finished packing 2002-04-24 Charlie Zender <zender@uci.edu> * Put warnings on some arbitrary limits in ncap * Modify ncap and Makefile to link to -lC AIX C++ library to access float intrinsics cosf()... AIX does not support gammaf() (nor does SGI). 2002-04-23 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_2_3 Changes since nco-2_2_2: Support unpacking variables in ncap * Altered ncap algorithm so that now binary var/att expressions take highest precision type rather than always using var type. This allows unpacking algorithm to work straightforwarly using scale_factor and add_offset convention. 2002-04-18 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_2_2 Changes since nco-2_2_1: fix rare ncwa weight bug * make tst gets test file from dust.ps.uci.edu rather than CGD * Avoid wgt portion of ncwa when DO_CONFORM_WGT = False This section was superfluous in this situation anyway. Avoiding it by testing for DO_CONFORM_WGT appears to fix some hard to identify bug when lon was being averaged with gw as weight. The two did not conform, but a problem occured anyway. This appears to cure the three mysterious ncwa crashes present in the TODO list so mark those as fixed too. 2002-04-02 Charlie Zender <zender@uci.edu> * Switch back to using pure_parser to prevent Linux core dumps on 2002-03-30 Charlie Zender <zender@uci.edu> * Prototype fabsf() where used to help Sun compilation 2002-03-27 Charlie Zender <zender@uci.edu> * Quiet output by only printing CONVENTION when dbg > 0 2002-03-21 Charlie Zender <zender@uci.edu> * Remove %pure_parser from ncap.y to preserve yacc compatibility 2002-02-27 Charlie Zender <zender@uci.edu> * Described LHS casting in manual 2002-02-24 Charlie Zender <zender@uci.edu> * ncap variable division had numerator and denominator swapped! * Removed ncap_lex.c ncap.tab.c ncap.tab.h because they are not portable across platforms so it is best to let these be generated on user's machine until portable x-platform versions can be generated. Immediate problem is that IRIX and Linux versions are interfering with eachother. * ncap -r does not require argument 2002-02-18 Charlie Zender <zender@uci.edu> * Began documenting ncap in nco.texi 2002-02-11 Charlie Zender <zender@uci.edu> * Remove ncap dependency on getopt.o, getopt1.o since getopt_long() is not currently used (but using it is on the TODO list). Re-implementing getopt_long() will be done eventually, but now just make ncap easier to build to facilitate more beta testing. 2002-02-04 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_2_0: Changes since nco-2_1_3: Henry made subscripts lexer tokens and improved dimension list handling so LHS casting works regardless of precedence or presence of LHS expressions. * Small ANSI fixes in ncap.c for AIX * Replace math float prototypes required for AIX, Solaris builds These are not required on Linux or IRIX, which keep them in math.h * Documentation changes 2002-02-02 Charlie Zender <zender@uci.edu> * Quieted output by removing verbose printing from debug level = 0 Execution is now silent except for WARNINGS and INFOs. 2002-01-30 Charlie Zender <zender@uci.edu> * Added some Compile Farm procedures to nco_src_frg.txt * Add FREEBSD support to Makefile, and allow environment to override CC and C++ compiler variables 2002-01-29 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_1_3 Changes since nco-2_1_2: LHS casting really works as long as dimensions are already in output file. Replaced var_add() in ncap, no known parser bugs, just missing features. * Created var_add_no_tally() by copying var_subtract() and changing minuses to pluses. This routine does not use tally, and does handle missing values appropriately. Changed ncap_var_var_add() to use var_add_no_tally() rather than var_add(). This fixed segfaults with prs_mdp definition, which now works as expected. * ncap variable addition scripts break in var_add() Possibly because not all ncap routines malloc() tally array Possibly because missing values handling is not failsafe in var_add() (although I believe it is fine for other NCO operators) * Allocate tally using sizeof(long) rather than nco_typ_lng(NC_INT) 2002-01-28 Charlie Zender <zender@uci.edu> * Move LHS_cst cleanup to statement_list rule so cleanup does not occur before last expression in statement is evaluated * Added ncap_var_stretch() to out_var_exp = att_exp action * Spread out debug levels to create more uniform distribution of diagnostics as dbg_lvl increases * LHS casting is semi-working, semi-broken. Seems to work when all required dimensions are already in input file and RHS is variable, not attribute. Thus a3[lat,lon,lev] = one works but a3[lat,lon,lev]=1.0 does not. Still thinking about how to stretch attributes. 2002-01-27 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_1_2 Changes since nco-2_1_1: ncap warning cleanups, icc conformance, plug most non-ncap leaks detected by insure++ * Move yylex() so only called once in ncap_initial_scan() 2002-01-26 Charlie Zender <zender@uci.edu> * Use <limits.h> to get INT_MIN, INT_MAX * Changed from gcc to icc on Intel, seems to work fine * Cast char to unsigned char and visa versa where necessary to prevent icc warnings. This solved all icc warnings on ncap, and solved all IRIX cc warnings for ncap too. Since unsigned char pointer and char pointer are same size, there should be no problem casting between them. However, casting between unsigned char value and char value could change answers. String operations should be checked to ensure they still work. 2002-01-23 Charlie Zender <zender@uci.edu> * Fix read overflow in ncks.c:prn_var_val_lmt() where non-NUL-terminated character array was being printed with %s format 2002-01-22 Charlie Zender <zender@uci.edu> * Fix non-NUL-terminated strings in hst_att_cat() * Added nco/doc/purify.txt to track memory problems reported by purify 2002-01-21 Charlie Zender <zender@uci.edu> * Remove templates for math functions from ncap.c * Implement CCOMMENT start state to recognize C comments /* */ * Explicitly declare yy_scan_string() to remove compiler warnings * Implement/use nco_set_fill() wrapper Remove architecture-dependent wrapper on usage Pass pointer to valid fll_md_old since NULL is not safe because nc_set_fill() may try to write to it 2002-01-17 Charlie Zender <zender@uci.edu> * Initialize RHS to False, add global LHS_cast 2002-01-16 Charlie Zender <zender@uci.edu> * Enable assertion macros in ncap * Changed Makefile default to OMP=N until further testing * Began implementation of lexer for subscripted variables with x[dmn1,dmn2,...dmnN] syntax. Currently creates list of dimension structures but does nothing with it. 2002-01-13 Charlie Zender <zender@uci.edu> * Verified ncap script gives same answers as odxc.ncl script * var_free() superceded variable in ncap_var_conform_dim() causes core dump (why?), but not free'ing must cause memory leak * Realized var_conform_dim() has bug of omission: does not abort when variables do not conform because dimension list of one is subset of other but ORDER of dimensions differs, e.g., a(lat,lev,lon) !~ b(lon,lev). NCO probably returns incorrect answers silently in this case! 2002-01-12 Charlie Zender <zender@uci.edu> * Added variable/variable division (which had been overlooked) * Created ncap_var_stretch() which generalizes var_conform_dim() by including convolution of variables. ncap_var_stretch() appears to work as drop in replacement for var_conform_dim(). Convolution is just a stub which does not do anything yet. Realized that expansion might work by creating arbitrary variable of convolution size and then separately calling var_conform_dim() with that as var and both vars as wgt. * First build of ncap on LINUXALPHA architecture 2001-12-31 Charlie Zender <zender@uci.edu> * More ncap.* cleanup. Alter some variable names to conform to rest of NCO. 2001-12-29 Charlie Zender <zender@uci.edu> * Succeeded in building ncap on 64 bit machines: SGI, Alpha, Sun All of these machines have bison installed Some of these builds require building getopt.o separately Build fails on IBM because yacc does not understand pure_parser 2001-12-28 Charlie Zender <zender@uci.edu> * More CEWI initializations * Changed copyright to 2002 * Remove C++ comment characters that crept into various routines Using // comments causes default to compilation to break on many architectures (Linux being an exception) * cvs tag -c nco-2_1_1 Changes since nco-2_1_0: ncap uses @ for attributes Mixed rank handling improved but still broken * Fix '@' handling for global attributes * Make handling of attribute indicator @ consistent in warnings * ncap.l: cast void to ptr_aed->type not undefined type * Begin clean up of ncap code, eliminate compiler warnings 2001-12-11 Henry Butowsky <henryb@ntlworld.com> * Changed ncap.l so attributes now use the @ symbol rather than : i.e var_nm@att_nm rather than var_nm:att_nm * Changed ncap.y So that 1-D variables can be saved in an attribute * Attempted to fix the rank problem. Added ncap_var_conform_dim() to ncap_utl.c, Its not working correctly at the mo. * Added att_lst_max to ncap.c * Updated ncap.in 2001-12-02 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_1_0 Changes since nco-2_0_3: ncrename interactive query limiter ncap extravaganza * Move Henry's tests into ncap.in script * Allow global:att_nm to signify global attributes in ncap * Removed file query cruft from ncrename and added maximum response count short circuit mechanism to prevent batch jobs from going bonkers when -O not specified and output file exists * Added hybrid coordinate info to in.cdl 2001-11-10 Charlie Zender <zender@uci.edu> * Added three_dim_var_crd to check COORDS storage convention ordering 2001-10-30 Charlie Zender <zender@uci.edu> * Fixed getopt dependencies and included nco_netcdf.h nco.h in ncap stuff 2001-10-28 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_0_3 Changes since nco-2_0_2: ncks bugfixes for units printing, and version information * Use NC_REC_DMN_UNDEFINED instead of -1 where appropriate * Fix argument list to nco_inq_varid() call in ncap * Fix netCDF3 migration bug that could cause core dumps because non-coordinate dimensions where being interpreted as coordinate dimensions in ncks due to changed nature of nc_inq_varid returns 2001-10-15 Charlie Zender <zender@uci.edu> * Added VERSION infrastructure for recording build information in executable * Check rcd before Exit_Gracefully() in main() routines so rcd does not generate "set but not used warnings" 2001-10-07 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_0_2 Changes since nco-2_0_1: No user-visible changes All changes are in netCDF3 compliant function semantics for the nco_netcdf wrappers. Altered many routines to call information-specific versions of inquire functions where possible. * Verified nco-2_0_2 passes test suite * nco_typ_sng(): NC_CHAR is unsigned char, moved nco_typ_sng(), c_type_nm(), fortran_typ_nm() to nco_netcdf library so that library may take advantage of these routines in printing useful diagnostics, yet still be independent of the rest of NCO. * Attempted to replace netCDF2-style use of -1 return codes by rcd and comparison to NC_NOERR everywhere. This is not exactly straightforward because there are many valid reasons to have literal constant -1 in the code. I think I did this correctly. However the use of -1, instead of NC_GLOBAL, for global attributes and, for ncatted, to indicate that attribute changes should apply to all variables, should be changed to NC_GLOBAL so code is more clear. * Change functions to implement netCDF 3.X interface Meaning pass answer holder as pointer and pass back return code nco_open(), nco_create(), nco_inq_[var,dim,att]*() * ncks.c prn_var_val_lmt(): Handle nco_inq_attid() correctly 2001-10-01 Charlie Zender <zender@uci.edu> * Build is clean and successful with g++/Linux, xlC/AIX * Return (int)0 instead of (int)NULL from nco_typ_lng() * Initialize nc_type variables with new netCDF 3.5 token NC_NAT This requires building NCO > 2.0.1 with netCDF >= 3.5 This change also permits successful builds with g++ again, which complained about initializing nc_type with non nc_type enums * cvs tag -c nco-2_0_1 Changes since nco-2_0_0: Cleaner separation of netCDF wrappers into nco_netcdf.c,.h Appearance of error messages slightly changed since prg_nm_get() no longer invoked by wrappers * ncap uses nco netCDF3 interface * Structure nco_netcdf.h as standard library header * Forbid multiple includes of nco_netcdf.h * Changed nc.h to nco.h for namespace consistency * Removed prg_nm_get() from all nco wrappers so wrappers have no external dependencies * Changed nc_err_exit() to nco_err_exit() * Removed superfluous headers from nco_netcdf.c 2001-08-10 Charlie Zender <zender@uci.edu> * Changed storage location of remotely accessible in.nc to .../nco/in.nc * Various Makefile changes for exotic NCAR platforms * Changed band to bnd in in.cdl 2001-07-25 Charlie Zender <zender@uci.edu> * TODO: Rearranged TODO list and put ~15 high-priority items up on the top 2001-05-28 Charlie Zender <zender@uci.edu> * Added HDF5 interface to nco_create(), using HDF5 preprocessor token to isolate native HDF code currently 2001-05-07 Charlie Zender <zender@uci.edu> * cvs tag -c nco-2_0_0 * This netcdf3 branch is now merged into the main trunk. The netcdf2 branch is now in maintenance mode. * cvs tag -c nco-1_3_4 2001-04-04 Charlie Zender <zender@uci.edu> * Changed to using nco_inq_att_flg() in ncar_csm_inq() 2001-03-26 Charlie Zender <zender@uci.edu> * cvs tag -c nco-1_3_3 * nco_tst.sh runs as expected when compiled with OPTS=D and OPTS=O on both Linux 32 bit and Irix 64 bit * Removed make options for NETCDF2_ONLY flag which is no longer supported with NCO 1.3 branch * Default build now compiles with -DNO_NETCDF_2 flag and tests successfully. * Replaced nclong with nco_long = long. There's no need to remove nco_long just yet until a long term strategy for dealing with long, size_t, int issues is decided upon. netCDF 3.x deprecates the whole nclong issue, but it might be useful to have reminders of where the issue was important, e.g., in the fortran arithmetic routines. * Changed in.cdl to use int rather than deprecated long and removed deprecated "l", "L" suffices to numeric constants * Added i18n headers to ncra.c and to Makefile with default being no internationalization. Hooks appear to work fine. * Changed NC_LONG to NC_INT to adhere closer to standard. Type NC_LONG is currently synonymous with NC_INT and will continue to be represented by native type long. 2001-03-07 Charlie Zender <zender@uci.edu> * Added Makefile option to disable OpenMP support on compilers that automatically support it: OMP=N 2001-03-06 Henry Butowsky <thin@thin.screaming.net> * fixed ncdiff problem. There was a conversion error in dmn_dfn * fixed ncrename problem. There was a typo error in nco_rename_var ( it was calling itself) This is the third such error in nco_netcdf.c. 2001-03-02 Charlie Zender <zender@uci.edu> * Code fails to compile with -DNO_NETCDF_2 * Code compiles but is buggy. This command fails: ncdiff -O -C -v three_dmn_rec_var in.nc in.nc foo.nc;ncks -H foo.nc As does ncdiff -O -C -v PS nco_tst.nc nco_tst.nc foo.nc * Removed NETCDF2_ONLY code 2001-02-16 Charlie Zender <zender@uci.edu> * cvs tag -c nco-1_3_2 * After many minor tweaks, made commits and retagged nco-1_2_2 and this is the version that will be merged into 1_3_1 to create 1_3_2 * Tagged changes until today as nco-1_3_1 and will merge with nco-1_2_2 2001-02-11 Charlie Zender <zender@uci.edu> * Figured out that texi2dvi --pdf really does work nicely and am now using it rather than ps2pdf * Tagged and released nco-1_2_2 2001-01-02 Charlie Zender <zender@uci.edu> * Clean up some function prototypes, removed crud from nc.h, implemented extern keyword on all functions * Changed ncks print formatting of NC_BYTE data (i.e., signed char) to print as unsigned char data to facilitate printing HDF4 datasets. 2000-12-29 Charlie Zender <zender@uci.edu> * Added make NETCDF2=Y to compare netCDF2 behavior with/without HDF libraries. HDF libraries cause ncwa to fail certain tests that work fine with Unidata netcdf2 libraries. All other operators appear fine with HDF libraries. * Added default statements to switch(type) constructs which emitted warnings when compiled with -DNETCDF2_ONLY since, apparently, there are a whole bunch of things besides numeric types in the old nctype enum and gcc warns about enums that are not exhaustively considered in switch() statements. All these default statements can be removed with netCDF3 interface so perhaps these should be surrounded with #ifdef NETCDF2_ONLY constructs, but they actually do make sense for netCDF3 as well so I have implemented a uniform error function, dfl_case_nctype_err(), to be called by all routines which emit errors only when compiled with NETCDF2_ONLY. This makes the behavior easy to modify or remove in the future. * Fixed echoing of Makefile diagnostics * Added HDF4 option to Makefile to facilitate compiling with HDF version of netCDF libraries 2000-11-25 Charlie Zender <zender@uci.edu> * Added -q ("quiet") switch to ncks to suppress printing of dimension indices and coordinate values 2000-11-04 Charlie Zender <zender@uci.edu> * Created netcdf3 branch with tag netcdf3 and additional version tag nco-1_3. This branch will hold the netcdf3 implmentation while it is in development. When it is ready it will become the main trunk of nco. 2000-10-29 Charlie Zender <zender@uci.edu> * Implemented new @command{}, @env{}, and @acronym{} keywords in nco.texi 2000-10-26 Charlie Zender <zender@uci.edu> * Fixed makeinfo problem with nco.texi 2000-10-15 Charlie Zender <zender@uci.edu> * Implemented Henry's ncrename and nco_tst patches: Former allows renaming of attribute for a single variable, latter cleans up nco_tst.sh and allows easier function-specific testing. 2000-10-01 Charlie Zender <zender@uci.edu> * Cast results of sqrt() to LHS type to eliminate new gcc 2.96 warnings. g++ 2.96 is less forgiving than gcc 2.2 * Fixed some miscast missing values in arithmetic routines for doubles, shorts, and longs. Odd that I had not noticed these miscasts before. g++ 2.96 flagged them. 2000-09-21 Charlie Zender <zender@uci.edu> * Fixed one-line bug where ncwa min,max,ttl cases fell through to error exit rather than doing nothing with weights * Adding critical regions about netCDF library calls appeared to work. ncwa and ncra/ncea now give repeatable, correct answers when multithreaded. Speedups appear to be about ~30--40% on small numbers of typical GCM files when using 4 threads. Performance appears to level out by 4 threads, although more timing is necessary. For example, ncwa is now spending about 50% on I/O and 50% on computation. 2000-09-20 Charlie Zender <zender@uci.edu> * Added OpenMP critical regions to var_get() and var_refresh() * Added OpenMP critical regions around thread-unsafe netCDF write calls in ncwa and ncra/ncea. * Finally removed 1995--1996-era SGI enum workarounds from nc.h SGI compiler on dataproc seems to behave sanely now * Jim Rosinski and Stacy Walters have both pointed out that netCDF is not thread-safe. Therefore the OpenMP implementation is bound to fail until critical locks are added. 2000-09-19 Charlie Zender <zender@uci.edu> * Added OpenMP pragmas to ncea,ncra,ncrcat. They probably don't work either. * Multi-threaded version of ncwa averages variables in T42 file correctly sometimes, but also incorrectly. Currently unpredictable what causes discrepancy. * Removed arg_cnt from ncwa.c. ncwa.c now uses standard fl_mk_lst() to diagnose incorrect argument counts 2000-09-18 Charlie Zender <zender@uci.edu> * Got OpenMP compiling but not completely working on ncwa under IRIX 2000-09-17 Charlie Zender <zender@uci.edu> * Added full OpenMP directives to ncwa. If this works then ncra is next. 2000-09-13 Charlie Zender <zender@uci.edu> * Contrary to the documentation, ncflint has always processed coordinate variables. Perhaps fixing coordinate variables is a better idea? Patched var_lst_divide() to fix all non-record coordinate variable to ncflint. Not sure if I will release this though. Perhaps it should be a switch? * Improved ncflint documentation, including its ability to add files 2000-08-31 Charlie Zender <zender@uci.edu> * Changed handling of netCDF convetions to search for "Conventions" attribute rather than "convention" attribute Not sure why I ever used "convention" in the first place! This should fix things like gasussian weights being subtracted 2000-08-30 Charlie Zender <zender@uci.edu> * Updated var_dup(), var_free() to handle scl_fct.vp, add_fst.vp, srd. That's right, these routines were not handling srd pointer correctly. 2000-08-29 Charlie Zender <zender@uci.edu> * Removed tally array from var_avg_reduce_max/min() routines * Added some clarifications to memory handling in the manual, and additional examples provoked by John Sheldon's GFDL netCDF website. * Integrated ncvarid_or_die() and ncdimid_or_die() into ncrename 2000-08-28 Charlie Zender <zender@uci.edu> * Tagged nco-1_2_1 * Will all previous changes to nco_malloc(), nco_realloc(), and memory management in general, make tst now seems to be well behaved with Electric Fence, and duplicates answers without Electric Fence. * Another nco_realloc() tweak to satisfy Electric Fence: Now realloc() is never called when size == 0 2000-08-27 Charlie Zender <zender@uci.edu> * Automatic type conversions for arithmetic now work in ncwa, ncea, ncra. Don't think ncflint would benefit from the capability. 2000-08-26 Charlie Zender <zender@uci.edu> * Changed meaning and name of typ_prv to typ_dsk in var_sct * Moved target dat_cln from cln to dst_cln, opposite for lib_cln * Simplified internals of nco_opr_drv() * Fixed nco_realloc() so it does not call realloc() when ptr==NULL and size==0. Allowing a realloc() call in this situation is ANSI-legal but triggers Electric Fence. 2000-08-25 Charlie Zender <zender@uci.edu> * Merged Makefile refinements by Ethan Davis to better support DODS compilations and less redundant building 2000-08-23 Charlie Zender <zender@uci.edu> * Functionalized type conversion routines nco_cnv_var_dbl() and nco_cnv_dbl_var() * Fixed x:y bug in opt arg of ncra.c introduced in 1.2 2000-08-14 Charlie Zender <zender@uci.edu> * Added this text to the license, provided by Steven G. Johnson "As a special exception to the terms of the GPL, you are permitted to link the NCO source code with the NetCDF and HDF libraries and distribute the resulting executables under the terms of the GPL, but in addition obeying the extra stipulations of the netCDF and HDF library licenses." This is intended to allow NCO to link to the non-GPL licenses of netCDF and HDF and avoid the "KDE problem". Apparently binaries are considered derived works of any libraries they are statically linked to, i.e., netCDF and/or HDF. This means both licenses must be obeyed, but that is impossible because no license except the GPL is compatible with the GPL. So we the developers of NCO must explicitly grant permission to all users to redistribute NCO under both licenses. 2000-08-03 Charlie Zender <zender@uci.edu> * Tagged nco-1_2 * Altered homepage * Added PDF version of manual 2000-07-31 Charlie Zender <zender@uci.edu> * Added mathematical definitions of all new operations to manual * Rearranged order of operations in ncwa so that non-linear operations may employ weighting correctly. Weighting is not well-defined for some operations (rmssdn) but I did my best. 2000-07-28 Charlie Zender <zender@uci.edu> * Applied final min/max/ttl patch so min/max/ttl capabilities should now work as expected on ncwa,ncea,ncra 2000-07-16 Charlie Zender <zender@uci.edu> * Added ncwa min/max/ttl patch 2000-07-09 Charlie Zender <zender@uci.edu> * Added section on Contributing/contributors to nco.texi * Fixed case when missing value is first in min and max operators * Added min, max, ttl cases to nco_tst.sh * Added temporary variable to all low-level arithmetic routines (e.g., var_add()) to store missing value in order to reduce dereferencing of pointers to missing values. This may result in measurable speedups for all arithmetic operators. 2000-07-08 Charlie Zender <zender@uci.edu> * Applied minmaxttl03 patch 2000-07-02 Charlie Zender <zender@uci.edu> * Added nco_op_typ to SGI enum section of nc.h * Update nco/doc/nco_src_frg.txt to include release instructions * Released nco-1.1.48.tar.gz on sourceforge * Switched all memory management to nco_malloc(), nco_realloc() * Implemented min/max/ttl patch (min and ttl not working yet) 2000-06-20 Charlie Zender <zender@uci.edu> * Implemented Schweitzer's 'make DODS=Y' patch in Makefile Only works on Linux as configured but that's a start * Added ncvarid_or_die() to replace simple ncvarid() on calls where failure to find variable is an error and a dianostic of which variable is missing would be nice. 2000-06-05 Charlie Zender <zender@uci.edu> * Changed ~/nc/nco paths to ~/nco 2000-06-02 Charlie Zender <zender@uci.edu> * Added explicity coercion to all lines which generated assignment errors under g++ C++ compiler. This might affect speed of conversions to long, short, char, and byte types from double and float types. These conversions are rare and considered unimportant so any performance penalty is acceptable. * Small changes to make code compile cleanly with g++ C++ compiler * Changed fl_mk_lcl() to try scp instead of rcp. scp should fall through to rcp is sshd is not running on remote machine. * Add Schweitzer's HTTP patch for DODS compatibility to fl_mk_lcl(). Will probably alter implementation in the future so fl_mk_lcl() works without netcdf library as before. NCO should now be DODS compliant when linked to DODS libraries. This allows reading, but not writing, of remote files using HTTP protocol. * Tagged nco1_1_49 as clean new version to start work from sourceforge without any overlaps with 1_1_48 * Had to re-tag nco1_1_48 because sourceforge CVS repository was slightly out of sync with CGD repository * Moved CVS repository to sourceforge 2000-05-16 Charlie Zender <zender@uci.edu> * Tagged this as nco1_1_48 * Found and fixed ncdiff bug in TODO #155. Bug had been inserted in ncdiff 1.5 (nco-1.1.15, November 1998) when I switched from using dimension IDs to using dimension names to identify and compare dimensions in var_conform_dim(). Problem was that there are two locations in var_conform_dim() where this needed to be done and I only changed the first location. Then I turned off dimension remapping code in ncdiff which used to take care of the problem. Fortunately I had left that code commented out in ncdiff() so I was able to check that turning it back on solves the problem. Now I have removed old commented-out block from ncdiff (because it was confusing) and all logic is now local to var_conform_dim(). This fix increases the number of strstr() comparisons of dimension names because names must now be compared twice in var_conform_dim() for each variable being expanded. Oh well. 2000-05-12 Charlie Zender <zender@uci.edu> * Fixed bug in -d dmn,,,srd case where lmt.end was incorrect 2000-05-10 Charlie Zender <zender@uci.edu> * Finished testing of feature allowing skipping of initial files when lmt_typ = dim_idx. Seems to work. Now dim_idx and crd_val hyperslabs may have arbitrary numbers of superfluous files at beginning and end. * Added rec_skp_nsh member to lmt structure to support keeping track of records skipped in superfluous initial files for lmt_typ = dim_idx on multi-file operators 2000-05-09 Charlie Zender <zender@uci.edu> * Altered behavior of single point hyperslabs so that single point hyperslabs in the record coordinate (i.e., -d time,1.0,1.0) may be treated differently than single point hyperslabs in other coordinates. Multifile operators will skip files if single point hyperslabs in record coordinate lays outside record coordinate range of file. For non-record coordinates (and for all operators besides ncra and ncrcat on record coordinates), single point hyperslabs will choose the closest value rather than skip the file (I believe). This should be verified. * Switched NCO from atof(), atol() to strtod(), strtol() * Added ability to skip superfluous trailing files to lmt_evl() when limits are coordinate values. Needs testing but appears to work. Had to use a goto statement, icky. Fixes TODO #157? 2000-05-04 Charlie Zender <zender@uci.edu> * More fixes to fix problems with Electric Fence 2000-04-18 Charlie Zender <zender@uci.edu> * Added ncra ncea to RPM 2000-03-22 Charlie Zender <zender@uci.edu> * Added code to prevent malloc'ing 0 bytes because, although it is perfectly legal, Electric Fence complains 2000-03-06 Charlie Zender <zender@uci.edu> * Added target rpmnet to Makefile. Usage is sudo make NCO_VRS=1.1.46 rpmnet This downloads specified NCO version, builds RPMs from it, and uploads the RPMs back to NCO FTP site 2000-03-01 Charlie Zender <zender@uci.edu> * Added nco.spec to bld directory * Adding rpm target to Makefile 2000-01-27 Charlie Zender <zender@uci.edu> * Tagged this as nco-1_1_45 * Fix to PID length bug in fl_out_open() on SGIs Now use dynamically allocated string to hold PID Kudos to Juliana Rew for finding this bug 2000-01-16 Charlie Zender <zender@uci.edu> * Changed all addresses except website to UCI ESS * Changed license to GNU GPL (!) 2000-01-14 Charlie Zender <zender@uci.edu> * Calling ncwa without arguments now causes operator to print usage and exit successfully * Print error message and hint when ncwa called with -a dim1 -a dim2 instead of -a dim1,dim2 2000-01-10 Charlie Zender <zender@z.ppp.uci.edu> * ncks now puts string values inside double quotes, character values inside single quotes, other slight formatting changes too 1999-12-26 Charlie Zender <zender@dust.ps.uci.edu> * Tagged this as nco1_1_44 * Documented record coordinate-stride capability for ncra, ncrcat in nco.texi examples and command-line usage routines * Added support for printing all character arrays as strings in ncks 1999-12-14 Charlie Zender <zender@dust.ps.uci.edu> * Tagged this as nco1_1_43 * Merged all SGI builds (SGI5,SGI64,SGIMP64) into single makefile block * Added support for C-language escape sequences to ncatted Moved all this code from ncks to new sng_ascii_trn() function 1999-12-06 Charlie Zender <zender@dust.ps.uci.edu> * Tagged this as nco1_1_42 * Fixed so that consecutive delimiter strings work, e.g., ncatted -O -a history,global,o,c,"hi,,,hi" in.nc 1999-12-04 Charlie Zender <zender@dust.ps.uci.edu> * Fixed bug where ncatted neglected string values after first comma, e.g., would drop ", NCAR" from test attribute in ncatted -O -h -a test,global,o,c,"NREL, NCAR" in.nc because the new attribute value followed the comma delimiter used internally by ncatted to delimit list elements * Fixed so that setting strings to zero length with ncatted works, e.g., ncatted -O -a history,global,o,c,"" in.nc * Fixed attribute printing of lists in ncks: terminal delimiter is no longer appended 1999-11-02 Charlie Zender <zender@dust.ps.uci.edu> * Fixed some problems with AIX build environment, switched to -DAIX on the IBM SP cluster environment 1999-10-21 Charlie Zender <zender@dust.ps.uci.edu> * Defined MY_BLD_DIR in Makefile before it is used Re-tagged this as nco1_1_41 1999-10-17 Charlie Zender <zender@dust.ps.uci.edu> * Tagged this as nco1_1_41 * Added dir to targets which do not rebuild *.d dependencies Doing this and installing latest make-3.78.1 fixes build on dataproc 1999-10-15 Charlie Zender <zender@dust.ps.uci.edu> * Tagged this as nco1_1_40 * Added two more ncwa test cases to nco_tst.sh * Fixed another bug in ncwa reported by Keith Lindsay. The denominator in normalized averages did not always account for missing values in the variable being averaged. This bug may have been introduced sometime during ncwa rewrites around 19981201. I'm sure it has not always been there. 1999-10-03 Charlie Zender <zender@dust.ps.uci.edu> * Tagged this as nco1_1_39 * Added test target in bld/Makefile. Now `make test' automatically downloads a small (66 kb) file on which it performs a small battery of tests (bld/nco_tst.sh). * Revamped build procedure to use GNU make dependencies rather than makdep perl script. * All *.c files now build cleanly with gcc -Wall * Fixed Makefile bug where Fortran routines were compiled when they were not needed and visa versa * Moved NETCDF_INC and NETCDF_LIB out of architecture specific portions of Makefile 1999-08-31 Charlie Zender <zender@dust.acd.ucar.edu> * Tagged this as nco1_1_38 * ncwa now averages over all dimensions when none are specified with -a * Patched avg_reduce...() to fix set averages equal to zero when tally is zero. This fixes Keith's problem. * Added ncwa test #12 to nco_tst.sh. This tests for a bug reported by Keith Lindsay where ncwa averaged a variable which is completely missing_value to a value of 0.0 rather than missing_value. * Removed C_ONLY token and made builds completely C-based by default. Introduced new token USE_FORTRAN_ARITHMETIC to build old style NCO with fortran arithmetic routines. Seems to work fine. Translated Fortran date routines newdate() and days2eom() to C as part of this. 1999-08-04 Charlie Zender <zender@dust.acd.ucar.edu> * Tagged this as nco1_1_37 * Implemented first attempt to use msrcp, seems to work * Tagged this as nco1_1_36 * Improved patch to fl_mk_lcl() so it "does the right thing" when determining whether to try to rcp files 1999-08-03 Charlie Zender <zender@dust.acd.ucar.edu> * Patched fl_mk_lcl() to exit gracefully on filenames with multiple colons (which are legal in UNIX). Files with single colons in their names, as opposed to rcp requests, still cause core dumps and will continue to do so until I write a routine which determines whether to treat the filename as an rcp request based on some sort of valid hostname recognition algorithm. 1999-07-29 Charlie Zender <zender@dust.acd.ucar.edu> * Tagged this as nco1_1_35 * Changed WARNING message to print in ncra only when insufficient records have been found and last file has been processed 1999-07-03 Charlie Zender <zender@z.ppp.ucar.edu> * Tagged this as nco1_1_34 * Altered ncatted behavior to replace missing data values with new missing_value when missing_value attribute changes 1999-05-12 Charlie Zender <zender@z.ppp.ucar.edu> * Tagged this as nco1_1_33 * Using the new lmt_evl() dim_idx code for operators besides ncra and ncrcat, e.g. ncks, causes problems with wrapped and wrapped stride limits. One line fix to lmt_evl() restored old lmt_evl() dim_idx code to all operators except ncra and ncrcat. The new lmt_evl() code simply will not (and should not) handle wrapped coordinates for the record dimension in multi-file operators. 1999-05-11 Charlie Zender <zender@z.ppp.ucar.edu> * Tagged this as nco1_1_31 * Fixed ncks problem caused by not initializing all boolean flags of lmt structures in lmt_prs(). Changed lim to lmt. * Tagged this as nco1_1_30 * Tentatively fixed record hyperslab problems introduced in 1_1_29 by extensively generalizing and rewriting lim_evl() * Clarified in User's Guide that default behavior of stride is that -d time,,,srd is syntactically equivalent to -d time,0,,srd * Realized new record dimension features of nco1_1_29 were buggy when min and max limits were not both user-specified because then total number of records cannot be not known a priori 1999-05-10 Charlie Zender <zender@z.ppp.ucar.edu> * Tagged this as nco1_1_29 * Cleaned up files for a clean compile with gcc -Wall 1999-05-09 Charlie Zender <zender@z.ppp.ucar.edu> * Toggled behavior of `ncks -a' so that default is now to alphabetize output * Fixed error (missing comma) in stride documentation in User's Guide * Implemented stride for the record dimension in ncra and ncrcat, e.g., ncra -d time,1,100,12 in1.nc in2.nc ... out.nc should now work correctly across files. Currently stride only works on the record dimension of ncra and ncrcat, however, not the rest of the dimensions. * Implemented index-based hyperslabbing across files in the record dimension in the multi-file operators ncra and ncrcat, e.g., ncra -d time,1,100 in1.nc in2.nc ... out.nc. The User's Guide said this feature had already been implemented, but that, apparently, was not true. The symptom was an "index out of range error" from netCDF. 1999-04-27 Charlie Zender <zender@sanitas-e0.cgd.ucar.edu> * Tagged and released this as nco1_1_28 * Isolated build procedure in nco_dst.pl with --bld option, off by default * Added --dat_cnt to nco_dst.pl to support dataproc.ucar.edu * Added SGIMP64 architecture to Makefile to support dataproc.ucar.edu. Default netCDF lib is r4i4 rather than r4i8 as on winterpark so I am avoiding the whole issue by using C_ONLY defaults and spoofing SGI64. nco_tst.sh executed without errors. 1999-04-20 Charlie Zender <zender@flagstaff.cgd.ucar.edu> * Tagged and released this as nco1_1_26 * Fixed so arguments to log10() are reasonable * Tagged and released this as nco1_1_25 * New version requires -lm for log() and ceil() functions Added -lm to Makefile for some architecture which were missing it * Tagged and released this as nco1_1_24 * Fixed bug where ncrcat and ncra omitted the last slice of the record dimension when -F (Fortran indexing) was user-specified AND user-specified hyperslab information was provided for some dimensions BUT not for the record dimension. This bug only affected ncrcat and ncra and only under these conditions. Thanks to John Sheldon <jps@server1.gfdl.gov> for pointing this out. 1999-04-04 Charlie Zender <zender@z.ppp.ucar.edu> * Tagged and released this as nco1_1_22 * Added -a switch to ncks for alphabetizing the output list. Added index_alpha() and changed lst_heapsort() to accept an boolean switch argument to accomplish this. 1999-02-25 Charlie Zender <zender@z.ppp.ucar.edu> * Added partial diagnostic message and workaround for TODO #116 1999-01-29 Charlie Zender <zender@z.ppp.ucar.edu> * Added nco_dst.pl to distribution * Changed ALPHA makefile to support native compilers (untested) * Added NETCDF2_ONLY option to circumvent ncdiff nc_inq_vartype() call. 1999-01-24 Charlie Zender <zender@z.ppp.ucar.edu> * Moved index.shtml to NCO doc directory Thu Jan 21 15:23:02 1999 Charlie Zender <zender@sanitas-e0.cgd.ucar.edu> * Tagged and released this as nco1_1_20 * Updated various portions of User's Guide * Added NETCDF2_ONLY token to eliminate nc_inq_libvers() call 1999-01-13 Charlie Zender <zender@garcia.acd.ucar.edu> * Fixed bug in var_def() where wrong branch was executed when called by ncwa for files with no output dimensions. Symptom was a core dump or "ncvardef: ncid 4: Invalid dimension id or name" error. Wed Jan 6 17:55:26 1999 Charlie Zender <zender@sanitas-e0.cgd.ucar.edu> * Tagged and released this as nco1_1_18 * Added kludge to ncwa to workaround bug in var_conform_dim() where var_conform_dim() sometimes allows the returned weight not to have same size tally array as the template variable. This caused core dumps in ncwa. Fri Dec 4 15:06:23 1998 Charlie Zender <zender@sanitas-e0.cgd.ucar.edu> * Tagged and released this as nco1_1_16 * Added new logic switch DO_CONFORM to var_conform_dim() to allow ncwa to tell when to mask variables. Reran test cases, everything seems to be working. Documentation seems up-to-date. 1998-12-03 Charlie Zender <zender@z.ppp.ucar.edu> * Tagged and released this as nco1_1_15 * Disabled ncwa -n and -W normalization options until I think of better way to implement them. -N option now causes ncwa not to divide by the denominator. Disabled -n and -W tests in nco_tst.sh (tests #2 and #3). 1998-12-02 Charlie Zender <zender@z.ppp.ucar.edu> * Changed ncwa to default to weighting coordinates just like variables, i.e., toggle default value of -I switch * Added more exhaustive ncwa tests to nco_tst.sh * Rewrote ncwa section of User's Guide 1998-12-01 Charlie Zender <zender@z.ppp.ucar.edu> * Made sure ncwa weights were being masked when normalization was invoked. Formerly, they were not. This was a bug. The bug is now fixed. * Rearranged normalization logic of ncwa * Removed special treatment of "gw" from ncwa 1998-11-25 Charlie Zender <zender@z.ppp.ucar.edu> * Allow hyperslab coordinate specification in exponential format even when decimal point is missing, e.g., "-d lon,36e1" is valid * var_conform_dim() now checks dimension names rather than IDs Mon Nov 23 17:29:02 1998 Charlie Zender <zender@sanitas-e0.cgd.ucar.edu> * Added -I switch to ncwa to enable weighting and masking of coordinates. Rewrote var_conform_dim() to allow mutually exclusive dimensions in ncwa. 1998-11-01 Charlie Zender <zender@z.ppp.ucar.edu> * Added wrapped coordinate caveats to hyperslabe section of manual, and implemented warning message whenever wrapped coordinates are used. Sat Oct 31 14:56:33 1998 Charlie Zender <zender@sanitas-e0.cgd.ucar.edu> * Tagged and released this as version nco1_1_10 1998-10-29 Charlie Zender <zender@z.ppp.ucar.edu> * Fixed bug in ncra which only occured when NC_CHAR and NC_BYTE types also had a record dimension. var_lst_divide() decided these should be fixed variables, thus the output file size included the record dimension. The timecom variable in LSM history files was triggering this bug. Solution was to make ncra, like ncrcat, simply process ALL record variable even NC_CHAR and NC_BYTE, though the averaging operation on these types is still ill-defined. Wed Aug 26 16:30:27 1998 Charlie Zender <zender@odin.cgd.ucar.edu> * Fixed bld/nco_tst.sh to work again at NCAR. `make test' now performs non-trivial tests of ncwa and ncdiff. Only works at NCAR (a necessary input file is rather large). Wed Aug 19 11:26:25 1998 Charlie Zender <zender@odin.cgd.ucar.edu> * Fixed bug where pvmgetarch returned "SGI" on UNICOS 10.0.0. * Improved Makefile fortran switches and added convenience housekeeping targets. * Fixed bug where cvs_vrs_prs() crashed with -kkv exports 1998-08-18 Charlie Zender <zender@z.ppp.ucar.edu> * Added nco_vrs_prs() to parse NCO versions. NCO version is now always printed with -r option. * Fixed bug where -n NINTAP list did not recognize `.cdf' suffixes. Mon Aug 10 21:53:34 1998 Charlie Zender <zender@sanitas-e0.cgd.ucar.edu> * Split ncks -m option into -m and -M so now global metadata can be avoided if desired. * Fixed bug in ncrename where fl_in could be free()'d before it was malloc()'d in fl_nm_prs(). * Fixed bug in ncdiff where variables of same rank but different types would cause core dump when being subtracted. Improved diagnostics for failures with ncdiff. 1998-07-07 Charlie Zender <zender@z.ppp.ucar.edu> * Added -h option (suppresses history concatentation) to all operators. Changed existing ncks -h option to -m (for metadata). Sun Jun 7 17:45:20 1998 Charlie Zender <zender@sanitas-e0.cgd.ucar.edu> * Added more descriptive error diagnostics to ncvarid() calls to var_lst_mk() ncks to report the names of user-specified variables which do not exist in input files. 1998-05-16 Charlie Zender <zender@sanitas-e0.cgd.ucar.edu> * Improved nc_lib_vers_prn() 1998-05-08 Charlie Zender <zender@sanitas-e0.cgd.ucar.edu> * Added improved error diagnostics to ncks when -A fails. Added improved error diagnostics to fl_out_close() and fl_mv(). -r now prints library version for all operators. -r and usg_prn() now prints NCO homepage URL. Wed Mar 11 11:16:39 1998 Charlie Zender <zender@odin.cgd.ucar.edu> * Removed warning message for processed text fields in ncecat. Mon Mar 2 22:00:59 1998 Charlie Zender <zender@odin.cgd.ucar.edu> * Added nc_inq_libvers() call to ncwa to diagnose SGI problems. This is the first netCDF 3.x call in NCO. There's no stopping the avalanche now, so user's must install netCDF3.x Mon Feb 9 09:50:26 1998 Charlie Zender <zender@odin.cgd.ucar.edu> * Merged all three ncwa loops that access output and tally buffers so that these buffers can be allocated and deleted inside the main loop over variables. This reduces peak memory consumption by a factor of three and sustained memory by a factor of two to three. ncwa performance should be noticeably enhanced by this. This problem affected, and this fix improves, only the ncwa operator. Sun Feb 8 21:50:32 1998 Charlie Zender <zender@odin.cgd.ucar.edu> * Downloaded and altered pvmgetarch from pvm3.4b6. This version was returning SGI instead of CRAY on Ouray (=Cray J90 Unicos 10.0.0) because pvmgetarch thinks machines with /bin/4D are SGI. Distributing this customized version of pvmgetarch with NCO. Also changed new pvmgetarch so it returns SUNMP on any Sun multiprocessor, regardless of whether PVM_SHMEM is set to ON. Mon Jan 19 13:13:08 1998 Charlie Zender <zender@odin.cgd.ucar.edu> * Added HTML keywords and section on large numbers of files to the NCO User's Guide. * Eliminated compiler warnings caused by type--format mismatches from all source code. Should now compile without warnings on SGI. * Removed -ansi switch from Linux CFLAGS in order to support glibc2. glibc2 considers resolv.h non ANSI so that using the -ansi swithc results in unresolved tokens in system #include files. Tue Dec 2 21:35:19 1997 Charlie Zender <zender@odin.cgd.ucar.edu> * Added fl_nm_nbr_max and fl_nm_nbr_min as optional arguments 4 and 5, respectively, to the -n switch on the multi-file operators. This allows the NINTAP automatic filename generation feature to handle input sets of cyclic filenames whose suffixes grow to fl_nm_nbr_max and then wrap back to fl_nm_nbr_min. Now, e.g., DJF files may be specified with -n 3,2,1,12,1. Fri Nov 14 14:20:23 1997 Charlie Zender <zender@odin.cgd.ucar.edu> * Added additional diagnostics concerning record dimensions to ncra and ncrcat. ncra and ncrcat now exit when called to operate on files without record dimensions. * Made ncar_csm_date() more fault tolerant for files which do not adhere to de facto atmospheric model time conventions involving date, time, nbdate. 1997-11-04 Charlie Zender <zender@z.ppp.ucar.edu> * Added error diagnostics for large malloc()'s in ncwa and var_dup in nc_utl.c. These should aid in identifying causes for core dumps when running with extremely large files or insufficient memory. Fri Oct 17 12:53:54 1997 Charlie Zender <zender@odin.cgd.ucar.edu> * Deprecated checking for "gw" for NCAR CSM conformance and replaced it with comparing global attribute "convention"'s value to "NCAR-CSM". Fixed up ncar_csm_date accordingly and rewrote warning messages. * Improved usg_prn for ncatted. 1997-10-12 Charlie Zender <zender@z.ppp.ucar.edu> * Fixed fl_out_open() to abort after receiving more than 10 incorrect user responses. This fixes a bug where NCO would crash in non-interactive shells when the -O or -A options were omitted. 1997-10-07 Charlie Zender <zender@z.ppp.ucar.edu> * Finished ncatted documentation. Cleaned up code to avoid compilation errors with SGI on memcpy(). Touched up manual. Releasing this as NCO Version 1.1. Sun Sep 21 15:08:49 1997 Charlie Zender <zender@z.ppp.ucar.edu> * Merged WIN32 modifications into distribution. Added WIN32 and C_ONLY tokens to code. Relinked with netCDF 3.3.1, had no problems. Tue Sep 16 23:53:44 1997 Charlie Zender <zender@z.ppp.ucar.edu> * Wrote ncatted. * Fixed ncks to omit trailing commas after printing scalar attribute values. Thu Aug 28 17:07:50 1997 Charlie Zender <zender@heinlein.cgd.ucar.edu> * Rewrote ncdiff documention, which had many errors and omissions. Sat Jun 28 23:26:40 1997 Charlie Zender <zender@z.ppp.ucar.edu> * Fixed bug in cpy_var_val_lim() in ncks.c which caused core dumps when hyperslabbing on multiple coordinates where at least one of the coordinates was wrapped. Tue Jun 17 14:15:53 1997 Charlie Zender <zender@heinlein.cgd.ucar.edu> * Changed Makefile compile rules for f90 on CRAY to explicitly preprocess *.F files. This had stopped working, perhaps when I switched CRAY default from f77 to f90. Fri May 30 13:12:25 1997 Charlie Zender <zender@heinlein.cgd.ucar.edu> * Improved mss_val_get to allow any NC_TYPE for missing_value attribute. This was needed for ARM files which store missing_value as an NCCHAR. Converting NCCHAR to short, long, float, or double now uses strdod() instead of implicit type conversion. * Added routines to process time variables in ARM files. * Added hyam,hybm,hyai,hybi to list of variables which will not be differenced by ncdiff. * ncdiff now chooses mss_val from either file where it's valid, and from fl_1 when it's valid in both. This fixed the last known bug in NCO. Tue May 27 00:08:23 1997 Charlie Zender <zender@z.ppp.ucar.edu> * Moved var_refresh() out of inner loop to file loop level in ncra.c. This should speed things up a little because var_refresh() can be an expensive call since it calls mss_val_get(). * Altered mss_val_get to accomodate missing_value attributes whose type did not match their variable's type. Also made allowance for missing_value's of type NC_CHAR, as in ARM data files. Added three routines, arm_inq(), arm_time_mk(), and arm_time_install(), which implemente the ARM convention time=base_time+time_offset convention in ncrcat. Fri May 16 16:28:02 1997 Charlie Zender <zender@heinlein.cgd.ucar.edu> * Fixed bug in srt vector in ncdiff. Now ncdiff should be working with hyperslabs. Mon May 5 17:55:00 1997 Charlie Zender <zender@heinlein.cgd.ucar.edu> * Implemented nclong fix for SGI64. This involved very slight changes: made defs.h:ptr_unn->lp point to nclong rather than long cast_void_nctype():ptr->lp evaluates to (nclong *). Sun Mar 30 15:26:10 1997 Charlie Zender <zender@z.ppp.ucar.edu> * added and removed some return() statements to nc_utl.c to stop compiler warnings from SGI cc. * fixed small memory leak and bug in var_srt_zero() where var->srt was being allocated twice, and the xrf sct was pointing to the copy that had not been zeroed. Thu Mar 27 15:13:18 1997 Charlie Zender <zender@z.ppp.ucar.edu> * Removed ncap from target all in Makefile * Added section describing differences between ncra, ncea, ncrcat, and ncecat to nco.texi. * Fixed memory leak in ncecat in which memory never seemed to be deallocated until the end of the file loop. Wed Mar 26 21:13:09 1997 Charlie Zender <zender@z.ppp.ucar.edu> * Added nco.ps, nco.dvi, nco.html, texi2html, texi2dvi to distribution * Changed ncks -s option to accept full printf() style format strings * Fixed memory leak in var_get() in nc_utl.c which caused memory to be allocated twice. This also fixed longstanding performance problem where ncrcat() allocated memory for entire array not just current record. Thanks to jps@GFDL.GOV (John Sheldon) for discovering this bug. * General release of nco-0.9 on netCDF and CCM mailing lists Thu Feb 8 23:01:04 MST 1996 Charlie Zender <zender@ncar.ucar.edu> * First internal CGD release of operator set known as ncz-0.9 Sat May 15 23:01:04 MST 1993 Charlie Zender <zender@ncar.ucar.edu> * First RCS snapshot of ncks.c in directory nc. ncks.c began life as c_template.c