=head1 NAME XBase - Perl module for reading and writing the dbf files =head1 POZOR! This is suggestion for new interface. Not current documentation, see normal perldoc XBase. =head1 SYNOPSIS use XBase; my $table = new XBase "dbase.dbf" or die XBase->errstr; for (0 .. $table->last_record) { my ($deleted, $id, $msg) = $table->get_record($_, "ID", "MSG"); print "$id:\t$msg\n" unless $deleted; } $table->{'RaiseError'} = 1; # new my $cur = $table->prepare_select_with_index("dbaseid.ndx", "id", "msg"); $cur->find_eq(156) or do { print "Value 156 not found.\n"; exit; }; my ($id, $msg) = $cur->fetch; =head1 DESCRIPTION This module can read and write XBase database files, known as dbf in dBase and FoxPro world. It also transparently reads memo fields from the dbt, fpt and smt files and works with index files (ndx, ntx, mdx, idx, cdx and SDBM). This module XBase.pm provides simple native interface to XBase files. For DBI compliant database access, see DBD::XBase and DBI modules and their man pages. To work with dbf and associated files, you first need to open the dbf file using my $table = new XBase 'dbase.dbf' or die XBase->errstr; type of call. This gives you an object to interact with the table. You can then access the records using their position in the file my ($deleted, $id, $name, $born) = $table->get_record($num, 'ID', 'NAME', 'DO_BIRTH'); if ($id == 436) { $table->update_record_hash($num, 'NAME' => 'Peter') } or via cursors that allow you to walk through the file my $cur = $table->prepare_select('ID', 'NAME', 'DO_BIRTH'); while (my ($id, $name, $born) = $cur->fetch) { # do some work } If there are index files for given table, they can be used to speedup the searches. You can either use them explicitely to open cursor based on the index my $cur = $table->prepare_select_with_index('dbaseid.ndx' 'ID', 'NAME', 'DO_BIRTH'); if ($cur->find_eq(436)) { my ($id, $name, $born) = $cur->fetch; } or you can attach the indexes to the table and they will be used when needed and also updated when the dbf table changes my $table = new XBase 'file' => 'dbase.dbf', 'RaiseError' => 1, 'index' => [ 'dbaseid.ndx', 'dbasename.ndx' ]; my $cur = $table->prepare_select_where('id = 436', 'NAME'); while (my ($name) = $cur->fetch) { print "Old value $name, new value 'Peter'\n"; $table->update_record_hash($cur->last_fetched, 'name' => 'Peter'); } The cdx, mdx and SDBM index files (with the same base name as the dbf) are attached by default. =head1 LIST OF METHODS The following methods are available for XBase.pm tables and their cursors, their meaning and parameters are in more detail described below: =over 4 =item General methods working with the table new close create drop attach_index pack errstr =item Methods returning information about the file last_record field_names last_field field_types header_info field_lengths field_type field_decimals field_length field_decimal =item Accessing and modifying the records get_record set_record get_record_nf set_record_hash get_record_as_hash update_record_hash get_all_records delete_record dump_records undelete_record =item Creating cursors and working with them prepare_select fetch prepare_select_with_index fetch_hashref prepare_select_where last_fetched find_eq cursor_uses_index =back =head1 General methods The general methods working with the whole files or tables. =head2 new Opens the existing dbf file and provides an object to interact with the table. Memo and index files are also opened transparently. If opening of the dbf file or any other needed file (memo, index) fails, C<new> returns undef and the error message may be retrieved via B<XBase-E<gt>errstr>. The parameters to B<new> are passed as hash: =over 4 =item name Name of the dbf file (dbf table). The C<.dbf> suffix may be omitted. The name of the file may also be passed as the very first single parameter. =item memofile Specifies non standard name for the associated memo file. By default it's the name of the dbf file, with suffix C<.dbt>, C<.fpt> or C<.smt>. =item ignorememo Makes B<new> and all subsequent operation to ignore memo file at all. This is usefull if you've lost the dbt file and you do not need it. The default is undef, not ignoring the memo file. =item memosep Separator of memo records in the dBase III memo files. The standard says it should be C<"\x1a\x1a">. There are however implementations that only put in one C<"\x1a">. XBase.pm tries hard to guess which is the case for your dbt but if it fails, you can tell it yourself. =item nolongchars Prevents B<new> to treat the decimal value of character fields as high byte of the length -- there are some broken products around producing character fields with decimal values set. XBase.pm tries hard to guess which is the case for your dbf, so again you need this option only if it fails. =item index Name of arrayref of names of index files to attach to the opened object. The cdx, mdx and SDBM indexes are attached by default. =item noindex Prevents any index file to be attached automatically (cdx, mdx, SDBM). Default is undef. =item PrintError If the B<new> or any subsequent call to the object fail, they generate a warning using warn. The default is undef (but future versions may default to 1). =item RaiseError If the B<new> or any subsequent call to the object fail, they raise an exception (die). The default is undef. =back Examples: my $table = new XBase "table.dbf" or die XBase->errstr; my $table = new XBase 'name' => 'table.dbf', 'index' => 'table.ndx', 'PrintError' => 1; =head2 create Creates new empty dbf file on disk; memo file will be also created if the table contains some memo fields. Parameters to create are passed as hash. =over 4 =item name Name of the new dbf file. =back You can call this method as method of another XBase object and then you only need to pass B<name> value of the hash; the structure (fields) of the new file will be the same as of the original object. If you call B<create> using class name (XBase), you have to (besides B<name>) also specify the structure of the file: =over 4 =item field_names Arrayref to list of field name. =item field_types Arrayref to list of field types, specified either by one letter strings (C, N, L, D, ...) or by long versions (char, numeric, date, ...) =item field_lengths Arrayref to list of field widths. =item field_decimals Arrayref to list of precissions, for numeric columns. =item version Force different version of the dbf or memo file. The default is version of the source table (if you call B<create> on an object), 3 (dBase III compatible) otherwise. =item memofile Specify nonstandard memo file name or location. =back If you keep some value undefined, B<create> will make it into some reasonable default. The new dbf file (nor memo file) mustn't exist yet -- B<create> will not allow you to overwrite existing table. Use B<drop> (or unlink) to delete it first. my $newtable = $table->create("name" => "copy.dbf"); my $newtable = XBase->create("name" => "new.dbf", "field_names" => [ "ID", "MSG" ], "field_types" => [ "N", "C" ], "field_lengths" => [ 6, 40 ], "field_decimals" => [ 0, undef ]); =head2 attach_index The index file may be attached during the B<new> call or additionally with this call. =head2 pack All records that were marked deleted in the table, will be purged from the file. Effectively does a fresh copy to new file and then moves it to original location, so is not aimed at efficiency. Also recreates all attached index files. =head2 close Closes the object and associated memo file and attached index files, no arguments. =head2 drop This method closes the table and deletes it on disk (including associated memo file and attached index files, if there are any). =head2 errstr Called either as a class method (after B<new> or B<create>) or on a table object, it returns error string describing the last error of previous failed method call. =head1 Information about dbf file =head2 last_record Returns number of the last record in the file. The records marked deleted but present in the file are included in this number. =head2 last_field Returns number of the last field in the file, number of fields minus 1. =head2 header_info Returns string with formated information about the file and about the fields. =head2 field_names, field_types, field_lengths, field_decimals Return list of field names and so on for the dbf file. =head2 field_type, field_length, field_decimal For a field name, returns the appropriate value. Returns undef if the field doesn't exist in the table. =head1 Accessing the records When dealing with the records one by one, reading or writing, you have to specify the number of the record in the file as the first argument. The valid range is from 0 to C<$table-E<gt>last_record> and C<$table-E<gt>last_record+1> to insert new record to the file. =head2 get_record Returns list of field values from the specified record. The first parameter in the call is the number of the record. If you do not specify any other parameters, all fields are returned in the same order as they appear in the file. You can also put list of field names after the record number and then only those will be returned. The first value of the returned list is always the 1/0 C<_DELETED> value saying whether the record is marked deleted or not, so on success, B<get_record> never returns empty list. =head2 get_record_as_hash Like B<get_record>, but returns the values as hash (in list context) or reference to hash (in scalar context) containing field values indexed by field names. The name of the deleted flag is C<_DELETED>. The field names are returned as uppercase. =head2 get_record_nf Like B<get_record> but instead if the names of the fields, you can pass list of numbers of the fields to read. =head2 get_all_records Returns reference to an array containing array of values for each undeleted record at once. As parameters, pass list of fields to return for each record. =head2 dump_records Prints to currently selected filehandle all non-deleted records from the file. By default, all fields are printed, separated by colons, one record on a row. The method can have parameters in a form of a hash with the following keys: =over 4 =item rs Record separator, string, newline by default. =item fs Field separator, string, one colon by default. =item fields Reference to a list of names of the fields to print. By default it's undef, meaning all fields. You can also pass in scalar where the field names are separated by commas, or by dashes to denote intervals. =item undef What to print for undefined (NULL) values, empty string by default. =back Example of use is use XBase; my $table = new XBase "table" or die XBase->errstr; $table->dump_records("fs" => " | ", "rs" => " <-+\n", "fields" => [ "id", "msg" ]);' Also note that there is a command line script dbfdump.pl(1) that does the printing. =head1 Writing the data All three writing methods always undelete the record. On success they return true -- the record number actually written. =head2 set_record As parameters, takes the number of the record and the list of values of the fields. It writes the record to the file. Unspecified fields (if you pass less than you should) are set to undef/empty. =head2 set_record_hash Takes number of the record and hash as parameters, sets the fields, unspecified fields are undefed/emptied. =head2 update_record_hash Like B<set_record_hash> but fields that do not have value specified in the hash retain their original value. =head2 delete_record, undelete_record Marks the specified record in the file deleted/undeleted. =head1 Sequentially reading the file If you plan to sequentially walk through the file, you can create a cursor first and then repeatedly call B<fetch> to get next record. =head2 prepare_select Creates and returns an cursor to walk through the file. As parameters, pass list of field names to return, by default all fields. =head2 prepare_select_with_index The first parameter is the file name of the index file, the rest is optional list of field names. For index types that can hold more index structures in one file (have tags), instead of file name use arrayref with the file name, the tag name and optionaly the index type (at the moment, expressions are not supported, so XBase.pm won't be able to determine type of the index unless you tell it). The B<fetch> will then return records in the ascending order, according to the index. =head2 prepare_select_where The first parameter is a string with boolean expression, the rest is optional list of field names. The B<fetch>es on the returned cursor will return only records matching the expression. If there are attached index files, they may be used to speed the search. The previous methods on the table object will return cursor object, the following methods are to be called on the cursor, not on the table. =head2 fetch Returns the fields of the next available undeleted record from the cursor. The list thus doesn't contain the C<_DELETED> flag since you are guaranteed that the record is not deleted. =item fetch_hashref Returns a hash reference of fields for the next undeleted record from the cursor. =item last_fetched Returns the record number of the record last fetched. =item find_eq This only works with cursor created via B<prepare_select_with_index> or B<prepare_select_where> that uses index. As a parameter it takes the cursor value to find. It returns 1 if there is matching record, or 0 otherwise. If there is a match, the next B<fetch>es will fetch the records matching, and continue with records greater than the specified value (walk the index). If there isn't match, B<fetch> returns next greater record. =item cursor_uses_index Returns true if the cursor created with B<prepare_select_where> uses index. =head1 EXAMPLES Assorted examples of reading and writing: my @data = $table->get_record(3, "jezek", "krtek"); my $hashref = $table->get_record_as_hash(38); $table->set_record_hash(8, "jezek" => "jezecek", "krtek" => 5); $table->undelete_record(4); This is a code to update field MSG in record where ID is 123. use XBase; my $table = new XBase "test.dbf" or die XBase->errstr; for (0 .. $table->last_record) { my ($deleted, $id) = $table->get_record($_, "ID") die $table->errstr unless defined $deleted; next if $deleted; $table->update_record_hash($_, "MSG" => "New message") if $id == 123; } Examples of using cursors: my $table = new XBase "names.dbf" or die XBase->errstr; my $cursor = $table->prepare_select("ID", "NAME", "STREET"); while (my @data = $cursor->fetch) { ### do something here, like print "@data\n"; } my $table = new XBase "employ.dbf"; my $cur = $table->prepare_select_with_index("empid.ndx"); ## my $cur = $table->prepare_select_with_index( ["empid.cdx", "ADDRES"], "id", "address"); $cur->find_eq(1097); while (my $hashref = $cur->fetch_hashref and $hashref->{"ID"} == 1097) { ### do something here with $hashref } The second example shows that after you have done B<find_eq>, the B<fetch>es continue untill the end of the index, so you have to check whether you are still on records with given value. And if there is no record with value 1097 in the indexed field, you will just get the next record in the order. The updating example can be rewritten to: use XBase; my $table = new XBase "test.dbf" or die XBase->errstr; my $cursor = $table->prepare_select("ID") while (my ($id) = $cursor->fetch) { $table->update_record_hash($cursor->last_fetched, "MSG" => "New message") if $id == 123 } =head1 DATA TYPES The character fields are returned "as is". No charset or other translation is done. The numbers are converted to Perl numbers. The date fields are returned as 8 character string of the 'YYYYMMDD' form and when inserting the date, you again have to provide it in this form. No checking for the validity of the date is done. The datetime field is returned in the number of seconds since 1970/1/1, possibly with decimal part (since it allows precision up to 1/1000 s). To get the fields, use the gmtime (or similar) Perl function. If there is a memo field in the dbf file, the module tries to open file with the same name but extension dbt, fpt or smt. It uses module XBase::Memo(3) for this. It reads and writes this memo field transparently (you do not know about it) and returns the data as normal scalar. =head1 INFORMATION SOURCE This module is built using information from and article XBase File Format Description by Erik Bachmann, URL http://www.e-bachmann.dk/docs/xbase.htm Thanks a lot. =head1 VERSION 0.162 =head1 AUTHOR (c) 1997--2001 Jan Pazdziora, adelton@fi.muni.cz, http://www.fi.muni.cz/~adelton/ at Faculty of Informatics, Masaryk University in Brno, Czech Republic All rights reserved. This package is free software; you can redistribute it and/or modify it under the same terms as Perl itself. =head1 THANKS Many people have provided information, test files, test results and patches. This project would not be so great without them. See the Changes file for (I hope) complete list. Thank you all! Special thanks go to Erik Bachmann for his great page about the file structures; to Frans van Loon, William McKee, Randy Kobes and Dan Albertsson for longtime cooperation and many emails we've exchanged when fixing and polishing the module's behaviour; and to Dan Albertsson for providing support for the project. =head1 SEE ALSO XBase::FAQ(3); XBase::Index(3); DBD::XBase(3) and DBI(3) for DBI interface; dbfdump.pl(1); perl(1) =cut