Changes in Pyrite 0.7.0
-----------------------

The changes in Pyrite 0.7.0, relative to previous releases, are very
substantial.  In fact, so many things have changed that it might be
wise to think of Pyrite 0.7.0 as an entirely new package.

(My thinking is this: since Pyrite is still young and the number of
applications using it appears to be small, it seems better to get all
of these major changes out of the way at the same time, instead of
spreading them out over a number of releases.  This way, the major
hassles will be over with until post-1.0; from here on out, any
changes will be refinements, not upheavals.)

Please read this entire document before installing or using 0.7.0.
If you have been developing with Pyrite 0.6.x or older, you WILL have
to rewrite your code to use future versions.  Although there will
undoubtedly be changes and additions to Pyrite's APIs, the basic
design of Pyrite 0.7.0 is the way it will be for 1.0 and beyond.

* Installation

Pyrite now expects to be a top-level package; you should now import
"Pyrite" rather than "PDA.Palm".  The Makefile will install a symlink
from .../PDA/Palm to .../Pyrite, but only if there isn't one there
already.  This means that if you have a previous version of Pyrite
installed, it should not be disturbed when you install the new one.

The binary modules are now installed in the "glue" subdirectory of
Pyrite, rather than being in a platform-specific directory.  The
rationale for this change is described in doc/platform-dir.txt.

The configure script now copies Makefile.pre.in from the existing
Python installation, instead of using one supplied in the package.  It
also runs the Makefile.pre.in automatically, so you should be able to
simply do "configure ; make ; make install" to install Pyrite.

* Configuration Files

Configuration files are now in a new format, which will hopefully be
more flexible.  (I never liked the .INI type format anyway.)  The new
format is inspired by the configuration file formats used by the BIND
nameserver, and Debian GNU/Linux's new package manager (apt).  The
format is very simple:

  - every option has a path; a C++-like scoping operator (::) is used
    as the path separator.  For example, the full path to the 'debug'
    option in the Backup conduit is 'Pyrite::Conduit::Backup::debug'.

  - the configuration file is a simple sequence of names and values,
    delimited by whitespace.  The only "punctuation" are semicolons,
    which separate name/value pairs, and curly braces, which denote
    scope.

  - names are case-sensitive.  (this might change in the future)

  - if you need to put whitespace, braces, semicolons, or anything
    other potentially confusing character in a name or value, you can
    enclose it in ordinary quotes ("..."); within a quoted string, the
    sequence \" will insert a quote.  (Other escapes may be added in
    the future.)

  - the parser is really simple, assuming that if you present it with
    a list of N strings, that the first N-1 of them are components of
    the name, and the Nth is the value.  Thus, you can do
    constructions like:

    string1 "string2" {
        string3 "value";
        ...
    }

    ... which sets "string1::string2" as the scope for everything in
    the braces.

    Of course, you can use the scoping operator explicitly; the above
    could be rewritten as:

    string1::string2 {
        string3 "value";
    }

  - If you specify values without names, they will be applied to the
    surrounding scope as a *list*, e.g.:

    ports {
        "/dev/ttyS0";
        "/dev/ttyS1";
        "/dev/ttyS2";
    }

  - There isn't any inheritance, or any smart stuff like that; this is
    really just a way to describe what amounts to a multi-leveled
    dictionary.

* Sulfur

Sulfur is the Pyrite application framework.  It has its own name and
its own subpackage because nothing in it is Pyrite-specific, and it
will soon be released on its own.  In essence, Sulfur contains the
functionality that used to be in the Plugin, Config, and Options
modules, plus a bunch of new stuff.

The core of Sulfur is the new Application class, which provides
runtime access to other application management functions (plugins,
mostly).  A more in-depth discussion of Sulfur is found in the file
Sulfur/README; a preliminary manual can be found in Sulfur/doc.

The Pyrite.Application module contains a Pyrite-specific subclass of
the Sulfur.Application class.  Pyrite-based applications should derive
their own subclasses from it, rather than from Sulfur.Application,
because the Pyrite-specific version has added functionality for
dealing with the Pyrite data repository, remote connections, and so
on.

* Stores

In the previous releases, you may have noticed some skeletal plug-ins
in the "Store" package.  They have been reorganized and fleshed out,
and are now useful.  Stores are a location-independent way of getting
at a collection of Palm OS databases -- the basic interface to a Store
is the same whether the database collection is in a local directory,
at the other end of a synchronization cable, or even someplace more
exotic like on an SQL server.  At present, two Store plug-ins are
available: "DLP", which connects to a docked handheld, and
"Directory", which accesses databases in an ordinary directory.

The Store database management interface is simple, much like the
Database Manager in the Palm OS -- each Store is a flat collection of
databases, referenced by their full name (and possibly creator and
type).  Translation between this simple interface and the underlying
data storage method is transparent to the caller.

The return from a Store's "open" or "create" method is a Database
object, just like earlier versions of Pyrite.  However, Stores *do
not* automatically subclass the database based on type information.
Instead, the equivalent functionality has been moved into the App
plug-ins; to open the Address database, for example, you don't open
the database by its filename and let Pyrite return the appropriate
class, you ask the Address plug-in to open its standard database,
whatever that happens to be.

One reason for this change is to simplify the Store code; another
reason is to support applications which don't have an exact mapping
between raw databases and the Database objects they return.  Some
handheld applications, for example, extend the capabilities of the
standard ones by adding extra databases in parallel with the standard
MemoDB, AddressDB, ToDoDB, etc.  Moving the app-specific opening code
into the application's plug-in will allow these multiple databases to
be virtualized behind a single Database object.

Incidentally, installation and retrieval of databases on the handheld
is now done in pure Python, instead of calling pilot-link.  A general
'copy' method on all Stores exists to move databases from one Store to
another, including retrieval from the handheld to a local Store.
Because installation requires a bit of extra behavior (handling of
forced resets and the like) the DLP Store has a pure Python 'install'
method as well.  Once the "prc" module is tested and shown to work, it
will be possible to eliminate all dependency on pilot-link for local
pdb/prc file access.

* Conduits and Synchronization

The addition of Sulfur and Stores has resulted in changes to the
synchronization process.  The "Sync" conduit is now obsolete; its
functionality (opening and closing connections, loading other
conduits) is now available in the Pyrite Application class, and the
actual synchronization loop is now in the sync utility.

Conduits are now called with the Application as their only parameter,
instead of a DLP connection object and username as before.  The
Application object contains user information and a DLP Store object,
which the conduit may use to access the handheld; if the conduit needs
access to local data, it may obtain a local Store object from the
application.

All three of the standard conduits have been rewritten to use this new
calling sequence.

* New App Plug-in API

The App module API has been extended to support the reorganization
described above.  Since 0.6.0, App modules have been plug-ins; with
this release, they become *useful* as plug-ins.

Each App plug-in has new methods to open, create, and list databases
(in a particular store).  The open and create methods use default
values when possible, meaning it is now easy to open the standard
database for a particular application, or create databases with the
proper creator and type.  For example, if you do not specify a
database name to the Memo plug-in, you will get the database called
'MemoDB', as an appropriately-classed Database object.

App plug-ins also have 'list' and 'listinfo' methods which do the same
thing as their counterparts in a Store, but which only list databases
which that particular plug-in can handle.  For example, you can now
list all Doc databases by calling the Doc plug-in's 'list' or
'listinfo' method.  In most cases this is equivalent to calling the
Store's 'listinfo' method with a particular creator and type, but
using the App plug-in is potentially more flexible.  Since the App
plug-in uses its own 'classify' method to choose databases to handle,
it is easy to deal with multiple creator/type pairs.  For example, the
Doc plug-in can handle both TealDoc and ordinary Doc databases, and
both will be returned in a single call to the Doc plug-in's 'list' or
'listinfo' method.

* The Future

My focus in the 0.7.x series will be replacing C with Python for local
file access, fleshing out the Backup conduit a bit more, adding App
modules for Mail and Expense, and working on the documentation.

After that, the 0.8.x series is likely to focus on record-level
synchronization, and new Store backends (a dbm-based one for
efficiency, at least).

And after that, it will be time to release 1.0, and fork development
into a "stable" and "unstable" branch...