WWWOFFLE - World Wide Web Offline Explorer - Version 2.6
          ========================================================


The progam UdmSearch is a free (GPL) internet indexing and search program.  The
UdmSearch documentation describes itself as follows:

        UdmSearch is a full-featured search engine software for intranet and
        internet servers. UdmSearch is a free software covered by the GNU
        General Public License.

        UdmSearch has a number of unique features, which makes it appropriate
        for a wide range of applications from searches within your site to
        specialized search systems such as cooking recipes or newspaper
        searches, ftp archive searches, news articles searches, etc.

I have written WWWOFFLE so that UdmSearch can be used with it to allow the
entire cache of pages can be indexed.  There are three stages to using the
program that are described in this document; installation, indexing and
searching.


Installing UdmSearch
--------------------

Note: If you already have version 3.0.0 or later of udmsearch installed and
      working then you can skip this section.

To be able to use this program it must be installed.  The instructions below
give a step-by-step guide to this process assuming that version 3.0.10 of
UdmSearch is used, later versions should also work.

1) Get the UdmSearch source code

Download the source for the UdmSearch programs from

        http://search.mnogo.ru/


2) Unpack the source code

Use

        tar -xvzf udmsearch-3.0.10.tar.gz

to create the directory udmsearch-3.1.10 with the program source files in.


3) Configure the UdmSearch program

Move to the udmsearch-3.0.10 directory and run the configuration program

        cd udmsearch-3.0.10
        ./configure --disable-syslog


4) Compile

Run make to compile UdmSearch

        make
        make install

This will compile and install it.  Any problems at this stage will require the
use of the UdmSearch documentation to solve.


Configure WWWOFFLE to run with UdmSearch
----------------------------------------

The configuration files for the UdmSearch programs as used with WWWOFFLE will
have been installed in /var/spool/wwwoffle/html/search/udmsearch/conf when
WWWOFFLE was installed.  The scripts used to run the udmsearch programs will
have been installed in /var/spool/wwwoffle/html/search/udmsearch/scripts when
WWWOFFLE was installed.

These files should be correct if the information in the WWWOFFLE Makefile
(LOCALHOST and SPOOLDIR) was set correctly.  Check them, they should have the
spool directory and the proxy hostname and port set correctly.

Also they should be checked to ensure that the UdmSearch programs are on the
path (you can edit the PATH variable here if they are not in /usr/local/bin).

One bug that appears to be in version 3.0.19 of UdmSearch means that the line in
the robots.txt file that says

        Disallow: /index

needs to be removed or commented out.


Configure database to work with UdmSearch
-----------------------------------------

MySQL
- - -

Create the MySQL database using the 'mysqladmin' command

$ mysqladmin create udmsearch

Setup the database structure for the udmsearch database.

$ mysql udmsearch < udmsearch-3.0.10/create/mysql/create.txt
$ mysql udmsearch < udmsearch-3.0.10/create/mysql/crc-multi.txt
$ mysql udmsearch < udmsearch-3.0.10/create/stopwords/stop.en.txt



Postgres SQL
- - - - - -

The Postgres database server needs to be configured so that it users TCP/IP and
so that access is allowed from the host that the udmsearch program will be run
from.

The option PGALLOWTCPIP=yes in postmaster.init needs to be set to allow TCP/IP
access.

The options PGFSYNC=no in postmaster.init needs to be set to get good
performance.

You will need to create a database user and set up the database for udmsearch.

$ /usr/bin/createuser -U postgres --createdb --no-adduser wwwoffle
$ createdb -U wwwoffle udmsearch

Setup the database structure for the udmsearch database.

$ psql -U wwwoffle udmsearch < udmsearch-3.0.10/create/pgsql/create.txt
$ psql -U wwwoffle udmsearch < udmsearch-3.0.10/create/pgsql/crc-multi.txt
$ psql -U wwwoffle udmsearch < udmsearch-3.0.10/create/stopwords/stop.en.txt


Indexing
--------

Indexing is the name that is given to the process of searching through the
web-pages to make the search database.

To work with WWWOFFLE I have produced my own scripts that should be used
to call the UdmSearch indexer.

   /var/spool/wwwoffle/html/search/udmsearch/scripts/wwwoffle-udmsearch-full
   /var/spool/wwwoffle/html/search/udmsearch/scripts/wwwoffle-udmsearch-incr

The first of these scripts will do a full search and index all of the URLs in
the cache.  The second one will do an incremental search and will only index
those that have changed since the last full search was done.


Searching
---------

The search page for UdmSearch is at http://localhost:8080/search/udmsearch/ and
is linked to from the "Welcome Page".  The word or words that you want to search
for should be entered here.

This form actually calls the script

   /var/spool/wwwoffle/html/search/udmsearch/scripts/wwwoffle-udmsearch

to do the searching so it is possible to edit this to modify it if required.


Thanks to
---------

Thanks to Volker Wysk <vw@volker-wysk.de> for providing the initial information
about using UdmSearch.  I have used his useful e-mail about how to configure the
UdmSearch program and MySQL in this document (with modifications).



Andrew M. Bishop
20th Aug 2000