.. _glossary:

========
Glossary
========

.. glossary::

    Analysis
        The process of breaking the text of a field into individual *terms*
        to be indexed. This consists of tokenizing the text into terms, and then optionally
        filtering the tokenized terms (for example, lowercasing and removing *stop words*).
        Whoosh includes several different analyzers.

    Corpus
        The set of documents you are indexing.

    Documents
        The individual pieces of content you want to make searchable.
        The word "documents" might imply files, but the data source could really be
        anything -- articles in a content management system, blog posts in a blogging
        system, chunks of a very large file, rows returned from an SQL query, individual
        email messages from a mailbox file, or whatever. When you get search results
        from Whoosh, the results are a list of documents, whatever "documents" means in
        your search engine.

    Fields
        Each document contains a set of fields. Typical fields might be "title", "content",
        "url", "keywords", "status", "date", etc. Fields can be indexed (so they're
        searchable) and/or stored with the document. Storing the field makes it available
        in search results. For example, you typically want to store the "title" field so
        your search results can display it.

    Forward index
        A table listing every document and the words that appear in the document.
        Whoosh lets you store *term vectors* that are a kind of forward index.

    Indexing
        The process of examining documents in the corpus and adding them to the
        *reverse index*.

    Postings
        The *reverse index* lists every word in the corpus, and for each word, a list
        of documents in which that word appears, along with some optional information
        (such as the number of times the word appears in that document). These items
        in the list, containing a document number and any extra information, are
        called *postings*. In Whoosh the information stored in postings is customizable
        for each *field*.

    Reverse index
        Basically a table listing every word in the corpus, and for each word, the
        list of documents in which it appears. It can be more complicated (the index can
        also list how many times the word appears in each document, the positions at which
        it appears, etc.) but that's how it basically works.

    Schema
        Whoosh requires that you specify the *fields* of the index before you begin
        indexing. The Schema associates field names with metadata about the field, such
        as the format of the *postings* and whether the contents of the field are stored
        in the index.

    Term vector
        A *forward index* for a certain field in a certain document. You can specify
        in the Schema that a given field should store term vectors.