INN Python Filtering Support
This is $Revision: 1.2 $, dated $Date: 1999/09/23 14:23:48 $.
This file documents INN's built-in optional support for Python
article filtering. It is patterned after the TCL and Perl hooks
previously added by Bob Heiney and Christophe Wolfhugel.
For this filter to work successfully, you will need to have Python
1.5.2 (the latest at this writing) installed. You can obtain it
from <URL:http://www.python.org>.
NOTE TO RED HAT LINUX USERS:
Python will be preinstalled, but it may not include all the
headers and libraries required for embedding into INN. You will
need to add the development package. Better yet, get the source
kit from the above URL and build it yourself. Be sure when
installing Python on Red Hat, to run configure with
'--prefix=/usr' so that there are no version conflicts with the
"factory" installation. You can also find a selection of well
made RPMs at <URL:ftp://starship.python.net/pub/crew/andrich/>
INSTALLATION:
Once you have built and installed Python, you can cause INN to use
it by adding the '--with-python' switch to your configure command.
See the ctlinnd(8) manual page to learn how to enable, disable and
reload Python filters on a running server ('ctlinnd mode',
'ctlinnd python y|n', 'ctlinnd reload filter.python').
Also, see the example filter_innd.py script in your filters
directory for a demonstration of how to get all this working.
WRITING AN INND FILTER:
You need to create a filter_innd.py module in INN's filter
directory (see the pathfilter setting in inn.conf). A
heavily-commented sample is provided that you can use as a
template for your own filter. There is also an INN.py module
there which is not actually used by INN; it is there so you
can test your module interactively.
First, define a class containing the methods you want to provide
to innd. Methods innd will use if present are:
__init__(self):
Not explicitly called by innd, but will run whenever the
filter module is (re)loaded. This is a good place to
initialize constants or pick up where filter_before_reload
or filter_close left off.
filter_before_reload(self):
This will execute any time a 'ctlinnd reload all' or
'ctlinnd reload filter.python' command is issued. You can
use it to save statistics or reports for use after
reloading.
filter_close(self):
This will run when a 'ctlinnd shutdown' command is received.
filter_art(self, art):
art is a dictionary containing an article's headers and
body. This method is called every time innd receives an
article. The following can be defined.
Approved, Control, Date, Distribution, Expires, From,
Lines, Message-ID, Newsgroups, Path, Reply-To, Sender,
Subject, Supersedes, Bytes, Also-Control, References,
Xref, Keywords, X-Trace, NNTP-Posting-Host,
Followup-To, Organization, Content-Type, Content-Base,
Content-Disposition, X-Newsreader, X-Mailer,
X-Newsposter, X-Cancelled-By, X-Canceled-By,
Cancel-Key, __LINES__, __BODY__
All the above values will be buffer objects holding the
contents of the same named article headers, except for the
special __BODY__ and __LINES__ items. Items not present
in the article will contain None.
__BODY__ is a buffer object containing the article's
entire body, and __LINES__ is an int holding innd's
reckoning of the number of lines in the article. All the
other elements will be buffers with the contents of the
same-named article headers.
If you want to accept an article, return None or an empty
string. To reject, return a non-empty string. The
rejection strings will be shown to local clients and your
peers, so keep that in mind when phrasing your rejection
responses.
filter_messageid(self, msgid):
msgid is a buffer object containing the ID of an article
being offered by IHAVE or CHECK. Like with filter_art(),
the message will be refused if you return a non-empty
string. If you use this feature, keep it light because it
is called at a rather busy place in innd's main loop.
Also, do not rely on this function alone to reject by ID;
you should repeat the tests in filter_art() to catch
articles sent with TAKETHIS but no CHECK.
filter_mode(self, oldmode, newmode, reason):
When the operator issues a ctlinnd pause, throttle or go
command, this function can be used to do something
sensible in accordance with the state change. Stamp a log
file, save your state on throttle, etc. oldmode and
newmode will be strings containing one of the values in
('running', 'throttled', 'paused', 'unknown') -- oldmode
is the state innd was in before ctlinnd was run, newmode
is the state innd will be in after the command finishes.
reason is the comment string provided on the ctlinnd
command line.
To register your methods with innd, you need to create an instance
of your class, import the built-in INN module, and pass the
instance to INN.set_filter_hook(). For example:
class Filter:
def filter_art(self, art):
...
blah blah
...
def filter_messageid(self, id):
...
yadda yadda
...
import INN
myfilter = Filter()
INN.set_filter_hook(myfilter)
When writing and testing your Python filter, don't be afraid to
make use of try:/except: and the provided INN.syslog() function.
stdout and stderr will be disabled, so your filter will die
silently otherwise.
Also, remember to try importing your module interactively before
loading it, to ensure there are no obvious errors. One typo can
ruin your whole filter. A dummy INND.py module is provided to
facilitate testing outside the server. To test, change into your
filter directory and use a command like:
python -ic 'import INN, filter_innd'
You can define as many or few of the methods listed above as you
want in your filter class (it's fine to define more methods for
your own use; innd won't use them but your filter can). If you
*do* define the above methods, GET THE PARAMETER COUNTS RIGHT.
There are checks in innd to see if the methods exist and are
callable, but if you define one and get the parameter counts
wrong, INND WILL DIE. You have been warned. Be careful with your
return values, too. The filter_art() and filter_messageid()
methods have to return strings, or None. If you return something
like an int, innd will *not* be happy.
WHAT'S THE DEAL WITH THESE BUFFER OBJECTS?
Buffer objects are cousins of strings, new in Python 1.5.2. They
are supported, but at this writing you won't yet find much about
them in the Python documentation. Using buffer objects may take
some getting used to, but we can create buffers much faster and
with less memory than strings.
For most of the operations you will perform in filters (like
re.search, string.find, md5.digest) you can treat buffers just
like strings, but there are a few important differences you should
know about:
# Make a string and a two buffers.
s = "abc"
b = buffer("def")
bs = buffer("abc")
s == bs # - This is false because the types differ...
buffer(s) == bs # - ...but this is true, the types now agree.
s == str(bs) # - This is also true, but buffer() is faster.
s[:2] == bs[:2] # - True. Buffer slices are strings.
# While most string methods will take either a buffer or string,
# string.join insists on using only strings.
string.join([str(b), s], '.') # returns 'def.abc'
e = s + b # This raises a TypeError, but...
# ...these two both return the string 'abcdef'. The first one
# is faster -- choose buffer() over str() whenever you can.
e = buffer(s) + b
f = s + str(b)
g = b + '>' # This is legal, returns the string 'def>'.
FUNCTIONS SUPPLIED BY THE BUILT-IN INN MODULE:
Not only can innd use Python, but your filter can use some of
innd's features too. Here is some sample Python code to show what
you get:
import INN
# Python's native syslog module isn't compiled in by default,
# so the INN module provides a replacement. The first parameter
# tells the Unix syslogger what severity to use; you can
# abbreviate down to one letter and it's case insensitive.
# Available levels are (in increasing levels of seriousness)
# Debug, Info, Notice, Warning, Err, Crit, and Alert. (If you
# provide any other string, it will be defaulted to Notice.) The
# second parameter is the message text. The syslog entries will
# go to the same log files innd itself uses, with a 'python:'
# prefix.
syslog('warning', 'I will not buy this record. It is scratched.')
animals = 'eels'
vehicle = 'hovercraft'
syslog('N', 'My %s is full of %s.' % (vehicle, animals))
# Let's cancel an article! This only deletes the message on the
# local server; it doesn't send out a control message or anything
# scary like that. Returns 1 if successful, else 0.
if INN.cancel('<meow$123.456@solvangpastries.edu>'):
canceled = "yup"
else:
canceled = "nope"
# Check if a given message is in history. This doesn't
# necessarily mean the article is on your spool; canceled and
# expired articles hang around in history for a while, and
# rejected articles will be in there if you have enabled
# remember_trash in inn.conf. Returns 1 if found, else 0.
if INN.havehist('<z456$789.abc@isc.org>'):
comment = "*yawn* I've already seen this article."
else:
comment = 'Mmm, fresh news.'
# Here we are running a local spam filter, so why eat all those
# cancels? We can add fake entries to history so they'll get
# refused. Returns 1 on success, 0 on failure.
canceled_id = buffer('<meow$123.456@isc.org>')
if INN.addhist("<cancel." + canceled_id[1:]):
thought = "Eat my dust, roadkill!"
else:
thought = "Darn, someone beat me to it."
# We can look at the header or all of an article already on spool,
# too. Might be useful for long-memory despamming or
# authentication things. Each is returned (if present) as a
# string object; otherwise you'll end up with an empty string.
artbody = INN.article('<foo$bar.baz@bungmunch.edu>')
artheader = INN.head('<foo$bar.baz@bungmunch.edu>')
# Finally, do you want to see if a given newsgroup is moderated or
# whatever? INN.newsgroup returns the last field of a group's
# entry in active as a string.
froupflag = INN.newsgroup('alt.fan.karl-malden.nose')
if froupflag == '':
moderated = 'no such newsgroup'
elif froupflag == 'y':
moderated = "nope"
elif froupflag == 'm':
moderated = "yep"
else:
moderated = "something else"
=-=-=
This document and the innd Python interface were written by Greg
Andruk (nee Fluffy) <gerglery@usa.net>.
