Changes in ModLogAn 0.7.0 againt 0.6.x $Id: Changes-0.7.0.txt,v 1.3 2001/08/12 21:10:27 jk Exp $ Starting with ModLogAn 070 the configfile format changes again. Some internal changes were neccesary to provide a way to make ModLogAn more flexible. The two important changes that result in these Changes are: - generic file inclusion - generic plugin support For the developer the last sections are more interessing as they describe the dataflow in modlogan 070 and how they are handled. generic file inclusion ---------------------- In versions before 070 the default values where taken from the file specified by 'default_configfile'. This system was sufficent for most cases as it provided a easy way to define a global set of options for all modlogan instances. This had one big disadvantage: It was only possible to include one file in a whole. There was no way to pick a specific portion from the file nor was it possible to include multiple files. This resulted in a very huge default configfile which redefined every option for the input-plugins which can now be share between them. Every config-section got a new option called 'include'. This option is part of the configfile handling and is not specific for a plugin. Let compare a old config file with the new scheme: --> (./doc/sample.def.conf) [input_msiis] inputfile=- matchos="^Win" matchos="^Linux" matchos="^FreeBSD" matchos="^Macintosh" matchos="^PPC" matchos="^AIX" matchos="^IRIX" matchos="^SunOS" matchos="^OSF1" matchos="^OpenBSD" matchos="^Mac_PowerPC" matchos="^Solaris" matchos="^OS/2" matchos="^HP-UX" matchos="^NetBSD" matchos="^BSD/OS" matchua="^iCab" matchua="^MSIE" matchua="^StarOffice" matchua="^Lotus-Notes" matchua="^BorderManager" matchua="^Powermarks" matchua="^NetMind-Minder" matchua="^WebCapture" matchua="^Squid" matchua="^Mozilla" matchua="^Konqueror" <-- The key value pair for matchua and matchos is defined for [input_msiis], [input_netscape] and [input_clf] as all handle logfile that provide this kind of information. With the new 'include' directive these options are share between a all section: --> [input_msiis] include = match.os.conf,matchos include = match.ua.conf,matchua inputfile = - <-- The options were exported to a external file 'match.os.conf' and 'match.ua.conf'. The section [matchos] from the external file is now included into the section [input_msiis]. The same procedure is used for the matchua options. The most important result for these kind of options (match*, group*) is a external database of strings which is already available for the searchengine strings. While the 'groupreferrer' section is specific for one site, the grouphost option can be share between all users that use ModLogAn. Currently this database is rather small. It contains three entries and is shipped with the regular version of ModLogAn. #grouphosts="\.dip\.t-dialin\.net$",dip.t-dialin.net #grouphosts="\.arcor-ip\.net$",arcor-ip.net #grouphosts="\.Concordia\.CA$",Concordia.CA These entries are disabled by default as they were just examples. Now it is time to extend this database and add more entries. But there is one side effect that effects the speed. The more strings are added to the database the more time spend in comparing these strings with the current 'requesting host'. This can result in a notable slowdown. That's why this database will be structured in some way. There is now sense in check for German provider if your site is commonly used by Japanese as most Germans can't read those symbols (most browser can't even display them, but that's another topic). That's why I propose a the following scheme: - the host groupings are grouped by country, region, continent or language - these groups are combined into grouping classes --> (group.host.conf) [group_host_all] <-- a grouping class include = group.host.conf,de include = group.host.conf,eng include = group.host.conf,fr [de] <-- host grouping by language grouphosts = "\.dip\.t-dialin\.net$",dip.t-dialin.net grouphosts = "\.arcor-ip\.net$",arcor-ip.net [eng] grouphosts = "\.Concordia\.CA$",Concordia.CA [fr] <-- In your configfile you just include either the 'group_host_all' section or the small sections (de, fr, eng). [processor_web] include = group.host.conf,group_host_all ... or if you don't need the german ones: [processor_web] include = group.host.conf,eng include = group.host.conf,fr ... So much for the topic 'generic file inclusion'. As you can see the new system is far more flexible then the old one. That's why the option 'default_configfile' is gone now. But let's take a look at another really nice new feature: generic plugin support ---------------------- If you take a look at the plugin handling you see three options in your configfile: [global] inputplugin = clf outputplugin = modlogan processorplugin = web It was possible to load ONE input plugin + ONE processor plugin + ONE output plugin. For most cases this is enough as you just have one logfile which shall be transformed into a nice HTML-graphic. Some users sent in a request for handling multiple logfiles which should merge together to generate a report for the whole server. In the old version you had to use an external programm like 'mergelog' to combine the logfiles before they where piped into ModLogAn. The new plugin handling code gives you the possiblity to do it directly in ModLogAn. You can load one plugin multiple times which then uses a different config section to setup it's state (inputfile). [global] loadplugin = input_clf, input_file_1 loadplugin = input_clf, input_file_2 loadplugin = input_clf, input_file_3 loadplugin = input_clf, input_file_4 loadplugin = processor_web loadplugin = output_modlogan ... After the global section is parsed the plugins are loaded and the corresponding configsections are parsed: 1. the input plugin clf is loaded and the section 'input_file_1' is used for the configuration 2. the input plugin clf is loaded again, but the section 'input_file_2' is used for the configuration, same for input_file_3 and input_file_4 3. the processor 'web' is loaded and the section 'processor_web' is used for the configuration 4. the report generator 'modlogan' is loaded and the section 'output_modlogan' is used for the configuration As you can see the strict binding between sectionname and plugin is gone. The pluginname is only used as sectionname if no explicit sectionname was provided. This change is not without a side effect. As the plugins have to be loaded at runtime the static version of modlogan is gone. The static version of ModLogAn existed for one reason: In most cases libgd was only available as a static lib (libgd.a). Starting with GD 2.0 libgd compiles as a shared library. The configure script already checks if your system provides the neccesary libraries as a shared-object file (.so). It will be intresting to see how the new scheme compiles on Windows. datatype handling ----------------- ...
