2010-08-10 Michael Schwendt <mschwendt@fedoraproject.org>
* Release of 0.4.5.8
* common/LockFile.py
- Don't use os.SEEK_SET which is available since Python 2.5 only.
2010-08-04 Michael Schwendt <mschwendt@fedoraproject.org>
* common/SSLConnection.py
- Fix makefile() default args for Python 2.7 httplib.
- Don't crash in sendall() with memoryview data.
* common/XMLRPCServerProxy.py
- Fix make_connection() for Python 2.7 xmlrpclib.
2010-05-05 Michael Schwendt <mschwendt@fedoraproject.org>
* server/ArchJob.py
- Let job fail for FileDownloader.FileNameException errors,
or else it would succeed without having downloaded all files
e.g. if Mock is run with createrepo_on_rpms = True.
* builder/builder.py
- Always ignore 'repodata' directory in build results dir.
2009-06-12 Michael Schwendt <mschwendt@fedoraproject.org>
* common/SSLConnection.py
- Catch SSL.SysCallError exceptions for unexpected EOF.
- Remove superfluous connection shutdown() call in close()
because it results in an SSL.Error "uninitialized" exception
with pyOpenSSL 0.7 and Python 2.6
2008-12-16 Michael Schwendt <mschwendt@fedoraproject.org>
* common/SSLCommon.py
- Catch EINTR in serve_forever to avoid breakage with
Python 2.6.
* common/FileDownloader.py
- Fix dl_callback number of args for test suite.
* builder/builder.py
- Fix mock state file timeout error msg traceback.
- With Python >= 2.6 use "hashlib" not "sha" to
avoid the ugly deprecation warning.
2008-12-10 Michael Schwendt <mschwendt@fedoraproject.org>
* server/Repo.py
- With Python >= 2.6 use "subprocess" not "popen2" to
avoid the ugly deprecation warning.
2008-11-17 Michael Schwendt <mschwendt@fedoraproject.org>
* common/SSLConnection.py
- Fix initial reference count (set to 1)
and the comments about shutdown().
2008-11-13 Michael Schwendt <mschwendt@fedoraproject.org>
* builder/builder.py
- In start_new_job() release building_jobs_lock already
before the max_jobs check.
2008-11-05 Michael Schwendt <mschwendt@fedoraproject.org>
* Release of 0.4.5.7
* server/BuildMaster.py
- Fix class definition of PeriodicJob for Python 2.4.
2008-09-29 Michael Schwendt <mschwendt@fedoraproject.org>
* server/Repo.py
- Replace the repository write-lock with the LockFile module.
- Add server option "locktype" in section "General".
Can be set to "posix" or "flock" (the default).
* common/LockFile.py
- Add this based on an older version used by the pushscript.
It adds POSIX fcntl() write-lock support.
2008-09-20 Michael Schwendt <mschwendt@fedoraproject.org>
* Release of 0.4.5.6
* server/DBManager.py
- PostgreSQL uses different column rename syntax, too.
- SQLite doesn't support column rename, so reconstruct
a new table from the old one (only if engine==sqlite).
2008-09-08 Michael Schwendt <mschwendt@fedoraproject.org>
* Release of 0.4.5.5
* server/DBManager.py
- Fix column rename if no tables exist.
- Fix traceback in db engine not found error message.
2008-09-07 Michael Schwendt <mschwendt@fedoraproject.org>
* Release of 0.4.5.4
* server/BuildMaster.py
- Ping the database periodically. Configurable via option
"ping_interval" (seconds) in section "Database".
This is a cheap way to work around MySQL idle disconnect.
- Make repodir a member (_repodir).
* www/job.psp, server/DBManager.py, server/BuildMaster.py,
* server/PackageJob.py, server/UserInterface.py
- Prefix jobs table columns "epoch", "version", "release"
with "rpm_" because RELEASE is a new reserved keyword in
MySQL 5 and the quote character differs from Postgresql.
At startup, DBManager alters the table automatically
for compatibility with existing databases.
* utils/user-manager.py
- Fix mod_user for SQLite2/SQLite3 API.
2008-09-04 Michael Schwendt <mschwendt@fedoraproject.org>
* Release of 0.4.5.3
* server/Repo.py
- Fix _update_repo args traceback introduced in 0.4.5.1.
- Merge repolock patch.
* server/User.py
- Apply the missing SQLite2/SQLite3 API fix.
2008-09-04 Dennis Gilmore <dennis@ausil.us>
* Release of 0.4.5.2
- fix bug in find option for plague-user-manager
2008-09-03 Dennis Gilmore <dennis@ausil.us>
* Release of 0.4.5.1
- require mock > 0.8
- require createrepo > 0.4.7
- apply patches from Michael Schwendt
- plague-0.4.5-logtail.patch
- plague-0.4.5-mock-0.8.patch
- create sqlite data for createrepo
- based on plague-0.4.5-sqlite3.patch fix sqlite so it will work as
shipped on RHEL5 and Fedora, tested on RHEL5 and rawhide
2008-09-02 Dennis Gilmore <dennis@ausil.us>
* Release of 0.4.5
2008-08-23 Michael Schwendt <mschwendt@fedoraproject.org>
* client/client.py
- Remove 8K file size limit in user-cert reader.
This is too short for the new Fedora certs.
2008-06-14 Michael Schwendt <mschwendt@fedoraproject.org>
* builder/Config.py
- Change builder_work default to /var/lib/plague/builder
2008-03-05 Michael Schwendt <mschwendt@fedoraproject.org>
* server/Repo.py
- Fix debuginfo rpms exclude in _update_repo().
2008-01-18 Michael Schwendt <mschwendt@fedoraproject.org>
* server/PackageJob.py
* www/css/style.css, www/indiv.psp, www/success.psp
- Fix inaccessible downloads from builders by adding a repowait
stage after add_to_repo. Else request_copy was called twice
under race-like conditions.
2007-12-26 Michael Schwendt <mschwendt@fedoraproject.org>
* server/ArchJob.py, server/Builder.py
- Speed up downloads by forwarding callbacks between threads
and not waiting 20 seconds prior to each download.
* builder/builder.py, common/FileDownloader.py, server/ArchJob.py
- Update file download error/message handling.
* www/job.psp
- Fix the "Result" that overlapped with double-quotes.
* client/client.py, server/UserInterface.py
- Fix update_builders traceback for non-admin users.
- s/PLAUGE/PLAGUE/ for typo in env var.
Merge minor patches applied to FE buildsys:
* server/Builder.py
- Treat the builder as timed out and ping it periodically
* server/BuildMaster.py, server/DBManager.py
* server/PackageJob.py, server/Repo.py
* server/UserInterface.py, server/User.py
- Mostly db manager closing and error handling.
2007-11-14 Michael Schwendt <mschwendt@fedoraproject.org>
* server/PackageJob.py
- Fix PrepError exception and messages.
2007-11-13 Michael Schwendt <mschwendt@fedoraproject.org>
* builder/builder.py
- Apply the race condition fix in _status_prepping() which
has been fixed in Fedora Extras buildsys in September.
- Merge the patch that relies on mock 0.7 exit code and
reads the mock state for the prepping stage.
- Let _handle_death wait for mock process group.
* etc/
- Update initscripts for LSB.
- Move file definition into sysconfig scripts.
- killproc with delay 5 as clean shutdown takes some time
2007-11-11 Michael Schwendt <mschwendt@fedoraproject.org>
* server/PackageJob.py
- Run cvs pkg checkout and make srpm with umask 0022.
* client/client.py
- Fix pause command.
2007-09-30 Ville Skyttä <scop@fedoraproject.org>
* server/PackageJob.py
- Fix rpmUtils imports
2007-09-18 Michael Schwendt <mschwendt@fedoraproject.org>
* etc/plague-builder.init, etc/plague-server.init
- Fix usage help output
2007-05-20 Ville Skyttä <scop@fedoraproject.org>
* common/AuthedXMLRPCServer.py
- Fix SimpleXMLRPCDispatcher initialization for Python 2.5
2006-09-14 Dennis Gilmore <dennis@ausil.us>
* common/daemonize.py
- set umask to 0002
2006-12-17 Dan Williams <dcbw@redhat.com>
Toshio Kuratomi <toshio@tiki-lounge.com>
* common/XMLRPCServerProxy.py
- Fix transport class initialization for Python 2.5 (RH #214687)
2006-03-13 Dan Williams <dcbw@redhat.com>
* Release of 0.4.4.1
* builder/builder.py
- (main): Fix usage of handle_request when serve_forever was intended
2006-03-12 Dan Williams <dcbw@redhat.com>
* Release of 0.4.4
2006-03-12 Dan Williams <dcbw@redhat.com>
Fix up kmod support.
* server/Config.py
- Fix splitting of arches for Additonal Package Arches
- Use wildcard matching (*, ?, [...], [!...]) for package names in
Additional Package Arches so we can support kmods easily
* server/PackageJob.py
- Add lots of comments in arch_handling() so people can tell what's
going on
2006-03-12 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- Add a SIGTERM handler for clean shutdown
- Clean up lifecycle management
* common/HTTPServer.py
- Clean up lifecycle management of PlgHTTPServerManager objects
* common/SSLCommon.py
- Clean up lifecycle management of PlgBaseServer objects
- Initialize correct parent object in PlgBaseSSLServer.__init__
* etc/plague-builder.init
etc/plague-server.init
- Cleanups
* server/main.py
- Add a SIGTERM handler for clean shutdown
- Clean up lifecycle management
2006-02-26 Dan Williams <dcbw@redhat.com>
* client/client.py
- Handle more errors, and print them more nicely
2006-02-19 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- Kill jobs in 'downloaded' state after 30 minutes, because it's
likely there was an error returning the archjob's UID to the
server. In this case, the server has already gone on and
assigned the job to another builder.
- (run): Move execution of the status function outside of the try/except
block so we don't mask AttributeErrors in the code
2006-02-19 Dan Williams <dcbw@redhat.com>
* server/Builder.py
- Fix mistaken occurances of self.address -> self._address
- Print a message when errors occur starting new jobs on builders
2006-02-16 Dan Williams <dcbw@redhat.com>
* server/ArchJob.py
- Fix a few print statements that were using an unknown variable
to get the builder's address from
- Handle socket timeout errors
2006-02-16 Dan Williams <dcbw@redhat.com>
* server/ArchJob.py
- (process): don't trap AttributeErrors triggered by running the status
worker function, only trap them when getting the status worker
function. Also don't set state to 'failed' when an internal error
occurs, since that's not a valid status.
- (internal_failure): new function
* server/PackageJob.py
- (_stage_building): trap and report internal failures too
2006-02-15 Dan Williams <dcbw@redhat.com>
Patch from Andreas Thienemann <andreas@bawue.net> #rh180555#
* etc/plague-server.init
etc/plague-builder.init
- Correct usage of pidof so it kills plague processes
executing under Python
2006-02-15 Dan Williams <dcbw@redhat.com>
* common/SSLConnection.py
- Don't use pyOpenSSL's sendall() call, since it simply
calls SSL_write() in a loop. Instead, we simulate the sendall
ourselves, allowing us to honor socket timeouts and to not
peg the CPU when we get WantReadError/WantWriteError exceptions due
to non-blocking IO
- Clean up the recv/sendall functions somewhat too
- Add some cosmetic inter-function spacing
- Increase WantReadError/WantWriteError sleeps from 0.1s -> 0.2s
- Return valid socket error message on timeout
2006-02-15 Dan Williams <dcbw@redhat.com>
* common/XMLRPCServerProxy.py
- Fix up some of the test code
2006-02-15 Dan Williams <dcbw@redhat.com>
* server/main.py
- remove pidfile on exit
2006-01-24 Dan Williams <dcbw@redhat.com>
* common/ExecUtils.py
- (exec_with_redirect): by default, have child start a new
process group
* builder/builder.py
- (_handle_death): kill child's entire process group, not just
the child itself
2006-01-23 Dan Williams <dcbw@redhat.com>
* common/SSLConnection.py
- Revert previous commit, causing too much trouble
2006-01-22 Dan Williams <dcbw@redhat.com>
* server/Builder.py
- Move most of the building_jobs() logic into another thread to
combat hanging issues
2006-01-22 Dan Williams <dcbw@redhat.com>
* common/SSLConnection.py
- Be smarter about SSL stuff by using pending() where possible in a
vain effort not to hang as much. Cribbed from rhn code by
misa@redhat
2006-01-22 Dan Williams <dcbw@redhat.com>
* server/ArchJob.py
- Don't traceback when killing jobs on builders
2006-01-22 Dan Williams <dcbw@redhat.com>
* server/Builder.py
- Move most of the building_jobs() logic into another thread to
combat hanging issues
2006-01-22 Dan Williams <dcbw@redhat.com>
* common/SSLConnection.py
- Be smarter about SSL stuff by using pending() where possible in a
vain effort not to hang as much. Cribbed from rhn code by
misa@redhat
2005-12-01 Dan Williams <dcbw@redhat.com>
Patch from Rudolf Kastl <che666@gmail.com> #rh174573#:
* etc/plague-server.init
etc/plague-builder.init
- Add nls support to initscripts
- initscript cleanups
2005-11-29 Dan Williams <dcbw@redhat.com>
* Add a traceback server that listens on a Unix socket
and writes backtraces for all threads to it. Disabled
by default.
2005-11-28 Dan Williams <dcbw@redhat.com>
Reported by <jeff@ollie.clive.ia.us> #rh174379#:
* server/UserInterface.py
- Accept '.' as valid package name character
2005-11-24 Dan Williams <dcbw@redhat.com>
* client/client.py
server/Builder.py
www/template/main.psp
- Use timeouts for XMLRPC clients
2005-11-24 Dan Williams <dcbw@redhat.com>
* common/AuthedXMLRPMServer.py
common/SSLCommon.py
common/SSLConnection.py
common/XMLRPCServerProxy.py
- Implement socket timeouts for XMLRPC client & server
2005-11-18 Dan Williams <dcbw@redhat.com>
* common/SSLConnection.py
- Fix behavior when socket timeout is set
- Increase default socket timeout
- Fix position of refcount decrease in close() so we
actually close the socket now
* common/SSLCommon.py
- Add optional socket timeout parameter for PlgHTTPSConnection
* common/HTTPSURLopener.py
- Pass along a socket timeout, if specified
* common/FileDownloader.py
- Specify a socket timeout for download connections
2005-11-18 Dan Williams <dcbw@redhat.com>
* server/ArchJob.py
- Log failed downloads before retrying
- Retry downloads from builder up to 5 times, not 3
* builder/builder.py
- Retry downloads from server up to 5 times, not 3
2005-11-15 Dan Williams <dcbw@redhat.com>
* server/Builder.py
- Suspend builders on hard errors like running out of disk
space or file descriptors or whatever
2005-11-15 Dan Williams <dcbw@redhat.com>
* server/ArchJob.py
server/PackageJob.py
- Log kill requests
2005-11-14 Dan Williams <dcbw@redhat.com>
* common/ExecUtils.py
- Close files opened in parent after exec() of child
2005-11-13 Dan Williams <dcbw@redhat.com>
* common/ExecUtils.py
- On python 2.3 and lower, unblock signals before execing
processes. Requires py_pthread_sigmask module.
2005-11-12 Dan Williams <dcbw@redhat.com>
* server/PackageJob.py
- (die): immediately kill jobs by starting up a controller
thread when the job is in 'waiting' stage
2005-11-12 Dan Williams <dcbw@redhat.com>
* server/ArchJob.py
- Lock access to self._downloads
- Make more stuff private to ArchJob
- Use variables for download dict to reduce possibility
of typing mistakes
2005-11-01 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- (dl_callback): retry failed downloads from the build
server up to 3 times before failing the job
2005-11-01 Dan Williams <dcbw@redhat.com>
Patch from Alexandr Kanevskiy <kad@crystalowl.net>
* builder/builder.py
- Remove unused imports
- Clean up usage of work_dir and other previous global
variables now that they are local to the BuilderMock class
2005-11-01 Dan Williams <dcbw@redhat.com>
* server/ArchJob.py
- (get_files): fix up download status dict to work with
new download-retry stuff
- (set_download_status): remove, unused
* server/Repo.py
- (_update_repo): remove ignore_errors argument from
shutil.copy(), since that's incorrect for that function
2005-10-31 Dan Williams <dcbw@redhat.com>
Patch from Alexandr Kanevskiy <kad@crystalowl.net>
* builder/builder.py
- (_status_prepping): catch mock failure case earlier
2005-10-27 Dan Williams <dcbw@redhat.com>
Patch from Jeff Sheltren <sheltren@cs.ucsb.edu>
* server/main.py
- Honor config file location passed in on command line
2005-10-25 Dan Williams <dcbw@redhat.com>
* server/PackageJob.py
- (_stage_prep): fail the job if we can't access the SRPM here. Can
happen if the server restarts and requeues waiting jobs, but the
SRPM isn't around anymore for some reason.
2005-10-25 Dan Williams <dcbw@redhat.com>
* server/ArchJob.py
- Retry finished job downloads from builders up to 3 times before
failing the job entirely
2005-10-25 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- (_handle_death): don't forget about a job when it gets killed,
but doesn't have an mock subprocess (for example, if the job
is in the 'downloaded' state)
2005-10-25 Dan Williams <dcbw@redhat.com>
* server/PackageJob.py
- (repo_add_callback): trap failures during repo addition of job
* server/Repo.py
- (_update_repo): recognize failures and notify the PackageJob
of them so the job can fail cleanly rather than half-copying
stuff to the repo
2005-10-24 Dan Williams <dcbw@redhat.com>
Patch from Jeff Sheltren <sheltren@cs.ucsb.edu>
* server/DBManager.py
server/Config.py
- Add MySQL support
2005-10-19 Dan Williams <dcbw@redhat.com>
* server/builder.py
- Make the server's builder-tracking thread more resistant
to builder errors
2005-10-19 Dan Williams <dcbw@redhat.com>
* client/client.py
- Don't traceback when the 'allow_uploads' config options
is not present
- Correctly print target in job detail command without a
traceback
2005-10-17 Dan Williams <dcbw@redhat.com>
* server/DBManager.py
- Increase 'username' and 'status' field sizes in the jobdb
2005-10-05 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- Fix race condition in which files would not be downloaded
to the build server
2005-09-16 Dan Williams <dcbw@redhat.com>
* builder/builder.py
common/FileDownloader.py
server/ArchJob.py
- Rename FileNameError -> FileNameException
* builder/builder.py
- Clean up exception handling on job creation so that we
don't end up calling start() before Thread.__init__() got
called.
2005-09-14 Dan Williams <dcbw@redhat.com>
* server/PackageJob.py
server/Config.py
- Rename "scratch" targets to "testing" targets to make their
purpose clearer
- Don't delete packages from the repodir if their target
is a testing target
2005-09-14 Dan Williams <dcbw@redhat.com>
* utils/distro-rebuild.py
- Optionally add a changelog entry to rebuilt specfile
2005-09-13 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- Try to tighten up child process handling to not
leave defunct mock processes around
2005-09-10 Dan Williams <dcbw@redhat.com>
* utils/distro-rebuild.py
- Don't delete the rebuilt SRPM until after everything
has been queued
2005-09-10 Dan Williams <dcbw@redhat.com>
* server/BuilderManager.py
- Fix python < 2.4 issues with sequence sort() call
2005-09-09 Dan Williams <dcbw@redhat.com>
* server/UserInterface.py
- Allow '+' in CVS tags
2005-09-09 Dan Williams <dcbw@redhat.com>
* Add support for PostgreSQL as the database backend.
Databases are now abstracted and support for others,
like MySQL, need to be added to server/DBManager.py.
Should be really easy though.
NOTE: sqlite support might be broken right now due to
SQL statement quoting changes
2005-09-08 Dan Williams <dcbw@redhat.com>
* server/PackageJob.py
server/ArchJob.py
- Fail job if downloads from builder fail
2005-09-07 Dan Williams <dcbw@redhat.com>
* utils/distro-rebuild.py
- Script that takes a directory of SRPMs, finds the
newest ones, adds a ".1" to their Release, then
queues them up using plague
2005-09-01 Dan Williams <dcbw@redhat.com>
* client/client.py
server/Repo.py
server/UserInterface.py
- Implement package upload functionality using 'scp' as
part of the 'build' command
2005-09-01 Dan Williams <dcbw@redhat.com>
* www/builders.psp
- Better unavailable builders display
2005-08-31 Dan Williams <dcbw@redhat.com>
* server/BuilderManager.py
- Load-balance builders somewhat
2005-08-31 Dan Williams <dcbw@redhat.com>
* www/
- Fix up web frontend for latest server code
2005-08-31 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- Reset self._die so we don't kill cleanup processes
2005-08-31 Dan Williams <dcbw@redhat.com>
* server/UserInterface.py
- Fix more target-related stuff
2005-08-31 Dan Williams <dcbw@redhat.com>
* builder/builder.py
server/Builder.py
- Expose "num slots" values to build server
2005-08-30 Dan Williams <dcbw@redhat.com>
* builder/builder.py
common/ExecUtils.py
- To execute mock, we now fork() and execv() the mock process
so that we can more reliably gather its output
- Correctly clean up the mock root directory
- Rename builder.log -> job.log so it can't get confused
with build.log from mock
2005-08-30 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- all self.log() -> self._log()
* server/Builder.py
- Fix condition where finished jobs wouldn't get noticed
by the server
* server/PackageJob.py
- Fix CVS commands
2005-08-29 Dan Williams <dcbw@redhat.com>
* More misc cleanups
2005-08-29 Dan Williams <dcbw@redhat.com>
* Make more stuff actually work
2005-08-29 Dan Williams <dcbw@redhat.com>
* server/Config.py
- Ensure duplicate user_aliases and duplicate target strings
cannot be loaded
- Move user target alias resolution into the Config stuff to
keep it centralized
* server/UserInterface.py
- Move target user alias resolution to Config stuff
- Consolidate 'enqueue' and 'enqueue_srpm' commands into one
'enqueue' command to enqueue packages that detects what you
are trying to enqueue by looking at the source
- Work with new target dict stuff
2005-08-25 Dan Williams <dcbw@redhat.com>
* Initial commit of reworked stuff:
- Each target gets separate config files on builder
and server
- Builders now run multiple jobs per builder instance
- Config files now ConfigParser based
- Target specifications are richer and require distro,
target, and repo names
- Builder's supported arches are autodetermined
- Various database fields renamed and/or removed
IT DOESN'T WORK YET
2005-08-24 Dan Williams <dcbw@redhat.com>
* common/BaseConfig.py
client/client.py
- Rewrite the client to make the code less stupid,
more easily followed, and to validate more
input options
- Also fixes RH #166692
2005-08-15 Dan Williams <dcbw@redhat.com>
* server/BuildMaster.py
- Clear out old job info when requeueing a job
2005-08-15 Dan Williams <dcbw@redhat.com>
* server/ArchJob.py
server/PackageJob.py
- Let "scratch" target jobs continue building even if one or more arches
has failed
2005-08-15 Dan Williams <dcbw@redhat.com>
* client/client.py
server/UserInterface.py
- Implement API versioning, and check the API version in the client
before doing anything else
2005-08-14 Dan Williams <dcbw@redhat.com>
* Switch to new server config file format using ConfigParser
- Server config files are no longer python scripts
- Server now takes a mandatory -c option for config file location
- Server will write out a default config file at the specified location
if none exists
- Each target has its own config file, by default in /etc/plague/targets/
- CVSROOT and CVS_RSH are now target-specific, not server-wide
- Additional Package Arches are now target-specific, and reside in each
target's config file rather than server-wide
* "Scratch" targets are now supported; ie, they are targets that do not
contribute packages to a repository. Use the option "scratch=yes" in
the target's config file in the "General" section to make the target a
scratch target.
* Repository Scripts: you may now specify a script that is run after packages
are copied to a repository. Use the "repo_script" option in each target's
config file to specify a script for that target/repo. The script must
exit with result 0 on success, and > 0 on error. The script is given
one argument, which is the name of the target/repo which has just copied
packages. If the script fails, those listed in "admin_emails" in the
server config file will be mailed with the script's output. The script
is also killed if it takes longer than 1 hour to complete, since packages
cannot be built for the target while the script is running.
2005-08-11 Dan Williams <dcbw@redhat.com>
* Tag plague-0_3_1_RELEASE
2005-08-11 Dan Williams <dcbw@redhat.com>
* plague.spec
- Clean up to Fedora Extras standards in preparation for package submission
* builder/builder.py
- Don't ignore mock output during the 'cleanup' stage
- Make the process() function less byzantine by moving each step to a
separate function and using getattr() to call them, like in
PackageJob.py on the server
* www/*
- Clean up error handling so it actually works all the time intead of
returning Error 500 Internal Server Error when, for example, the
build server isn't running
Patch from Ville Skyttä <ville.skytta@iki.fi>
* www/template/head.psp
www/indiv.psp
- Use HTTP GET instead of POST to stop nags about form data resubmission
2005-08-08 Dan Williams <dcbw@redhat.com>
* Tag plague-0_3_RELEASE
2005-08-08 Dan Williams <dcbw@redhat.com>
* README
plague.spec
utils/Makefile
utils/certhelper.py
- Make creation of SSL certificates not suck
2005-08-07 Dan Williams <dcbw@redhat.com>
Play nicely when running more than one builder instance on the same
machine.
* builder/builder.py
- Try to delete actual mock work dir rather than just the buildroot
* client/client.py
- Prettier printing of job detail command output
- Show builder port numbers
* server/ArchJob.py
- Save builder port number
* server/BuildMaster.py
- Add a builder port # to the archjobs table, and save builder port #
when writing archjob status to the DB
* server/Builder.py
- (to_dict): return builder port # too
* server/BuilderManager.py
- Don't exit when adding new builders that have wrong connection
protocol, just ignore them
* server/UserInterface.py
- Retrieve and return builder port numbers too
* www/builders.psp
- Differentiate builders and their jobs by port #
- Only show building archjobs, not 'downloading' or 'downloaded' ones
* www/job.psp
- Show builder port numbers
2005-08-05 Dan Williams <dcbw@redhat.com>
* Rework builder tracking code to always keep Builder objects around, and
to mark them as active/unavailable rather than dropping unavailable
builders. Unavailable builders are pinged every 5 minutes to see if
they are alive or not. Admins can still manually ping builders.
* Remove lots of locking code in between the Builders, ArchJobs, and the
BuildMaster since it was only relevant when pyOpenSSL still sucked.
Communication with the builder's XMLRPC server only happens from each
Builder object's thread now.
* Consolidate job-killing code in both the ArchJob and the PackageJob
2005-08-05 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- (_mock_done): hand-wavy attempt to ensure the mock child process
is always reaped
* client/client.py
- (enqueue, enqueue_srpm): Print out returned job UID
- (detail_job): remove bogus email argument
* server/BuildMaster.py
- (create_job_request): new function to consolidate job request creation
- (enqueue, enqueue_srpm): consolidate to one enqueue() function since
the database really makes no distinction between CVS and SRPM jobs
- (_start_new_jobs): return UID to requester when we become aware of it
* server/UserInterface.py
- (_wait_for_uid): new function, waits max 3 seconds for the BuildMaster
to return the new job's UID
- (enqueue, enqueue_srpm): use create_job_request(), and wait for the
job's UID to be returned from the BuildMaster, which we then pass
back to the client
- (detail_job): remove bogus email argument
- (_kill_job): warn user if the job is no longer building or waiting to
be added to the repo. We can't kill jobs if they are no longer building.
* www/job.psp
- detail_job()'s email argument got removed, remove it here to
2005-08-04 Dan Williams <dcbw@redhat.com>
* server/UserInterface.py
- Accept asc/desc ordering in addition to field order. You now
do e.x.: args['orderby'] = ['package asc', 'endtime desc']
* www/indiv.psp
- Stick in-progress jobs at the top of the list
* www/template/head.psp
- Keep user's email address through the success/failed pages
2005-08-04 Dan Williams <dcbw@redhat.com>
* Attempts to reduce network bandwidth usage between client & server
- for listing jobs, 'status' item in the arg list is now a list itself
which is combined using OR.
- New 'orderby' arg for listing jobs
- Fixes for both of the above in the web interface, the client,
and the server
* Small fixes to the web interface to display 'result' as well as 'status'
2005-08-04 Dan Williams <dcbw@redhat.com>
* server/PackageJob.py
- Clean up job files when the job is done. We now only
keep logs and 1 SRPM in 'server_work_dir', the RPMs
get copied to the repo and then deleted from server_work_dir
2005-08-04 Seth Vidal <skvidal@linux.duke.edu>
* add in success and failed pages for www report
2005-08-02 Dan Williams <dcbw@redhat.com>
Patch from Josh Boyer <jwboyer@jdub.homelinux.org>
* client/client.py
- Fix up usage printout
2005-08-01 Dan Williams <dcbw@redhat.com>
* Implement 'finished' state, allow users to move failed/needsign jobs
to the 'finished' state
* Random cleanups of database access code to make it more readable
* Simplify user permissions
* server/PackageJob.py
- Each state/stage now has a private method called "_stage_<the-stage>"
that gets called when the job is in that stage. This simplifies
the job's process routine quite a bit
- Jobs now record a 'result', either 'success', 'failed', or 'killed'
2005-08-01 Dan Williams <dcbw@redhat.com>
* common/AuthedXMLRPCServer.py
common/SSLConnection.py
server/Builder.py
- Simulate socket timeouts with select()
2005-07-29 Dan Williams <dcbw@redhat.com>
* server/BuilderManager.py
server/BuildMaster.py
server/UserInterface.py
client/client.py
- Add admin-only pause/unpause commands to the server
2005-07-29 Dan Williams <dcbw@redhat.com>
* www/job.psp
- Line up arch jobs
- Only show build times for successful jobs
2005-07-29 Seth Vidal <skvidal@linux.duke.edu>
* www/index.psp - despam email addresses
2005-07-29 Seth Vidal <skvidal@linux.duke.edu>
* server/UserInterface.py - fix resolved target to succeed if a match
comes from the client_target_map
also simplify the function a bit.
2005-07-28 Jeremy Katz <katzj@redhat.com>
* plague.spec: Create builder user and work dir in -build package
* builder/CONFIG.py: Set builder cert by hostname.
Listen on all interfaces.
2005-07-27 Jeremy Katz <katzj@redhat.com>
* builder/builder.py (PPCArch.__init__): Use setarch for ppc32
2005-07-26 Dan Williams <dcbw@redhat.com>
- www/config.psp
www/templates/main.psp
- Move config-type stuff for SSL to config.psp
2005-07-26 Dan Williams <dcbw@redhat.com>
- Throttle CVS checkouts to 5 jobs at a time
- Validate package and cvs_tag inputs to enqueue functions since they
get passed directly to commands.getstatusoutput()
2005-07-25 Dan Williams <dcbw@redhat.com>
* client/client.py
server/UserInterface.py
server/BuildMaster.py
- Add a "requeue" command to restart a failed or killed job
2005-07-25 Dan Williams <dcbw@redhat.com>
* server/CONFIG.py
- Move stuff around, two new options for admin_emails and success_emails
* server/BuildMaster.py
server/PackageJob.py
- Set endtime when all builds complete or fail, to not include all
the repo time in the job's build time
* server/PackageJob.py
server/UserInterface.py
server/BuilderManager.py
server/EmailUtils.py
- Notify admins when a builder times out
- Consolidate email sending into one place
- Notify success_emails when a build job succeeds
2005-07-22 Dan Williams <dcbw@redhat.com>
* server/UserInterface.py
- Fix traceback in job detail code
* www/*
- Add an individual job view
- Cleanups for Safari
2005-07-22 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- Fix traceback when killing jobs on shutdown of the builder
* client/client.py
- Add a job detail command
* server/ArchJob.py
- Store job start and end times in the database
* server/BuildMaster.py
- Remove JobsQuery class, no longer used
- Store more info in the job database to support web front end features
- Don't try to restart jobs in the 'initialize' state since they'll
get restarted anyway
* server/PackageJob.py
- Generalize log URL construction so it can be accessed from the user interface
- Pass more job info to the BuildMaster to be written to the DB
- Don't use tempfile.mkdtemp(), it seems to have issues
- Return 30 lines of log rather than 20
* server/User.py
- Grab a new connection to the database on every access, so that
user addition/modification can happen when the server is running
* server/UserInterface.py
- Add a "job detail" interface that returns information about a single
specific job
* server/main.py
- Greatly reduce buffer size for the logfile, now it actually gets written
out in a timely fashion
2005-07-22 Dan Williams <dcbw@redhat.com>
* Web front-end cleanup
- Use Diana Fong's new design. Delicious!
2005-07-21 Dan Williams <dcbw@redhat.com>
* builder/CONFIG.py
builder/builder.py
- You must now specify a target/arch -> mock buildroot name
mapping in the config file
* server/UserInterface.py
server/CONFIG.py
- Add client target alias support so people don't have to
know the exact target name
- Add rowlimit support to job listing
2005-07-21 Dan Williams <dcbw@redhat.com>
* common/FileDownloader.py
server/PackageJob.py
- Remove usage of os.chdir() since it's process-wide
- Lock usage of tempfile.mkdtemp() since it seems to have
threading issues
2005-07-21 Dan Williams <dcbw@redhat.com>
* www/*
- Add the web front-end
2005-07-20 Dan Williams <dcbw@redhat.com>
* builder/builder.py
server/main.py
- Don't use lighttpd any more
- Fix some issues with non-SSL build systems (Oliver Falk)
* client/client.py
server/UserInterface.py
- Return job lists as dicts for easier use
- Return archjobs embedded in the job list rather than a separate array
2005-07-18 Dan Williams <dcbw@redhat.com>
* server/UserInterface.py
client/client.py
- Rework argument handling and passing so that listing jobs
is more flexible. Can now list jobs by UID as well.
2005-07-18 Dan Williams <dcbw@redhat.com>
* builder/builder.py
common/ArchUtils.py
server/PackageJob.py
- Allow the server and builder to actually build "sub" arches
like i486/sparcv9/etc, which broke after the the builder
changes on 2005-07-16
2005-07-18 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- Don't start a cleanup over top of another cleanup
when we're told to kill the job
* server/BuildMaster.py
server/Repo.py
- Clear repo additions array after each repo update, fixes problem
where jobs would switch back from 'needsign' -> 'repodone' after
the job was already complete
2005-07-18 Dan Williams <dcbw@redhat.com>
* server/PackageJob.py
server/BuilderManager.py
- Restart orphaned archjobs immediately rather than
sticking them at the back of the build queue
2005-07-18 Dan Williams <dcbw@redhat.com>
* builder/builder.py
common/FileDownloader.py
server/ArchJob.py
- Allow '+' character in RPM names
- Clean up file name handling in URLs
2005-07-16 Dan Williams <dcbw@redhat.com>
* builder/builder.py
server/Builder.py
server/BuilderManager.py
- Make the builder aware of what mock targets & arches it can actually
build, and expose that information to the build server
- On the server, make sure that we ask the builder to only build for
arches that it supports for the target we request
(This makes noarch jobs work correctly everywhere)
2005-07-16 Dan Williams <dcbw@redhat.com>
* server/PackageJob.py
- Include last 20 lines of relevant log file for failed build jobs
2005-07-14 Dan Williams <dcbw@redhat.com>
* etc/plague-builder.init
etc/plague-builder.config
- Allow multiple builders to be started/stopped at one time.
plague-builder.config must be copied to /etc/sysconfig/plague-builder
and have the CONFIGS variable defined to be a space-separated
list of builder config files, with the whole list enclosed in quotes
2005-07-14 Dan Williams <dcbw@redhat.com>
* server/PackageJob.py
- Include log URL in email notifications
2005-07-14 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- Make the builder not hang if you interrupt it in the middle
of a time.sleep()
- Fix misdeclaration of log() which caused a traceback in some
instances
* server/Builder.py
- Kill any job the builder is running when the server starts up
* server/BuildMaster.py
- Restart interrupted jobs when the server starts up
- Remove old query queue code that's no longer used
* server/PackageJob.py
server/UserInterface.py
- is_build_job_stage_valid -> is_package_job_stage_valid
2005-07-13 Dan Williams <dcbw@redhat.com>
* builder/builder.py
builder/CONFIG.py
etc/plague-builder.config
etc/plague-builder.init
- Move arches and hostname into the CONFIG.py file, since
its already instance-specific. Change the initscripts
to reference the config file, which gets passed
to the builder on the command line now.
2005-07-13 Dan Williams <dcbw@redhat.com>
* common/AuthedXMLRPCServer.py
common/SSLCommon.py
common/SSLConnection.py
- Play better with Python 2.2
2005-07-12 Dan Williams <dcbw@redhat.com>
* Warn user and exit if server or builder is already running
* Add aliasing/mapping of plague targets to CVS targets, to
allow for different CVS directory names than what our
targets are
2005-07-10 Dan Williams <dcbw@redhat.com>
Patch from Ignacio Vazquez-Abrams <ivazquez@ivazquez.net>
* Add initscript/daemon support for the server
2005-07-10 Dan Williams <dcbw@redhat.com>
* server/PackageJob.py
server/BuildMaster.py
- Reduce total threadcount by breaking PackageJob runs into
two threads, so that no thread executes while the package
is waiting for a builder
2005-07-09 Dan Williams <dcbw@redhat.com>
Patch from Ignacio Vazquez-Abrams <ivazquez@ivazquez.net>
* Add initscript/daemonize support for the builder
2005-07-09 Dan Williams <dcbw@redhat.com>
* Rework the authorization framework so we don't need to
subclass/override subclass so much stuff
* Make the specfile Python version independent
2005-07-08 Dan Williams <dcbw@redhat.com>
* Pass archjob information along to the client,
and move query into UserInterface's thread
2005-07-07 Dan Williams <dcbw@redhat.com>
* builder/builder.py
- Clean up the buildroot no matter what
* Make archjobs write their status to the database too
2005-07-07 Dan Williams <dcbw@redhat.com>
* server/PackageJob.py
- Make some prep error emails more informative
2005-07-07 Dan Williams <dcbw@redhat.com>
* server/PackageJob.py
- Use Python exceptions for prep and build errors
- Clean up a bunch of stuff
- Specify dest file when copying files to the repo
* server/Repo.py
- Ensure that the destination file is actually the file rather
than a path, since shutil.copy() just copies the file over top
of the directory in some instances, corrupting the repo
2005-07-06 Dan Williams <dcbw@redhat.com>
* Convert M2Crypto code to pyOpenSSL so that stuff actually works
2005-07-05 Dan Williams <dcbw@redhat.com>
* Rework a bunch of stuff so the build server doesn't use quite as much CPU,
also split out stuff from server/client_manager.py
* Generalize the common/ classes to provide both SSL and non-SSL facilities,
renaming a lot of those files in the process
* Fix non-SSL builder/server and client/server communication
Note: At this time, SSL may be broken.
2005-07-01 Dan Williams <dcbw@redhat.com>
* We now use lighttpd as the fileserver in both the server and the builder. It's
a lot more robust than the python implementation, but the python implementation
code still exists and can be used.
* The server key and certificate must now be in the same file. You can 'cat' them
together with 'cat key.pem cert.pem > key_and_cert.pem'
* Initialize m2crypto threading in buildservery.py and builder.py
* Change some config options for the builder that said 'client' to 'builder'
* Add some testing code to SimpleHTTPSServer.py and FileDownloader.py
2005-06-29 Dan Williams <dcbw@redhat.com>
* Add the ability to search jobs on current job status from plague-client
2005-06-29 Dan Williams <dcbw@redhat.com>
* Copy finished SRPMs to the correct directories in the repo
2005-06-29 Dan Williams <dcbw@redhat.com>
* Implement job kill functionality
2005-06-28 Dan Williams <dcbw@redhat.com>
* README
- Fix up readme for current CVS
Also fix up some makefiles to create the /etc/plague/[server|builder]/certs directories.
2005-06-26 Dan Williams <dcbw@redhat.com>
* Move stuff around. The client that package maintainers will use
to submit jobs is now in client/, and the actual build daemon
has moved to builder/
2005-06-26 Dan Williams <dcbw@redhat.com>
* plague.spec
Makefile
- Make RPM builds using the specfile work
2005-06-26 Dan Williams <dcbw@redhat.com>
* client/buildclient.py
common/SSLCommon.py
common/SimpleHTTPSServer.py
common/SimpleSSLXMLRPCServer.py
- Set a much lower socket read timeout than the default, which
is 600 seconds. We don't want clients blocking the server
for too long, and non-blocking sockets are a bit too
complicated at this time.
2005-06-26 Dan Williams <dcbw@redhat.com>
* utils/package-builder.py
- Report server errors
* Fix paths in config files
2005-06-26 Dan Williams <dcbw@redhat.com>
* We have a name: "(The) Plague". We come to eat your packages.
* Add Makefiles, install common stuff in python site-packages,
and convert files to use config from /etc
2005-06-25 Dan Williams <dcbw@redhat.com>
* server/BuildMaster.py
server/UserInterface.py
- Make queries work again
2005-06-25 Dan Williams <dcbw@redhat.com>
* server/client_manager.py
- Lock access to build client queries because they could be
called from different threads
2005-06-25 Dan Williams <dcbw@redhat.com>
* client/buildclient.py
- Break out some functions from process() to make code easier to follow
- Use unique extensions on buildroots so more than one build client
can run at the same time on the same machine
- Execute mock after the build to clean up the buildroot before
deleting the buildroot directory
Requires latest mock from CVS
2005-06-24 Dan Williams <dcbw@redhat.com>
* Rework job database handling. Only 1 thread now has access to
the job database, the BuildMaster thread. This should fix issues
with database locking, timeouts, etc. sqlite doesn't really like
multiple threads, plus it doesn't have row locking, only table
locking.
So, when things want the BuildMaster to do something, they queue
up a request and the BuildMaster gets around to it. Once consequence
of this change is that job UIDs are not known until the job gets
added to the database, so we can no longer return the job's UID
to the client enqueueing the job.
* BuildClients are also in their own thread now so they don't block the
BuildMaster. There are interesting cases where clients can block
the server while the server is reading data from the client, or if
the client tracebacks in the middle of an SSL connection. This should
help keep the server more robust. Operational latencies are also
reduced by this change and the one to the BuildMaster.
2005-06-24 Dan Williams <dcbw@redhat.com>
* common/CommonErrors.py
- Try to trap actual error strings. Does this actually work?
2005-06-24 Dan Williams <dcbw@redhat.com>
* client/buildclient.py
- Remove some debug strings
- Fix undefined variable usage
2005-06-24 Dan Williams <dcbw@redhat.com>
* server/Repo.py
- Deal with repo locking during createrepo stages
* Ensure repositories can run createrepo without being accessed
by clients during the operation. Since createrepo runs can take
a long time, during that time clients cannot try to install their
buildroots or do any operations against the repository, otherwise
they may fail randomly with yum errors. So we lock access to the
repository on a 2-level lock: when a build job is done, it asks the
repo to copy its finished RPMs, and the repo enters lock level 1.
Level 1 prevents new build jobs from entering their 'prep' state.
When all currently running jobs have finished their prep state, and
the repo is in lock level 1, the repo promotes to lock leve 2 and
is able to run createrepo after copying any new RPMs into the repo.
When this is done, all waiting clients are released into their
'prep' states.
This requires an absolute latest mock from CVS.
2005-06-24 Dan Williams <dcbw@redhat.com>
* server/BuildMaster.py
server/client_manager.py
- Trap more errors
2005-06-21 Dan Williams <dcbw@redhat.com>
* common/FileDownloader.py
- Trap more errors, simplify code a bit
2005-06-17 Dan Williams <dcbw@redhat.com>
* common/CommonErrors.py
- Oops, "e" isn't an error, its the data. Treat it as such.
* server/client_manager.py
- Trap more errors in BuildClient._update_cur_job()
2005-06-17 Dan Williams <dcbw@redhat.com>
* common/CommonErrors.py
server/client_manager.py
common/SSLCommon.py
common/FileDownloader.py
* server/BuildJob.py
- better check for unspawned jobs
- Add a '/' to SRPM URLs that clients download to protect against CONFIG.py
errors
- Mark the repo as invalid after we copy RPMs to it
* server/BuildMaster.py
- Only run createrepo when the repository has actually changed
* server/client_manager.py
- Fix bug that caused jobs to simultaneously get started on all clients
that supported an architecture
2005-06-17 Dan Williams <dcbw@redhat.com>
* common/SSLCommon.py
- Trap some more SSLErrors
2005-06-17 Dan Williams <dcbw@redhat.com>
* server/client_manager.py
server/BuildJob.py
- Fix name clash of builder_gone
2005-06-17 Dan Williams <dcbw@redhat.com>
* utils/package-builder.py
- Simply arguments a bit, same command now for CVS or SRPM builds. Script
assumes that if the cvs/srpm argument ends with .src.rpm and exists
locally, that its an SRPM build.
* utils/repoconv.py
- Copy debuginfo RPMs too
2005-06-17 Dan Williams <dcbw@redhat.com>
* utils/repoconv.py
- Converts between flat directories of RPMs, and the buildserver repo
format of <repo>/name/v-r/arch/package
* server/client_manager.py
- Accept error 61 (connection refused) and increment unavailable_count
when we get it
2005-06-16 Dan Williams <dcbw@redhat.com>
* common/SSLCommon.py
- Override SSL.SSLServer's handle_error() method since it doesn't
take the right number of arguments
* common/SimpleSSLXMLRPCServer.py
- Add proxy lcl_resolve_dotted_attribute() function to deal with
different versions of python
* server/BuildJob.py
- email_to -> self.username in email_result()
* server/UserInterface.py
- email_to -> email in email_result()
- Fix typo that broke simple SRPM building (cvs_tag -> srpm_file)
* server/buildserver.py
- Deal with m2crypto 0.09 X509 cert handling of 'emailAddress'/'Email'
* utils/package-builder.py
- Deal with m2crypto 0.09 X509 cert handling of 'emailAddress'/'Email'
- Fix simple SRPM building, our local enqueue_srpm() function was calling
just enqueue() on the server
2005-06-15 Dan Williams <dcbw@redhat.com>
* client/buildclient.py
client/CONFIG.py
- Fix up the XMLRPC server so its port is configurable, to support
more than one build client running on the same machine
2005-06-15 Dan Williams <dcbw@redhat.com>
* server/UserInterface.py
- Add a list_clients call that returns a list of currentl active build clients
* server/client_manager.py
- Return list of active build clients
* utils/package-builder.py
- Hook up list_clients call, and refactor client printing code into new function
use by both list_clients and update_clients
2005-06-15 Dan Williams <dcbw@redhat.com>
* server/BuildJob.py
- Fix up build failure email to include status on each sub-job
- Fix email_result since we use email addresses as usernames
* server/UserInterface.py
- Fix email_result since we use email addresses as usernames
2005-06-14 Dan Williams <dcbw@redhat.com>
* client/buildclient.py
- Write mock output to a log on failure
* server/buildmaster.py -> server/BuildMaster.py
* server/BuildJob.py
server/UserInterface.py
server/buildserver.py
- Fix for buildmaster.py -> BuildMaster.py
* server/BuildJob.py
server/BuildMaster.py
- BuildMaster object now has a 'createrepo' method, which BuildJobs
call when they need to update the repo. BuildMaster.createrepo() is
locked so that we never run two createrepos at the same time
* server/CONFIG.py
- Clarify some options
* server/UserInterface.py
- Remove list_waiting_jobs and list_building_jobs, we now have a more
flexible "list_jobs" interface that will be extended to allow more
search parameters
- In update_clients(), return new clients that we've found
* server/client_manager.py
- Print out clients we find when we start up
- In update_clients(), return new clients that we've found
* utils/package-builder.py
- Implement most of remaining functionality:
- hook up enqueue and enqueue_srpm commands
- Add a list_own_jobs command
- grab user's email address from certificate or ~/.package-builder
- Allow non-SSL connections to build server
2005-06-14 Dan Williams <dcbw@redhat.com>
* common/SimpleSSLXMLRPCServer.py
- Fix for usage on Python 2.2.x, which doesn't have SimpleXMLRPCDispatcher
2005-06-14 Dan Williams <dcbw@redhat.com>
* server/CONFIG.py
- New options "ssl_frontend" and "ssl_buildclients". Only ssl_frontend is
used at this time. Setting it to True requires use of SSL to submit
build jobs and get server status. Setting it to False allows use of
xmlrpclib connections over HTTP, no certificates involved.
* server/User.py
- Add guest member to User class
* server/buildserver.py
- Move UserInterface class to UserInterface.py
- Re-enable the non-SSL XMLRPC server code
- Switch XMLRPC servers between SSL/non-SSL depending on CONFIG's
ssl_frontend value
- VerifiableSSLXMLRPCServer -> AuthenticatedSSLXMLRPCServer
* server/UserInterface.py
- Generalize UserInterface into a base class
- Implement UserInterfaceSSLAuth to handle SSL authenticated requests
- Implement UserInterfaceNoAuth to handle non-SSL requests
2005-06-13 Dan Williams <dcbw@redhat.com>
* client/buildclient.py
common/FileDownloader.py
common/HTTPSURLopener.py
common/SSLCommon.py
common/SSLXMLRPCServerProxy.py
common/SimpleHTTPSServer.py
common/SimpleSSLXMLRPCServer.py
server/client_manager.py
- Clean up cert handling by stuffing cert file paths into a dict
* common/SSLCommon.py
- QuietSSLServer: new class to quiet certain SSL errors we don't care about
* common/SimpleSSLXMLRPCServer.py
- Add an authorization framework. The actual request gets authorized
in SimpleSSLXMLRPCServer, but since we need to pass the authorization
object back up to the Instance class, we need to override a bunch
of functions in superclasses just to pass the object through. Subclasses
of SimpleSSLXMLRPCServer don't need to use authorization, they just don't
set an auth callback. If authorization is in use, the subclass is asked
to authorize the connection, and passes back an arbitrary authorization
object, which gets passed to each Handler method.
* server/buildmaster.py
server/buildserver.py
- Change buildmaster_db -> jobdb
* server/buildserver.py
- XMLRPCBuildMaster -> UserInterface
- Attach authorization to all public XMLRPC methods
- VerifiableSSLXMLRPCServer: new class to handle request authorization
- Add "guest" user support. If the user's cert checks out but they are not
in the user database, they have read-only permission to the server
* server/User.py
- New User object: encapsulates all permissions that a user might
have.
- New UserAuthentication object: looks up user authentication info in
a user database and creates User objects
* utils/user-manager.py
- Local user administration tool
* utils/package-builder.py
- New CLI front-end to the build system that support XMLRPC over SSL
* utils/certs/fedora-upload-ca.cert
- Fedora CVS Upload CA certificate, required to validate users
2005-06-13 Dan Williams <dcbw@redhat.com>
* common/HTTPSURLopener.py
common/SSLXMLRPCServerProxy.py
common/SimpleHTTPSServer.py
common/SimpleSSLXMLRPCServer.py
common/SSLCommon.py
- Refactor _initSSLContext() usage into SSLCommon.py
* common/SimpleSSLXMLRPCServer.py
- Route all requests through new class VerifiableSimpleXMLRPCRequestHander
which has the ability to callback the server for client certificate
verification.
2005-06-12 Dan Williams <dcbw@redhat.com>
* Require a "hostname" argument to the build server. But we get to remove
it as a config option from CONFIG.py then, which is good :)
* Implement Additional Package Architectures support. This is essentially
an look-aside file listing packages and other arches these packages should
build on, but that a whole distribution isn't built on. Examples include
OpenSSL on SPARC, where sparcv8 and sparcv9 optimized builds are produced,
but not an entire sparcv9 distro. See the "addl_pkg_arches" directory
for an example file, and the CONFIG.py file for more information.
* Simplify build client tracking on the server. There is now only 1 instance
of a BuildClient (formerly BuildClientInstance) class per build client.
Previously there was one instance for each arch the client supported.
Since at this time we don't allow more than one build job per client,
its un-necessary to track anything per-client-per-arch.
* Return the build job UID when queuing a job
* Start jobs on clients immediately when creating them, instead of putting
the server-side tracking object in 'initialize' state first and then
starting the job on the client in the next process() iteration.
* Do some locking when the BuildClient class gets the job the client
is currently running. Because BuildJob instances run in their own threads,
the BuildJob could be starting a new job on the BuildClient while the
BuildClient instance is getting status. This could result in a race
condition and potentially two jobs thinking they have started concurrently.
* Add a "Getting Started" section to the README file
2005-06-09 Dan Williams <dcbw@redhat.com>
* client/buildclient.py
- Add sparc32 and sparc64 bits
2005-06-09 Dan Williams <dcbw@redhat.com>
* common/FileDownloader.py
- Don't traceback on refused or dropped connections
* server/buildjob.py -> server/BuildJob.py
- BuildJob objects are now threads
- Clean up console output a lot
- Delete arch-specific jobs that don't actually start
* server/buildmaster.py
- Clean up console output
* server/buildserver.py
- Clean up 'enqueue' command return messages
* server/client_manager.py
- Try to tighten up exception handling around socket operations
- Don't print so much stuff to the console
- Prettier printing of downloaded file lists from client
2005-06-09 Dan Williams <dcbw@redhat.com>
* client/buildclient.py
- Fix up some log messages to be shorter
2005-06-08 Dan Williams <dcbw@redhat.com>
* Fix SRPM only mode:
- server/buildjob.py: In 'prep' stage, make sure we know where the SRPM is, and
don't try to delete the checkout dir when in SRPM-only mode (since there is
no checkout dir)
2005-06-08 Dan Williams <dcbw@redhat.com>
* Convert all client/server communication to SSL. You will now need certificates
(see the README file for how to set them all up) to get them to talk to each
other. Now requires m2crypto module as well.
2005-06-08 Dan Williams <dcbw@redhat.com>
* Refactor FileDownload.py and FileServer.py, moving them into a shared
directory common/. Fix client & server to point to these modules and
implement callbacks in each that are required by FileDownload.py. Since
at this time we don't install anything to site-packages, the build-client
and build-server scripts are necessary to properly set up PYTHONPATH to
find stuff in the common/ directory.
2005-06-07 Dan Williams <dcbw@redhat.com>
* Rework much of the build system to support HTTP transfer of SRPMs to
build clients, and of logs & RPMs back to the build server. Simple
SRPM building is broken right now but will be fixed soon.
2005-05-22 Dan Williams <dcbw@redhat.com>
* client/fileserver.py
- Add a simple HTTP server for package download to build server. Its
integration into the archwelder is not yet complete.
2005-05-12 Dan Williams <dcbw@redhat.com>
* Add an SRPM build mode that short-circuits CVS checkouts to build an SRPM
queued by the user. This mode CANNOT be enabled at the same time as CVS
checkout mode, since its less secure. It should only be used for local
building and development. To enable this mode, set the 'use_srpm_not_cvs'
config option to True.
enqueue_srpm(<user>, <package name>, <path to SRPM>, <target>)
2005-05-12 Dan Williams <dcbw@redhat.com>
* Add ability to drop ArchWelders and the jobs they are currently building,
and to find them again when kicked to do so. To do this, config file for
the server got changed to CONFIG so we can reload(CONFIG) and get new
builders on the fly
2005-05-11 Dan Williams <dcbw@redhat.com>
* Add README file explaining stuff about build system architecture
