0.20 (2013-06-28) ================= Improvements ------------ - A new CaptureTracer has been added to storm.testing, for capturing all SQL statements executed by Storm. It can be used like this: with CaptureTracer() as tracer: # Run queries pass print tracer.queries # Print all queries run in the context manager block You will need the python-fixtures package in order to use this feature. - Setuptools <http://pypi.python.org/pypi/setuptools> is now required to run setup.py. This makes it much easier to install the majority of the dependencies required to run the test suite in its entirety. - Disconnection errors arising from PostgreSQL are now more reliably detected, especially with regard to recent libpq updates in Ubuntu. There are also disconnection tests that simulate sudden termination of pgbouncer <http://pgfoundry.org/projects/pgbouncer/>. - Insert expressions now support multi-row and subquery INSERT statements. - Support in the postgres backend to use the RETURNING extension for UPDATE statement, optionally specifying the columns to return. (PostgreSQL >= 8.2 only) - Add a new Distinct expression for pre-pending the 'DISTINCT' token to SQL expressions that support it (like columns). - Switch to REPEATABLE READ as isolation level for Postgres. If you use psycopg2 < 2.4, it doesn't change anything. For psycopg2 2.4 and newer, it will keep the same behavior on Postgres < 9 as it's equivalent to SERIALIZABLE, but it will be less strict on Postgres >= 9. - Add support for two-phase commits, using the DB API version 2.0, which is only supported by PostgreSQL. - ZStormResourceManager now has a schema_stamp_dir optional instance attribute that, if set, will be used to to save timestamps of the schema's patch packages, so schema upgrades will be performed only when needed. Bug fixes --------- - When a SQLObjectResultSet object was sliced with slice.start and slice.end both zero (sqlobject[0:0]), the full, unsliced resultset was returned (bug #872086). - Fixes some test failures around Django disconnections from PostgreSQL stores. - Some of the proxy-less disconnection tests were causing segfaults, so they're now run in a subprocess via subunit's IsolatedTestCase, when it's available. - Fix a few non-uses of super() in TestHelper. - Abort the transaction and reset ZStorm at the end of tests/zope/README.txt. - Fix the ./test script to not import tests until after local eggs have been added to sys.path. This ensures that all possible features are available to the tests. - If transaction.commit() fails, call transaction.abort() to rollback everything and leave the transaction in a clean state when using Django's ZopeTransactionMiddleware and DatabaseWrapper. - It's now again possible to use the Desc() expression when passing an order_by parameter to a ReferenceSet (bug #620369). 0.19 (2011-10-03) ================= Improvements ------------ - A new Cast expressions compiles an input and the type to cast it to into a call the CAST function (bug #681121). - The storm.zope.testing.ZStormResourceManager now supports applying database schemas using a custom URI, typically for connecting to the database using a different user with greater privileges than the user running the tests. Note that the format of the 'databases' parameter passed to the constructor of the ZStormResourceManager class has changed. So now you have to create your resource manager roughly like this:: databases = [{"name": "test", "uri": "postgres://user@host/db", "schema": Schema(...), "schema-uri: "postgres://schema_user@host/db"}] manager = ZStormResourceManager(databases) where the "schema-uri" key is optional and would default to "uri" if not given. The old format of the 'databases' parameter is still supported but deprecated. (bug #772258) - A new storm.twisted.transact module has been added with facilities to integrate Storm with twisted, by running transactions in a separate thread from the main one in order to not block the reactor. (bug #777047) - ResultSet.config's "distinct" argument now also accepts a tuple of columns, which will be turned into a DISTINCT ON clause. - Provide wrapped cursor objects in the Django integration layer. Catch some disconnection errors that might otherwise be missed, leading to broken connections. (bug #816049) - A new JSON property is available. It's similar to the existing Pickle property, except that it serializes data as JSON, and must back onto a text column rather than a byte column. (bug #726799, #846867) - Cache the compilation of columns and tables (bug #826170, #848925). - Add two new tracers extracted from the Launchpad codebase. BaseStatementTracer provides statements with parameters substituted to its subclasses. TimelineTracer records queries in a timeline (useful for OOPS reports). - New ROW constructor (bug #698344). - Add support for Postgres DISTINCT ON queries. (bug #374777) Bug fixes --------- - When retrieving and using an object with Store.get(), Storm will no longer issue two queries when there is a live invalidated object (bug #697513). - When a datetime object is returned by the database driver, DateVariable failed to detect and convert it to a date() object (bug #391601). - The ISQLObjectResultSet declares an is_empty method, which matches the existing implementation. This makes it possible to call the method in security proxied environments (bug #759384). - The UUIDVariable correctly converts inputs to unicode before sending them to the database. This makes the UUID property usable (bug #691752). - Move the firing of the register-transaction event in Connection.execute before the connection checking, to make sure that the store gets registered properly for future rollbacks (bug #819282). - Skip tests/zope/README.txt when zope.security is not found. (bug #848848) - Fix the handling of disconnection errors in the storm.django bridge. (bug #854787) 0.18 (2010-10-25) ================= Improvements ------------ - Include code to manage and migrate database schemas. See the storm.schema sub-package (bug #250412). - Added a storm.zope.testing.ZStormResourceManager class to manage a set of stores registered with ZStorm (bug #618704). It can be used roughly like this:: from testresources import ResourcedTestCase from storm.zope.testing import ZStormResourceManager from storm.schema import Schema name = "test" uri = "sqlite:" schema = Schema(...) manager = ZStormResourceManager({name: (uri, schema)}) class MyTest(ResourcedTestCase): resources = [("zstorm", manager)] def test_stuff(self): store = self.zstorm.get("test") store.execute(...) - When a TimeoutError is raised it includes a description about why the exception was raised, to help make it easier to reason about timeout-related issues (bug #617973). - Improved the IResultSet interface to document the rationale of why some attributes are not included (bug #659883). Bug fixes --------- - Make storm compatible with psycopg2 2.2 (bug #585704). - Fix bug #620615, which caused lazy expressions to cause subsequent loading of objects to explode if unflushed. - Fix bug #620508, which caused slicing a ResultSet to break subsequent count() calls. - Fix bug #659708, correcting the behavior of the sqlobject is_empty and __nonzero__ methods. - Fix bug #619017, which caused __storm_loaded__ to be called without its object's variables defined if the object were in the alive cache but disappeared. 0.17 (2010-08-05) ================= Improvements ------------ - The order_by parameter defined on ReferenceSet can now be specified as a string, to workaround circular dependency issues. The order by property column will be resolved after import time (bug #580037). - The Store and Connection classes have block_access() and unblock_access() methods that can be used to block access to the database connection. This can be used to ensure that an application doesn't access the database at unexpected times. - When using stores managed by ZStorm, a ZStormError will be raised on attempts to use a per-thread store from the wrong thread (bug #348815). - ResultSet.is_empty strips the ORDER BY clause, when present, to provide a performance boost for queries that would match a large number of rows (bug #246200). - A new ResultSet.get_select_expr method returns a Select expression built for a specified set of columns, based on the settings of the result set (bug #337494). - ResultSet.any and ReferenceSet.any strips the ORDER BY clause, when present, to provide a performance boost for queries that would match a large number of rows (bug #608825). - SQLObjectResultSet has a new is_empty method which should be used in preference to __nonzero__, because it is compatible with ResultSet. Bug fixes --------- - SQLite reserved words are handled properly (bug #593633). - A bug in change checkpointing logic has been fixed to detect changes in mutable objects correctly and to prevent useless (or potentially harmful!) columns to be specified in updates (bug #553334). 0.16 (2009-11-28) ================= Improvements ------------ - The set expression constructor will now flatten its first argument if it is of the same type. The resulting expression tree uses less stack when compiling so reduces the chance of hitting Python's recursion limit (bug #242813). - Add startswith(), endswith() and contains_string() methods to Comparable. These methods perform prefix, suffix and substring checks respectively using the LIKE operator, taking care of escaping for you (bug #387840). - C extensions are enabled by default. Define the STORM_CEXTENSIONS=0 environment variable to disable them (bug #410592). - The README file contains information about Storm's license and detailed instructions on setting up a development environment suitable for running the entire test suite. - 'make doc' uses Pydoctor to generate API documentation. - Integration tests for Django now work with Django 1.1. Bug fixes --------- - Remove a leak when mutable variables (ListVariable or PickleVariable instances) are collected before store.flush, leaving hooks behind them. - The ResultSet min, max and sum methods now work correctly when the result set is empty and the column has allow_none=False set. Previously this resulted in a NoneError (bug #457801). - MySQL reserved words are handled properly (bug #433833). - Test loading code has been simplified. Support for py.test has been removed in this process, as it was not functioning correctly and didn't fit into the PyUnit framework (bug #331905). - Remote diverged and remote deleted references now use a weak (Python) reference to the local object. This prevents a leak when the remote object stays in memory (bug #475148). - Check for invalidated state when returning the remote object of a relation: it fixes a bug if the local key of the Reference is the primary key (bug #435962). - The default Cache instance created for a Store honours Cache's default size. Store's docstring has been updated to reflect this (bug #374180). 0.15 (2009-08-07) ================= Improvements ------------ - Add support for latest version on Django by not checking arguments passed to _cursor. - Added support for using Expressions with ResultSet.set(). - The default cache size was changed from 100 to 1000 objects. - Added the new GenerationalCache with a faster implementation of the caching API (by Jeroen Vermeulen). This will likely become the default implementation in the future, so please test it if possible. - A new UUID property type has been added. It depends on the uuid module introduced in Python 2.5 to represent values. - The StoreDataManager now gets passed a transaction manager from the ZStorm utility. This will make it easier to support non-default transaction managers in the future. - An adapter is now available for converting ISQLObjectResultSet objects to IResultSet. This is intended to help in gradual porting SQLObject applications to Storm's native API (bug #338184, by Gavin Panella). - If a disconnection occurs outside of Storm's control, a DisconnectionError will still be raised next time the connection is used through Storm. This is useful if the connection is being shared with another framework like Django (bug #374909). - The PostgreSQL backend now requires psycopg2 >= 2.0.7. The work around for broken quoting behaviour in older psycopg2 versions has been removed. (bug #322206). - A new Neg expression is available. It provides unary minus by prepending a minus sign to whatever expression is passed to it (bug #397654 by Michael Hudson). - A new Coalese expressions is available. - ResultSets now have a find(). It acts similar to Store.find(), but without the first argument (it uses the same classes as the original result), and only returns results found in the original result set. (bug #338255). - Result.rowcount exposes the number of rows affected by the query, when known. - ResultSet.remove returns the number of rows deleted from the database. (bug #180122). Bug fixes --------- - The fix for ResultSet.count() on distinct results from last release has been improved, and the fix has been extended to the other aggregates. This may change the result of some count(), min(), max(), sum() and avg() calls for results using distinct, limit or offset. - The test suite now passes when run with Python 2.6. - ListVariable now converts its elements to database representation correctly (bug #136806 reported by Marc Tardif). - compile_python now works for values that don't produce valid Python expressions with repr(). - The C extension should now build under Windows. 0.14 (2009-01-09) ================= Improvements ------------ - A new doctest describing the 'Infoheritance' pattern is integrated into the test suite. - A new storm.django package has been added to allow use of Storm within Django applications. - The way Storm interacts with the Zope transaction manager has changed. Rather than using a synchronizer to join each new transaction, it now delays joining the transaction until the store is actually used. - The Store constructor takes an optional cache keyword argument. - ResultSets now offer an is_empty() method. Bug fixes --------- - Manage better row full of NULL in case of LEFT JOIN, without validating against object constraints. - The Reference class now has an __ne__() method, so inequality checks against a reference now work in find expressions (bug #244768 reported by Stuart Bishop). - Make ResultSet.count() handles the distinct, limit and offset flags, to really reflect the length of the current ResultSet. - The store doesn't iterate on all the alive objects anymore, using instead events that objects can subscribe to. This improves performance drastically when lots of objects are in the cache. 0.13 (2008-08-28) ================= Improvements ------------ - Add group_by/having methods on ResultSet objects, to allow access to the "GROUP BY" and "HAVING" statements. - Change tests/store to keep the connection during the tests to make it faster. - Implemented support for plugging generic "tracers". Statement debugging is now implemented using a tracer, and easily enabled with storm.tracer.debug(True) (storm.database.DEBUG = True is gone). - All properties now accept a "validator" parameter. When used, a function like validate(object, attribute_name, value) should be given, and it may validate or modify the value before it's set in the property. The value assigned to the property is the result of the validator, so the original value should be returned if changing it isn't intended. - Expressions can be passed to Store.find() as well as classes. This makes it possible to request individual columns from a table, computed expressions or aggregates. - Objects will be flushed in the order they become dirty by default. This means that generally the order in which Python operations are performed will be used to define the order in which flushes are done, which is generally the most expected. - The cextensions module was fixed, optimized, and improved. It's now built by default, but to actually enable it the environment variable STORM_CEXTENSIONS=1 must be defined at runtime. The module will likely be enabled by default on a future release. - ClassAlias will now cache all explicitly named aliases, to prevent the cost of rebuilding it. - Result sets and reference sets now have a __contains__() method. While code like "item in set" was previously possible, it involved iterating over the result set, which is expensive for large databases. - The storm.zope.zstorm code can now be used with only the zope.interface and transaction packages installed. This makes it easier to reuse the per-thread store management and global transaction handling from other web frameworks. Bug fixes --------- - Make is_in returns False instead of NULL on an empty result set. - ZStorm now keeps strong references to named stores. Previously it only kept weak ones, so stores were recreated more often than necessary. - References now won't flush the store or query the database when the foreign key is None. - When a reference is set to an object that wasn't yet inserted in the database, the foreign key is immediately unset instead of keeping the old value up to the flushing. - Setting a reference to None works even if the previously referenced object isn't in memory. - When setting a reference, flush ordering is only enforced if the key is dirty. This allows a number of changes that would previously raise OrderLoopError. - If the remote object in a back reference is removed, the reference will now be broken. - Fixed a race condition when two threads try to initialize the ClassInfo for a given class at the same time. - Improve handling of AUTO INCREMENT columns in the MySQL backend to remove an unnecessary query when adding objects to a store. 0.12 (2008-01-28) ================= Improvements ------------ - The Connection will reconnect automatically when connection drops are detected and a rollback is performed. As a result, the Store should handle reconnections in a seamless way in most circumstances (#94986, by James Henstridge). This is supported in the MySQL and PostgreSQL backends. - Store.flush() will not load values inserted in the database. Instead, undefined variables are set to AutoReload, and resolved once first accessed. This won't be noticeable in normal usage, but will boost the performance of inserts. - Support in the postgres backend to use the RETURNING extension of the INSERT statement to retrieve the primary key on inserts for object identity mapping (PostgreSQL >= 8.2 only) - Introduced a cache mechanism that keeps the N last retrieved objects in memory to optimize cases where the same object is retrieved often while no strong references are kept elsewhere. Implemented by Bernd Dorn, from Lovely Systems. - Improved support for TimeDelta properties on all backends. Many more formats are accepted now, and some issues were fixed. Bug fixes --------- - TimeDelta was added to storm.locals. - Fixed TimeDelta support in SQLite, MySQL, and PostgreSQL, and enabled base test for all backends to ensure that it continues to work. - Schema names are accepted in __storm_table__ when using PostgreSQL as the database (e.g. "schema.table"). (#146580, reported by James Mayfield) - Test runner handles path correctly on Windows, and SQLite tests won't break (patch by Ali Afshar). - In the SQLite backend, ensure that we're able to recommit a transaction after "database is locked" errors. Also make sure that when this happens the timeout is actually the expected one (patch by Richard Boulton) - TransactionFailedError is now imported from the public place: ZODB.POSException (#129715, by James Henstridge). - TimeDelta was added to storm.locals. - Tables named with reserved keywords are properly escaped. - Reserved keywords on column names are properly escaped when part of an insert or update statement (#184849, by Thomas Herve). - Prevent cached objects from issuing selects to retrieve their data when find()s were previously made and already brough their data (#174388, reported and fixed by Bernd Dorn). - Fixed bug which caused an object to be readded to the store when a reference of an object that had already been removed was looked up. - Prevent pathological case which happens when a statement like "SELECT ... WHERE table.id = currval(...)" is executed in PostgreSQL. The change is only meaningful on PostgreSQL < 8.2, as newer versions will use the RETURNING extension instead. - Specify both of the joining tables explicitly when compiling Proxy, so that it doesn't break due to incorrect references in the ON clause when multiple tables are used (reported in #162528 by S3ym0ur and Hamilton Tran) - MySQL client charset now defaults to UTF-8 (reported by Brad Crittenden). 0.11 (2007-10-08) ================= Improvements ------------ - Added case-insensitive support to the Like expression and Class.attr.like() constructions. - ZStorm.get_name() for obtaining the the name of the given store. Bug fixes --------- - storm.zope wasn't included on the tarball due to an error in setup.py. - Binary strings are now properly quoted with the E'' formatting if needed, in the postgres backend. 0.10 (2007-08-08) ================= Improvements ------------ - Improvements were made to the tutorial. - There is now a setup.py script for installing Storm. - Count and ClassAlias is now available through the storm.locals module. - A new hook, __storm_pre_flush__, can be implemented on objects in a Store. It is called before an object is flushed to the database. - Storm can now use the built-in sqlite support in Python 2.5 and above. - There is now a storm.properties.Decimal, which allows you to store Decimal (as opposed to binary) floating point values. - storm.zope was added, which offers a simple integration mechanism with the Zope transaction machinery. - Complex expressions other than simple Columns can now be passed to the aggregation methods of ResultSet (avg,max,min,sum). - Backend implementors can now override preset_primary_key on their Database object to come up with primary key values before an Insert. - A large amount of API documentation was added. Bug fixes --------- - SQL reserved words are now properly escaped in SQL statements. - GROUP BY and ORDER BY statements are now ordered correctly. - Running the tests with trial now works. - All backends are now initialized such that their transactions are truly SERIALIZABLE. Psycopg2 and Pysqlite2 both did not previously have serializable transactions by default, but this has been fixed. - A bug in ResultSet.cached which could occasionally cause inconsistencies in ResultSet.set was fixed. API Changes ----------- Most changes are backwards compatible. There were some incompatible changes which may affect alternative database backends. - Chars was renamed to RawStr. Chars still exists, but is deprecated. All raw 8-bit data in your database should be represented with RawStr. - compiler handlers have had their arguments reordered. - The Compile.__call__ method now returns only the Statement. - Compile.fork was renamed to Compile.create_child. - Many methods which previously had underscores were renamed to get rid of the underscores to reflect their status as things which can be safely touched in subclasses. Documentation was added clarifying their intended use.