==========================
Frequently Asked Questions
==========================
General Questions
=================
What is Django Evolution?
-------------------------
When you run ``./manage.py syncdb``, Django will look for any new models that
have been defined, and add a database table to represent those new models.
However, if you make a change to an existing model, ``./manage.py syncdb`` will
not make any changes to the database.
This is where **Django Evolution** fits in. Django Evolution is an extension to
Django that allows you to track changes in your models over time, and to update
the database to reflect those changes.
When will Django Evolution be merged into Django trunk?
-------------------------------------------------------
Possibly never, and certainly not before it's ready. Django Evolution is
still in the early stages of development; considering plans for merging to the
Django trunk would be premature.
Installation
============
How do I install Django Evolution?
----------------------------------
There have not been any official releases of Django Evolution. If you want to
use Django Evolution, you will need to use a subversion checkout.
Check out the Django Evolution sources, and put the checkout directory into
your Python Path.
Then add 'django_evolution' to the list of installed applications for the
project in which you want to use Django Evolution.
What version of Django does Django Evolution support?
-----------------------------------------------------
Django Evolution requires that you are using Django v1.0. You should also be able
to use any SVN checkout from revision 8296 or higher. You cannot use Django
Evolution with a 0.96 (or earlier) release of Django.
Management
==========
Who is behind Django Evolution?
-------------------------------
Django Evolution was originally developed as a part time project by
two guys based in Perth, Western Australia: `Ben Khoo`_ and `Russell
Keith-Magee`_. Russell is also a core developer of Django itself.
Russell is now an emeritus contributor. the project is maintained by
`Christian Hammond`_.
.. _`Ben Khoo`: mailto:khoobks@westnet.com.au
.. _`Russell Keith-Magee`: mailto:russell@keith-magee.com
.. _`Christian Hammond`: mailto:chipx86@chipx86.com
Where should I ask questions about Django Evolution?
----------------------------------------------------
Django Evolution has a `Google Groups mailing list`_ for discussing
the usage and development of the project.
.. _`Google Groups mailing list`: http://groups.google.com/group/django-evolution
There is also a read-only mailing list if you wish to be notified
whenever code is checked into the Subversion repository, or whenever
a ticket is added or modified.
.. _`read-only mailing list`: http://groups.google.com/group/django-evolution-changes
I think I've found a bug! What should I do?
-------------------------------------------
Django Evolution has a `bug tracker`_. Check the list of bugs that are
already known; if your bug doesn't appear to be logged, please add it
to the list.
When you lodge your bug, *please* remember to provide clear instructions
on how to reproduce your problem. This means providing sample models,
sample migrations, and any error logs that appeared during the evolution
process.
.. _`bug tracker`: http://code.google.com/p/django-evolution/issues/list
Status
======
What works?
-----------
The general framework should be stable. Any difficulties installing
Django Evolution into your project, running the evolve command, or identifying
changes in your models should be reported as a bug.
The following is a list of the model changes that we believe to be working:
* Addition of simple fields (e.g., a new CharField).
* Addition of Many To Many Fields.
* Deletion of most fields (of any type).
* Deletion of models.
* Deletion of entire applications.
* Renaming of a field. Renames won't ever appear in a hinted evolution,
but they can be manually added as a replacement for the Add+Delete pair
that a hinted evolution will provide.
Primary development for Django Evolution has used the Psycopg1 interface to
PostgreSQL. As a result, this is the most stable database interface. A
Psycopg2 PostgreSQL backend also exists, and should work without difficulty.
The SQLite backend is also fairly reliable. The MySQL backend works for most
mutations, with some exceptions (details below).
What doesn't work?
------------------
Django Evolution is a work in progress, and as a result, there are
things that don't work, or don't work as expected. The following model
changes are known to cause difficulty:
* Addition of ForeignKey fields. A field will be added, but the foreign
key constraints are not correctly constructed.
* Deletion of primary keys (including adding a manual primary key as
a replacement for an automatically installed primary key)
* Deletion of fields referenced by other database constraints.
Dependencies between applications can also cause difficulties. If you have
changes in one application that are dependent on changes in another
application, you may experience difficulties in applying evolutions.
Django Evolution is known to work well for PostgreSQL and SQLite. This is
because the core developers both use PostgreSQL and SQLite on a daily basis.
Support for MySQL is less complete. The following features are known to be
buggy or broken under MySQL:
* Renaming of primary key fields and foreign key fields.
Improving MySQL support (as well as adding support for other database backends)
is one big area where contributions would be most welcome.
Usage
=====
How do I remove an entire application from my project?
------------------------------------------------------
If you remove an application from the ``INSTALLED_APPS`` list, Django
Evolution will not attempt to remove the tables for this application.
If you want to remove any tables associated with a deleted application,
you must specify the ``--purge`` option to evolve.
Why does Django Evolution generate an error when hinting an evolution?
----------------------------------------------------------------------
Django Evolution is a work in progress. The infrastructure necessary to
identify model changes is in place and is (to the best of our knowledge)
complete. However, the definitions required to handle every possible type
of model change have not yet been developed. As a result, there are some types
of change that you can make to a model for which Django Evolution cannot
produce a corresponding SQL mutation.
We hope to eventually support these changes with a hinted mutation. However,
in the interim, the workaround is to manually define an SQL mutation
and include it as a stored evolution in your project.
My OneToOneField has changed as a result of the QuerySet Refactor merge! What should I do?
------------------------------------------------------------------------------------------
First - some background. `Django revision 7477`_ saw the merging of the
`Queryset Refactor branch`_ back into the Django trunk. This change introduced a
lot of behind-the-scenes improvements to the way queries are composed.
However, this also introduced some subtle `backwards-incompatible changes`_
that can affect your database definitions. In turn, these changes have an
effect on the way Django Evolution treats your database.
For example, consider the case of the following model defining a user profile::
class UserProfile(models.Model):
user = models.OneToOneField()
foobar = models.CharField(max_length=40)
If you were to run syncdb on this model prior to Django revision 7477,
you would get a database definition that looks something like::
CREATE TABLE "test_app_userprofile" (
"user_id" integer NOT NULL PRIMARY KEY REFERENCES "auth_user" ("id") DEFERRABLE INITIALLY DEFERRED,
"foobar" varchar(40) NOT NULL
);
However, if you were to run syncdb using a version of Django after
revision 7477, you would get the following::
CREATE TABLE "test_app_userprofile" (
"id" serial NOT NULL PRIMARY KEY,
"user_id" integer NOT NULL UNIQUE REFERENCES "auth_user" ("id") DEFERRABLE INITIALLY DEFERRED,
"foobar" varchar(40) NOT NULL
);
The suggested response to this change is to add "primary_key=True" to your
OneToOneField defintion. While this will result in the same functionality at
runtime, it will be identified by Django Evolution as a change in model
signature - even though no database change is required.
What is required is a way to ignore the change of signature. This can be
achieved using an empty SQL mutation that provides a signature update
function. To update the previous example, put the following into your
mutation sequence definition::
def qsrf_update(app_label, proj_sig):
app_sig = proj_sig[app_label]
model_sig = app_sig['UserProfile']
model_sig['fields']['user'] = {
'primary_key': True
}
SEQUENCE = [
...
SQLMutation('qsrf-update', [], qsrf_update)
...
]
This defines a signature update function called "qsrf_update()" that puts
a "primary_key" definition into the signature for the OneToOneField. This
function is used by an SQL mutation called 'qsrf-update' that contains
no actual SQL statements. So, when this mutation is applied, the signature
will be updated, but no changes will be made to the database.
.. _`Django revision 7477`: http://code.djangoproject.com/changeset/7477
.. _`Queryset Refactor branch`: http://code.djangoproject.com/wiki/QuerysetRefactorBranch
.. _`backwards-incompatible changes`: http://code.djangoproject.com/wiki/QuerysetRefactorBranch#Backwardsincompatiblechanges
Design
======
What are the design goals for Django Evolution?
-----------------------------------------------
Django Evolution has the following design goals:
1. Provide a way for simple changes to be applied to a database
without significant user intervention.
2. Provide an entry point for using raw SQL as a migration tactic.
3. Provide a way to 'hint and tweak' - i.e., use an automatic method to
generate a first pass at a migration, but provide a way for a
developer to fine tune the migration before application.
4. Provide a way to be self-documenting and self-auditing - i.e., keep
a permanent archive of the changes that have been made, which ones
have been applied to which database, etc.
5. Provide some manner of validation that a migration, when applied,
will be correct.
6. Provide an easy way for end users to integrate their own custom
migration needs into the framework.
Isn't this just ActiveRecord::Migration in Python?
--------------------------------------------------
Superficially, yes, but not really.
There is a degree of similarity - the Django Evolution syntax is
a DSL for describing the changes that have been made to a model.
However, there are also some significant differences. In addition to
ActiveRecord::Migration, Django Evolution provides:
1. An audit trail - a permanent archive in the database of the
changes that have been applied, and when.
2. An automated hinting scheme. If you don't want to write the
migrations yourself, you don't have to.
3. Validation that an evolution script will leave the database
in a state consistent with the current model definition.
These differences are largely afforded by the model-centric design of
Django itself. Whereas a Ruby on Rails model is a description of a
database that has been created by hand, Django uses the Python model
to creates the database. As a result, the model definition is canonical
- not the database.
This means that audit, hinting, and verification schemes can use the
Django model as a point of reference. A Django Evolution script is
much more than just an alternate syntax for SQL ALTER statements - it
is an evolution scheme that is bound to the canonical model definition.
.. admonition:: Ultracrepidation
We're not Rails developers, so when we talk about Ruby on Rails and
ActiveRecord::Migration, it's not from extensive personal experience.
If we have misrepresented or misunderstood Ruby on Rails or
ActiveRecord::Migration, feel free to correct us.
Why do you generate signatures based on the model, rather than introspecting the database?
------------------------------------------------------------------------------------------
There are two arguments for using model-based signatures rather than database
introspection: One is practical; the other is philosophical.
The practical argument is that introspection is _much_ harder to do
properly. To prove the point: ``inspectdb`` has been in Django since
the beginning, and it still doesn't work right 100% of the time.
Introspection of fields with custom column names or custom field types
will always be very difficult to resolve. By using model-based signatures,
we always have access to all the properties that Django considers significant,
in a format directly relevant to Django - as a result, it was possible to
get model-based signatures working with pretty much 100% accuracy in
relatively short order.
The philosophical argument is that by making the model the canonical
reference point (rather than the database) it puts the emphasis on
Django as the centre of the process, and discourages external
modification. This has the side effect of encouraging users to store
(which is to say *document*) all the changes they make to the database,
rather than just poking the schema as they see fit.
This also has the advantage of being consistent with general Django
philosophy - for example, in default usage, you write a Django model,
and Django creates the tables. You *can* manually create the tables if
you want, and you *can* manually tweak the tables if you need to - but
if you do, Django doesn't guarantee that that anything will work as
expected. Django provides initial SQL hooks that implicitly encourage
you to store/document any external modifications. Django doesn't go
out of its way to make life difficult in this regard, but the core
design of Django isn't going to change fundamentally in order to
accommodate externalities.
Contributing code
=================
How can I get started contributing code to Django Evolution?
------------------------------------------------------------
Thanks for asking! If you're interested in contributing to Django Evolution,
please send a private email to Christian. However, be advised that if you
you want your request to be taken seriously, you need to have a track record
of producing high quality patches.
A good place to start would be to pick a non-trivial missing feature from the
ticket database, and implement a fix.
I submitted a bug fix in the ticket system several weeks ago. Why are you ignoring my patch?
--------------------------------------------------------------------------------------------
Don't worry: We're not ignoring you!
It's important to understand there is a difference between "a ticket is being
ignored" and "a ticket has not been attended to yet." Django Evolution is
a volunteer project, and the core developers may sometimes get distracted by
other things; if this happens, a ticket may sit unattended for a while. Don't
worry - we'll get to your issue eventually.
Besides, if your feature request stands no chance of inclusion in Django Evolution,
we won't ignore it -- we'll just close the ticket. So if your ticket is still
open, it doesn't mean we're ignoring you; it just means we haven't had time to
look at it yet.
