=====================
How is Django Formed?
=====================
.. highlight:: console
This document explains how to release Django.
**Please, keep these instructions up-to-date if you make changes!** The point
here is to be descriptive, not prescriptive, so feel free to streamline or
otherwise make changes, but **update this document accordingly!**
Overview
========
There are three types of releases that you might need to make:
* Security releases: disclosing and fixing a vulnerability. This'll
generally involve two or three simultaneous releases -- e.g.
1.5.x, 1.6.x, and, depending on timing, perhaps a 1.7 alpha/beta/rc.
* Regular version releases: either a final release (e.g. 1.5) or a
bugfix update (e.g. 1.5.1).
* Pre-releases: e.g. 1.6 alpha, beta, or rc.
The short version of the steps involved is:
#. If this is a security release, pre-notify the security distribution list
one week before the actual release.
#. Proofread the release notes, looking for organization and writing errors.
Draft a blog post and email announcement.
#. Update version numbers and create the release package(s).
#. Upload the package(s) to the ``djangoproject.com`` server.
#. Upload the new version(s) to PyPI.
#. Declare the new version in the admin on ``djangoproject.com``.
#. Post the blog entry and send out the email announcements.
#. Update version numbers post-release.
There are a lot of details, so please read on.
Prerequisites
=============
You'll need a few things before getting started:
* A GPG key. If the key you want to use is not your default signing key, you'll
need to add ``-u you@example.com`` to every GPG signing command below, where
``you@example.com`` is the email address associated with the key you want to
use.
* An install of some required Python packages::
$ pip install wheel twine
* Access to Django's record on PyPI. Create a file with your credentials:
.. snippet::
:filename: ~/.pypirc
[pypi]
username:YourUsername
password:YourPassword
* Access to the ``djangoproject.com`` server to upload files.
* Access to the admin on ``djangoproject.com`` as a "Site maintainer".
* Access to post to ``django-announce``.
* If this is a security release, access to the pre-notification distribution
list.
If this is your first release, you'll need to coordinate with James and/or
Jacob to get all these things lined up.
Pre-release tasks
=================
A few items need to be taken care of before even beginning the release process.
This stuff starts about a week before the release; most of it can be done
any time leading up to the actual release:
#. If this is a security release, send out pre-notification **one week** before
the release. We maintain a list of who gets these pre-notification emails in
the private ``django-core`` repository. Send the mail to
``security@djangoproject.com`` and BCC the pre-notification recipients.
This email should be signed by the key you'll use for the release and
should include `CVE IDs <https://cveform.mitre.org/>`_ (requested with
Vendor: djangoproject, Product: django) and patches for each issue being
fixed. Also, :ref:`notify django-announce <security-disclosure>` of the
upcoming security release.
#. As the release approaches, watch Trac to make sure no release blockers
are left for the upcoming release.
#. Check with the other committers to make sure they don't have any
uncommitted changes for the release.
#. Proofread the release notes, including looking at the online
version to catch any broken links or reST errors, and make sure the
release notes contain the correct date.
#. Double-check that the release notes mention deprecation timelines
for any APIs noted as deprecated, and that they mention any changes
in Python version support.
#. Double-check that the release notes index has a link to the notes
for the new release; this will be in ``docs/releases/index.txt``.
#. If this is a feature release, ensure translations from Transifex have been
integrated. This is typically done by a separate translation's manager
rather than the releaser, but here are the steps. Provided you have an
account on Transifex::
$ python scripts/manage_translations.py fetch
and then commit the changed/added files (both .po and .mo). Sometimes there
are validation errors which need to be debugged, so avoid doing this task
immediately before a release is needed.
#. :ref:`Update the django-admin manual page <django-admin-manpage>`::
$ cd docs
$ make man
$ man _build/man/django-admin.1 # do a quick sanity check
$ cp _build/man/django-admin.1 man/django-admin.1
and then commit the changed man page.
Preparing for release
=====================
Write the announcement blog post for the release. You can enter it into the
admin at any time and mark it as inactive. Here are a few examples: `example
security release announcement`__, `example regular release announcement`__,
`example pre-release announcement`__.
__ https://www.djangoproject.com/weblog/2013/feb/19/security/
__ https://www.djangoproject.com/weblog/2012/mar/23/14/
__ https://www.djangoproject.com/weblog/2012/nov/27/15-beta-1/
Actually rolling the release
============================
OK, this is the fun part, where we actually push out a release!
#. Check `Jenkins`__ is green for the version(s) you're putting out. You
probably shouldn't issue a release until it's green.
__ https://djangoci.com
#. A release always begins from a release branch, so you should make sure
you're on a stable branch and up-to-date. For example::
$ git checkout stable/1.5.x
$ git pull
#. If this is a security release, merge the appropriate patches from
``django-private``. Rebase these patches as necessary to make each one a
simple commit on the release branch rather than a merge commit. To ensure
this, merge them with the ``--ff-only`` flag; for example::
$ git checkout stable/1.5.x
$ git merge --ff-only security/1.5.x
(This assumes ``security/1.5.x`` is a branch in the ``django-private`` repo
containing the necessary security patches for the next release in the 1.5
series.)
If git refuses to merge with ``--ff-only``, switch to the security-patch
branch and rebase it on the branch you are about to merge it into (``git
checkout security/1.5.x; git rebase stable/1.5.x``) and then switch back and
do the merge. Make sure the commit message for each security fix explains
that the commit is a security fix and that an announcement will follow
(:commit:`example security commit <bf39978a53f117ca02e9a0c78b76664a41a54745>`).
#. For a feature release, remove the ``UNDER DEVELOPMENT`` header at the
top of the release notes and add the release date on the next line. For a
patch release, replace ``*Under Development*`` with the release date. Make
this change on all branches where the release notes for a particular version
are located.
#. Update the version number in ``django/__init__.py`` for the release.
Please see `notes on setting the VERSION tuple`_ below for details
on ``VERSION``.
#. If this is a pre-release package, update the "Development Status" trove
classifier in ``setup.py`` to reflect this. Otherwise, make sure the
classifier is set to ``Development Status :: 5 - Production/Stable``.
#. Tag the release using ``git tag``. For example::
$ git tag --sign --message="Tag 1.5.1" 1.5.1
You can check your work by running ``git tag --verify <tag>``.
#. Push your work, including the tag: ``git push --tags``.
#. Make sure you have an absolutely clean tree by running ``git clean -dfx``.
#. Run ``make -f extras/Makefile`` to generate the release packages. This will
create the release packages in a ``dist/`` directory.
#. Generate the hashes of the release packages::
$ cd dist
$ md5sum *
$ sha1sum *
$ sha256sum *
#. Create a "checksums" file, ``Django-<<VERSION>>.checksum.txt`` containing
the hashes and release information. Start with this template and insert the
correct version, date, GPG key ID (from
``gpg --list-keys --keyid-format LONG``), release URL, and checksums:
.. code-block:: text
This file contains MD5, SHA1, and SHA256 checksums for the source-code
tarball and wheel files of Django <<VERSION>>, released <<DATE>>.
To use this file, you will need a working install of PGP or other
compatible public-key encryption software. You will also need to have
the Django release manager's public key in your keyring; this key has
the ID ``XXXXXXXXXXXXXXXX`` and can be imported from the MIT
keyserver. For example, if using the open-source GNU Privacy Guard
implementation of PGP:
gpg --keyserver pgp.mit.edu --recv-key XXXXXXXXXXXXXXXX
Once the key is imported, verify this file::
gpg --verify <<THIS FILENAME>>
Once you have verified this file, you can use normal MD5, SHA1, or SHA256
checksumming applications to generate the checksums of the Django
package and compare them to the checksums listed below.
Release packages:
=================
https://www.djangoproject.com/m/releases/<<RELEASE TAR.GZ FILENAME>>
https://www.djangoproject.com/m/releases/<<RELEASE WHL FILENAME>>
MD5 checksums:
==============
<<MD5SUM>> <<RELEASE TAR.GZ FILENAME>>
<<MD5SUM>> <<RELEASE WHL FILENAME>>
SHA1 checksums:
===============
<<SHA1SUM>> <<RELEASE TAR.GZ FILENAME>>
<<SHA1SUM>> <<RELEASE WHL FILENAME>>
SHA256 checksums:
=================
<<SHA256SUM>> <<RELEASE TAR.GZ FILENAME>>
<<SHA256SUM>> <<RELEASE WHL FILENAME>>
#. Sign the checksum file (``gpg --clearsign --digest-algo SHA256
Django-<version>.checksum.txt``). This generates a signed document,
``Django-<version>.checksum.txt.asc`` which you can then verify using ``gpg
--verify Django-<version>.checksum.txt.asc``.
If you're issuing multiple releases, repeat these steps for each release.
Making the release(s) available to the public
=============================================
Now you're ready to actually put the release out there. To do this:
#. Upload the release package(s) to the djangoproject server, replacing
A.B. with the appropriate version number, e.g. 1.5 for a 1.5.x release::
$ scp Django-* djangoproject.com:/home/www/www/media/releases/A.B
#. Upload the checksum file(s)::
$ scp Django-A.B.C.checksum.txt.asc djangoproject.com:/home/www/www/media/pgp/Django-A.B.C.checksum.txt
#. Test that the release packages install correctly using ``easy_install``
and ``pip``. Here's one method (which requires `virtualenvwrapper`__)::
$ RELEASE_VERSION='1.7.2'
$ MAJOR_VERSION=`echo $RELEASE_VERSION| cut -c 1-3`
$ mktmpenv
$ easy_install https://www.djangoproject.com/m/releases/$MAJOR_VERSION/Django-$RELEASE_VERSION.tar.gz
$ deactivate
$ mktmpenv
$ pip install https://www.djangoproject.com/m/releases/$MAJOR_VERSION/Django-$RELEASE_VERSION.tar.gz
$ deactivate
$ mktmpenv
$ pip install https://www.djangoproject.com/m/releases/$MAJOR_VERSION/Django-$RELEASE_VERSION-py2.py3-none-any.whl
$ deactivate
This just tests that the tarballs are available (i.e. redirects are up) and
that they install correctly, but it'll catch silly mistakes.
__ https://pypi.python.org/pypi/virtualenvwrapper
#. Ask a few people on IRC to verify the checksums by visiting the checksums
file (e.g. https://www.djangoproject.com/m/pgp/Django-1.5b1.checksum.txt)
and following the instructions in it. For bonus points, they can also unpack
the downloaded release tarball and verify that its contents appear to be
correct (proper version numbers, no stray ``.pyc`` or other undesirable
files).
#. Upload the release packages to PyPI (for pre-releases, only upload the wheel
file)::
$ twine upload -s dist/*
#. Go to the `Add release page in the admin`__, enter the new release number
exactly as it appears in the name of the tarball (Django-<version>.tar.gz).
So for example enter "1.5.1" or "1.4c2", etc. If the release is part of
an LTS branch, mark it so.
__ https://www.djangoproject.com/admin/releases/release/add/
#. Make the blog post announcing the release live.
#. For a new version release (e.g. 1.5, 1.6), update the default stable version
of the docs by flipping the ``is_default`` flag to ``True`` on the
appropriate ``DocumentRelease`` object in the ``docs.djangoproject.com``
database (this will automatically flip it to ``False`` for all
others); you can do this using the site's admin.
#. Post the release announcement to the |django-announce|, |django-developers|,
and |django-users| mailing lists. This should include a link to the
announcement blog post. If this is a security release, also include
oss-security@lists.openwall.com.
#. Add a link to the blog post in the topic of the `#django` IRC channel:
``/msg chanserv TOPIC #django new topic goes here``.
Post-release
============
You're almost done! All that's left to do now is:
#. Update the ``VERSION`` tuple in ``django/__init__.py`` again,
incrementing to whatever the next expected release will be. For
example, after releasing 1.5.1, update ``VERSION`` to
``VERSION = (1, 5, 2, 'alpha', 0)``.
#. Add the release in `Trac's versions list`_ if necessary (and make it the
default if it's a final release). Not all versions are declared;
take example on previous releases.
#. If this was a security release, update :doc:`/releases/security` with
details of the issues addressed.
.. _Trac's versions list: https://code.djangoproject.com/admin/ticket/versions
New stable branch tasks
=======================
There are several items to do in the time following the creation of a new
stable branch (often following an alpha release). Some of these tasks don't
need to be done by the releaser.
#. Create a new ``DocumentRelease`` object in the ``docs.djangoproject.com``
database for the new version's docs, and update the
``docs/fixtures/doc_releases.json`` JSON fixture, so people without access
to the production DB can still run an up-to-date copy of the docs site.
#. Create a stub release note for the new feature version. Use the stub from
the previous feature release version or copy the contents from the previous
feature version and delete most of the contents leaving only the headings.
#. Increase the default PBKDF2 iterations in
``django.contrib.auth.hashers.PBKDF2PasswordHasher`` by about 20%
(pick a round number). Run the tests, and update the 3 failing
hasher tests with the new values. Make sure this gets noted in the
release notes (see the 1.8 release notes for an example).
#. Remove features that have reached the end of their deprecation cycle. Each
removal should be done in a separate commit for clarity. In the commit
message, add a "refs #XXXX" to the original ticket where the deprecation
began if possible.
#. Remove ``.. versionadded::``, ``.. versionadded::``, and ``.. deprecated::``
annotations in the documentation from two releases ago. For example, in
Django 1.9, notes for 1.7 will be removed.
#. Add the new branch to `Read the Docs
<https://readthedocs.org/projects/django/>`_. Since the automatically
generated version names ("stable-A.B.x") differ from the version numbers
we've used historically in Read the Docs ("A.B.x"), we currently ask Eric
Holscher to add the version for us. Someday the alias functionality may be
built-in to the Read the Docs UI.
Notes on setting the VERSION tuple
==================================
Django's version reporting is controlled by the ``VERSION`` tuple in
``django/__init__.py``. This is a five-element tuple, whose elements
are:
#. Major version.
#. Minor version.
#. Micro version.
#. Status -- can be one of "alpha", "beta", "rc" or "final".
#. Series number, for alpha/beta/RC packages which run in sequence
(allowing, for example, "beta 1", "beta 2", etc.).
For a final release, the status is always "final" and the series
number is always 0. A series number of 0 with an "alpha" status will
be reported as "pre-alpha".
Some examples:
* ``(1, 2, 1, 'final', 0)`` â "1.2.1"
* ``(1, 3, 0, 'alpha', 0)`` â "1.3 pre-alpha"
* ``(1, 3, 0, 'beta', 2)`` â "1.3 beta 2"
