<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>
What’s New in SQLAlchemy 1.1?
—
SQLAlchemy 1.2 Documentation
</title>
<!-- begin iterate through site-imported + sphinx environment css_files -->
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../_static/docs.css" type="text/css" />
<link rel="stylesheet" href="../_static/changelog.css" type="text/css" />
<link rel="stylesheet" href="../_static/sphinx_paramlinks.css" type="text/css" />
<!-- end iterate through site-imported + sphinx environment css_files -->
<!-- begin layout.mako headers -->
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="copyright" title="Copyright" href="../copyright.html" />
<link rel="top" title="SQLAlchemy 1.2 Documentation" href="../index.html" />
<link rel="up" title="Changes and Migration" href="index.html" />
<link rel="next" title="What’s New in SQLAlchemy 1.0?" href="migration_10.html" />
<link rel="prev" title="0.1 Changelog" href="changelog_01.html" />
<!-- end layout.mako headers -->
</head>
<body>
<div id="docs-container">
<div id="docs-top-navigation-container" class="body-background">
<div id="docs-header">
<div id="docs-version-header">
Release: <span class="version-num">1.2.19</span>
| Release Date: April 15, 2019
</div>
<h1>SQLAlchemy 1.2 Documentation</h1>
</div>
</div>
<div id="docs-body-container">
<div id="fixed-sidebar" class="withsidebar">
<div id="docs-sidebar-popout">
<h3><a href="../index.html">SQLAlchemy 1.2 Documentation</a></h3>
<p id="sidebar-topnav">
<a href="../contents.html">Contents</a> |
<a href="../genindex.html">Index</a>
</p>
<div id="sidebar-search">
<form class="search" action="../search.html" method="get">
<label>
Search terms:
<input type="text" placeholder="search..." name="q" size="12" />
</label>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div id="docs-sidebar">
<div id="sidebar-banner">
</div>
<div id="docs-sidebar-inner">
<h3>
<a href="index.html" title="Changes and Migration">Changes and Migration</a>
</h3>
<ul>
<li><span class="link-container"><a class="reference external" href="migration_12.html">What’s New in SQLAlchemy 1.2?</a></span></li>
<li><span class="link-container"><a class="reference external" href="changelog_12.html">1.2 Changelog</a></span></li>
<li><span class="link-container"><a class="reference external" href="changelog_11.html">1.1 Changelog</a></span></li>
<li><span class="link-container"><a class="reference external" href="changelog_10.html">1.0 Changelog</a></span></li>
<li><span class="link-container"><a class="reference external" href="changelog_09.html">0.9 Changelog</a></span></li>
<li><span class="link-container"><a class="reference external" href="changelog_08.html">0.8 Changelog</a></span></li>
<li><span class="link-container"><a class="reference external" href="changelog_07.html">0.7 Changelog</a></span></li>
<li><span class="link-container"><a class="reference external" href="changelog_06.html">0.6 Changelog</a></span></li>
<li><span class="link-container"><a class="reference external" href="changelog_05.html">0.5 Changelog</a></span></li>
<li><span class="link-container"><a class="reference external" href="changelog_04.html">0.4 Changelog</a></span></li>
<li><span class="link-container"><a class="reference external" href="changelog_03.html">0.3 Changelog</a></span></li>
<li><span class="link-container"><a class="reference external" href="changelog_02.html">0.2 Changelog</a></span></li>
<li><span class="link-container"><a class="reference external" href="changelog_01.html">0.1 Changelog</a></span></li>
<li class="selected"><span class="link-container"><strong>What’s New in SQLAlchemy 1.1?</strong><a class="paramlink headerlink reference internal" href="#">¶</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#introduction">Introduction</a></span></li>
<li><span class="link-container"><a class="reference external" href="#platform-installer-changes">Platform / Installer Changes</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#setuptools-is-now-required-for-install">Setuptools is now required for install</a></span></li>
<li><span class="link-container"><a class="reference external" href="#enabling-disabling-c-extension-builds-is-only-via-environment-variable">Enabling / Disabling C Extension builds is only via environment variable</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#new-features-and-improvements-orm">New Features and Improvements - ORM</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#new-session-lifecycle-events">New Session lifecycle events</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#new-state-transition-events">New State Transition Events</a></span></li>
<li><span class="link-container"><a class="reference external" href="#new-object-state-deleted-is-added-deleted-objects-no-longer-persistent">New Object State “deleted” is added, deleted objects no longer “persistent”</a></span></li>
<li><span class="link-container"><a class="reference external" href="#strong-identity-map-is-deprecated">Strong Identity Map is Deprecated</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#new-init-scalar-event-intercepts-default-values-at-orm-level">New init_scalar() event intercepts default values at ORM level</a></span></li>
<li><span class="link-container"><a class="reference external" href="#changes-regarding-unhashable-types-impacts-deduping-of-orm-rows">Changes regarding “unhashable” types, impacts deduping of ORM rows</a></span></li>
<li><span class="link-container"><a class="reference external" href="#specific-checks-added-for-passing-mapped-classes-instances-as-sql-literals">Specific checks added for passing mapped classes, instances as SQL literals</a></span></li>
<li><span class="link-container"><a class="reference external" href="#new-indexable-orm-extension">New Indexable ORM extension</a></span></li>
<li><span class="link-container"><a class="reference external" href="#new-options-allowing-explicit-persistence-of-null-over-a-default">New options allowing explicit persistence of NULL over a default</a></span></li>
<li><span class="link-container"><a class="reference external" href="#further-fixes-to-single-table-inheritance-querying">Further Fixes to single-table inheritance querying</a></span></li>
<li><span class="link-container"><a class="reference external" href="#improved-session-state-when-a-savepoint-is-cancelled-by-the-database">Improved Session state when a SAVEPOINT is cancelled by the database</a></span></li>
<li><span class="link-container"><a class="reference external" href="#erroneous-new-instance-x-conflicts-with-persistent-instance-y-flush-errors-fixed">Erroneous “new instance X conflicts with persistent instance Y” flush errors fixed</a></span></li>
<li><span class="link-container"><a class="reference external" href="#passive-deletes-feature-for-joined-inheritance-mappings">passive_deletes feature for joined-inheritance mappings</a></span></li>
<li><span class="link-container"><a class="reference external" href="#same-named-backrefs-will-not-raise-an-error-when-applied-to-concrete-inheritance-subclasses">Same-named backrefs will not raise an error when applied to concrete inheritance subclasses</a></span></li>
<li><span class="link-container"><a class="reference external" href="#same-named-relationships-on-inheriting-mappers-no-longer-warn">Same-named relationships on inheriting mappers no longer warn</a></span></li>
<li><span class="link-container"><a class="reference external" href="#hybrid-properties-and-methods-now-propagate-the-docstring-as-well-as-info">Hybrid properties and methods now propagate the docstring as well as .info</a></span></li>
<li><span class="link-container"><a class="reference external" href="#session-merge-resolves-pending-conflicts-the-same-as-persistent">Session.merge resolves pending conflicts the same as persistent</a></span></li>
<li><span class="link-container"><a class="reference external" href="#fix-involving-many-to-one-object-moves-with-user-initiated-foreign-key-manipulations">Fix involving many-to-one object moves with user-initiated foreign key manipulations</a></span></li>
<li><span class="link-container"><a class="reference external" href="#improvements-to-the-query-correlate-method-with-polymorphic-entities">Improvements to the Query.correlate method with polymorphic entities</a></span></li>
<li><span class="link-container"><a class="reference external" href="#stringify-of-query-will-consult-the-session-for-the-correct-dialect">Stringify of Query will consult the Session for the correct dialect</a></span></li>
<li><span class="link-container"><a class="reference external" href="#joined-eager-loading-where-the-same-entity-is-present-multiple-times-in-one-row">Joined eager loading where the same entity is present multiple times in one row</a></span></li>
<li><span class="link-container"><a class="reference external" href="#new-mutablelist-and-mutableset-helpers-added-to-the-mutation-tracking-extension">New MutableList and MutableSet helpers added to the mutation tracking extension</a></span></li>
<li><span class="link-container"><a class="reference external" href="#new-raise-raise-on-sql-loader-strategies">New “raise” / “raise_on_sql” loader strategies</a></span></li>
<li><span class="link-container"><a class="reference external" href="#mapper-order-by-is-deprecated">Mapper.order_by is deprecated</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#new-features-and-improvements-core">New Features and Improvements - Core</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#engines-now-invalidate-connections-run-error-handlers-for-baseexception">Engines now invalidate connections, run error handlers for BaseException</a></span></li>
<li><span class="link-container"><a class="reference external" href="#cte-support-for-insert-update-delete">CTE Support for INSERT, UPDATE, DELETE</a></span></li>
<li><span class="link-container"><a class="reference external" href="#support-for-range-and-rows-specification-within-window-functions">Support for RANGE and ROWS specification within window functions</a></span></li>
<li><span class="link-container"><a class="reference external" href="#support-for-the-sql-lateral-keyword">Support for the SQL LATERAL keyword</a></span></li>
<li><span class="link-container"><a class="reference external" href="#support-for-tablesample">Support for TABLESAMPLE</a></span></li>
<li><span class="link-container"><a class="reference external" href="#the-autoincrement-directive-is-no-longer-implicitly-enabled-for-a-composite-primary-key-column">The <code class="docutils literal notranslate"><span class="pre">.autoincrement</span></code> directive is no longer implicitly enabled for a composite primary key column</a></span></li>
<li><span class="link-container"><a class="reference external" href="#support-for-is-distinct-from-and-is-not-distinct-from">Support for IS DISTINCT FROM and IS NOT DISTINCT FROM</a></span></li>
<li><span class="link-container"><a class="reference external" href="#core-and-orm-support-for-full-outer-join">Core and ORM support for FULL OUTER JOIN</a></span></li>
<li><span class="link-container"><a class="reference external" href="#resultset-column-matching-enhancements-positional-column-setup-for-textual-sql">ResultSet column matching enhancements; positional column setup for textual SQL</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#textasfrom-columns-now-works-positionally">TextAsFrom.columns() now works positionally</a></span></li>
<li><span class="link-container"><a class="reference external" href="#positional-matching-is-trusted-over-name-based-matching-for-core-orm-sql-constructs">Positional matching is trusted over name-based matching for Core/ORM SQL constructs</a></span></li>
<li><span class="link-container"><a class="reference external" href="#much-less-likely-to-get-an-ambiguous-column-error-message">Much less likely to get an “ambiguous column” error message</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#support-for-python-s-native-enum-type-and-compatible-forms">Support for Python’s native <code class="docutils literal notranslate"><span class="pre">enum</span></code> type and compatible forms</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#the-enum-enums-collection-is-now-a-list-instead-of-a-tuple">The <code class="docutils literal notranslate"><span class="pre">Enum.enums</span></code> collection is now a list instead of a tuple</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#negative-integer-indexes-accommodated-by-core-result-rows">Negative integer indexes accommodated by Core result rows</a></span></li>
<li><span class="link-container"><a class="reference external" href="#the-enum-type-now-does-in-python-validation-of-values">The <code class="docutils literal notranslate"><span class="pre">Enum</span></code> type now does in-Python validation of values</a></span></li>
<li><span class="link-container"><a class="reference external" href="#non-native-boolean-integer-values-coerced-to-zero-one-none-in-all-cases">Non-native boolean integer values coerced to zero/one/None in all cases</a></span></li>
<li><span class="link-container"><a class="reference external" href="#large-parameter-and-row-values-are-now-truncated-in-logging-and-exception-displays">Large parameter and row values are now truncated in logging and exception displays</a></span></li>
<li><span class="link-container"><a class="reference external" href="#json-support-added-to-core">JSON support added to Core</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#json-null-is-inserted-as-expected-with-orm-operations-omitted-when-not-present">JSON “null” is inserted as expected with ORM operations, omitted when not present</a></span></li>
<li><span class="link-container"><a class="reference external" href="#new-json-null-constant-added">New JSON.NULL Constant Added</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#array-support-added-to-core-new-any-and-all-operators">Array support added to Core; new ANY and ALL operators</a></span></li>
<li><span class="link-container"><a class="reference external" href="#new-function-features-within-group-array-agg-and-set-aggregate-functions">New Function features, “WITHIN GROUP”, array_agg and set aggregate functions</a></span></li>
<li><span class="link-container"><a class="reference external" href="#typedecorator-now-works-with-enum-boolean-schema-types-automatically">TypeDecorator now works with Enum, Boolean, “schema” types automatically</a></span></li>
<li><span class="link-container"><a class="reference external" href="#multi-tenancy-schema-translation-for-table-objects">Multi-Tenancy Schema Translation for Table objects</a></span></li>
<li><span class="link-container"><a class="reference external" href="#friendly-stringification-of-core-sql-constructs-without-a-dialect">“Friendly” stringification of Core SQL constructs without a dialect</a></span></li>
<li><span class="link-container"><a class="reference external" href="#the-type-coerce-function-is-now-a-persistent-sql-element">The type_coerce function is now a persistent SQL element</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#key-behavioral-changes-orm">Key Behavioral Changes - ORM</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#json-columns-will-not-insert-json-null-if-no-value-is-supplied-and-no-default-is-established">JSON Columns will not insert JSON NULL if no value is supplied and no default is established</a></span></li>
<li><span class="link-container"><a class="reference external" href="#columns-no-longer-added-redundantly-with-distinct-order-by">Columns no longer added redundantly with DISTINCT + ORDER BY</a></span></li>
<li><span class="link-container"><a class="reference external" href="#same-named-validates-decorators-will-now-raise-an-exception">Same-named @validates decorators will now raise an exception</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#key-behavioral-changes-core">Key Behavioral Changes - Core</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#textclause-columns-will-match-columns-positionally-not-by-name-when-passed-positionally">TextClause.columns() will match columns positionally, not by name, when passed positionally</a></span></li>
<li><span class="link-container"><a class="reference external" href="#string-server-default-now-literal-quoted">String server_default now literal quoted</a></span></li>
<li><span class="link-container"><a class="reference external" href="#a-union-or-similar-of-selects-with-limit-offset-order-by-now-parenthesizes-the-embedded-selects">A UNION or similar of SELECTs with LIMIT/OFFSET/ORDER BY now parenthesizes the embedded selects</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#dialect-improvements-and-changes-postgresql">Dialect Improvements and Changes - PostgreSQL</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#support-for-insert-on-conflict-do-update-do-nothing">Support for INSERT..ON CONFLICT (DO UPDATE | DO NOTHING)</a></span></li>
<li><span class="link-container"><a class="reference external" href="#array-and-json-types-now-correctly-specify-unhashable">ARRAY and JSON types now correctly specify “unhashable”</a></span></li>
<li><span class="link-container"><a class="reference external" href="#correct-sql-types-are-established-from-indexed-access-of-array-json-hstore">Correct SQL Types are Established from Indexed Access of ARRAY, JSON, HSTORE</a></span></li>
<li><span class="link-container"><a class="reference external" href="#the-json-cast-operation-now-requires-astext-is-called-explicitly">The JSON cast() operation now requires <code class="docutils literal notranslate"><span class="pre">.astext</span></code> is called explicitly</a></span></li>
<li><span class="link-container"><a class="reference external" href="#array-with-enum-will-now-emit-create-type-for-the-enum">ARRAY with ENUM will now emit CREATE TYPE for the ENUM</a></span></li>
<li><span class="link-container"><a class="reference external" href="#check-constraints-now-reflect">Check constraints now reflect</a></span></li>
<li><span class="link-container"><a class="reference external" href="#plain-and-materialized-views-can-be-inspected-separately">“Plain” and “Materialized” views can be inspected separately</a></span></li>
<li><span class="link-container"><a class="reference external" href="#added-tablespace-option-to-index">Added tablespace option to Index</a></span></li>
<li><span class="link-container"><a class="reference external" href="#support-for-pygresql">Support for PyGreSQL</a></span></li>
<li><span class="link-container"><a class="reference external" href="#the-postgres-module-is-removed">The “postgres” module is removed</a></span></li>
<li><span class="link-container"><a class="reference external" href="#support-for-for-update-skip-locked-for-no-key-update-for-key-share">Support for FOR UPDATE SKIP LOCKED / FOR NO KEY UPDATE / FOR KEY SHARE</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#dialect-improvements-and-changes-mysql">Dialect Improvements and Changes - MySQL</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#mysql-json-support">MySQL JSON Support</a></span></li>
<li><span class="link-container"><a class="reference external" href="#added-support-for-autocommit-isolation-level">Added support for AUTOCOMMIT “isolation level”</a></span></li>
<li><span class="link-container"><a class="reference external" href="#no-more-generation-of-an-implicit-key-for-composite-primary-key-w-auto-increment">No more generation of an implicit KEY for composite primary key w/ AUTO_INCREMENT</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#dialect-improvements-and-changes-sqlite">Dialect Improvements and Changes - SQLite</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#right-nested-join-workaround-lifted-for-sqlite-version-3-7-16">Right-nested join workaround lifted for SQLite version 3.7.16</a></span></li>
<li><span class="link-container"><a class="reference external" href="#dotted-column-names-workaround-lifted-for-sqlite-version-3-10-0">Dotted column names workaround lifted for SQLite version 3.10.0</a></span></li>
<li><span class="link-container"><a class="reference external" href="#improved-support-for-remote-schemas">Improved Support for Remote Schemas</a></span></li>
<li><span class="link-container"><a class="reference external" href="#reflection-of-the-name-of-primary-key-constraints">Reflection of the name of PRIMARY KEY constraints</a></span></li>
<li><span class="link-container"><a class="reference external" href="#id1">Check constraints now reflect</a></span></li>
<li><span class="link-container"><a class="reference external" href="#on-delete-and-on-update-foreign-key-phrases-now-reflect">ON DELETE and ON UPDATE foreign key phrases now reflect</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#dialect-improvements-and-changes-sql-server">Dialect Improvements and Changes - SQL Server</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#added-transaction-isolation-level-support-for-sql-server">Added transaction isolation level support for SQL Server</a></span></li>
<li><span class="link-container"><a class="reference external" href="#string-varlength-types-no-longer-represent-max-explicitly-on-reflection">String / varlength types no longer represent “max” explicitly on reflection</a></span></li>
<li><span class="link-container"><a class="reference external" href="#support-for-non-clustered-on-primary-key-to-allow-clustered-elsewhere">Support for “non clustered” on primary key to allow clustered elsewhere</a></span></li>
<li><span class="link-container"><a class="reference external" href="#the-legacy-schema-aliasing-flag-is-now-set-to-false">The legacy_schema_aliasing flag is now set to False</a></span></li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="#dialect-improvements-and-changes-oracle">Dialect Improvements and Changes - Oracle</a></span><ul>
<li><span class="link-container"><a class="reference external" href="#support-for-skip-locked">Support for SKIP LOCKED</a></span></li>
</ul>
</li>
</ul>
</li>
<li><span class="link-container"><a class="reference external" href="migration_10.html">What’s New in SQLAlchemy 1.0?</a></span></li>
<li><span class="link-container"><a class="reference external" href="migration_09.html">What’s New in SQLAlchemy 0.9?</a></span></li>
<li><span class="link-container"><a class="reference external" href="migration_08.html">What’s New in SQLAlchemy 0.8?</a></span></li>
<li><span class="link-container"><a class="reference external" href="migration_07.html">What’s New in SQLAlchemy 0.7?</a></span></li>
<li><span class="link-container"><a class="reference external" href="migration_06.html">What’s New in SQLAlchemy 0.6?</a></span></li>
<li><span class="link-container"><a class="reference external" href="migration_05.html">What’s new in SQLAlchemy 0.5?</a></span></li>
<li><span class="link-container"><a class="reference external" href="migration_04.html">What’s new in SQLAlchemy 0.4?</a></span></li>
</ul>
</div>
</div>
</div>
<div id="docs-body" class="withsidebar" >
<div class="section" id="what-s-new-in-sqlalchemy-1-1">
<h1>What’s New in SQLAlchemy 1.1?<a class="headerlink" href="#what-s-new-in-sqlalchemy-1-1" title="Permalink to this headline">¶</a></h1>
<div class="admonition-about-this-document admonition">
<p class="admonition-title">About this Document</p>
<p>This document describes changes between SQLAlchemy version 1.0
and SQLAlchemy version 1.1.</p>
</div>
<div class="section" id="introduction">
<h2>Introduction<a class="headerlink" href="#introduction" title="Permalink to this headline">¶</a></h2>
<p>This guide introduces what’s new in SQLAlchemy version 1.1,
and also documents changes which affect users migrating
their applications from the 1.0 series of SQLAlchemy to 1.1.</p>
<p>Please carefully review the sections on behavioral changes for
potentially backwards-incompatible changes in behavior.</p>
</div>
<div class="section" id="platform-installer-changes">
<h2>Platform / Installer Changes<a class="headerlink" href="#platform-installer-changes" title="Permalink to this headline">¶</a></h2>
<div class="section" id="setuptools-is-now-required-for-install">
<h3>Setuptools is now required for install<a class="headerlink" href="#setuptools-is-now-required-for-install" title="Permalink to this headline">¶</a></h3>
<p>SQLAlchemy’s <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> file has for many years supported operation
both with Setuptools installed and without; supporting a “fallback” mode
that uses straight Distutils. As a Setuptools-less Python environment is
now unheard of, and in order to support the featureset of Setuptools
more fully, in particular to support py.test’s integration with it as well
as things like “extras”, <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> now depends on Setuptools fully.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../intro.html#installation"><span class="std std-ref">Installation Guide</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3489">#3489</a></p>
</div>
<div class="section" id="enabling-disabling-c-extension-builds-is-only-via-environment-variable">
<h3>Enabling / Disabling C Extension builds is only via environment variable<a class="headerlink" href="#enabling-disabling-c-extension-builds-is-only-via-environment-variable" title="Permalink to this headline">¶</a></h3>
<p>The C Extensions build by default during install as long as it is possible.
To disable C extension builds, the <code class="docutils literal notranslate"><span class="pre">DISABLE_SQLALCHEMY_CEXT</span></code> environment
variable was made available as of SQLAlchemy 0.8.6 / 0.9.4. The previous
approach of using the <code class="docutils literal notranslate"><span class="pre">--without-cextensions</span></code> argument has been removed,
as it relies on deprecated features of setuptools.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../intro.html#c-extensions"><span class="std std-ref">Installing the C Extensions</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3500">#3500</a></p>
</div>
</div>
<div class="section" id="new-features-and-improvements-orm">
<h2>New Features and Improvements - ORM<a class="headerlink" href="#new-features-and-improvements-orm" title="Permalink to this headline">¶</a></h2>
<div class="section" id="new-session-lifecycle-events">
<span id="change-2677"></span><h3>New Session lifecycle events<a class="headerlink" href="#new-session-lifecycle-events" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a> has long supported events that allow some degree
of tracking of state changes to objects, including
<a class="reference internal" href="../orm/events.html#sqlalchemy.orm.events.SessionEvents.before_attach" title="sqlalchemy.orm.events.SessionEvents.before_attach"><code class="xref py py-meth docutils literal notranslate"><span class="pre">SessionEvents.before_attach()</span></code></a>, <a class="reference internal" href="../orm/events.html#sqlalchemy.orm.events.SessionEvents.after_attach" title="sqlalchemy.orm.events.SessionEvents.after_attach"><code class="xref py py-meth docutils literal notranslate"><span class="pre">SessionEvents.after_attach()</span></code></a>,
and <a class="reference internal" href="../orm/events.html#sqlalchemy.orm.events.SessionEvents.before_flush" title="sqlalchemy.orm.events.SessionEvents.before_flush"><code class="xref py py-meth docutils literal notranslate"><span class="pre">SessionEvents.before_flush()</span></code></a>. The Session documentation also
documents major object states at <a class="reference internal" href="../orm/session_state_management.html#session-object-states"><span class="std std-ref">Quickie Intro to Object States</span></a>. However,
there has never been system of tracking objects specifically as they
pass through these transitions. Additionally, the status of “deleted” objects
has historically been murky as the objects act somewhere between
the “persistent” and “detached” states.</p>
<p>To clean up this area and allow the realm of session state transition
to be fully transparent, a new series of events have been added that
are intended to cover every possible way that an object might transition
between states, and additionally the “deleted” status has been given
its own official state name within the realm of session object states.</p>
<div class="section" id="new-state-transition-events">
<h4>New State Transition Events<a class="headerlink" href="#new-state-transition-events" title="Permalink to this headline">¶</a></h4>
<p>Transitions between all states of an object such as <a class="reference internal" href="../glossary.html#term-persistent"><span class="xref std std-term">persistent</span></a>,
<a class="reference internal" href="../glossary.html#term-pending"><span class="xref std std-term">pending</span></a> and others can now be intercepted in terms of a
session-level event intended to cover a specific transition.
Transitions as objects move into a <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a>, move out of a
<a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a>, and even all the transitions which occur when the
transaction is rolled back using <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session.rollback" title="sqlalchemy.orm.session.Session.rollback"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Session.rollback()</span></code></a>
are explicitly present in the interface of <a class="reference internal" href="../orm/events.html#sqlalchemy.orm.events.SessionEvents" title="sqlalchemy.orm.events.SessionEvents"><code class="xref py py-class docutils literal notranslate"><span class="pre">SessionEvents</span></code></a>.</p>
<p>In total, there are <strong>ten new events</strong>. A summary of these events is in a
newly written documentation section <a class="reference internal" href="../orm/session_events.html#session-lifecycle-events"><span class="std std-ref">Object Lifecycle Events</span></a>.</p>
</div>
<div class="section" id="new-object-state-deleted-is-added-deleted-objects-no-longer-persistent">
<h4>New Object State “deleted” is added, deleted objects no longer “persistent”<a class="headerlink" href="#new-object-state-deleted-is-added-deleted-objects-no-longer-persistent" title="Permalink to this headline">¶</a></h4>
<p>The <a class="reference internal" href="../glossary.html#term-persistent"><span class="xref std std-term">persistent</span></a> state of an object in the <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a> has
always been documented as an object that has a valid database identity;
however in the case of objects that were deleted within a flush, they
have always been in a grey area where they are not really “detached”
from the <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a> yet, because they can still be restored
within a rollback, but are not really “persistent” because their database
identity has been deleted and they aren’t present in the identity map.</p>
<p>To resolve this grey area given the new events, a new object state
<a class="reference internal" href="../glossary.html#term-deleted"><span class="xref std std-term">deleted</span></a> is introduced. This state exists between the “persistent” and
“detached” states. An object that is marked for deletion via
<a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session.delete" title="sqlalchemy.orm.session.Session.delete"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Session.delete()</span></code></a> remains in the “persistent” state until a flush
proceeds; at that point, it is removed from the identity map, moves
to the “deleted” state, and the <a class="reference internal" href="../orm/events.html#sqlalchemy.orm.events.SessionEvents.persistent_to_deleted" title="sqlalchemy.orm.events.SessionEvents.persistent_to_deleted"><code class="xref py py-meth docutils literal notranslate"><span class="pre">SessionEvents.persistent_to_deleted()</span></code></a>
hook is invoked. If the <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a> object’s transaction is rolled
back, the object is restored as persistent; the
<a class="reference internal" href="../orm/events.html#sqlalchemy.orm.events.SessionEvents.deleted_to_persistent" title="sqlalchemy.orm.events.SessionEvents.deleted_to_persistent"><code class="xref py py-meth docutils literal notranslate"><span class="pre">SessionEvents.deleted_to_persistent()</span></code></a> transition is called. Otherwise
if the <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a> object’s transaction is committed,
the <a class="reference internal" href="../orm/events.html#sqlalchemy.orm.events.SessionEvents.deleted_to_detached" title="sqlalchemy.orm.events.SessionEvents.deleted_to_detached"><code class="xref py py-meth docutils literal notranslate"><span class="pre">SessionEvents.deleted_to_detached()</span></code></a> transition is invoked.</p>
<p>Additionally, the <a class="reference internal" href="../orm/internals.html#sqlalchemy.orm.state.InstanceState.persistent" title="sqlalchemy.orm.state.InstanceState.persistent"><code class="xref py py-attr docutils literal notranslate"><span class="pre">InstanceState.persistent</span></code></a> accessor <strong>no longer returns
True</strong> for an object that is in the new “deleted” state; instead, the
<a class="reference internal" href="../orm/internals.html#sqlalchemy.orm.state.InstanceState.deleted" title="sqlalchemy.orm.state.InstanceState.deleted"><code class="xref py py-attr docutils literal notranslate"><span class="pre">InstanceState.deleted</span></code></a> accessor has been enhanced to reliably
report on this new state. When the object is detached, the <a class="reference internal" href="../orm/internals.html#sqlalchemy.orm.state.InstanceState.deleted" title="sqlalchemy.orm.state.InstanceState.deleted"><code class="xref py py-attr docutils literal notranslate"><span class="pre">InstanceState.deleted</span></code></a>
returns False and the <a class="reference internal" href="../orm/internals.html#sqlalchemy.orm.state.InstanceState.detached" title="sqlalchemy.orm.state.InstanceState.detached"><code class="xref py py-attr docutils literal notranslate"><span class="pre">InstanceState.detached</span></code></a> accessor is True
instead. To determine if an object was deleted either in the current
transaction or in a previous transaction, use the
<a class="reference internal" href="../orm/internals.html#sqlalchemy.orm.state.InstanceState.was_deleted" title="sqlalchemy.orm.state.InstanceState.was_deleted"><code class="xref py py-attr docutils literal notranslate"><span class="pre">InstanceState.was_deleted</span></code></a> accessor.</p>
</div>
<div class="section" id="strong-identity-map-is-deprecated">
<h4>Strong Identity Map is Deprecated<a class="headerlink" href="#strong-identity-map-is-deprecated" title="Permalink to this headline">¶</a></h4>
<p>One of the inspirations for the new series of transition events was to enable
leak-proof tracking of objects as they move in and out of the identity map,
so that a “strong reference” may be maintained mirroring the object
moving in and out of this map. With this new capability, there is no longer
any need for the <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session.params.weak_identity_map" title="sqlalchemy.orm.session.Session"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">Session.weak_identity_map</span></code></a> parameter and the
corresponding <code class="xref py py-class docutils literal notranslate"><span class="pre">StrongIdentityMap</span></code> object. This option has remained
in SQLAlchemy for many years as the “strong-referencing” behavior used to be
the only behavior available, and many applications were written to assume
this behavior. It has long been recommended that strong-reference tracking
of objects not be an intrinsic job of the <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a> and instead
be an application-level construct built as needed by the application; the
new event model allows even the exact behavior of the strong identity map
to be replicated. See <a class="reference internal" href="../orm/session_state_management.html#session-referencing-behavior"><span class="std std-ref">Session Referencing Behavior</span></a> for a new
recipe illustrating how to replace the strong identity map.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/2677">#2677</a></p>
</div>
</div>
<div class="section" id="new-init-scalar-event-intercepts-default-values-at-orm-level">
<span id="change-1311"></span><h3>New init_scalar() event intercepts default values at ORM level<a class="headerlink" href="#new-init-scalar-event-intercepts-default-values-at-orm-level" title="Permalink to this headline">¶</a></h3>
<p>The ORM produces a value of <code class="docutils literal notranslate"><span class="pre">None</span></code> when an attribute that has not been
set is first accessed, for a non-persistent object:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">obj</span> <span class="o">=</span> <span class="n">MyObj</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">obj</span><span class="o">.</span><span class="n">some_value</span>
<span class="go">None</span></pre></div>
</div>
<p>There’s a use case for this in-Python value to correspond to that of a
Core-generated default value, even before the object is persisted.
To suit this use case a new event <a class="reference internal" href="../orm/events.html#sqlalchemy.orm.events.AttributeEvents.init_scalar" title="sqlalchemy.orm.events.AttributeEvents.init_scalar"><code class="xref py py-meth docutils literal notranslate"><span class="pre">AttributeEvents.init_scalar()</span></code></a>
is added. The new example <code class="docutils literal notranslate"><span class="pre">active_column_defaults.py</span></code> at
<a class="reference internal" href="../orm/examples.html#examples-instrumentation"><span class="std std-ref">Attribute Instrumentation</span></a> illustrates a sample use, so the effect
can instead be:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">obj</span> <span class="o">=</span> <span class="n">MyObj</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">obj</span><span class="o">.</span><span class="n">some_value</span>
<span class="go">"my default"</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/1311">#1311</a></p>
</div>
<div class="section" id="changes-regarding-unhashable-types-impacts-deduping-of-orm-rows">
<span id="change-3499"></span><h3>Changes regarding “unhashable” types, impacts deduping of ORM rows<a class="headerlink" href="#changes-regarding-unhashable-types-impacts-deduping-of-orm-rows" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../orm/query.html#sqlalchemy.orm.query.Query" title="sqlalchemy.orm.query.Query"><code class="xref py py-class docutils literal notranslate"><span class="pre">Query</span></code></a> object has a well-known behavior of “deduping”
returned rows that contain at least one ORM-mapped entity (e.g., a
full mapped object, as opposed to individual column values). The
primary purpose of this is so that the handling of entities works
smoothly in conjunction with the identity map, including to
accommodate for the duplicate entities normally represented within
joined eager loading, as well as when joins are used for the purposes
of filtering on additional columns.</p>
<p>This deduplication relies upon the hashability of the elements within
the row. With the introduction of PostgreSQL’s special types like
<a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.ARRAY</span></code></a>, <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.HSTORE" title="sqlalchemy.dialects.postgresql.HSTORE"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.HSTORE</span></code></a> and
<a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON" title="sqlalchemy.dialects.postgresql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.JSON</span></code></a>, the experience of types within rows being
unhashable and encountering problems here is more prevalent than
it was previously.</p>
<p>In fact, SQLAlchemy has since version 0.8 included a flag on datatypes that
are noted as “unhashable”, however this flag was not used consistently
on built in types. As described in <a class="reference internal" href="#change-3499-postgresql"><span class="std std-ref">ARRAY and JSON types now correctly specify “unhashable”</span></a>, this
flag is now set consistently for all of PostgreSQL’s “structural” types.</p>
<p>The “unhashable” flag is also set on the <a class="reference internal" href="../core/type_api.html#sqlalchemy.types.NullType" title="sqlalchemy.types.NullType"><code class="xref py py-class docutils literal notranslate"><span class="pre">NullType</span></code></a> type,
as <a class="reference internal" href="../core/type_api.html#sqlalchemy.types.NullType" title="sqlalchemy.types.NullType"><code class="xref py py-class docutils literal notranslate"><span class="pre">NullType</span></code></a> is used to refer to any expression of unknown
type.</p>
<p>Since <a class="reference internal" href="../core/type_api.html#sqlalchemy.types.NullType" title="sqlalchemy.types.NullType"><code class="xref py py-class docutils literal notranslate"><span class="pre">NullType</span></code></a> is applied to most
usages of <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.Over.func" title="sqlalchemy.sql.expression.Over.func"><code class="xref py py-attr docutils literal notranslate"><span class="pre">func</span></code></a>, as <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.Over.func" title="sqlalchemy.sql.expression.Over.func"><code class="xref py py-attr docutils literal notranslate"><span class="pre">func</span></code></a> doesn’t actually know anything
about the function names given in most cases, <strong>using func() will
often disable row deduping unless explicit typing is applied</strong>.
The following examples illustrate <code class="docutils literal notranslate"><span class="pre">func.substr()</span></code> applied to a string
expression, and <code class="docutils literal notranslate"><span class="pre">func.date()</span></code> applied to a datetime expression; both
examples will return duplicate rows due to the joined eager load unless
explicit typing is applied:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">result</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span>
<span class="n">func</span><span class="o">.</span><span class="n">substr</span><span class="p">(</span><span class="n">A</span><span class="o">.</span><span class="n">some_thing</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="n">A</span>
<span class="p">)</span><span class="o">.</span><span class="n">options</span><span class="p">(</span><span class="n">joinedload</span><span class="p">(</span><span class="n">A</span><span class="o">.</span><span class="n">bs</span><span class="p">))</span><span class="o">.</span><span class="n">all</span><span class="p">()</span>
<span class="n">users</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span>
<span class="n">func</span><span class="o">.</span><span class="n">date</span><span class="p">(</span>
<span class="n">User</span><span class="o">.</span><span class="n">date_created</span><span class="p">,</span> <span class="s1">'start of month'</span>
<span class="p">)</span><span class="o">.</span><span class="n">label</span><span class="p">(</span><span class="s1">'month'</span><span class="p">),</span>
<span class="n">User</span><span class="p">,</span>
<span class="p">)</span><span class="o">.</span><span class="n">options</span><span class="p">(</span><span class="n">joinedload</span><span class="p">(</span><span class="n">User</span><span class="o">.</span><span class="n">orders</span><span class="p">))</span><span class="o">.</span><span class="n">all</span><span class="p">()</span></pre></div>
</div>
<p>The above examples, in order to retain deduping, should be specified as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">result</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span>
<span class="n">func</span><span class="o">.</span><span class="n">substr</span><span class="p">(</span><span class="n">A</span><span class="o">.</span><span class="n">some_thing</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="n">type_</span><span class="o">=</span><span class="n">String</span><span class="p">),</span> <span class="n">A</span>
<span class="p">)</span><span class="o">.</span><span class="n">options</span><span class="p">(</span><span class="n">joinedload</span><span class="p">(</span><span class="n">A</span><span class="o">.</span><span class="n">bs</span><span class="p">))</span><span class="o">.</span><span class="n">all</span><span class="p">()</span>
<span class="n">users</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span>
<span class="n">func</span><span class="o">.</span><span class="n">date</span><span class="p">(</span>
<span class="n">User</span><span class="o">.</span><span class="n">date_created</span><span class="p">,</span> <span class="s1">'start of month'</span><span class="p">,</span> <span class="n">type_</span><span class="o">=</span><span class="n">DateTime</span>
<span class="p">)</span><span class="o">.</span><span class="n">label</span><span class="p">(</span><span class="s1">'month'</span><span class="p">),</span>
<span class="n">User</span><span class="p">,</span>
<span class="p">)</span><span class="o">.</span><span class="n">options</span><span class="p">(</span><span class="n">joinedload</span><span class="p">(</span><span class="n">User</span><span class="o">.</span><span class="n">orders</span><span class="p">))</span><span class="o">.</span><span class="n">all</span><span class="p">()</span></pre></div>
</div>
<p>Additionally, the treatment of a so-called “unhashable” type is slightly
different than its been in previous releases; internally we are using
the <code class="docutils literal notranslate"><span class="pre">id()</span></code> function to get a “hash value” from these structures, just
as we would any ordinary mapped object. This replaces the previous
approach which applied a counter to the object.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3499">#3499</a></p>
</div>
<div class="section" id="specific-checks-added-for-passing-mapped-classes-instances-as-sql-literals">
<span id="change-3321"></span><h3>Specific checks added for passing mapped classes, instances as SQL literals<a class="headerlink" href="#specific-checks-added-for-passing-mapped-classes-instances-as-sql-literals" title="Permalink to this headline">¶</a></h3>
<p>The typing system now has specific checks for passing of SQLAlchemy
“inspectable” objects in contexts where they would otherwise be handled as
literal values. Any SQLAlchemy built-in object that is legal to pass as a
SQL value (which is not already a <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.ClauseElement" title="sqlalchemy.sql.expression.ClauseElement"><code class="xref py py-class docutils literal notranslate"><span class="pre">ClauseElement</span></code></a> instance)
includes a method <code class="docutils literal notranslate"><span class="pre">__clause_element__()</span></code> which provides a
valid SQL expression for that object. For SQLAlchemy objects that
don’t provide this, such as mapped classes, mappers, and mapped
instances, a more informative error message is emitted rather than
allowing the DBAPI to receive the object and fail later. An example
is illustrated below, where a string-based attribute <code class="docutils literal notranslate"><span class="pre">User.name</span></code> is
compared to a full instance of <code class="docutils literal notranslate"><span class="pre">User()</span></code>, rather than against a
string value:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">some_user</span> <span class="o">=</span> <span class="n">User</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">q</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">User</span><span class="p">)</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">User</span><span class="o">.</span><span class="n">name</span> <span class="o">==</span> <span class="n">some_user</span><span class="p">)</span>
<span class="gp">...</span>
<span class="go">sqlalchemy.exc.ArgumentError: Object <__main__.User object at 0x103167e90> is not legal as a SQL literal value</span></pre></div>
</div>
<p>The exception is now immediate when the comparison is made between
<code class="docutils literal notranslate"><span class="pre">User.name</span> <span class="pre">==</span> <span class="pre">some_user</span></code>. Previously, a comparison like the above
would produce a SQL expression that would only fail once resolved
into a DBAPI execution call; the mapped <code class="docutils literal notranslate"><span class="pre">User</span></code> object would
ultimately become a bound parameter that would be rejected by the
DBAPI.</p>
<p>Note that in the above example, the expression fails because
<code class="docutils literal notranslate"><span class="pre">User.name</span></code> is a string-based (e.g. column oriented) attribute.
The change does <em>not</em> impact the usual case of comparing a many-to-one
relationship attribute to an object, which is handled distinctly:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="c1"># Address.user refers to the User mapper, so</span>
<span class="gp">>>> </span><span class="c1"># this is of course still OK!</span>
<span class="gp">>>> </span><span class="n">q</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">Address</span><span class="p">)</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">Address</span><span class="o">.</span><span class="n">user</span> <span class="o">==</span> <span class="n">some_user</span><span class="p">)</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3321">#3321</a></p>
</div>
<div class="section" id="new-indexable-orm-extension">
<span id="feature-indexable"></span><h3>New Indexable ORM extension<a class="headerlink" href="#new-indexable-orm-extension" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../orm/extensions/indexable.html"><span class="std std-ref">Indexable</span></a> extension is an extension to the hybrid
attribute feature which allows the construction of attributes which
refer to specific elements of an “indexable” data type, such as an array
or JSON field:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Person</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'person'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">JSON</span><span class="p">)</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">index_property</span><span class="p">(</span><span class="s1">'data'</span><span class="p">,</span> <span class="s1">'name'</span><span class="p">)</span></pre></div>
</div>
<p>Above, the <code class="docutils literal notranslate"><span class="pre">name</span></code> attribute will read/write the field <code class="docutils literal notranslate"><span class="pre">"name"</span></code>
from the JSON column <code class="docutils literal notranslate"><span class="pre">data</span></code>, after initializing it to an
empty dictionary:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">person</span> <span class="o">=</span> <span class="n">Person</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">'foobar'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">person</span><span class="o">.</span><span class="n">name</span>
<span class="go">foobar</span></pre></div>
</div>
<p>The extension also triggers a change event when the attribute is modified,
so that there’s no need to use <a class="reference internal" href="../orm/extensions/mutable.html#sqlalchemy.ext.mutable.MutableDict" title="sqlalchemy.ext.mutable.MutableDict"><code class="xref py py-class docutils literal notranslate"><span class="pre">MutableDict</span></code></a> in order
to track this change.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../orm/extensions/indexable.html"><span class="std std-ref">Indexable</span></a></p>
</div>
</div>
<div class="section" id="new-options-allowing-explicit-persistence-of-null-over-a-default">
<span id="change-3250"></span><h3>New options allowing explicit persistence of NULL over a default<a class="headerlink" href="#new-options-allowing-explicit-persistence-of-null-over-a-default" title="Permalink to this headline">¶</a></h3>
<p>Related to the new JSON-NULL support added to PostgreSQL as part of
<a class="reference internal" href="#change-3514"><span class="std std-ref">JSON “null” is inserted as expected with ORM operations, omitted when not present</span></a>, the base <a class="reference internal" href="../core/type_api.html#sqlalchemy.types.TypeEngine" title="sqlalchemy.types.TypeEngine"><code class="xref py py-class docutils literal notranslate"><span class="pre">TypeEngine</span></code></a> class now supports
a method <a class="reference internal" href="../core/type_api.html#sqlalchemy.types.TypeEngine.evaluates_none" title="sqlalchemy.types.TypeEngine.evaluates_none"><code class="xref py py-meth docutils literal notranslate"><span class="pre">TypeEngine.evaluates_none()</span></code></a> which allows a positive set
of the <code class="docutils literal notranslate"><span class="pre">None</span></code> value on an attribute to be persisted as NULL, rather than
omitting the column from the INSERT statement, which has the effect of using
the column-level default. This allows a mapper-level
configuration of the existing object-level technique of assigning
<code class="xref py py-func docutils literal notranslate"><span class="pre">sql.null()</span></code> to the attribute.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../orm/persistence_techniques.html#session-forcing-null"><span class="std std-ref">Forcing NULL on a column with a default</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3250">#3250</a></p>
</div>
<div class="section" id="further-fixes-to-single-table-inheritance-querying">
<span id="change-3582"></span><h3>Further Fixes to single-table inheritance querying<a class="headerlink" href="#further-fixes-to-single-table-inheritance-querying" title="Permalink to this headline">¶</a></h3>
<p>Continuing from 1.0’s <a class="reference internal" href="migration_10.html#migration-3177"><span class="std std-ref">Change to single-table-inheritance criteria when using from_self(), count()</span></a>, the <a class="reference internal" href="../orm/query.html#sqlalchemy.orm.query.Query" title="sqlalchemy.orm.query.Query"><code class="xref py py-class docutils literal notranslate"><span class="pre">Query</span></code></a> should
no longer inappropriately add the “single inheritance” criteria when the
query is against a subquery expression such as an exists:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Widget</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'widget'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="nb">type</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">String</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">String</span><span class="p">)</span>
<span class="n">__mapper_args__</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'polymorphic_on'</span><span class="p">:</span> <span class="nb">type</span><span class="p">}</span>
<span class="k">class</span> <span class="nc">FooWidget</span><span class="p">(</span><span class="n">Widget</span><span class="p">):</span>
<span class="n">__mapper_args__</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'polymorphic_identity'</span><span class="p">:</span> <span class="s1">'foo'</span><span class="p">}</span>
<span class="n">q</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">FooWidget</span><span class="p">)</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">FooWidget</span><span class="o">.</span><span class="n">data</span> <span class="o">==</span> <span class="s1">'bar'</span><span class="p">)</span><span class="o">.</span><span class="n">exists</span><span class="p">()</span>
<span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">q</span><span class="p">)</span><span class="o">.</span><span class="n">all</span><span class="p">()</span></pre></div>
</div>
<p>Produces:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SELECT</span> <span class="n">EXISTS</span> <span class="p">(</span><span class="n">SELECT</span> <span class="mi">1</span>
<span class="n">FROM</span> <span class="n">widget</span>
<span class="n">WHERE</span> <span class="n">widget</span><span class="o">.</span><span class="n">data</span> <span class="o">=</span> <span class="p">:</span><span class="n">data_1</span> <span class="n">AND</span> <span class="n">widget</span><span class="o">.</span><span class="n">type</span> <span class="n">IN</span> <span class="p">(:</span><span class="n">type_1</span><span class="p">))</span> <span class="n">AS</span> <span class="n">anon_1</span></pre></div>
</div>
<p>The IN clause on the inside is appropriate, in order to limit to FooWidget
objects, however previously the IN clause would also be generated a second
time on the outside of the subquery.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3582">#3582</a></p>
</div>
<div class="section" id="improved-session-state-when-a-savepoint-is-cancelled-by-the-database">
<span id="change-3680"></span><h3>Improved Session state when a SAVEPOINT is cancelled by the database<a class="headerlink" href="#improved-session-state-when-a-savepoint-is-cancelled-by-the-database" title="Permalink to this headline">¶</a></h3>
<p>A common case with MySQL is that a SAVEPOINT is cancelled when a deadlock
occurs within the transaction. The <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a> has been modified
to deal with this failure mode slightly more gracefully, such that the
outer, non-savepoint transaction still remains usable:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">s</span> <span class="o">=</span> <span class="n">Session</span><span class="p">()</span>
<span class="n">s</span><span class="o">.</span><span class="n">begin_nested</span><span class="p">()</span>
<span class="n">s</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">SomeObject</span><span class="p">())</span>
<span class="k">try</span><span class="p">:</span>
<span class="c1"># assume the flush fails, flush goes to rollback to the</span>
<span class="c1"># savepoint and that also fails</span>
<span class="n">s</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">err</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"Something broke, and our SAVEPOINT vanished too"</span><span class="p">)</span>
<span class="c1"># this is the SAVEPOINT transaction, marked as</span>
<span class="c1"># DEACTIVE so the rollback() call succeeds</span>
<span class="n">s</span><span class="o">.</span><span class="n">rollback</span><span class="p">()</span>
<span class="c1"># this is the outermost transaction, remains ACTIVE</span>
<span class="c1"># so rollback() or commit() can succeed</span>
<span class="n">s</span><span class="o">.</span><span class="n">rollback</span><span class="p">()</span></pre></div>
</div>
<p>This issue is a continuation of <a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/2696">#2696</a> where we emit a warning
so that the original error can be seen when running on Python 2, even though
the SAVEPOINT exception takes precedence. On Python 3, exceptions are chained
so both failures are reported individually.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3680">#3680</a></p>
</div>
<div class="section" id="erroneous-new-instance-x-conflicts-with-persistent-instance-y-flush-errors-fixed">
<span id="change-3677"></span><h3>Erroneous “new instance X conflicts with persistent instance Y” flush errors fixed<a class="headerlink" href="#erroneous-new-instance-x-conflicts-with-persistent-instance-y-flush-errors-fixed" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session.rollback" title="sqlalchemy.orm.session.Session.rollback"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Session.rollback()</span></code></a> method is responsible for removing objects
that were INSERTed into the database, e.g. moved from pending to persistent,
within that now rolled-back transaction. Objects that make this state
change are tracked in a weak-referencing collection, and if an object is
garbage collected from that collection, the <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a> no longer worries
about it (it would otherwise not scale for operations that insert many new
objects within a transaction). However, an issue arises if the application
re-loads that same garbage-collected row within the transaction, before the
rollback occurs; if a strong reference to this object remains into the next
transaction, the fact that this object was not inserted and should be
removed would be lost, and the flush would incorrectly raise an error:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">Column</span><span class="p">,</span> <span class="n">create_engine</span>
<span class="kn">from</span> <span class="nn">sqlalchemy.orm</span> <span class="k">import</span> <span class="n">Session</span>
<span class="kn">from</span> <span class="nn">sqlalchemy.ext.declarative</span> <span class="k">import</span> <span class="n">declarative_base</span>
<span class="n">Base</span> <span class="o">=</span> <span class="n">declarative_base</span><span class="p">()</span>
<span class="k">class</span> <span class="nc">A</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'a'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">e</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s2">"sqlite://"</span><span class="p">,</span> <span class="n">echo</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">Base</span><span class="o">.</span><span class="n">metadata</span><span class="o">.</span><span class="n">create_all</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
<span class="n">s</span> <span class="o">=</span> <span class="n">Session</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
<span class="c1"># persist an object</span>
<span class="n">s</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">A</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span>
<span class="n">s</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="c1"># rollback buffer loses reference to A</span>
<span class="c1"># load it again, rollback buffer knows nothing</span>
<span class="c1"># about it</span>
<span class="n">a1</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">A</span><span class="p">)</span><span class="o">.</span><span class="n">first</span><span class="p">()</span>
<span class="c1"># roll back the transaction; all state is expired but the</span>
<span class="c1"># "a1" reference remains</span>
<span class="n">s</span><span class="o">.</span><span class="n">rollback</span><span class="p">()</span>
<span class="c1"># previous "a1" conflicts with the new one because we aren't</span>
<span class="c1"># checking that it never got committed</span>
<span class="n">s</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">A</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span>
<span class="n">s</span><span class="o">.</span><span class="n">commit</span><span class="p">()</span></pre></div>
</div>
<p>The above program would raise:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>FlushError: New instance <User at 0x7f0287eca4d0> with identity key
(<class 'test.orm.test_transaction.User'>, ('u1',)) conflicts
with persistent instance <User at 0x7f02889c70d0></pre></div>
</div>
<p>The bug is that when the above exception is raised, the unit of work
is operating upon the original object assuming it’s a live row, when in
fact the object is expired and upon testing reveals that it’s gone. The
fix tests this condition now, so in the SQL log we see:</p>
<div class="highlight-sql notranslate"><div class="highlight"><pre><span></span><span class="k">BEGIN</span> <span class="p">(</span><span class="k">implicit</span><span class="p">)</span>
<span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">a</span> <span class="p">(</span><span class="n">id</span><span class="p">)</span> <span class="k">VALUES</span> <span class="p">(</span><span class="o">?</span><span class="p">)</span>
<span class="p">(</span><span class="mi">1</span><span class="p">,)</span>
<span class="k">SELECT</span> <span class="n">a</span><span class="p">.</span><span class="n">id</span> <span class="k">AS</span> <span class="n">a_id</span> <span class="k">FROM</span> <span class="n">a</span> <span class="k">LIMIT</span> <span class="o">?</span> <span class="k">OFFSET</span> <span class="o">?</span>
<span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
<span class="k">ROLLBACK</span>
<span class="k">BEGIN</span> <span class="p">(</span><span class="k">implicit</span><span class="p">)</span>
<span class="k">SELECT</span> <span class="n">a</span><span class="p">.</span><span class="n">id</span> <span class="k">AS</span> <span class="n">a_id</span> <span class="k">FROM</span> <span class="n">a</span> <span class="k">WHERE</span> <span class="n">a</span><span class="p">.</span><span class="n">id</span> <span class="o">=</span> <span class="o">?</span>
<span class="p">(</span><span class="mi">1</span><span class="p">,)</span>
<span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">a</span> <span class="p">(</span><span class="n">id</span><span class="p">)</span> <span class="k">VALUES</span> <span class="p">(</span><span class="o">?</span><span class="p">)</span>
<span class="p">(</span><span class="mi">1</span><span class="p">,)</span>
<span class="k">COMMIT</span></pre></div>
</div>
<p>Above, the unit of work now does a SELECT for the row we’re about to report
as a conflict for, sees that it doesn’t exist, and proceeds normally.
The expense of this SELECT is only incurred in the case when we would have
erroneously raised an exception in any case.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3677">#3677</a></p>
</div>
<div class="section" id="passive-deletes-feature-for-joined-inheritance-mappings">
<span id="change-2349"></span><h3>passive_deletes feature for joined-inheritance mappings<a class="headerlink" href="#passive-deletes-feature-for-joined-inheritance-mappings" title="Permalink to this headline">¶</a></h3>
<p>A joined-table inheritance mapping may now allow a DELETE to proceed
as a result of <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session.delete" title="sqlalchemy.orm.session.Session.delete"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Session.delete()</span></code></a>, which only emits DELETE for the
base table, and not the subclass table, allowing configured ON DELETE CASCADE
to take place for the configured foreign keys. This is configured using
the <a class="reference internal" href="../orm/mapping_api.html#sqlalchemy.orm.mapper.params.passive_deletes" title="sqlalchemy.orm.mapper"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">orm.mapper.passive_deletes</span></code></a> option:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">Column</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">String</span><span class="p">,</span> <span class="n">ForeignKey</span><span class="p">,</span> <span class="n">create_engine</span>
<span class="kn">from</span> <span class="nn">sqlalchemy.orm</span> <span class="k">import</span> <span class="n">Session</span>
<span class="kn">from</span> <span class="nn">sqlalchemy.ext.declarative</span> <span class="k">import</span> <span class="n">declarative_base</span>
<span class="n">Base</span> <span class="o">=</span> <span class="n">declarative_base</span><span class="p">()</span>
<span class="k">class</span> <span class="nc">A</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s2">"a"</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="s1">'id'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="nb">type</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">String</span><span class="p">)</span>
<span class="n">__mapper_args__</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">'polymorphic_on'</span><span class="p">:</span> <span class="nb">type</span><span class="p">,</span>
<span class="s1">'polymorphic_identity'</span><span class="p">:</span> <span class="s1">'a'</span><span class="p">,</span>
<span class="s1">'passive_deletes'</span><span class="p">:</span> <span class="kc">True</span>
<span class="p">}</span>
<span class="k">class</span> <span class="nc">B</span><span class="p">(</span><span class="n">A</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'b'</span>
<span class="n">b_table_id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="s1">'b_table_id'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">bid</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="s1">'bid'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">ForeignKey</span><span class="p">(</span><span class="s1">'a.id'</span><span class="p">,</span> <span class="n">ondelete</span><span class="o">=</span><span class="s2">"CASCADE"</span><span class="p">))</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="s1">'data'</span><span class="p">,</span> <span class="n">String</span><span class="p">)</span>
<span class="n">__mapper_args__</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">'polymorphic_identity'</span><span class="p">:</span> <span class="s1">'b'</span>
<span class="p">}</span></pre></div>
</div>
<p>With the above mapping, the <a class="reference internal" href="../orm/mapping_api.html#sqlalchemy.orm.mapper.params.passive_deletes" title="sqlalchemy.orm.mapper"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">orm.mapper.passive_deletes</span></code></a> option
is configured on the base mapper; it takes effect for all non-base mappers
that are descendants of the mapper with the option set. A DELETE for
an object of type <code class="docutils literal notranslate"><span class="pre">B</span></code> no longer needs to retrieve the primary key value
of <code class="docutils literal notranslate"><span class="pre">b_table_id</span></code> if unloaded, nor does it need to emit a DELETE statement
for the table itself:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">session</span><span class="o">.</span><span class="n">delete</span><span class="p">(</span><span class="n">some_b</span><span class="p">)</span>
<span class="n">session</span><span class="o">.</span><span class="n">commit</span><span class="p">()</span></pre></div>
</div>
<p>Will emit SQL as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">DELETE</span> <span class="n">FROM</span> <span class="n">a</span> <span class="n">WHERE</span> <span class="n">a</span><span class="o">.</span><span class="n">id</span> <span class="o">=</span> <span class="o">%</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span><span class="n">s</span>
<span class="p">{</span><span class="s1">'id'</span><span class="p">:</span> <span class="mi">1</span><span class="p">}</span>
<span class="n">COMMIT</span></pre></div>
</div>
<p>As always, the target database must have foreign key support with
ON DELETE CASCADE enabled.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/2349">#2349</a></p>
</div>
<div class="section" id="same-named-backrefs-will-not-raise-an-error-when-applied-to-concrete-inheritance-subclasses">
<span id="change-3630"></span><h3>Same-named backrefs will not raise an error when applied to concrete inheritance subclasses<a class="headerlink" href="#same-named-backrefs-will-not-raise-an-error-when-applied-to-concrete-inheritance-subclasses" title="Permalink to this headline">¶</a></h3>
<p>The following mapping has always been possible without issue:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">A</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'a'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">b</span> <span class="o">=</span> <span class="n">relationship</span><span class="p">(</span><span class="s2">"B"</span><span class="p">,</span> <span class="n">foreign_keys</span><span class="o">=</span><span class="s2">"B.a_id"</span><span class="p">,</span> <span class="n">backref</span><span class="o">=</span><span class="s2">"a"</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">A1</span><span class="p">(</span><span class="n">A</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'a1'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">b</span> <span class="o">=</span> <span class="n">relationship</span><span class="p">(</span><span class="s2">"B"</span><span class="p">,</span> <span class="n">foreign_keys</span><span class="o">=</span><span class="s2">"B.a1_id"</span><span class="p">,</span> <span class="n">backref</span><span class="o">=</span><span class="s2">"a1"</span><span class="p">)</span>
<span class="n">__mapper_args__</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'concrete'</span><span class="p">:</span> <span class="kc">True</span><span class="p">}</span>
<span class="k">class</span> <span class="nc">B</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'b'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">a_id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">ForeignKey</span><span class="p">(</span><span class="s1">'a.id'</span><span class="p">))</span>
<span class="n">a1_id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">ForeignKey</span><span class="p">(</span><span class="s1">'a1.id'</span><span class="p">))</span></pre></div>
</div>
<p>Above, even though class <code class="docutils literal notranslate"><span class="pre">A</span></code> and class <code class="docutils literal notranslate"><span class="pre">A1</span></code> have a relationship
named <code class="docutils literal notranslate"><span class="pre">b</span></code>, no conflict warning or error occurs because class <code class="docutils literal notranslate"><span class="pre">A1</span></code> is
marked as “concrete”.</p>
<p>However, if the relationships were configured the other way, an error
would occur:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">A</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'a'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">A1</span><span class="p">(</span><span class="n">A</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'a1'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">__mapper_args__</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'concrete'</span><span class="p">:</span> <span class="kc">True</span><span class="p">}</span>
<span class="k">class</span> <span class="nc">B</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'b'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">a_id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">ForeignKey</span><span class="p">(</span><span class="s1">'a.id'</span><span class="p">))</span>
<span class="n">a1_id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">ForeignKey</span><span class="p">(</span><span class="s1">'a1.id'</span><span class="p">))</span>
<span class="n">a</span> <span class="o">=</span> <span class="n">relationship</span><span class="p">(</span><span class="s2">"A"</span><span class="p">,</span> <span class="n">backref</span><span class="o">=</span><span class="s2">"b"</span><span class="p">)</span>
<span class="n">a1</span> <span class="o">=</span> <span class="n">relationship</span><span class="p">(</span><span class="s2">"A1"</span><span class="p">,</span> <span class="n">backref</span><span class="o">=</span><span class="s2">"b"</span><span class="p">)</span></pre></div>
</div>
<p>The fix enhances the backref feature so that an error is not emitted,
as well as an additional check within the mapper logic to bypass warning
for an attribute being replaced.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3630">#3630</a></p>
</div>
<div class="section" id="same-named-relationships-on-inheriting-mappers-no-longer-warn">
<span id="change-3749"></span><h3>Same-named relationships on inheriting mappers no longer warn<a class="headerlink" href="#same-named-relationships-on-inheriting-mappers-no-longer-warn" title="Permalink to this headline">¶</a></h3>
<p>When creating two mappers in an inheritance scenario, placing a relationship
on both with the same name would emit the warning
“relationship ‘<name>’ on mapper <name> supersedes the same relationship
on inherited mapper ‘<name>’; this can cause dependency issues during flush”.
An example is as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">A</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'a'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">bs</span> <span class="o">=</span> <span class="n">relationship</span><span class="p">(</span><span class="s2">"B"</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">ASub</span><span class="p">(</span><span class="n">A</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'a_sub'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">ForeignKey</span><span class="p">(</span><span class="s1">'a.id'</span><span class="p">),</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">bs</span> <span class="o">=</span> <span class="n">relationship</span><span class="p">(</span><span class="s2">"B"</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">B</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'b'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">a_id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">ForeignKey</span><span class="p">(</span><span class="s1">'a.id'</span><span class="p">))</span></pre></div>
</div>
<p>This warning dates back to the 0.4 series in 2007 and is based on a version of
the unit of work code that has since been entirely rewritten. Currently, there
is no known issue with the same-named relationships being placed on a base
class and a descendant class, so the warning is lifted. However, note that
this use case is likely not prevalent in real world use due to the warning.
While rudimentary test support is added for this use case, it is possible that
some new issue with this pattern may be identified.</p>
<div class="versionadded">
<p><span class="versionmodified added">New in version 1.1.0b3.</span></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3749">#3749</a></p>
</div>
<div class="section" id="hybrid-properties-and-methods-now-propagate-the-docstring-as-well-as-info">
<span id="change-3653"></span><h3>Hybrid properties and methods now propagate the docstring as well as .info<a class="headerlink" href="#hybrid-properties-and-methods-now-propagate-the-docstring-as-well-as-info" title="Permalink to this headline">¶</a></h3>
<p>A hybrid method or property will now reflect the <code class="docutils literal notranslate"><span class="pre">__doc__</span></code> value
present in the original docstring:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">A</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'a'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">String</span><span class="p">)</span>
<span class="nd">@hybrid_property</span>
<span class="k">def</span> <span class="nf">some_name</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">"""The name field"""</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span></pre></div>
</div>
<p>The above value of <code class="docutils literal notranslate"><span class="pre">A.some_name.__doc__</span></code> is now honored:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">A</span><span class="o">.</span><span class="n">some_name</span><span class="o">.</span><span class="vm">__doc__</span>
<span class="go">The name field</span></pre></div>
</div>
<p>However, to accomplish this, the mechanics of hybrid properties necessarily
becomes more complex. Previously, the class-level accessor for a hybrid
would be a simple pass-thru, that is, this test would succeed:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">assert</span> <span class="n">A</span><span class="o">.</span><span class="n">name</span> <span class="ow">is</span> <span class="n">A</span><span class="o">.</span><span class="n">some_name</span></pre></div>
</div>
<p>With the change, the expression returned by <code class="docutils literal notranslate"><span class="pre">A.some_name</span></code> is wrapped inside
of its own <code class="docutils literal notranslate"><span class="pre">QueryableAttribute</span></code> wrapper:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">A</span><span class="o">.</span><span class="n">some_name</span>
<span class="go"><sqlalchemy.orm.attributes.hybrid_propertyProxy object at 0x7fde03888230></span></pre></div>
</div>
<p>A lot of testing went into making sure this wrapper works correctly, including
for elaborate schemes like that of the
<a class="reference external" href="http://techspot.zzzeek.org/2011/10/21/hybrids-and-value-agnostic-types/">Custom Value Object</a>
recipe, however we’ll be looking to see that no other regressions occur for
users.</p>
<p>As part of this change, the <code class="xref py py-attr docutils literal notranslate"><span class="pre">hybrid_property.info</span></code> collection is now
also propagated from the hybrid descriptor itself, rather than from the underlying
expression. That is, accessing <code class="docutils literal notranslate"><span class="pre">A.some_name.info</span></code> now returns the same
dictionary that you’d get from <code class="docutils literal notranslate"><span class="pre">inspect(A).all_orm_descriptors['some_name'].info</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">A</span><span class="o">.</span><span class="n">some_name</span><span class="o">.</span><span class="n">info</span><span class="p">[</span><span class="s1">'foo'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'bar'</span>
<span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">inspect</span>
<span class="gp">>>> </span><span class="n">inspect</span><span class="p">(</span><span class="n">A</span><span class="p">)</span><span class="o">.</span><span class="n">all_orm_descriptors</span><span class="p">[</span><span class="s1">'some_name'</span><span class="p">]</span><span class="o">.</span><span class="n">info</span>
<span class="go">{'foo': 'bar'}</span></pre></div>
</div>
<p>Note that this <code class="docutils literal notranslate"><span class="pre">.info</span></code> dictionary is <strong>separate</strong> from that of a mapped attribute
which the hybrid descriptor may be proxying directly; this is a behavioral
change from 1.0. The wrapper will still proxy other useful attributes
of a mirrored attribute such as <a class="reference internal" href="../orm/internals.html#sqlalchemy.orm.attributes.QueryableAttribute.property" title="sqlalchemy.orm.attributes.QueryableAttribute.property"><code class="xref py py-attr docutils literal notranslate"><span class="pre">QueryableAttribute.property</span></code></a> and
<code class="xref py py-attr docutils literal notranslate"><span class="pre">QueryableAttribute.class_</span></code>.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3653">#3653</a></p>
</div>
<div class="section" id="session-merge-resolves-pending-conflicts-the-same-as-persistent">
<span id="change-3601"></span><h3>Session.merge resolves pending conflicts the same as persistent<a class="headerlink" href="#session-merge-resolves-pending-conflicts-the-same-as-persistent" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session.merge" title="sqlalchemy.orm.session.Session.merge"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Session.merge()</span></code></a> method will now track the identities of objects given
within a graph to maintain primary key uniqueness before emitting an INSERT.
When duplicate objects of the same identity are encountered, non-primary-key
attributes are <strong>overwritten</strong> as the objects are encountered, which is
essentially non-deterministic. This behavior matches that of how persistent
objects, that is objects that are already located in the database via
primary key, are already treated, so this behavior is more internally
consistent.</p>
<p>Given:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">u1</span> <span class="o">=</span> <span class="n">User</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="mi">7</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s1">'x'</span><span class="p">)</span>
<span class="n">u1</span><span class="o">.</span><span class="n">orders</span> <span class="o">=</span> <span class="p">[</span>
<span class="n">Order</span><span class="p">(</span><span class="n">description</span><span class="o">=</span><span class="s1">'o1'</span><span class="p">,</span> <span class="n">address</span><span class="o">=</span><span class="n">Address</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">email_address</span><span class="o">=</span><span class="s1">'a'</span><span class="p">)),</span>
<span class="n">Order</span><span class="p">(</span><span class="n">description</span><span class="o">=</span><span class="s1">'o2'</span><span class="p">,</span> <span class="n">address</span><span class="o">=</span><span class="n">Address</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">email_address</span><span class="o">=</span><span class="s1">'b'</span><span class="p">)),</span>
<span class="n">Order</span><span class="p">(</span><span class="n">description</span><span class="o">=</span><span class="s1">'o3'</span><span class="p">,</span> <span class="n">address</span><span class="o">=</span><span class="n">Address</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">email_address</span><span class="o">=</span><span class="s1">'c'</span><span class="p">))</span>
<span class="p">]</span>
<span class="n">sess</span> <span class="o">=</span> <span class="n">Session</span><span class="p">()</span>
<span class="n">sess</span><span class="o">.</span><span class="n">merge</span><span class="p">(</span><span class="n">u1</span><span class="p">)</span></pre></div>
</div>
<p>Above, we merge a <code class="docutils literal notranslate"><span class="pre">User</span></code> object with three new <code class="docutils literal notranslate"><span class="pre">Order</span></code> objects, each referring to
a distinct <code class="docutils literal notranslate"><span class="pre">Address</span></code> object, however each is given the same primary key.
The current behavior of <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session.merge" title="sqlalchemy.orm.session.Session.merge"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Session.merge()</span></code></a> is to look in the identity
map for this <code class="docutils literal notranslate"><span class="pre">Address</span></code> object, and use that as the target. If the object
is present, meaning that the database already has a row for <code class="docutils literal notranslate"><span class="pre">Address</span></code> with
primary key “1”, we can see that the <code class="docutils literal notranslate"><span class="pre">email_address</span></code> field of the <code class="docutils literal notranslate"><span class="pre">Address</span></code>
will be overwritten three times, in this case with the values a, b and finally
c.</p>
<p>However, if the <code class="docutils literal notranslate"><span class="pre">Address</span></code> row for primary key “1” were not present, <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session.merge" title="sqlalchemy.orm.session.Session.merge"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Session.merge()</span></code></a>
would instead create three separate <code class="docutils literal notranslate"><span class="pre">Address</span></code> instances, and we’d then get
a primary key conflict upon INSERT. The new behavior is that the proposed
primary key for these <code class="docutils literal notranslate"><span class="pre">Address</span></code> objects are tracked in a separate dictionary
so that we merge the state of the three proposed <code class="docutils literal notranslate"><span class="pre">Address</span></code> objects onto
one <code class="docutils literal notranslate"><span class="pre">Address</span></code> object to be inserted.</p>
<p>It may have been preferable if the original case emitted some kind of warning
that conflicting data were present in a single merge-tree, however the
non-deterministic merging of values has been the behavior for many
years for the persistent case; it now matches for the pending case. A
feature that warns for conflicting values could still be feasible for both
cases but would add considerable performance overhead as each column value
would have to be compared during the merge.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3601">#3601</a></p>
</div>
<div class="section" id="fix-involving-many-to-one-object-moves-with-user-initiated-foreign-key-manipulations">
<span id="change-3708"></span><h3>Fix involving many-to-one object moves with user-initiated foreign key manipulations<a class="headerlink" href="#fix-involving-many-to-one-object-moves-with-user-initiated-foreign-key-manipulations" title="Permalink to this headline">¶</a></h3>
<p>A bug has been fixed involving the mechanics of replacing a many-to-one
reference to an object with another object. During the attribute operation,
the location of the object that was previously referred to now makes use of the
database-committed foreign key value, rather than the current foreign key
value. The main effect of the fix is that a backref event towards a collection
will fire off more accurately when a many-to-one change is made, even if the
foreign key attribute was manually moved to the new value beforehand. Assume a
mapping of the classes <code class="docutils literal notranslate"><span class="pre">Parent</span></code> and <code class="docutils literal notranslate"><span class="pre">SomeClass</span></code>, where <code class="docutils literal notranslate"><span class="pre">SomeClass.parent</span></code>
refers to <code class="docutils literal notranslate"><span class="pre">Parent</span></code> and <code class="docutils literal notranslate"><span class="pre">Parent.items</span></code> refers to the collection of
<code class="docutils literal notranslate"><span class="pre">SomeClass</span></code> objects:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">some_object</span> <span class="o">=</span> <span class="n">SomeClass</span><span class="p">()</span>
<span class="n">session</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">some_object</span><span class="p">)</span>
<span class="n">some_object</span><span class="o">.</span><span class="n">parent_id</span> <span class="o">=</span> <span class="n">some_parent</span><span class="o">.</span><span class="n">id</span>
<span class="n">some_object</span><span class="o">.</span><span class="n">parent</span> <span class="o">=</span> <span class="n">some_parent</span></pre></div>
</div>
<p>Above, we’ve made a pending object <code class="docutils literal notranslate"><span class="pre">some_object</span></code>, manipulated its foreign key
towards <code class="docutils literal notranslate"><span class="pre">Parent</span></code> to refer to it, <em>then</em> we actually set up the relationship.
Before the bug fix, the backref would not have fired off:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># before the fix</span>
<span class="k">assert</span> <span class="n">some_object</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">some_parent</span><span class="o">.</span><span class="n">items</span></pre></div>
</div>
<p>The fix now is that when we seek to locate the previous value of
<code class="docutils literal notranslate"><span class="pre">some_object.parent</span></code>, we disregard the parent id that’s been manually set,
and we look for the database-committed value. In this case, it’s None because
the object is pending, so the event system logs <code class="docutils literal notranslate"><span class="pre">some_object.parent</span></code>
as a net change:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># after the fix, backref fired off for some_object.parent = some_parent</span>
<span class="k">assert</span> <span class="n">some_object</span> <span class="ow">in</span> <span class="n">some_parent</span><span class="o">.</span><span class="n">items</span></pre></div>
</div>
<p>While it is discouraged to manipulate foreign key attributes that are managed
by relationships, there is limited support for this use case. Applications
that manipulate foreign keys in order to allow loads to proceed will often make
use of the <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session.enable_relationship_loading" title="sqlalchemy.orm.session.Session.enable_relationship_loading"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Session.enable_relationship_loading()</span></code></a> and
<code class="xref py py-attr docutils literal notranslate"><span class="pre">RelationshipProperty.load_on_pending</span></code> features, which cause
relationships to emit lazy loads based on in-memory foreign key values that
aren’t persisted. Whether or not these features are in use, this behavioral
improvement will now be apparent.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3708">#3708</a></p>
</div>
<div class="section" id="improvements-to-the-query-correlate-method-with-polymorphic-entities">
<span id="change-3662"></span><h3>Improvements to the Query.correlate method with polymorphic entities<a class="headerlink" href="#improvements-to-the-query-correlate-method-with-polymorphic-entities" title="Permalink to this headline">¶</a></h3>
<p>In recent SQLAlchemy versions, the SQL generated by many forms of
“polymorphic” queries has a more “flat” form than it used to, where
a JOIN of several tables is no longer bundled into a subquery unconditionally.
To accommodate this, the <a class="reference internal" href="../orm/query.html#sqlalchemy.orm.query.Query.correlate" title="sqlalchemy.orm.query.Query.correlate"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Query.correlate()</span></code></a> method now extracts the
individual tables from such a polymorphic selectable and ensures that all
are part of the “correlate” for the subquery. Assuming the
<code class="docutils literal notranslate"><span class="pre">Person/Manager/Engineer->Company</span></code> setup from the mapping documentation,
using with_polymorphic:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sess</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">Person</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
<span class="o">.</span><span class="n">filter</span><span class="p">(</span>
<span class="n">sess</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">Company</span><span class="o">.</span><span class="n">name</span><span class="p">)</span><span class="o">.</span>
<span class="nb">filter</span><span class="p">(</span><span class="n">Company</span><span class="o">.</span><span class="n">company_id</span> <span class="o">==</span> <span class="n">Person</span><span class="o">.</span><span class="n">company_id</span><span class="p">)</span><span class="o">.</span>
<span class="n">correlate</span><span class="p">(</span><span class="n">Person</span><span class="p">)</span><span class="o">.</span><span class="n">as_scalar</span><span class="p">()</span> <span class="o">==</span> <span class="s2">"Elbonia, Inc."</span><span class="p">)</span></pre></div>
</div>
<p>The above query now produces:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>SELECT people.name AS people_name
FROM people
LEFT OUTER JOIN engineers ON people.person_id = engineers.person_id
LEFT OUTER JOIN managers ON people.person_id = managers.person_id
WHERE (SELECT companies.name
FROM companies
WHERE companies.company_id = people.company_id) = ?</pre></div>
</div>
<p>Before the fix, the call to <code class="docutils literal notranslate"><span class="pre">correlate(Person)</span></code> would inadvertently
attempt to correlate to the join of <code class="docutils literal notranslate"><span class="pre">Person</span></code>, <code class="docutils literal notranslate"><span class="pre">Engineer</span></code> and <code class="docutils literal notranslate"><span class="pre">Manager</span></code>
as a single unit, so <code class="docutils literal notranslate"><span class="pre">Person</span></code> wouldn’t be correlated:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>-- old, incorrect query
SELECT people.name AS people_name
FROM people
LEFT OUTER JOIN engineers ON people.person_id = engineers.person_id
LEFT OUTER JOIN managers ON people.person_id = managers.person_id
WHERE (SELECT companies.name
FROM companies, people
WHERE companies.company_id = people.company_id) = ?</pre></div>
</div>
<p>Using correlated subqueries against polymorphic mappings still has some
unpolished edges. If for example <code class="docutils literal notranslate"><span class="pre">Person</span></code> is polymorphically linked
to a so-called “concrete polymorphic union” query, the above subquery
may not correctly refer to this subquery. In all cases, a way to refer
to the “polymorphic” entity fully is to create an <a class="reference internal" href="../orm/query.html#sqlalchemy.orm.aliased" title="sqlalchemy.orm.aliased"><code class="xref py py-func docutils literal notranslate"><span class="pre">aliased()</span></code></a> object
from it first:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># works with all SQLAlchemy versions and all types of polymorphic</span>
<span class="c1"># aliasing.</span>
<span class="n">paliased</span> <span class="o">=</span> <span class="n">aliased</span><span class="p">(</span><span class="n">Person</span><span class="p">)</span>
<span class="n">sess</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">paliased</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
<span class="o">.</span><span class="n">filter</span><span class="p">(</span>
<span class="n">sess</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">Company</span><span class="o">.</span><span class="n">name</span><span class="p">)</span><span class="o">.</span>
<span class="nb">filter</span><span class="p">(</span><span class="n">Company</span><span class="o">.</span><span class="n">company_id</span> <span class="o">==</span> <span class="n">paliased</span><span class="o">.</span><span class="n">company_id</span><span class="p">)</span><span class="o">.</span>
<span class="n">correlate</span><span class="p">(</span><span class="n">paliased</span><span class="p">)</span><span class="o">.</span><span class="n">as_scalar</span><span class="p">()</span> <span class="o">==</span> <span class="s2">"Elbonia, Inc."</span><span class="p">)</span></pre></div>
</div>
<p>The <a class="reference internal" href="../orm/query.html#sqlalchemy.orm.aliased" title="sqlalchemy.orm.aliased"><code class="xref py py-func docutils literal notranslate"><span class="pre">aliased()</span></code></a> construct guarantees that the “polymorphic selectable”
is wrapped in a subquery. By referring to it explicitly in the correlated
subquery, the polymorphic form is correctly used.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3662">#3662</a></p>
</div>
<div class="section" id="stringify-of-query-will-consult-the-session-for-the-correct-dialect">
<span id="change-3081"></span><h3>Stringify of Query will consult the Session for the correct dialect<a class="headerlink" href="#stringify-of-query-will-consult-the-session-for-the-correct-dialect" title="Permalink to this headline">¶</a></h3>
<p>Calling <code class="docutils literal notranslate"><span class="pre">str()</span></code> on a <a class="reference internal" href="../orm/query.html#sqlalchemy.orm.query.Query" title="sqlalchemy.orm.query.Query"><code class="xref py py-class docutils literal notranslate"><span class="pre">Query</span></code></a> object will consult the <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a>
for the correct “bind” to use, in order to render the SQL that would be
passed to the database. In particular this allows a <a class="reference internal" href="../orm/query.html#sqlalchemy.orm.query.Query" title="sqlalchemy.orm.query.Query"><code class="xref py py-class docutils literal notranslate"><span class="pre">Query</span></code></a> that
refers to dialect-specific SQL constructs to be renderable, assuming the
<a class="reference internal" href="../orm/query.html#sqlalchemy.orm.query.Query" title="sqlalchemy.orm.query.Query"><code class="xref py py-class docutils literal notranslate"><span class="pre">Query</span></code></a> is associated with an appropriate <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a>.
Previously, this behavior would only take effect if the <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.MetaData" title="sqlalchemy.schema.MetaData"><code class="xref py py-class docutils literal notranslate"><span class="pre">MetaData</span></code></a>
to which the mappings were associated were itself bound to the target
<a class="reference internal" href="../core/connections.html#sqlalchemy.engine.Engine" title="sqlalchemy.engine.Engine"><code class="xref py py-class docutils literal notranslate"><span class="pre">Engine</span></code></a>.</p>
<p>If neither the underlying <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.MetaData" title="sqlalchemy.schema.MetaData"><code class="xref py py-class docutils literal notranslate"><span class="pre">MetaData</span></code></a> nor the <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a> are
associated with any bound <a class="reference internal" href="../core/connections.html#sqlalchemy.engine.Engine" title="sqlalchemy.engine.Engine"><code class="xref py py-class docutils literal notranslate"><span class="pre">Engine</span></code></a>, then the fallback to the
“default” dialect is used to generate the SQL string.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="#change-3631"><span class="std std-ref">“Friendly” stringification of Core SQL constructs without a dialect</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3081">#3081</a></p>
</div>
<div class="section" id="joined-eager-loading-where-the-same-entity-is-present-multiple-times-in-one-row">
<span id="change-3431"></span><h3>Joined eager loading where the same entity is present multiple times in one row<a class="headerlink" href="#joined-eager-loading-where-the-same-entity-is-present-multiple-times-in-one-row" title="Permalink to this headline">¶</a></h3>
<p>A fix has been made to the case has been made whereby an attribute will be
loaded via joined eager loading, even if the entity was already loaded from the
row on a different “path” that doesn’t include the attribute. This is a
deep use case that’s hard to reproduce, but the general idea is as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">A</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'a'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">b_id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">ForeignKey</span><span class="p">(</span><span class="s1">'b.id'</span><span class="p">))</span>
<span class="n">c_id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">ForeignKey</span><span class="p">(</span><span class="s1">'c.id'</span><span class="p">))</span>
<span class="n">b</span> <span class="o">=</span> <span class="n">relationship</span><span class="p">(</span><span class="s2">"B"</span><span class="p">)</span>
<span class="n">c</span> <span class="o">=</span> <span class="n">relationship</span><span class="p">(</span><span class="s2">"C"</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">B</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'b'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">c_id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">ForeignKey</span><span class="p">(</span><span class="s1">'c.id'</span><span class="p">))</span>
<span class="n">c</span> <span class="o">=</span> <span class="n">relationship</span><span class="p">(</span><span class="s2">"C"</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">C</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'c'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">d_id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">ForeignKey</span><span class="p">(</span><span class="s1">'d.id'</span><span class="p">))</span>
<span class="n">d</span> <span class="o">=</span> <span class="n">relationship</span><span class="p">(</span><span class="s2">"D"</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">D</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'d'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">c_alias_1</span> <span class="o">=</span> <span class="n">aliased</span><span class="p">(</span><span class="n">C</span><span class="p">)</span>
<span class="n">c_alias_2</span> <span class="o">=</span> <span class="n">aliased</span><span class="p">(</span><span class="n">C</span><span class="p">)</span>
<span class="n">q</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">A</span><span class="p">)</span>
<span class="n">q</span> <span class="o">=</span> <span class="n">q</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">A</span><span class="o">.</span><span class="n">b</span><span class="p">)</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">c_alias_1</span><span class="p">,</span> <span class="n">B</span><span class="o">.</span><span class="n">c</span><span class="p">)</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">c_alias_1</span><span class="o">.</span><span class="n">d</span><span class="p">)</span>
<span class="n">q</span> <span class="o">=</span> <span class="n">q</span><span class="o">.</span><span class="n">options</span><span class="p">(</span><span class="n">contains_eager</span><span class="p">(</span><span class="n">A</span><span class="o">.</span><span class="n">b</span><span class="p">)</span><span class="o">.</span><span class="n">contains_eager</span><span class="p">(</span><span class="n">B</span><span class="o">.</span><span class="n">c</span><span class="p">,</span> <span class="n">alias</span><span class="o">=</span><span class="n">c_alias_1</span><span class="p">)</span><span class="o">.</span><span class="n">contains_eager</span><span class="p">(</span><span class="n">C</span><span class="o">.</span><span class="n">d</span><span class="p">))</span>
<span class="n">q</span> <span class="o">=</span> <span class="n">q</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">c_alias_2</span><span class="p">,</span> <span class="n">A</span><span class="o">.</span><span class="n">c</span><span class="p">)</span>
<span class="n">q</span> <span class="o">=</span> <span class="n">q</span><span class="o">.</span><span class="n">options</span><span class="p">(</span><span class="n">contains_eager</span><span class="p">(</span><span class="n">A</span><span class="o">.</span><span class="n">c</span><span class="p">,</span> <span class="n">alias</span><span class="o">=</span><span class="n">c_alias_2</span><span class="p">))</span></pre></div>
</div>
<p>The above query emits SQL like this:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SELECT</span>
<span class="n">d</span><span class="o">.</span><span class="n">id</span> <span class="n">AS</span> <span class="n">d_id</span><span class="p">,</span>
<span class="n">c_1</span><span class="o">.</span><span class="n">id</span> <span class="n">AS</span> <span class="n">c_1_id</span><span class="p">,</span> <span class="n">c_1</span><span class="o">.</span><span class="n">d_id</span> <span class="n">AS</span> <span class="n">c_1_d_id</span><span class="p">,</span>
<span class="n">b</span><span class="o">.</span><span class="n">id</span> <span class="n">AS</span> <span class="n">b_id</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">c_id</span> <span class="n">AS</span> <span class="n">b_c_id</span><span class="p">,</span>
<span class="n">c_2</span><span class="o">.</span><span class="n">id</span> <span class="n">AS</span> <span class="n">c_2_id</span><span class="p">,</span> <span class="n">c_2</span><span class="o">.</span><span class="n">d_id</span> <span class="n">AS</span> <span class="n">c_2_d_id</span><span class="p">,</span>
<span class="n">a</span><span class="o">.</span><span class="n">id</span> <span class="n">AS</span> <span class="n">a_id</span><span class="p">,</span> <span class="n">a</span><span class="o">.</span><span class="n">b_id</span> <span class="n">AS</span> <span class="n">a_b_id</span><span class="p">,</span> <span class="n">a</span><span class="o">.</span><span class="n">c_id</span> <span class="n">AS</span> <span class="n">a_c_id</span>
<span class="n">FROM</span>
<span class="n">a</span>
<span class="n">JOIN</span> <span class="n">b</span> <span class="n">ON</span> <span class="n">b</span><span class="o">.</span><span class="n">id</span> <span class="o">=</span> <span class="n">a</span><span class="o">.</span><span class="n">b_id</span>
<span class="n">JOIN</span> <span class="n">c</span> <span class="n">AS</span> <span class="n">c_1</span> <span class="n">ON</span> <span class="n">c_1</span><span class="o">.</span><span class="n">id</span> <span class="o">=</span> <span class="n">b</span><span class="o">.</span><span class="n">c_id</span>
<span class="n">JOIN</span> <span class="n">d</span> <span class="n">ON</span> <span class="n">d</span><span class="o">.</span><span class="n">id</span> <span class="o">=</span> <span class="n">c_1</span><span class="o">.</span><span class="n">d_id</span>
<span class="n">JOIN</span> <span class="n">c</span> <span class="n">AS</span> <span class="n">c_2</span> <span class="n">ON</span> <span class="n">c_2</span><span class="o">.</span><span class="n">id</span> <span class="o">=</span> <span class="n">a</span><span class="o">.</span><span class="n">c_id</span></pre></div>
</div>
<p>We can see that the <code class="docutils literal notranslate"><span class="pre">c</span></code> table is selected from twice; once in the context
of <code class="docutils literal notranslate"><span class="pre">A.b.c</span> <span class="pre">-></span> <span class="pre">c_alias_1</span></code> and another in the context of <code class="docutils literal notranslate"><span class="pre">A.c</span> <span class="pre">-></span> <span class="pre">c_alias_2</span></code>.
Also, we can see that it is quite possible that the <code class="docutils literal notranslate"><span class="pre">C</span></code> identity for a
single row is the <strong>same</strong> for both <code class="docutils literal notranslate"><span class="pre">c_alias_1</span></code> and <code class="docutils literal notranslate"><span class="pre">c_alias_2</span></code>, meaning
two sets of columns in one row result in only one new object being added
to the identity map.</p>
<p>The query options above only call for the attribute <code class="docutils literal notranslate"><span class="pre">C.d</span></code> to be loaded
in the context of <code class="docutils literal notranslate"><span class="pre">c_alias_1</span></code>, and not <code class="docutils literal notranslate"><span class="pre">c_alias_2</span></code>. So whether or not
the final <code class="docutils literal notranslate"><span class="pre">C</span></code> object we get in the identity map has the <code class="docutils literal notranslate"><span class="pre">C.d</span></code> attribute
loaded depends on how the mappings are traversed, which while not completely
random, is essentially non-deterministic. The fix is that even if the
loader for <code class="docutils literal notranslate"><span class="pre">c_alias_1</span></code> is processed after that of <code class="docutils literal notranslate"><span class="pre">c_alias_2</span></code> for a
single row where they both refer to the same identity, the <code class="docutils literal notranslate"><span class="pre">C.d</span></code>
element will still be loaded. Previously, the loader did not seek to
modify the load of an entity that was already loaded via a different path.
The loader that reaches the entity first has always been non-deterministic,
so this fix may be detectable as a behavioral change in some situations and
not others.</p>
<p>The fix includes tests for two variants of the “multiple paths to one entity”
case, and the fix should hopefully cover all other scenarios of this nature.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3431">#3431</a></p>
</div>
<div class="section" id="new-mutablelist-and-mutableset-helpers-added-to-the-mutation-tracking-extension">
<h3>New MutableList and MutableSet helpers added to the mutation tracking extension<a class="headerlink" href="#new-mutablelist-and-mutableset-helpers-added-to-the-mutation-tracking-extension" title="Permalink to this headline">¶</a></h3>
<p>New helper classes <a class="reference internal" href="../orm/extensions/mutable.html#sqlalchemy.ext.mutable.MutableList" title="sqlalchemy.ext.mutable.MutableList"><code class="xref py py-class docutils literal notranslate"><span class="pre">MutableList</span></code></a> and <a class="reference internal" href="../orm/extensions/mutable.html#sqlalchemy.ext.mutable.MutableSet" title="sqlalchemy.ext.mutable.MutableSet"><code class="xref py py-class docutils literal notranslate"><span class="pre">MutableSet</span></code></a> have been
added to the <a class="reference internal" href="../orm/extensions/mutable.html"><span class="std std-ref">Mutation Tracking</span></a> extension, to complement the existing
<a class="reference internal" href="../orm/extensions/mutable.html#sqlalchemy.ext.mutable.MutableDict" title="sqlalchemy.ext.mutable.MutableDict"><code class="xref py py-class docutils literal notranslate"><span class="pre">MutableDict</span></code></a> helper.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3297">#3297</a></p>
</div>
<div class="section" id="new-raise-raise-on-sql-loader-strategies">
<span id="change-3512"></span><h3>New “raise” / “raise_on_sql” loader strategies<a class="headerlink" href="#new-raise-raise-on-sql-loader-strategies" title="Permalink to this headline">¶</a></h3>
<p>To assist with the use case of preventing unwanted lazy loads from occurring
after a series of objects are loaded, the new “lazy=’raise’” and
“lazy=’raise_on_sql’” strategies and
corresponding loader option <a class="reference internal" href="../orm/loading_relationships.html#sqlalchemy.orm.raiseload" title="sqlalchemy.orm.raiseload"><code class="xref py py-func docutils literal notranslate"><span class="pre">orm.raiseload()</span></code></a> may be applied to a
relationship attribute which will cause it to raise <code class="docutils literal notranslate"><span class="pre">InvalidRequestError</span></code>
when a non-eagerly-loaded attribute is accessed for read. The two variants
test for either a lazy load of any variety, including those that would
only return None or retrieve from the identity map:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy.orm</span> <span class="k">import</span> <span class="n">raiseload</span>
<span class="gp">>>> </span><span class="n">a1</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">A</span><span class="p">)</span><span class="o">.</span><span class="n">options</span><span class="p">(</span><span class="n">raiseload</span><span class="p">(</span><span class="n">A</span><span class="o">.</span><span class="n">some_b</span><span class="p">))</span><span class="o">.</span><span class="n">first</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">a1</span><span class="o">.</span><span class="n">some_b</span>
<span class="gt">Traceback (most recent call last):</span>
<span class="c">...</span>
<span class="gr">sqlalchemy.exc.InvalidRequestError</span>: <span class="n">'A.some_b' is not available due to lazy='raise'</span></pre></div>
</div>
<p>Or a lazy load only where SQL would be emitted:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy.orm</span> <span class="k">import</span> <span class="n">raiseload</span>
<span class="gp">>>> </span><span class="n">a1</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">A</span><span class="p">)</span><span class="o">.</span><span class="n">options</span><span class="p">(</span><span class="n">raiseload</span><span class="p">(</span><span class="n">A</span><span class="o">.</span><span class="n">some_b</span><span class="p">,</span> <span class="n">sql_only</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span><span class="o">.</span><span class="n">first</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">a1</span><span class="o">.</span><span class="n">some_b</span>
<span class="gt">Traceback (most recent call last):</span>
<span class="c">...</span>
<span class="gr">sqlalchemy.exc.InvalidRequestError</span>: <span class="n">'A.bs' is not available due to lazy='raise_on_sql'</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3512">#3512</a></p>
</div>
<div class="section" id="mapper-order-by-is-deprecated">
<span id="change-3394"></span><h3>Mapper.order_by is deprecated<a class="headerlink" href="#mapper-order-by-is-deprecated" title="Permalink to this headline">¶</a></h3>
<p>This old parameter from the very first versions of SQLAlchemy was part of
the original design of the ORM which featured the <a class="reference internal" href="../orm/mapping_api.html#sqlalchemy.orm.mapper.Mapper" title="sqlalchemy.orm.mapper.Mapper"><code class="xref py py-class docutils literal notranslate"><span class="pre">Mapper</span></code></a> object
as a public-facing query structure. This role has long since been replaced
by the <a class="reference internal" href="../orm/query.html#sqlalchemy.orm.query.Query" title="sqlalchemy.orm.query.Query"><code class="xref py py-class docutils literal notranslate"><span class="pre">Query</span></code></a> object, where we use <a class="reference internal" href="../orm/query.html#sqlalchemy.orm.query.Query.order_by" title="sqlalchemy.orm.query.Query.order_by"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Query.order_by()</span></code></a> to
indicate the ordering of results in a way that works consistently for any
combination of SELECT statements, entities and SQL expressions. There are
many areas in which <a class="reference internal" href="../orm/mapping_api.html#sqlalchemy.orm.mapper.Mapper.params.order_by" title="sqlalchemy.orm.mapper.Mapper"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">Mapper.order_by</span></code></a> doesn’t work as expected
(or what would be expected is not clear), such as when queries are combined
into unions; these cases are not supported.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3394">#3394</a></p>
</div>
</div>
<div class="section" id="new-features-and-improvements-core">
<h2>New Features and Improvements - Core<a class="headerlink" href="#new-features-and-improvements-core" title="Permalink to this headline">¶</a></h2>
<div class="section" id="engines-now-invalidate-connections-run-error-handlers-for-baseexception">
<span id="change-3803"></span><h3>Engines now invalidate connections, run error handlers for BaseException<a class="headerlink" href="#engines-now-invalidate-connections-run-error-handlers-for-baseexception" title="Permalink to this headline">¶</a></h3>
<div class="versionadded">
<p><span class="versionmodified added">New in version 1.1: </span>this change is a late add to the 1.1 series just
prior to 1.1 final, and is not present in the 1.1 beta releases.</p>
</div>
<p>The Python <code class="docutils literal notranslate"><span class="pre">BaseException</span></code> class is below that of <code class="docutils literal notranslate"><span class="pre">Exception</span></code> but is the
identifiable base for system-level exceptions such as <code class="docutils literal notranslate"><span class="pre">KeyboardInterrupt</span></code>,
<code class="docutils literal notranslate"><span class="pre">SystemExit</span></code>, and notably the <code class="docutils literal notranslate"><span class="pre">GreenletExit</span></code> exception that’s used by
eventlet and gevent. This exception class is now intercepted by the exception-
handling routines of <a class="reference internal" href="../core/connections.html#sqlalchemy.engine.Connection" title="sqlalchemy.engine.Connection"><code class="xref py py-class docutils literal notranslate"><span class="pre">Connection</span></code></a>, and includes handling by the
<a class="reference internal" href="../core/events.html#sqlalchemy.events.ConnectionEvents.handle_error" title="sqlalchemy.events.ConnectionEvents.handle_error"><code class="xref py py-meth docutils literal notranslate"><span class="pre">handle_error()</span></code></a> event. The <a class="reference internal" href="../core/connections.html#sqlalchemy.engine.Connection" title="sqlalchemy.engine.Connection"><code class="xref py py-class docutils literal notranslate"><span class="pre">Connection</span></code></a> is now
<strong>invalidated</strong> by default in the case of a system level exception that is not
a subclass of <code class="docutils literal notranslate"><span class="pre">Exception</span></code>, as it is assumed an operation was interrupted and
the connection may be in an unusable state. The MySQL drivers are most
targeted by this change however the change is across all DBAPIs.</p>
<p>Note that upon invalidation, the immediate DBAPI connection used by
<a class="reference internal" href="../core/connections.html#sqlalchemy.engine.Connection" title="sqlalchemy.engine.Connection"><code class="xref py py-class docutils literal notranslate"><span class="pre">Connection</span></code></a> is disposed, and the <a class="reference internal" href="../core/connections.html#sqlalchemy.engine.Connection" title="sqlalchemy.engine.Connection"><code class="xref py py-class docutils literal notranslate"><span class="pre">Connection</span></code></a>, if still
being used subsequent to the exception raise, will use a new
DBAPI connection for subsequent operations upon next use; however, the state of
any transaction in progress is lost and the appropriate <code class="docutils literal notranslate"><span class="pre">.rollback()</span></code> method
must be called if applicable before this re-use can proceed.</p>
<p>In order to identify this change, it was straightforward to demonstrate a pymysql or
mysqlclient / MySQL-Python connection moving into a corrupted state when
these exceptions occur in the middle of the connection doing its work;
the connection would then be returned to the connection pool where subsequent
uses would fail, or even before returning to the pool would cause secondary
failures in context managers that call <code class="docutils literal notranslate"><span class="pre">.rollback()</span></code> upon the exception
catch. The behavior here is expected to reduce
the incidence of the MySQL error “commands out of sync”, as well as the
<code class="docutils literal notranslate"><span class="pre">ResourceClosedError</span></code> which can occur when the MySQL driver fails to
report <code class="docutils literal notranslate"><span class="pre">cursor.description</span></code> correctly, when running under greenlet
conditions where greenlets are killed, or where <code class="docutils literal notranslate"><span class="pre">KeyboardInterrupt</span></code> exceptions
are handled without exiting the program entirely.</p>
<p>The behavior is distinct from the usual auto-invalidation feature, in that it
does not assume that the backend database itself has been shut down or
restarted; it does not recycle the entire connection pool as is the case
for usual DBAPI disconnect exceptions.</p>
<p>This change should be a net improvement for all users with the exception
of <strong>any application that currently intercepts ``KeyboardInterrupt`` or
``GreenletExit`` and wishes to continue working within the same transaction</strong>.
Such an operation is theoretically possible with other DBAPIs that do not appear to be
impacted by <code class="docutils literal notranslate"><span class="pre">KeyboardInterrupt</span></code> such as psycopg2. For these DBAPIs,
the following workaround will disable the connection from being recycled
for specific exceptions:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">engine</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s2">"postgresql+psycopg2://"</span><span class="p">)</span>
<span class="nd">@event</span><span class="o">.</span><span class="n">listens_for</span><span class="p">(</span><span class="n">engine</span><span class="p">,</span> <span class="s2">"handle_error"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">cancel_disconnect</span><span class="p">(</span><span class="n">ctx</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ctx</span><span class="o">.</span><span class="n">original_exception</span><span class="p">,</span> <span class="ne">KeyboardInterrupt</span><span class="p">):</span>
<span class="n">ctx</span><span class="o">.</span><span class="n">is_disconnect</span> <span class="o">=</span> <span class="kc">False</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3803">#3803</a></p>
</div>
<div class="section" id="cte-support-for-insert-update-delete">
<span id="change-2551"></span><h3>CTE Support for INSERT, UPDATE, DELETE<a class="headerlink" href="#cte-support-for-insert-update-delete" title="Permalink to this headline">¶</a></h3>
<p>One of the most widely requested features is support for common table
expressions (CTE) that work with INSERT, UPDATE, DELETE, and is now implemented.
An INSERT/UPDATE/DELETE can both draw from a WITH clause that’s stated at the
top of the SQL, as well as can be used as a CTE itself in the context of
a larger statement.</p>
<p>As part of this change, an INSERT from SELECT that includes a CTE will now
render the CTE at the top of the entire statement, rather than nested
in the SELECT statement as was the case in 1.0.</p>
<p>Below is an example that renders UPDATE, INSERT and SELECT all in one
statement:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">table</span><span class="p">,</span> <span class="n">column</span><span class="p">,</span> <span class="n">select</span><span class="p">,</span> <span class="n">literal</span><span class="p">,</span> <span class="n">exists</span>
<span class="gp">>>> </span><span class="n">orders</span> <span class="o">=</span> <span class="n">table</span><span class="p">(</span>
<span class="gp">... </span> <span class="s1">'orders'</span><span class="p">,</span>
<span class="gp">... </span> <span class="n">column</span><span class="p">(</span><span class="s1">'region'</span><span class="p">),</span>
<span class="gp">... </span> <span class="n">column</span><span class="p">(</span><span class="s1">'amount'</span><span class="p">),</span>
<span class="gp">... </span> <span class="n">column</span><span class="p">(</span><span class="s1">'product'</span><span class="p">),</span>
<span class="gp">... </span> <span class="n">column</span><span class="p">(</span><span class="s1">'quantity'</span><span class="p">)</span>
<span class="gp">... </span><span class="p">)</span>
<span class="go">>>></span>
<span class="gp">>>> </span><span class="n">upsert</span> <span class="o">=</span> <span class="p">(</span>
<span class="gp">... </span> <span class="n">orders</span><span class="o">.</span><span class="n">update</span><span class="p">()</span>
<span class="gp">... </span> <span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">orders</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">region</span> <span class="o">==</span> <span class="s1">'Region1'</span><span class="p">)</span>
<span class="gp">... </span> <span class="o">.</span><span class="n">values</span><span class="p">(</span><span class="n">amount</span><span class="o">=</span><span class="mf">1.0</span><span class="p">,</span> <span class="n">product</span><span class="o">=</span><span class="s1">'Product1'</span><span class="p">,</span> <span class="n">quantity</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="gp">... </span> <span class="o">.</span><span class="n">returning</span><span class="p">(</span><span class="o">*</span><span class="p">(</span><span class="n">orders</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">_all_columns</span><span class="p">))</span><span class="o">.</span><span class="n">cte</span><span class="p">(</span><span class="s1">'upsert'</span><span class="p">))</span>
<span class="go">>>></span>
<span class="gp">>>> </span><span class="n">insert</span> <span class="o">=</span> <span class="n">orders</span><span class="o">.</span><span class="n">insert</span><span class="p">()</span><span class="o">.</span><span class="n">from_select</span><span class="p">(</span>
<span class="gp">... </span> <span class="n">orders</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">keys</span><span class="p">(),</span>
<span class="gp">... </span> <span class="n">select</span><span class="p">([</span>
<span class="gp">... </span> <span class="n">literal</span><span class="p">(</span><span class="s1">'Region1'</span><span class="p">),</span> <span class="n">literal</span><span class="p">(</span><span class="mf">1.0</span><span class="p">),</span>
<span class="gp">... </span> <span class="n">literal</span><span class="p">(</span><span class="s1">'Product1'</span><span class="p">),</span> <span class="n">literal</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="gp">... </span> <span class="p">])</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="o">~</span><span class="n">exists</span><span class="p">(</span><span class="n">upsert</span><span class="o">.</span><span class="n">select</span><span class="p">()))</span>
<span class="gp">... </span><span class="p">)</span>
<span class="go">>>></span>
<span class="gp">>>> </span><span class="nb">print</span><span class="p">(</span><span class="n">insert</span><span class="p">)</span> <span class="c1"># note formatting added for clarity</span>
<span class="go">WITH upsert AS</span>
<span class="go">(UPDATE orders SET amount=:amount, product=:product, quantity=:quantity</span>
<span class="go"> WHERE orders.region = :region_1</span>
<span class="go"> RETURNING orders.region, orders.amount, orders.product, orders.quantity</span>
<span class="go">)</span>
<span class="go">INSERT INTO orders (region, amount, product, quantity)</span>
<span class="go">SELECT</span>
<span class="go"> :param_1 AS anon_1, :param_2 AS anon_2,</span>
<span class="go"> :param_3 AS anon_3, :param_4 AS anon_4</span>
<span class="go">WHERE NOT (</span>
<span class="go"> EXISTS (</span>
<span class="go"> SELECT upsert.region, upsert.amount,</span>
<span class="go"> upsert.product, upsert.quantity</span>
<span class="go"> FROM upsert))</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/2551">#2551</a></p>
</div>
<div class="section" id="support-for-range-and-rows-specification-within-window-functions">
<span id="change-3049"></span><h3>Support for RANGE and ROWS specification within window functions<a class="headerlink" href="#support-for-range-and-rows-specification-within-window-functions" title="Permalink to this headline">¶</a></h3>
<p>New <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.over.params.range_" title="sqlalchemy.sql.expression.over"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">expression.over.range_</span></code></a> and <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.over.params.rows" title="sqlalchemy.sql.expression.over"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">expression.over.rows</span></code></a> parameters allow
RANGE and ROWS expressions for window functions:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">func</span>
<span class="gp">>>> </span><span class="nb">print</span> <span class="n">func</span><span class="o">.</span><span class="n">row_number</span><span class="p">()</span><span class="o">.</span><span class="n">over</span><span class="p">(</span><span class="n">order_by</span><span class="o">=</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">range_</span><span class="o">=</span><span class="p">(</span><span class="o">-</span><span class="mi">5</span><span class="p">,</span> <span class="mi">10</span><span class="p">))</span>
<span class="go">row_number() OVER (ORDER BY x RANGE BETWEEN :param_1 PRECEDING AND :param_2 FOLLOWING)</span>
<span class="gp">>>> </span><span class="nb">print</span> <span class="n">func</span><span class="o">.</span><span class="n">row_number</span><span class="p">()</span><span class="o">.</span><span class="n">over</span><span class="p">(</span><span class="n">order_by</span><span class="o">=</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="mi">0</span><span class="p">))</span>
<span class="go">row_number() OVER (ORDER BY x ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)</span>
<span class="gp">>>> </span><span class="nb">print</span> <span class="n">func</span><span class="o">.</span><span class="n">row_number</span><span class="p">()</span><span class="o">.</span><span class="n">over</span><span class="p">(</span><span class="n">order_by</span><span class="o">=</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">range_</span><span class="o">=</span><span class="p">(</span><span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="kc">None</span><span class="p">))</span>
<span class="go">row_number() OVER (ORDER BY x RANGE BETWEEN :param_1 PRECEDING AND UNBOUNDED FOLLOWING)</span></pre></div>
</div>
<p><a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.over.params.range_" title="sqlalchemy.sql.expression.over"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">expression.over.range_</span></code></a> and <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.over.params.rows" title="sqlalchemy.sql.expression.over"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">expression.over.rows</span></code></a> are specified as
2-tuples and indicate negative and positive values for specific ranges,
0 for “CURRENT ROW”, and None for UNBOUNDED.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../core/tutorial.html#window-functions"><span class="std std-ref">Window Functions</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3049">#3049</a></p>
</div>
<div class="section" id="support-for-the-sql-lateral-keyword">
<span id="change-2857"></span><h3>Support for the SQL LATERAL keyword<a class="headerlink" href="#support-for-the-sql-lateral-keyword" title="Permalink to this headline">¶</a></h3>
<p>The LATERAL keyword is currently known to only be supported by PostgreSQL 9.3
and greater, however as it is part of the SQL standard support for this keyword
is added to Core. The implementation of <a class="reference internal" href="../core/selectable.html#sqlalchemy.sql.expression.Select.lateral" title="sqlalchemy.sql.expression.Select.lateral"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Select.lateral()</span></code></a> employs
special logic beyond just rendering the LATERAL keyword to allow for
correlation of tables that are derived from the same FROM clause as the
selectable, e.g. lateral correlation:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">table</span><span class="p">,</span> <span class="n">column</span><span class="p">,</span> <span class="n">select</span><span class="p">,</span> <span class="n">true</span>
<span class="gp">>>> </span><span class="n">people</span> <span class="o">=</span> <span class="n">table</span><span class="p">(</span><span class="s1">'people'</span><span class="p">,</span> <span class="n">column</span><span class="p">(</span><span class="s1">'people_id'</span><span class="p">),</span> <span class="n">column</span><span class="p">(</span><span class="s1">'age'</span><span class="p">),</span> <span class="n">column</span><span class="p">(</span><span class="s1">'name'</span><span class="p">))</span>
<span class="gp">>>> </span><span class="n">books</span> <span class="o">=</span> <span class="n">table</span><span class="p">(</span><span class="s1">'books'</span><span class="p">,</span> <span class="n">column</span><span class="p">(</span><span class="s1">'book_id'</span><span class="p">),</span> <span class="n">column</span><span class="p">(</span><span class="s1">'owner_id'</span><span class="p">))</span>
<span class="gp">>>> </span><span class="n">subq</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">books</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">book_id</span><span class="p">])</span><span class="o">.</span>\
<span class="gp">... </span> <span class="n">where</span><span class="p">(</span><span class="n">books</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">owner_id</span> <span class="o">==</span> <span class="n">people</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">people_id</span><span class="p">)</span><span class="o">.</span><span class="n">lateral</span><span class="p">(</span><span class="s2">"book_subq"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="nb">print</span><span class="p">(</span><span class="n">select</span><span class="p">([</span><span class="n">people</span><span class="p">])</span><span class="o">.</span><span class="n">select_from</span><span class="p">(</span><span class="n">people</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">subq</span><span class="p">,</span> <span class="n">true</span><span class="p">())))</span>
<span class="go">SELECT people.people_id, people.age, people.name</span>
<span class="go">FROM people JOIN LATERAL (SELECT books.book_id AS book_id</span>
<span class="go">FROM books WHERE books.owner_id = people.people_id)</span>
<span class="go">AS book_subq ON true</span></pre></div>
</div>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../core/tutorial.html#lateral-selects"><span class="std std-ref">LATERAL correlation</span></a></p>
<p><a class="reference internal" href="../core/selectable.html#sqlalchemy.sql.expression.Lateral" title="sqlalchemy.sql.expression.Lateral"><code class="xref py py-class docutils literal notranslate"><span class="pre">Lateral</span></code></a></p>
<p><a class="reference internal" href="../core/selectable.html#sqlalchemy.sql.expression.Select.lateral" title="sqlalchemy.sql.expression.Select.lateral"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Select.lateral()</span></code></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/2857">#2857</a></p>
</div>
<div class="section" id="support-for-tablesample">
<span id="change-3718"></span><h3>Support for TABLESAMPLE<a class="headerlink" href="#support-for-tablesample" title="Permalink to this headline">¶</a></h3>
<p>The SQL standard TABLESAMPLE can be rendered using the
<a class="reference internal" href="../core/selectable.html#sqlalchemy.sql.expression.FromClause.tablesample" title="sqlalchemy.sql.expression.FromClause.tablesample"><code class="xref py py-meth docutils literal notranslate"><span class="pre">FromClause.tablesample()</span></code></a> method, which returns a <a class="reference internal" href="../core/selectable.html#sqlalchemy.sql.expression.TableSample" title="sqlalchemy.sql.expression.TableSample"><code class="xref py py-class docutils literal notranslate"><span class="pre">TableSample</span></code></a>
construct similar to an alias:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">func</span>
<span class="n">selectable</span> <span class="o">=</span> <span class="n">people</span><span class="o">.</span><span class="n">tablesample</span><span class="p">(</span>
<span class="n">func</span><span class="o">.</span><span class="n">bernoulli</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span>
<span class="n">name</span><span class="o">=</span><span class="s1">'alias'</span><span class="p">,</span>
<span class="n">seed</span><span class="o">=</span><span class="n">func</span><span class="o">.</span><span class="n">random</span><span class="p">())</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">selectable</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">people_id</span><span class="p">])</span></pre></div>
</div>
<p>Assuming <code class="docutils literal notranslate"><span class="pre">people</span></code> with a column <code class="docutils literal notranslate"><span class="pre">people_id</span></code>, the above
statement would render as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SELECT</span> <span class="n">alias</span><span class="o">.</span><span class="n">people_id</span> <span class="n">FROM</span>
<span class="n">people</span> <span class="n">AS</span> <span class="n">alias</span> <span class="n">TABLESAMPLE</span> <span class="n">bernoulli</span><span class="p">(:</span><span class="n">bernoulli_1</span><span class="p">)</span>
<span class="n">REPEATABLE</span> <span class="p">(</span><span class="n">random</span><span class="p">())</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3718">#3718</a></p>
</div>
<div class="section" id="the-autoincrement-directive-is-no-longer-implicitly-enabled-for-a-composite-primary-key-column">
<span id="change-3216"></span><h3>The <code class="docutils literal notranslate"><span class="pre">.autoincrement</span></code> directive is no longer implicitly enabled for a composite primary key column<a class="headerlink" href="#the-autoincrement-directive-is-no-longer-implicitly-enabled-for-a-composite-primary-key-column" title="Permalink to this headline">¶</a></h3>
<p>SQLAlchemy has always had the convenience feature of enabling the backend database’s
“autoincrement” feature for a single-column integer primary key; by “autoincrement”
we mean that the database column will include whatever DDL directives the
database provides in order to indicate an auto-incrementing integer identifier,
such as the SERIAL keyword on PostgreSQL or AUTO_INCREMENT on MySQL, and additionally
that the dialect will receive these generated values from the execution
of a <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Table.insert" title="sqlalchemy.schema.Table.insert"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Table.insert()</span></code></a> construct using techniques appropriate to that
backend.</p>
<p>What’s changed is that this feature no longer turns on automatically for a
<em>composite</em> primary key; previously, a table definition such as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Table</span><span class="p">(</span>
<span class="s1">'some_table'</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'y'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="p">)</span></pre></div>
</div>
<p>Would have “autoincrement” semantics applied to the <code class="docutils literal notranslate"><span class="pre">'x'</span></code> column, only
because it’s first in the list of primary key columns. In order to
disable this, one would have to turn off <code class="docutils literal notranslate"><span class="pre">autoincrement</span></code> on all columns:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># old way</span>
<span class="n">Table</span><span class="p">(</span>
<span class="s1">'some_table'</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">autoincrement</span><span class="o">=</span><span class="kc">False</span><span class="p">),</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'y'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">autoincrement</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
<span class="p">)</span></pre></div>
</div>
<p>With the new behavior, the composite primary key will not have autoincrement
semantics unless a column is marked explicitly with <code class="docutils literal notranslate"><span class="pre">autoincrement=True</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># column 'y' will be SERIAL/AUTO_INCREMENT/ auto-generating</span>
<span class="n">Table</span><span class="p">(</span>
<span class="s1">'some_table'</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'y'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">autoincrement</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="p">)</span></pre></div>
</div>
<p>In order to anticipate some potential backwards-incompatible scenarios,
the <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Table.insert" title="sqlalchemy.schema.Table.insert"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Table.insert()</span></code></a> construct will perform more thorough checks
for missing primary key values on composite primary key columns that don’t
have autoincrement set up; given a table such as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Table</span><span class="p">(</span>
<span class="s1">'b'</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'y'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="p">)</span></pre></div>
</div>
<p>An INSERT emitted with no values for this table will produce this warning:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SAWarning</span><span class="p">:</span> <span class="n">Column</span> <span class="s1">'b.x'</span> <span class="ow">is</span> <span class="n">marked</span> <span class="k">as</span> <span class="n">a</span> <span class="n">member</span> <span class="n">of</span> <span class="n">the</span> <span class="n">primary</span>
<span class="n">key</span> <span class="k">for</span> <span class="n">table</span> <span class="s1">'b'</span><span class="p">,</span> <span class="n">but</span> <span class="n">has</span> <span class="n">no</span> <span class="n">Python</span><span class="o">-</span><span class="n">side</span> <span class="ow">or</span> <span class="n">server</span><span class="o">-</span><span class="n">side</span> <span class="n">default</span>
<span class="n">generator</span> <span class="n">indicated</span><span class="p">,</span> <span class="n">nor</span> <span class="n">does</span> <span class="n">it</span> <span class="n">indicate</span> <span class="s1">'autoincrement=True'</span><span class="p">,</span>
<span class="ow">and</span> <span class="n">no</span> <span class="n">explicit</span> <span class="n">value</span> <span class="ow">is</span> <span class="n">passed</span><span class="o">.</span> <span class="n">Primary</span> <span class="n">key</span> <span class="n">columns</span> <span class="n">may</span> <span class="ow">not</span>
<span class="n">store</span> <span class="n">NULL</span><span class="o">.</span> <span class="n">Note</span> <span class="n">that</span> <span class="k">as</span> <span class="n">of</span> <span class="n">SQLAlchemy</span> <span class="mf">1.1</span><span class="p">,</span> <span class="s1">'autoincrement=True'</span>
<span class="n">must</span> <span class="n">be</span> <span class="n">indicated</span> <span class="n">explicitly</span> <span class="k">for</span> <span class="n">composite</span> <span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">g</span><span class="o">.</span> <span class="n">multicolumn</span><span class="p">)</span>
<span class="n">primary</span> <span class="n">keys</span> <span class="k">if</span> <span class="n">AUTO_INCREMENT</span><span class="o">/</span><span class="n">SERIAL</span><span class="o">/</span><span class="n">IDENTITY</span> <span class="n">behavior</span> <span class="ow">is</span>
<span class="n">expected</span> <span class="k">for</span> <span class="n">one</span> <span class="n">of</span> <span class="n">the</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">the</span> <span class="n">primary</span> <span class="n">key</span><span class="o">.</span> <span class="n">CREATE</span> <span class="n">TABLE</span>
<span class="n">statements</span> <span class="n">are</span> <span class="n">impacted</span> <span class="n">by</span> <span class="n">this</span> <span class="n">change</span> <span class="k">as</span> <span class="n">well</span> <span class="n">on</span> <span class="n">most</span> <span class="n">backends</span><span class="o">.</span></pre></div>
</div>
<p>For a column that is receiving primary key values from a server-side
default or something less common such as a trigger, the presence of a
value generator can be indicated using <a class="reference internal" href="../core/defaults.html#sqlalchemy.schema.FetchedValue" title="sqlalchemy.schema.FetchedValue"><code class="xref py py-class docutils literal notranslate"><span class="pre">FetchedValue</span></code></a>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Table</span><span class="p">(</span>
<span class="s1">'b'</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">server_default</span><span class="o">=</span><span class="n">FetchedValue</span><span class="p">()),</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'y'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">server_default</span><span class="o">=</span><span class="n">FetchedValue</span><span class="p">())</span>
<span class="p">)</span></pre></div>
</div>
<p>For the very unlikely case where a composite primary key is actually intended
to store NULL in one or more of its columns (only supported on SQLite and MySQL),
specify the column with <code class="docutils literal notranslate"><span class="pre">nullable=True</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Table</span><span class="p">(</span>
<span class="s1">'b'</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'y'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="p">)</span></pre></div>
</div>
<p>In a related change, the <code class="docutils literal notranslate"><span class="pre">autoincrement</span></code> flag may be set to True
on a column that has a client-side or server-side default. This typically
will not have much impact on the behavior of the column during an INSERT.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="#change-mysql-3216"><span class="std std-ref">No more generation of an implicit KEY for composite primary key w/ AUTO_INCREMENT</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3216">#3216</a></p>
</div>
<div class="section" id="support-for-is-distinct-from-and-is-not-distinct-from">
<span id="change-is-distinct-from"></span><h3>Support for IS DISTINCT FROM and IS NOT DISTINCT FROM<a class="headerlink" href="#support-for-is-distinct-from-and-is-not-distinct-from" title="Permalink to this headline">¶</a></h3>
<p>New operators <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.operators.ColumnOperators.is_distinct_from" title="sqlalchemy.sql.operators.ColumnOperators.is_distinct_from"><code class="xref py py-meth docutils literal notranslate"><span class="pre">ColumnOperators.is_distinct_from()</span></code></a> and
<a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.operators.ColumnOperators.isnot_distinct_from" title="sqlalchemy.sql.operators.ColumnOperators.isnot_distinct_from"><code class="xref py py-meth docutils literal notranslate"><span class="pre">ColumnOperators.isnot_distinct_from()</span></code></a> allow the IS DISTINCT
FROM and IS NOT DISTINCT FROM sql operation:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="nb">print</span> <span class="n">column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">)</span><span class="o">.</span><span class="n">is_distinct_from</span><span class="p">(</span><span class="kc">None</span><span class="p">)</span>
<span class="go">x IS DISTINCT FROM NULL</span></pre></div>
</div>
<p>Handling is provided for NULL, True and False:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="nb">print</span> <span class="n">column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">)</span><span class="o">.</span><span class="n">isnot_distinct_from</span><span class="p">(</span><span class="kc">False</span><span class="p">)</span>
<span class="go">x IS NOT DISTINCT FROM false</span></pre></div>
</div>
<p>For SQLite, which doesn’t have this operator, “IS” / “IS NOT” is rendered,
which on SQLite works for NULL unlike other backends:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy.dialects</span> <span class="k">import</span> <span class="n">sqlite</span>
<span class="gp">>>> </span><span class="nb">print</span> <span class="n">column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">)</span><span class="o">.</span><span class="n">is_distinct_from</span><span class="p">(</span><span class="kc">None</span><span class="p">)</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="n">dialect</span><span class="o">=</span><span class="n">sqlite</span><span class="o">.</span><span class="n">dialect</span><span class="p">())</span>
<span class="go">x IS NOT NULL</span></pre></div>
</div>
</div>
<div class="section" id="core-and-orm-support-for-full-outer-join">
<span id="change-1957"></span><h3>Core and ORM support for FULL OUTER JOIN<a class="headerlink" href="#core-and-orm-support-for-full-outer-join" title="Permalink to this headline">¶</a></h3>
<p>The new flag <a class="reference internal" href="../core/selectable.html#sqlalchemy.sql.expression.FromClause.outerjoin.params.full" title="sqlalchemy.sql.expression.FromClause.outerjoin"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">FromClause.outerjoin.full</span></code></a>, available at the Core
and ORM level, instructs the compiler to render <code class="docutils literal notranslate"><span class="pre">FULL</span> <span class="pre">OUTER</span> <span class="pre">JOIN</span></code>
where it would normally render <code class="docutils literal notranslate"><span class="pre">LEFT</span> <span class="pre">OUTER</span> <span class="pre">JOIN</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">stmt</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">t1</span><span class="p">])</span><span class="o">.</span><span class="n">select_from</span><span class="p">(</span><span class="n">t1</span><span class="o">.</span><span class="n">outerjoin</span><span class="p">(</span><span class="n">t2</span><span class="p">,</span> <span class="n">full</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span></pre></div>
</div>
<p>The flag also works at the ORM level:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">q</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">MyClass</span><span class="p">)</span><span class="o">.</span><span class="n">outerjoin</span><span class="p">(</span><span class="n">MyOtherClass</span><span class="p">,</span> <span class="n">full</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/1957">#1957</a></p>
</div>
<div class="section" id="resultset-column-matching-enhancements-positional-column-setup-for-textual-sql">
<span id="change-3501"></span><h3>ResultSet column matching enhancements; positional column setup for textual SQL<a class="headerlink" href="#resultset-column-matching-enhancements-positional-column-setup-for-textual-sql" title="Permalink to this headline">¶</a></h3>
<p>A series of improvements were made to the <a class="reference internal" href="../core/connections.html#sqlalchemy.engine.ResultProxy" title="sqlalchemy.engine.ResultProxy"><code class="xref py py-class docutils literal notranslate"><span class="pre">ResultProxy</span></code></a> system
in the 1.0 series as part of <a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/918">#918</a>, which reorganizes the internals
to match cursor-bound result columns with table/ORM metadata positionally,
rather than by matching names, for compiled SQL constructs that contain full
information about the result rows to be returned. This allows a dramatic savings
on Python overhead as well as much greater accuracy in linking ORM and Core
SQL expressions to result rows. In 1.1, this reorganization has been taken
further internally, and also has been made available to pure-text SQL
constructs via the use of the recently added <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.TextClause.columns" title="sqlalchemy.sql.expression.TextClause.columns"><code class="xref py py-meth docutils literal notranslate"><span class="pre">TextClause.columns()</span></code></a> method.</p>
<div class="section" id="textasfrom-columns-now-works-positionally">
<h4>TextAsFrom.columns() now works positionally<a class="headerlink" href="#textasfrom-columns-now-works-positionally" title="Permalink to this headline">¶</a></h4>
<p>The <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.TextClause.columns" title="sqlalchemy.sql.expression.TextClause.columns"><code class="xref py py-meth docutils literal notranslate"><span class="pre">TextClause.columns()</span></code></a> method, added in 0.9, accepts column-based arguments
positionally; in 1.1, when all columns are passed positionally, the correlation
of these columns to the ultimate result set is also performed positionally.
The key advantage here is that textual SQL can now be linked to an ORM-
level result set without the need to deal with ambiguous or duplicate column
names, or with having to match labeling schemes to ORM-level labeling schemes. All
that’s needed now is the same ordering of columns within the textual SQL
and the column arguments passed to <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.TextClause.columns" title="sqlalchemy.sql.expression.TextClause.columns"><code class="xref py py-meth docutils literal notranslate"><span class="pre">TextClause.columns()</span></code></a>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">text</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">text</span><span class="p">(</span><span class="s2">"SELECT users.id, addresses.id, users.id, "</span>
<span class="s2">"users.name, addresses.email_address AS email "</span>
<span class="s2">"FROM users JOIN addresses ON users.id=addresses.user_id "</span>
<span class="s2">"WHERE users.id = 1"</span><span class="p">)</span><span class="o">.</span><span class="n">columns</span><span class="p">(</span>
<span class="n">User</span><span class="o">.</span><span class="n">id</span><span class="p">,</span>
<span class="n">Address</span><span class="o">.</span><span class="n">id</span><span class="p">,</span>
<span class="n">Address</span><span class="o">.</span><span class="n">user_id</span><span class="p">,</span>
<span class="n">User</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
<span class="n">Address</span><span class="o">.</span><span class="n">email_address</span>
<span class="p">)</span>
<span class="n">query</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">User</span><span class="p">)</span><span class="o">.</span><span class="n">from_statement</span><span class="p">(</span><span class="n">stmt</span><span class="p">)</span><span class="o">.</span>\
<span class="n">options</span><span class="p">(</span><span class="n">contains_eager</span><span class="p">(</span><span class="n">User</span><span class="o">.</span><span class="n">addresses</span><span class="p">))</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">query</span><span class="o">.</span><span class="n">all</span><span class="p">()</span></pre></div>
</div>
<p>Above, the textual SQL contains the column “id” three times, which would
normally be ambiguous. Using the new feature, we can apply the mapped
columns from the <code class="docutils literal notranslate"><span class="pre">User</span></code> and <code class="docutils literal notranslate"><span class="pre">Address</span></code> class directly, even linking
the <code class="docutils literal notranslate"><span class="pre">Address.user_id</span></code> column to the <code class="docutils literal notranslate"><span class="pre">users.id</span></code> column in textual SQL
for fun, and the <a class="reference internal" href="../orm/query.html#sqlalchemy.orm.query.Query" title="sqlalchemy.orm.query.Query"><code class="xref py py-class docutils literal notranslate"><span class="pre">Query</span></code></a> object will receive rows that are correctly
targetable as needed, including for an eager load.</p>
<p>This change is <strong>backwards incompatible</strong> with code that passes the columns
to the method with a different ordering than is present in the textual statement.
It is hoped that this impact will be low due to the fact that this
method has always been documented illustrating the columns being passed in the same order as that of the
textual SQL statement, as would seem intuitive, even though the internals
weren’t checking for this. The method itself was only added as of 0.9 in
any case and may not yet have widespread use. Notes on exactly how to handle
this behavioral change for applications using it are at <a class="reference internal" href="#behavior-change-3501"><span class="std std-ref">TextClause.columns() will match columns positionally, not by name, when passed positionally</span></a>.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../core/tutorial.html#sqlexpression-text-columns"><span class="std std-ref">Specifying Result-Column Behaviors</span></a> - in the Core tutorial</p>
<p><a class="reference internal" href="#behavior-change-3501"><span class="std std-ref">TextClause.columns() will match columns positionally, not by name, when passed positionally</span></a> - backwards compatibility remarks</p>
</div>
</div>
<div class="section" id="positional-matching-is-trusted-over-name-based-matching-for-core-orm-sql-constructs">
<h4>Positional matching is trusted over name-based matching for Core/ORM SQL constructs<a class="headerlink" href="#positional-matching-is-trusted-over-name-based-matching-for-core-orm-sql-constructs" title="Permalink to this headline">¶</a></h4>
<p>Another aspect of this change is that the rules for matching columns have also been modified
to rely upon “positional” matching more fully for compiled SQL constructs
as well. Given a statement like the following:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ua</span> <span class="o">=</span> <span class="n">users</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s1">'ua'</span><span class="p">)</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">users</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">user_id</span><span class="p">,</span> <span class="n">ua</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">user_id</span><span class="p">])</span></pre></div>
</div>
<p>The above statement will compile to:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SELECT</span> <span class="n">users</span><span class="o">.</span><span class="n">user_id</span><span class="p">,</span> <span class="n">ua</span><span class="o">.</span><span class="n">user_id</span> <span class="n">FROM</span> <span class="n">users</span><span class="p">,</span> <span class="n">users</span> <span class="n">AS</span> <span class="n">ua</span></pre></div>
</div>
<p>In 1.0, the above statement when executed would be matched to its original
compiled construct using positional matching, however because the statement
contains the <code class="docutils literal notranslate"><span class="pre">'user_id'</span></code> label duplicated, the “ambiguous column” rule
would still get involved and prevent the columns from being fetched from a row.
As of 1.1, the “ambiguous column” rule does not affect an exact match from
a column construct to the SQL column, which is what the ORM uses to
fetch columns:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">result</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">stmt</span><span class="p">)</span>
<span class="n">row</span> <span class="o">=</span> <span class="n">result</span><span class="o">.</span><span class="n">first</span><span class="p">()</span>
<span class="c1"># these both match positionally, so no error</span>
<span class="n">user_id</span> <span class="o">=</span> <span class="n">row</span><span class="p">[</span><span class="n">users</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">user_id</span><span class="p">]</span>
<span class="n">ua_id</span> <span class="o">=</span> <span class="n">row</span><span class="p">[</span><span class="n">ua</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">user_id</span><span class="p">]</span>
<span class="c1"># this still raises, however</span>
<span class="n">user_id</span> <span class="o">=</span> <span class="n">row</span><span class="p">[</span><span class="s1">'user_id'</span><span class="p">]</span></pre></div>
</div>
</div>
<div class="section" id="much-less-likely-to-get-an-ambiguous-column-error-message">
<h4>Much less likely to get an “ambiguous column” error message<a class="headerlink" href="#much-less-likely-to-get-an-ambiguous-column-error-message" title="Permalink to this headline">¶</a></h4>
<p>As part of this change, the wording of the error message <code class="docutils literal notranslate"><span class="pre">Ambiguous</span> <span class="pre">column</span>
<span class="pre">name</span> <span class="pre">'<name>'</span> <span class="pre">in</span> <span class="pre">result</span> <span class="pre">set!</span> <span class="pre">try</span> <span class="pre">'use_labels'</span> <span class="pre">option</span> <span class="pre">on</span> <span class="pre">select</span> <span class="pre">statement.</span></code>
has been dialed back; as this message should now be extremely rare when using
the ORM or Core compiled SQL constructs, it merely states
<code class="docutils literal notranslate"><span class="pre">Ambiguous</span> <span class="pre">column</span> <span class="pre">name</span> <span class="pre">'<name>'</span> <span class="pre">in</span> <span class="pre">result</span> <span class="pre">set</span> <span class="pre">column</span> <span class="pre">descriptions</span></code>, and
only when a result column is retrieved using the string name that is actually
ambiguous, e.g. <code class="docutils literal notranslate"><span class="pre">row['user_id']</span></code> in the above example. It also now refers
to the actual ambiguous name from the rendered SQL statement itself,
rather than indicating the key or name that was local to the construct being
used for the fetch.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3501">#3501</a></p>
</div>
</div>
<div class="section" id="support-for-python-s-native-enum-type-and-compatible-forms">
<span id="change-3292"></span><h3>Support for Python’s native <code class="docutils literal notranslate"><span class="pre">enum</span></code> type and compatible forms<a class="headerlink" href="#support-for-python-s-native-enum-type-and-compatible-forms" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Enum" title="sqlalchemy.types.Enum"><code class="xref py py-class docutils literal notranslate"><span class="pre">Enum</span></code></a> type can now be constructed using any
PEP-435 compliant enumerated type. When using this mode, input values
and return values are the actual enumerated objects, not the
string/integer/etc values:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">enum</span>
<span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">Table</span><span class="p">,</span> <span class="n">MetaData</span><span class="p">,</span> <span class="n">Column</span><span class="p">,</span> <span class="n">Enum</span><span class="p">,</span> <span class="n">create_engine</span>
<span class="k">class</span> <span class="nc">MyEnum</span><span class="p">(</span><span class="n">enum</span><span class="o">.</span><span class="n">Enum</span><span class="p">):</span>
<span class="n">one</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">two</span> <span class="o">=</span> <span class="mi">2</span>
<span class="n">three</span> <span class="o">=</span> <span class="mi">3</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span>
<span class="s1">'data'</span><span class="p">,</span> <span class="n">MetaData</span><span class="p">(),</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'value'</span><span class="p">,</span> <span class="n">Enum</span><span class="p">(</span><span class="n">MyEnum</span><span class="p">))</span>
<span class="p">)</span>
<span class="n">e</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s2">"sqlite://"</span><span class="p">)</span>
<span class="n">t</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
<span class="n">e</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">t</span><span class="o">.</span><span class="n">insert</span><span class="p">(),</span> <span class="p">{</span><span class="s2">"value"</span><span class="p">:</span> <span class="n">MyEnum</span><span class="o">.</span><span class="n">two</span><span class="p">})</span>
<span class="k">assert</span> <span class="n">e</span><span class="o">.</span><span class="n">scalar</span><span class="p">(</span><span class="n">t</span><span class="o">.</span><span class="n">select</span><span class="p">())</span> <span class="ow">is</span> <span class="n">MyEnum</span><span class="o">.</span><span class="n">two</span></pre></div>
</div>
<div class="section" id="the-enum-enums-collection-is-now-a-list-instead-of-a-tuple">
<h4>The <code class="docutils literal notranslate"><span class="pre">Enum.enums</span></code> collection is now a list instead of a tuple<a class="headerlink" href="#the-enum-enums-collection-is-now-a-list-instead-of-a-tuple" title="Permalink to this headline">¶</a></h4>
<p>As part of the changes to <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Enum" title="sqlalchemy.types.Enum"><code class="xref py py-class docutils literal notranslate"><span class="pre">Enum</span></code></a>, the <code class="xref py py-attr docutils literal notranslate"><span class="pre">Enum.enums</span></code> collection
of elements is now a list instead of a tuple. This because lists
are appropriate for variable length sequences of homogeneous items where
the position of the element is not semantically significant.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3292">#3292</a></p>
</div>
</div>
<div class="section" id="negative-integer-indexes-accommodated-by-core-result-rows">
<span id="change-gh-231"></span><h3>Negative integer indexes accommodated by Core result rows<a class="headerlink" href="#negative-integer-indexes-accommodated-by-core-result-rows" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../core/connections.html#sqlalchemy.engine.RowProxy" title="sqlalchemy.engine.RowProxy"><code class="xref py py-class docutils literal notranslate"><span class="pre">RowProxy</span></code></a> object now accommodates single negative integer indexes
like a regular Python sequence, both in the pure Python and C-extension
version. Previously, negative values would only work in slices:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">create_engine</span>
<span class="gp">>>> </span><span class="n">e</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s2">"sqlite://"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">row</span> <span class="o">=</span> <span class="n">e</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"select 1, 2, 3"</span><span class="p">)</span><span class="o">.</span><span class="n">first</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">row</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">row</span><span class="p">[</span><span class="o">-</span><span class="mi">2</span><span class="p">],</span> <span class="n">row</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">row</span><span class="p">[</span><span class="o">-</span><span class="mi">2</span><span class="p">:</span><span class="mi">2</span><span class="p">]</span>
<span class="go">3 2 2 (2,)</span></pre></div>
</div>
</div>
<div class="section" id="the-enum-type-now-does-in-python-validation-of-values">
<span id="change-3095"></span><h3>The <code class="docutils literal notranslate"><span class="pre">Enum</span></code> type now does in-Python validation of values<a class="headerlink" href="#the-enum-type-now-does-in-python-validation-of-values" title="Permalink to this headline">¶</a></h3>
<p>To accommodate for Python native enumerated objects, as well as for edge
cases such as that of where a non-native ENUM type is used within an ARRAY
and a CHECK constraint is infeasible, the <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Enum" title="sqlalchemy.types.Enum"><code class="xref py py-class docutils literal notranslate"><span class="pre">Enum</span></code></a> datatype now adds
in-Python validation of input values when the <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Enum.params.validate_strings" title="sqlalchemy.types.Enum"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">Enum.validate_strings</span></code></a>
flag is used (1.1.0b2):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">Table</span><span class="p">,</span> <span class="n">MetaData</span><span class="p">,</span> <span class="n">Column</span><span class="p">,</span> <span class="n">Enum</span><span class="p">,</span> <span class="n">create_engine</span>
<span class="gp">>>> </span><span class="n">t</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span>
<span class="gp">... </span> <span class="s1">'data'</span><span class="p">,</span> <span class="n">MetaData</span><span class="p">(),</span>
<span class="gp">... </span> <span class="n">Column</span><span class="p">(</span><span class="s1">'value'</span><span class="p">,</span> <span class="n">Enum</span><span class="p">(</span><span class="s2">"one"</span><span class="p">,</span> <span class="s2">"two"</span><span class="p">,</span> <span class="s2">"three"</span><span class="p">,</span> <span class="n">validate_strings</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
<span class="gp">... </span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">e</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s2">"sqlite://"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">t</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">e</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">t</span><span class="o">.</span><span class="n">insert</span><span class="p">(),</span> <span class="p">{</span><span class="s2">"value"</span><span class="p">:</span> <span class="s2">"four"</span><span class="p">})</span>
<span class="gt">Traceback (most recent call last):</span>
<span class="c">...</span>
<span class="gr">sqlalchemy.exc.StatementError</span>: <span class="n">(exceptions.LookupError)</span>
<span class="go">"four" is not among the defined enum values</span>
<span class="go">[SQL: u'INSERT INTO data (value) VALUES (?)']</span>
<span class="go">[parameters: [{'value': 'four'}]]</span></pre></div>
</div>
<p>This validation is turned off by default as there are already use cases
identified where users don’t want such validation (such as string comparisons).
For non-string types, it necessarily takes place in all cases. The
check also occurs unconditionally on the result-handling side as well, when
values coming from the database are returned.</p>
<p>This validation is in addition to the existing behavior of creating a
CHECK constraint when a non-native enumerated type is used. The creation of
this CHECK constraint can now be disabled using the new
<a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Enum.params.create_constraint" title="sqlalchemy.types.Enum"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">Enum.create_constraint</span></code></a> flag.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3095">#3095</a></p>
</div>
<div class="section" id="non-native-boolean-integer-values-coerced-to-zero-one-none-in-all-cases">
<span id="change-3730"></span><h3>Non-native boolean integer values coerced to zero/one/None in all cases<a class="headerlink" href="#non-native-boolean-integer-values-coerced-to-zero-one-none-in-all-cases" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Boolean" title="sqlalchemy.types.Boolean"><code class="xref py py-class docutils literal notranslate"><span class="pre">Boolean</span></code></a> datatype coerces Python booleans to integer values
for backends that don’t have a native boolean type, such as SQLite and
MySQL. On these backends, a CHECK constraint is normally set up which
ensures the values in the database are in fact one of these two values.
However, MySQL ignores CHECK constraints, the constraint is optional, and
an existing database might not have this constraint. The <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Boolean" title="sqlalchemy.types.Boolean"><code class="xref py py-class docutils literal notranslate"><span class="pre">Boolean</span></code></a>
datatype has been repaired such that an incoming Python-side value that is
already an integer value is coerced to zero or one, not just passed as-is;
additionally, the C-extension version of the int-to-boolean processor for
results now uses the same Python boolean interpretation of the value,
rather than asserting an exact one or zero value. This is now consistent
with the pure-Python int-to-boolean processor and is more forgiving of
existing data already within the database. Values of None/NULL are as before
retained as None/NULL.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>this change had an unintended side effect that the interpretation of non-
integer values, such as strings, also changed in behavior such that the
string value <code class="docutils literal notranslate"><span class="pre">"0"</span></code> would be interpreted as “true”, but only on backends
that don’t have a native boolean datatype - on “native boolean” backends
like PostgreSQL, the string value <code class="docutils literal notranslate"><span class="pre">"0"</span></code> is passed directly to the driver
and is interpreted as “false”. This is an inconsistency that did not occur
with the previous implementation. It should be noted that passing strings or
any other value outside of <code class="docutils literal notranslate"><span class="pre">None</span></code>, <code class="docutils literal notranslate"><span class="pre">True</span></code>, <code class="docutils literal notranslate"><span class="pre">False</span></code>, <code class="docutils literal notranslate"><span class="pre">1</span></code>, <code class="docutils literal notranslate"><span class="pre">0</span></code> to
the <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Boolean" title="sqlalchemy.types.Boolean"><code class="xref py py-class docutils literal notranslate"><span class="pre">Boolean</span></code></a> datatype is <strong>not supported</strong> and version 1.2 will
raise an error for this scenario (or possibly just emit a warning, TBD).
See also <a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/4102">#4102</a>.</p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3730">#3730</a></p>
</div>
<div class="section" id="large-parameter-and-row-values-are-now-truncated-in-logging-and-exception-displays">
<span id="change-2837"></span><h3>Large parameter and row values are now truncated in logging and exception displays<a class="headerlink" href="#large-parameter-and-row-values-are-now-truncated-in-logging-and-exception-displays" title="Permalink to this headline">¶</a></h3>
<p>A large value present as a bound parameter for a SQL statement, as well as a
large value present in a result row, will now be truncated during display
within logging, exception reporting, as well as <code class="docutils literal notranslate"><span class="pre">repr()</span></code> of the row itself:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">create_engine</span>
<span class="gp">>>> </span><span class="kn">import</span> <span class="nn">random</span>
<span class="gp">>>> </span><span class="n">e</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s2">"sqlite://"</span><span class="p">,</span> <span class="n">echo</span><span class="o">=</span><span class="s1">'debug'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">some_value</span> <span class="o">=</span> <span class="s1">''</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="nb">chr</span><span class="p">(</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">52</span><span class="p">,</span> <span class="mi">85</span><span class="p">))</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">5000</span><span class="p">))</span>
<span class="gp">>>> </span><span class="n">row</span> <span class="o">=</span> <span class="n">e</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"select ?"</span><span class="p">,</span> <span class="p">[</span><span class="n">some_value</span><span class="p">])</span><span class="o">.</span><span class="n">first</span><span class="p">()</span>
<span class="gp">... </span><span class="p">(</span><span class="n">lines</span> <span class="n">are</span> <span class="n">wrapped</span> <span class="k">for</span> <span class="n">clarity</span><span class="p">)</span> <span class="o">...</span>
<span class="go">2016-02-17 13:23:03,027 INFO sqlalchemy.engine.base.Engine select ?</span>
<span class="go">2016-02-17 13:23:03,027 INFO sqlalchemy.engine.base.Engine</span>
<span class="go">('E6@?>9HPOJB<<BHR:@=TS:5ILU=;JLM<4?B9<S48PTNG9>:=TSTLA;9K;9FPM4M8M@;NM6GU</span>
<span class="go">LUAEBT9QGHNHTHR5EP75@OER4?SKC;D:TFUMD:M>;C6U:JLM6R67GEK<A6@S@C@J7>4=4:P</span>
<span class="go">GJ7HQ6 ... (4702 characters truncated) ... J6IK546AJMB4N6S9L;;9AKI;=RJP</span>
<span class="go">HDSSOTNBUEEC9@Q:RCL:I@5?FO<9K>KJAGAO@E6@A7JI8O:J7B69T6<8;F:S;4BEIJS9HM</span>
<span class="go">K:;5OLPM@JR;R:J6<SOTTT=>Q>7T@I::OTDC:CC<=NGP6C>BC8N',)</span>
<span class="go">2016-02-17 13:23:03,027 DEBUG sqlalchemy.engine.base.Engine Col ('?',)</span>
<span class="go">2016-02-17 13:23:03,027 DEBUG sqlalchemy.engine.base.Engine</span>
<span class="go">Row (u'E6@?>9HPOJB<<BHR:@=TS:5ILU=;JLM<4?B9<S48PTNG9>:=TSTLA;9K;9FPM4M8M@;</span>
<span class="go">NM6GULUAEBT9QGHNHTHR5EP75@OER4?SKC;D:TFUMD:M>;C6U:JLM6R67GEK<A6@S@C@J7</span>
<span class="go">>4=4:PGJ7HQ ... (4703 characters truncated) ... J6IK546AJMB4N6S9L;;9AKI;=</span>
<span class="go">RJPHDSSOTNBUEEC9@Q:RCL:I@5?FO<9K>KJAGAO@E6@A7JI8O:J7B69T6<8;F:S;4BEIJS9HM</span>
<span class="go">K:;5OLPM@JR;R:J6<SOTTT=>Q>7T@I::OTDC:CC<=NGP6C>BC8N',)</span>
<span class="gp">>>> </span><span class="nb">print</span><span class="p">(</span><span class="n">row</span><span class="p">)</span>
<span class="go">(u'E6@?>9HPOJB<<BHR:@=TS:5ILU=;JLM<4?B9<S48PTNG9>:=TSTLA;9K;9FPM4M8M@;NM6</span>
<span class="go">GULUAEBT9QGHNHTHR5EP75@OER4?SKC;D:TFUMD:M>;C6U:JLM6R67GEK<A6@S@C@J7>4</span>
<span class="go">=4:PGJ7HQ ... (4703 characters truncated) ... J6IK546AJMB4N6S9L;;9AKI;</span>
<span class="go">=RJPHDSSOTNBUEEC9@Q:RCL:I@5?FO<9K>KJAGAO@E6@A7JI8O:J7B69T6<8;F:S;4BEIJS9H</span>
<span class="go">MK:;5OLPM@JR;R:J6<SOTTT=>Q>7T@I::OTDC:CC<=NGP6C>BC8N',)</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/2837">#2837</a></p>
</div>
<div class="section" id="json-support-added-to-core">
<span id="change-3619"></span><h3>JSON support added to Core<a class="headerlink" href="#json-support-added-to-core" title="Permalink to this headline">¶</a></h3>
<p>As MySQL now has a JSON datatype in addition to the PostgreSQL JSON datatype,
the core now gains a <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON" title="sqlalchemy.types.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">sqlalchemy.types.JSON</span></code></a> datatype that is the basis
for both of these. Using this type allows access to the “getitem” operator
as well as the “getpath” operator in a way that is agnostic across PostgreSQL
and MySQL.</p>
<p>The new datatype also has a series of improvements to the handling of
NULL values as well as expression handling.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="#change-3547"><span class="std std-ref">MySQL JSON Support</span></a></p>
<p><a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON" title="sqlalchemy.types.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.JSON</span></code></a></p>
<p><a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON" title="sqlalchemy.dialects.postgresql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.JSON</span></code></a></p>
<p><a class="reference internal" href="../dialects/mysql.html#sqlalchemy.dialects.mysql.JSON" title="sqlalchemy.dialects.mysql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">mysql.JSON</span></code></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3619">#3619</a></p>
<div class="section" id="json-null-is-inserted-as-expected-with-orm-operations-omitted-when-not-present">
<span id="change-3514"></span><h4>JSON “null” is inserted as expected with ORM operations, omitted when not present<a class="headerlink" href="#json-null-is-inserted-as-expected-with-orm-operations-omitted-when-not-present" title="Permalink to this headline">¶</a></h4>
<p>The <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON" title="sqlalchemy.types.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.JSON</span></code></a> type and its descendant types <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON" title="sqlalchemy.dialects.postgresql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.JSON</span></code></a>
and <a class="reference internal" href="../dialects/mysql.html#sqlalchemy.dialects.mysql.JSON" title="sqlalchemy.dialects.mysql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">mysql.JSON</span></code></a> have a flag <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON.params.none_as_null" title="sqlalchemy.types.JSON"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">types.JSON.none_as_null</span></code></a> which
when set to True indicates that the Python value <code class="docutils literal notranslate"><span class="pre">None</span></code> should translate
into a SQL NULL rather than a JSON NULL value. This flag defaults to False,
which means that the Python value <code class="docutils literal notranslate"><span class="pre">None</span></code> should result in a JSON NULL value.</p>
<p>This logic would fail, and is now corrected, in the following circumstances:</p>
<p>1. When the column also contained a default or server_default value,
a positive value of <code class="docutils literal notranslate"><span class="pre">None</span></code> on the mapped attribute that expects to persist
JSON “null” would still result in the column-level default being triggered,
replacing the <code class="docutils literal notranslate"><span class="pre">None</span></code> value:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyObject</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="c1"># ...</span>
<span class="n">json_value</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">JSON</span><span class="p">(</span><span class="n">none_as_null</span><span class="o">=</span><span class="kc">False</span><span class="p">),</span> <span class="n">default</span><span class="o">=</span><span class="s2">"some default"</span><span class="p">)</span>
<span class="c1"># would insert "some default" instead of "'null'",</span>
<span class="c1"># now will insert "'null'"</span>
<span class="n">obj</span> <span class="o">=</span> <span class="n">MyObject</span><span class="p">(</span><span class="n">json_value</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
<span class="n">session</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
<span class="n">session</span><span class="o">.</span><span class="n">commit</span><span class="p">()</span></pre></div>
</div>
<p>2. When the column <em>did not</em> contain a default or server_default value, a missing
value on a JSON column configured with none_as_null=False would still render
JSON NULL rather than falling back to not inserting any value, behaving
inconsistently vs. all other datatypes:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyObject</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="c1"># ...</span>
<span class="n">some_other_value</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">String</span><span class="p">(</span><span class="mi">50</span><span class="p">))</span>
<span class="n">json_value</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">JSON</span><span class="p">(</span><span class="n">none_as_null</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
<span class="c1"># would result in NULL for some_other_value,</span>
<span class="c1"># but json "'null'" for json_value. Now results in NULL for both</span>
<span class="c1"># (the json_value is omitted from the INSERT)</span>
<span class="n">obj</span> <span class="o">=</span> <span class="n">MyObject</span><span class="p">()</span>
<span class="n">session</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
<span class="n">session</span><span class="o">.</span><span class="n">commit</span><span class="p">()</span></pre></div>
</div>
<p>This is a behavioral change that is backwards incompatible for an application
that was relying upon this to default a missing value as JSON null. This
essentially establishes that a <strong>missing value is distinguished from a present
value of None</strong>. See <a class="reference internal" href="#behavior-change-3514"><span class="std std-ref">JSON Columns will not insert JSON NULL if no value is supplied and no default is established</span></a> for further detail.</p>
<p>3. When the <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session.bulk_insert_mappings" title="sqlalchemy.orm.session.Session.bulk_insert_mappings"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Session.bulk_insert_mappings()</span></code></a> method were used, <code class="docutils literal notranslate"><span class="pre">None</span></code>
would be ignored in all cases:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># would insert SQL NULL and/or trigger defaults,</span>
<span class="c1"># now inserts "'null'"</span>
<span class="n">session</span><span class="o">.</span><span class="n">bulk_insert_mappings</span><span class="p">(</span>
<span class="n">MyObject</span><span class="p">,</span>
<span class="p">[{</span><span class="s2">"json_value"</span><span class="p">:</span> <span class="kc">None</span><span class="p">}])</span></pre></div>
</div>
<p>The <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON" title="sqlalchemy.types.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.JSON</span></code></a> type now implements the
<a class="reference internal" href="../core/type_api.html#sqlalchemy.types.TypeEngine.should_evaluate_none" title="sqlalchemy.types.TypeEngine.should_evaluate_none"><code class="xref py py-attr docutils literal notranslate"><span class="pre">TypeEngine.should_evaluate_none</span></code></a> flag,
indicating that <code class="docutils literal notranslate"><span class="pre">None</span></code> should not be ignored here; it is configured
automatically based on the value of <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON.params.none_as_null" title="sqlalchemy.types.JSON"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">types.JSON.none_as_null</span></code></a>.
Thanks to <a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3061">#3061</a>, we can differentiate when the value <code class="docutils literal notranslate"><span class="pre">None</span></code> is actively
set by the user versus when it was never set at all.</p>
<p>The feature applies as well to the new base <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON" title="sqlalchemy.types.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.JSON</span></code></a> type
and its descendant types.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3514">#3514</a></p>
</div>
<div class="section" id="new-json-null-constant-added">
<span id="change-3514-jsonnull"></span><h4>New JSON.NULL Constant Added<a class="headerlink" href="#new-json-null-constant-added" title="Permalink to this headline">¶</a></h4>
<p>To ensure that an application can always have full control at the value level
of whether a <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON" title="sqlalchemy.types.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.JSON</span></code></a>, <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON" title="sqlalchemy.dialects.postgresql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.JSON</span></code></a>, <a class="reference internal" href="../dialects/mysql.html#sqlalchemy.dialects.mysql.JSON" title="sqlalchemy.dialects.mysql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">mysql.JSON</span></code></a>,
or <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSONB" title="sqlalchemy.dialects.postgresql.JSONB"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.JSONB</span></code></a> column
should receive a SQL NULL or JSON <code class="docutils literal notranslate"><span class="pre">"null"</span></code> value, the constant
<a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON.NULL" title="sqlalchemy.types.JSON.NULL"><code class="xref py py-attr docutils literal notranslate"><span class="pre">types.JSON.NULL</span></code></a> has been added, which in conjunction with
<a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.null" title="sqlalchemy.sql.expression.null"><code class="xref py py-func docutils literal notranslate"><span class="pre">null()</span></code></a> can be used to determine fully between SQL NULL and
JSON <code class="docutils literal notranslate"><span class="pre">"null"</span></code>, regardless of what <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON.params.none_as_null" title="sqlalchemy.types.JSON"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">types.JSON.none_as_null</span></code></a> is set
to:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">null</span>
<span class="kn">from</span> <span class="nn">sqlalchemy.dialects.postgresql</span> <span class="k">import</span> <span class="n">JSON</span>
<span class="n">obj1</span> <span class="o">=</span> <span class="n">MyObject</span><span class="p">(</span><span class="n">json_value</span><span class="o">=</span><span class="n">null</span><span class="p">())</span> <span class="c1"># will *always* insert SQL NULL</span>
<span class="n">obj2</span> <span class="o">=</span> <span class="n">MyObject</span><span class="p">(</span><span class="n">json_value</span><span class="o">=</span><span class="n">JSON</span><span class="o">.</span><span class="n">NULL</span><span class="p">)</span> <span class="c1"># will *always* insert JSON string "null"</span>
<span class="n">session</span><span class="o">.</span><span class="n">add_all</span><span class="p">([</span><span class="n">obj1</span><span class="p">,</span> <span class="n">obj2</span><span class="p">])</span>
<span class="n">session</span><span class="o">.</span><span class="n">commit</span><span class="p">()</span></pre></div>
</div>
<p>The feature applies as well to the new base <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON" title="sqlalchemy.types.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.JSON</span></code></a> type
and its descendant types.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3514">#3514</a></p>
</div>
</div>
<div class="section" id="array-support-added-to-core-new-any-and-all-operators">
<span id="change-3516"></span><h3>Array support added to Core; new ANY and ALL operators<a class="headerlink" href="#array-support-added-to-core-new-any-and-all-operators" title="Permalink to this headline">¶</a></h3>
<p>Along with the enhancements made to the PostgreSQL <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.ARRAY</span></code></a>
type described in <a class="reference internal" href="#change-3503"><span class="std std-ref">Correct SQL Types are Established from Indexed Access of ARRAY, JSON, HSTORE</span></a>, the base class of <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.ARRAY</span></code></a>
itself has been moved to Core in a new class <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.ARRAY" title="sqlalchemy.types.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.ARRAY</span></code></a>.</p>
<p>Arrays are part of the SQL standard, as are several array-oriented functions
such as <code class="docutils literal notranslate"><span class="pre">array_agg()</span></code> and <code class="docutils literal notranslate"><span class="pre">unnest()</span></code>. In support of these constructs
for not just PostgreSQL but also potentially for other array-capable backends
in the future such as DB2, the majority of array logic for SQL expressions
is now in Core. The <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.ARRAY" title="sqlalchemy.types.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.ARRAY</span></code></a> type still <strong>only works on
PostgreSQL</strong>, however it can be used directly, supporting special array
use cases such as indexed access, as well as support for the ANY and ALL:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">mytable</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span><span class="s2">"mytable"</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span>
<span class="n">Column</span><span class="p">(</span><span class="s2">"data"</span><span class="p">,</span> <span class="n">ARRAY</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">dimensions</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span>
<span class="p">)</span>
<span class="n">expr</span> <span class="o">=</span> <span class="n">mytable</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="mi">5</span><span class="p">][</span><span class="mi">6</span><span class="p">]</span>
<span class="n">expr</span> <span class="o">=</span> <span class="n">mytable</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="mi">5</span><span class="p">]</span><span class="o">.</span><span class="n">any</span><span class="p">(</span><span class="mi">12</span><span class="p">)</span></pre></div>
</div>
<p>In support of ANY and ALL, the <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.ARRAY" title="sqlalchemy.types.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.ARRAY</span></code></a> type retains the same
<a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.ARRAY.Comparator.any" title="sqlalchemy.types.ARRAY.Comparator.any"><code class="xref py py-meth docutils literal notranslate"><span class="pre">types.ARRAY.Comparator.any()</span></code></a> and <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.ARRAY.Comparator.all" title="sqlalchemy.types.ARRAY.Comparator.all"><code class="xref py py-meth docutils literal notranslate"><span class="pre">types.ARRAY.Comparator.all()</span></code></a> methods
from the PostgreSQL type, but also exports these operations to new
standalone operator functions <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.any_" title="sqlalchemy.sql.expression.any_"><code class="xref py py-func docutils literal notranslate"><span class="pre">sql.expression.any_()</span></code></a> and
<a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.all_" title="sqlalchemy.sql.expression.all_"><code class="xref py py-func docutils literal notranslate"><span class="pre">sql.expression.all_()</span></code></a>. These two functions work in more
of the traditional SQL way, allowing a right-side expression form such
as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">any_</span><span class="p">,</span> <span class="n">all_</span>
<span class="n">select</span><span class="p">([</span><span class="n">mytable</span><span class="p">])</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="mi">12</span> <span class="o">==</span> <span class="n">any_</span><span class="p">(</span><span class="n">mytable</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="mi">5</span><span class="p">]))</span></pre></div>
</div>
<p>For the PostgreSQL-specific operators “contains”, “contained_by”, and
“overlaps”, one should continue to use the <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.ARRAY</span></code></a>
type directly, which provides all functionality of the <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.ARRAY" title="sqlalchemy.types.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.ARRAY</span></code></a>
type as well.</p>
<p>The <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.any_" title="sqlalchemy.sql.expression.any_"><code class="xref py py-func docutils literal notranslate"><span class="pre">sql.expression.any_()</span></code></a> and <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.all_" title="sqlalchemy.sql.expression.all_"><code class="xref py py-func docutils literal notranslate"><span class="pre">sql.expression.all_()</span></code></a> operators
are open-ended at the Core level, however their interpretation by backend
databases is limited. On the PostgreSQL backend, the two operators
<strong>only accept array values</strong>. Whereas on the MySQL backend, they
<strong>only accept subquery values</strong>. On MySQL, one can use an expression
such as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">any_</span><span class="p">,</span> <span class="n">all_</span>
<span class="n">subq</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">mytable</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">value</span><span class="p">])</span>
<span class="n">select</span><span class="p">([</span><span class="n">mytable</span><span class="p">])</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="mi">12</span> <span class="o">></span> <span class="n">any_</span><span class="p">(</span><span class="n">subq</span><span class="p">))</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3516">#3516</a></p>
</div>
<div class="section" id="new-function-features-within-group-array-agg-and-set-aggregate-functions">
<span id="change-3132"></span><h3>New Function features, “WITHIN GROUP”, array_agg and set aggregate functions<a class="headerlink" href="#new-function-features-within-group-array-agg-and-set-aggregate-functions" title="Permalink to this headline">¶</a></h3>
<p>With the new <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.ARRAY" title="sqlalchemy.types.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.ARRAY</span></code></a> type we can also implement a pre-typed
function for the <code class="docutils literal notranslate"><span class="pre">array_agg()</span></code> SQL function that returns an array,
which is now available using <a class="reference internal" href="../core/functions.html#sqlalchemy.sql.functions.array_agg" title="sqlalchemy.sql.functions.array_agg"><code class="xref py py-class docutils literal notranslate"><span class="pre">array_agg</span></code></a>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">func</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">func</span><span class="o">.</span><span class="n">array_agg</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">value</span><span class="p">)])</span></pre></div>
</div>
<p>A PostgreSQL element for an aggregate ORDER BY is also added via
<a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.aggregate_order_by" title="sqlalchemy.dialects.postgresql.aggregate_order_by"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.aggregate_order_by</span></code></a>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy.dialects.postgresql</span> <span class="k">import</span> <span class="n">aggregate_order_by</span>
<span class="n">expr</span> <span class="o">=</span> <span class="n">func</span><span class="o">.</span><span class="n">array_agg</span><span class="p">(</span><span class="n">aggregate_order_by</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">a</span><span class="p">,</span> <span class="n">table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">b</span><span class="o">.</span><span class="n">desc</span><span class="p">()))</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">expr</span><span class="p">])</span></pre></div>
</div>
<p>Producing:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SELECT</span> <span class="n">array_agg</span><span class="p">(</span><span class="n">table1</span><span class="o">.</span><span class="n">a</span> <span class="n">ORDER</span> <span class="n">BY</span> <span class="n">table1</span><span class="o">.</span><span class="n">b</span> <span class="n">DESC</span><span class="p">)</span> <span class="n">AS</span> <span class="n">array_agg_1</span> <span class="n">FROM</span> <span class="n">table1</span></pre></div>
</div>
<p>The PG dialect itself also provides an <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.array_agg" title="sqlalchemy.dialects.postgresql.array_agg"><code class="xref py py-func docutils literal notranslate"><span class="pre">postgresql.array_agg()</span></code></a> wrapper to
ensure the <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.ARRAY</span></code></a> type:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy.dialects.postgresql</span> <span class="k">import</span> <span class="n">array_agg</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">array_agg</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">value</span><span class="p">)</span><span class="o">.</span><span class="n">contains</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">)])</span></pre></div>
</div>
<p>Additionally, functions like <code class="docutils literal notranslate"><span class="pre">percentile_cont()</span></code>, <code class="docutils literal notranslate"><span class="pre">percentile_disc()</span></code>,
<code class="docutils literal notranslate"><span class="pre">rank()</span></code>, <code class="docutils literal notranslate"><span class="pre">dense_rank()</span></code> and others that require an ordering via
<code class="docutils literal notranslate"><span class="pre">WITHIN</span> <span class="pre">GROUP</span> <span class="pre">(ORDER</span> <span class="pre">BY</span> <span class="pre"><expr>)</span></code> are now available via the
<a class="reference internal" href="../core/functions.html#sqlalchemy.sql.functions.FunctionElement.within_group" title="sqlalchemy.sql.functions.FunctionElement.within_group"><code class="xref py py-meth docutils literal notranslate"><span class="pre">FunctionElement.within_group()</span></code></a> modifier:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">func</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span>
<span class="n">department</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">id</span><span class="p">,</span>
<span class="n">func</span><span class="o">.</span><span class="n">percentile_cont</span><span class="p">(</span><span class="mf">0.5</span><span class="p">)</span><span class="o">.</span><span class="n">within_group</span><span class="p">(</span>
<span class="n">department</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">salary</span><span class="o">.</span><span class="n">desc</span><span class="p">()</span>
<span class="p">)</span>
<span class="p">])</span></pre></div>
</div>
<p>The above statement would produce SQL similar to:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SELECT</span> <span class="n">department</span><span class="o">.</span><span class="n">id</span><span class="p">,</span> <span class="n">percentile_cont</span><span class="p">(</span><span class="mf">0.5</span><span class="p">)</span>
<span class="n">WITHIN</span> <span class="n">GROUP</span> <span class="p">(</span><span class="n">ORDER</span> <span class="n">BY</span> <span class="n">department</span><span class="o">.</span><span class="n">salary</span> <span class="n">DESC</span><span class="p">)</span></pre></div>
</div>
<p>Placeholders with correct return types are now provided for these functions,
and include <a class="reference internal" href="../core/functions.html#sqlalchemy.sql.functions.percentile_cont" title="sqlalchemy.sql.functions.percentile_cont"><code class="xref py py-class docutils literal notranslate"><span class="pre">percentile_cont</span></code></a>, <a class="reference internal" href="../core/functions.html#sqlalchemy.sql.functions.percentile_disc" title="sqlalchemy.sql.functions.percentile_disc"><code class="xref py py-class docutils literal notranslate"><span class="pre">percentile_disc</span></code></a>,
<a class="reference internal" href="../core/functions.html#sqlalchemy.sql.functions.rank" title="sqlalchemy.sql.functions.rank"><code class="xref py py-class docutils literal notranslate"><span class="pre">rank</span></code></a>, <a class="reference internal" href="../core/functions.html#sqlalchemy.sql.functions.dense_rank" title="sqlalchemy.sql.functions.dense_rank"><code class="xref py py-class docutils literal notranslate"><span class="pre">dense_rank</span></code></a>, <a class="reference internal" href="../core/functions.html#sqlalchemy.sql.functions.mode" title="sqlalchemy.sql.functions.mode"><code class="xref py py-class docutils literal notranslate"><span class="pre">mode</span></code></a>, <a class="reference internal" href="../core/functions.html#sqlalchemy.sql.functions.percent_rank" title="sqlalchemy.sql.functions.percent_rank"><code class="xref py py-class docutils literal notranslate"><span class="pre">percent_rank</span></code></a>,
and <a class="reference internal" href="../core/functions.html#sqlalchemy.sql.functions.cume_dist" title="sqlalchemy.sql.functions.cume_dist"><code class="xref py py-class docutils literal notranslate"><span class="pre">cume_dist</span></code></a>.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3132">#3132</a> <a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/1370">#1370</a></p>
</div>
<div class="section" id="typedecorator-now-works-with-enum-boolean-schema-types-automatically">
<span id="change-2919"></span><h3>TypeDecorator now works with Enum, Boolean, “schema” types automatically<a class="headerlink" href="#typedecorator-now-works-with-enum-boolean-schema-types-automatically" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.SchemaType" title="sqlalchemy.types.SchemaType"><code class="xref py py-class docutils literal notranslate"><span class="pre">SchemaType</span></code></a> types include types such as <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Enum" title="sqlalchemy.types.Enum"><code class="xref py py-class docutils literal notranslate"><span class="pre">Enum</span></code></a>
and <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Boolean" title="sqlalchemy.types.Boolean"><code class="xref py py-class docutils literal notranslate"><span class="pre">Boolean</span></code></a> which, in addition to corresponding to a database
type, also generate either a CHECK constraint or in the case of PostgreSQL
ENUM a new CREATE TYPE statement, will now work automatically with
<a class="reference internal" href="../core/custom_types.html#sqlalchemy.types.TypeDecorator" title="sqlalchemy.types.TypeDecorator"><code class="xref py py-class docutils literal notranslate"><span class="pre">TypeDecorator</span></code></a> recipes. Previously, a <a class="reference internal" href="../core/custom_types.html#sqlalchemy.types.TypeDecorator" title="sqlalchemy.types.TypeDecorator"><code class="xref py py-class docutils literal notranslate"><span class="pre">TypeDecorator</span></code></a> for
an <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ENUM" title="sqlalchemy.dialects.postgresql.ENUM"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.ENUM</span></code></a> had to look like this:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># old way</span>
<span class="k">class</span> <span class="nc">MyEnum</span><span class="p">(</span><span class="n">TypeDecorator</span><span class="p">,</span> <span class="n">SchemaType</span><span class="p">):</span>
<span class="n">impl</span> <span class="o">=</span> <span class="n">postgresql</span><span class="o">.</span><span class="n">ENUM</span><span class="p">(</span><span class="s1">'one'</span><span class="p">,</span> <span class="s1">'two'</span><span class="p">,</span> <span class="s1">'three'</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s1">'myenum'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">_set_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">impl</span><span class="o">.</span><span class="n">_set_table</span><span class="p">(</span><span class="n">table</span><span class="p">)</span></pre></div>
</div>
<p>The <a class="reference internal" href="../core/custom_types.html#sqlalchemy.types.TypeDecorator" title="sqlalchemy.types.TypeDecorator"><code class="xref py py-class docutils literal notranslate"><span class="pre">TypeDecorator</span></code></a> now propagates those additional events so it
can be done like any other type:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># new way</span>
<span class="k">class</span> <span class="nc">MyEnum</span><span class="p">(</span><span class="n">TypeDecorator</span><span class="p">):</span>
<span class="n">impl</span> <span class="o">=</span> <span class="n">postgresql</span><span class="o">.</span><span class="n">ENUM</span><span class="p">(</span><span class="s1">'one'</span><span class="p">,</span> <span class="s1">'two'</span><span class="p">,</span> <span class="s1">'three'</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s1">'myenum'</span><span class="p">)</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/2919">#2919</a></p>
</div>
<div class="section" id="multi-tenancy-schema-translation-for-table-objects">
<span id="change-2685"></span><h3>Multi-Tenancy Schema Translation for Table objects<a class="headerlink" href="#multi-tenancy-schema-translation-for-table-objects" title="Permalink to this headline">¶</a></h3>
<p>To support the use case of an application that uses the same set of
<a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Table" title="sqlalchemy.schema.Table"><code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code></a> objects in many schemas, such as schema-per-user, a new
execution option <a class="reference internal" href="../core/connections.html#sqlalchemy.engine.Connection.execution_options.params.schema_translate_map" title="sqlalchemy.engine.Connection.execution_options"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">Connection.execution_options.schema_translate_map</span></code></a>
is added. Using this mapping, a set of <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Table" title="sqlalchemy.schema.Table"><code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code></a>
objects can be made on a per-connection basis to refer to any set of schemas
instead of the <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Table.params.schema" title="sqlalchemy.schema.Table"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">Table.schema</span></code></a> to which they were assigned. The
translation works for DDL and SQL generation, as well as with the ORM.</p>
<p>For example, if the <code class="docutils literal notranslate"><span class="pre">User</span></code> class were assigned the schema “per_user”:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">User</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'user'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">__table_args__</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'schema'</span><span class="p">:</span> <span class="s1">'per_user'</span><span class="p">}</span></pre></div>
</div>
<p>On each request, the <a class="reference internal" href="../orm/session_api.html#sqlalchemy.orm.session.Session" title="sqlalchemy.orm.session.Session"><code class="xref py py-class docutils literal notranslate"><span class="pre">Session</span></code></a> can be set up to refer to a
different schema each time:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">session</span> <span class="o">=</span> <span class="n">Session</span><span class="p">()</span>
<span class="n">session</span><span class="o">.</span><span class="n">connection</span><span class="p">(</span><span class="n">execution_options</span><span class="o">=</span><span class="p">{</span>
<span class="s2">"schema_translate_map"</span><span class="p">:</span> <span class="p">{</span><span class="s2">"per_user"</span><span class="p">:</span> <span class="s2">"account_one"</span><span class="p">}})</span>
<span class="c1"># will query from the ``account_one.user`` table</span>
<span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">User</span><span class="p">)</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span></pre></div>
</div>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../core/connections.html#schema-translating"><span class="std std-ref">Translation of Schema Names</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/2685">#2685</a></p>
</div>
<div class="section" id="friendly-stringification-of-core-sql-constructs-without-a-dialect">
<span id="change-3631"></span><h3>“Friendly” stringification of Core SQL constructs without a dialect<a class="headerlink" href="#friendly-stringification-of-core-sql-constructs-without-a-dialect" title="Permalink to this headline">¶</a></h3>
<p>Calling <code class="docutils literal notranslate"><span class="pre">str()</span></code> on a Core SQL construct will now produce a string
in more cases than before, supporting various SQL constructs not normally
present in default SQL such as RETURNING, array indexes, and non-standard
datatypes:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">table</span><span class="p">,</span> <span class="n">column</span>
<span class="go">t>>> t = table('x', column('a'), column('b'))</span>
<span class="gp">>>> </span><span class="nb">print</span><span class="p">(</span><span class="n">t</span><span class="o">.</span><span class="n">insert</span><span class="p">()</span><span class="o">.</span><span class="n">returning</span><span class="p">(</span><span class="n">t</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">a</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">b</span><span class="p">))</span>
<span class="go">INSERT INTO x (a, b) VALUES (:a, :b) RETURNING x.a, x.b</span></pre></div>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">str()</span></code> function now calls upon an entirely separate dialect / compiler
intended just for plain string printing without a specific dialect set up,
so as more “just show me a string!” cases come up, these can be added
to this dialect/compiler without impacting behaviors on real dialects.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="#change-3081"><span class="std std-ref">Stringify of Query will consult the Session for the correct dialect</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3631">#3631</a></p>
</div>
<div class="section" id="the-type-coerce-function-is-now-a-persistent-sql-element">
<span id="change-3531"></span><h3>The type_coerce function is now a persistent SQL element<a class="headerlink" href="#the-type-coerce-function-is-now-a-persistent-sql-element" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.type_coerce" title="sqlalchemy.sql.expression.type_coerce"><code class="xref py py-func docutils literal notranslate"><span class="pre">expression.type_coerce()</span></code></a> function previously would return
an object either of type <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.BindParameter" title="sqlalchemy.sql.expression.BindParameter"><code class="xref py py-class docutils literal notranslate"><span class="pre">BindParameter</span></code></a> or <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.Label" title="sqlalchemy.sql.expression.Label"><code class="xref py py-class docutils literal notranslate"><span class="pre">Label</span></code></a>, depending
on the input. An effect this would have was that in the case where expression
transformations were used, such as the conversion of an element from a
<a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Column" title="sqlalchemy.schema.Column"><code class="xref py py-class docutils literal notranslate"><span class="pre">Column</span></code></a> to a <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.BindParameter" title="sqlalchemy.sql.expression.BindParameter"><code class="xref py py-class docutils literal notranslate"><span class="pre">BindParameter</span></code></a> that’s critical to ORM-level
lazy loading, the type coercion information would not be used since it would
have been lost already.</p>
<p>To improve this behavior, the function now returns a persistent
<a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.TypeCoerce" title="sqlalchemy.sql.expression.TypeCoerce"><code class="xref py py-class docutils literal notranslate"><span class="pre">TypeCoerce</span></code></a> container around the given expression, which itself
remains unaffected; this construct is evaluated explicitly by the
SQL compiler. This allows for the coercion of the inner expression
to be maintained no matter how the statement is modified, including if
the contained element is replaced with a different one, as is common
within the ORM’s lazy loading feature.</p>
<p>The test case illustrating the effect makes use of a heterogeneous
primaryjoin condition in conjunction with custom types and lazy loading.
Given a custom type that applies a CAST as a “bind expression”:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">StringAsInt</span><span class="p">(</span><span class="n">TypeDecorator</span><span class="p">):</span>
<span class="n">impl</span> <span class="o">=</span> <span class="n">String</span>
<span class="k">def</span> <span class="nf">column_expression</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">col</span><span class="p">):</span>
<span class="k">return</span> <span class="n">cast</span><span class="p">(</span><span class="n">col</span><span class="p">,</span> <span class="n">Integer</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">bind_expression</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="k">return</span> <span class="n">cast</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="n">String</span><span class="p">)</span></pre></div>
</div>
<p>Then, a mapping where we are equating a string “id” column on one
table to an integer “id” column on the other:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Person</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'person'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">StringAsInt</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">pets</span> <span class="o">=</span> <span class="n">relationship</span><span class="p">(</span>
<span class="s1">'Pets'</span><span class="p">,</span>
<span class="n">primaryjoin</span><span class="o">=</span><span class="p">(</span>
<span class="s1">'foreign(Pets.person_id)'</span>
<span class="s1">'==cast(type_coerce(Person.id, Integer), Integer)'</span>
<span class="p">)</span>
<span class="p">)</span>
<span class="k">class</span> <span class="nc">Pets</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'pets'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="s1">'id'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">person_id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="s1">'person_id'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">)</span></pre></div>
</div>
<p>Above, in the <a class="reference internal" href="../orm/relationship_api.html#sqlalchemy.orm.relationship.params.primaryjoin" title="sqlalchemy.orm.relationship"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">relationship.primaryjoin</span></code></a> expression, we are
using <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.type_coerce" title="sqlalchemy.sql.expression.type_coerce"><code class="xref py py-func docutils literal notranslate"><span class="pre">type_coerce()</span></code></a> to handle bound parameters passed via
lazyloading as integers, since we already know these will come from
our <code class="docutils literal notranslate"><span class="pre">StringAsInt</span></code> type which maintains the value as an integer in
Python. We are then using <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.cast" title="sqlalchemy.sql.expression.cast"><code class="xref py py-func docutils literal notranslate"><span class="pre">cast()</span></code></a> so that as a SQL expression,
the VARCHAR “id” column will be CAST to an integer for a regular non-
converted join as with <a class="reference internal" href="../orm/query.html#sqlalchemy.orm.query.Query.join" title="sqlalchemy.orm.query.Query.join"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Query.join()</span></code></a> or <a class="reference internal" href="../orm/loading_relationships.html#sqlalchemy.orm.joinedload" title="sqlalchemy.orm.joinedload"><code class="xref py py-func docutils literal notranslate"><span class="pre">orm.joinedload()</span></code></a>.
That is, a joinedload of <code class="docutils literal notranslate"><span class="pre">.pets</span></code> looks like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SELECT</span> <span class="n">person</span><span class="o">.</span><span class="n">id</span> <span class="n">AS</span> <span class="n">person_id</span><span class="p">,</span> <span class="n">pets_1</span><span class="o">.</span><span class="n">id</span> <span class="n">AS</span> <span class="n">pets_1_id</span><span class="p">,</span>
<span class="n">pets_1</span><span class="o">.</span><span class="n">person_id</span> <span class="n">AS</span> <span class="n">pets_1_person_id</span>
<span class="n">FROM</span> <span class="n">person</span>
<span class="n">LEFT</span> <span class="n">OUTER</span> <span class="n">JOIN</span> <span class="n">pets</span> <span class="n">AS</span> <span class="n">pets_1</span>
<span class="n">ON</span> <span class="n">pets_1</span><span class="o">.</span><span class="n">person_id</span> <span class="o">=</span> <span class="n">CAST</span><span class="p">(</span><span class="n">person</span><span class="o">.</span><span class="n">id</span> <span class="n">AS</span> <span class="n">INTEGER</span><span class="p">)</span></pre></div>
</div>
<p>Without the CAST in the ON clause of the join, strongly-typed databases
such as PostgreSQL will refuse to implicitly compare the integer and fail.</p>
<p>The lazyload case of <code class="docutils literal notranslate"><span class="pre">.pets</span></code> relies upon replacing
the <code class="docutils literal notranslate"><span class="pre">Person.id</span></code> column at load time with a bound parameter, which receives
a Python-loaded value. This replacement is specifically where the intent
of our <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.type_coerce" title="sqlalchemy.sql.expression.type_coerce"><code class="xref py py-func docutils literal notranslate"><span class="pre">type_coerce()</span></code></a> function would be lost. Prior to the change,
this lazy load comes out as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SELECT</span> <span class="n">pets</span><span class="o">.</span><span class="n">id</span> <span class="n">AS</span> <span class="n">pets_id</span><span class="p">,</span> <span class="n">pets</span><span class="o">.</span><span class="n">person_id</span> <span class="n">AS</span> <span class="n">pets_person_id</span>
<span class="n">FROM</span> <span class="n">pets</span>
<span class="n">WHERE</span> <span class="n">pets</span><span class="o">.</span><span class="n">person_id</span> <span class="o">=</span> <span class="n">CAST</span><span class="p">(</span><span class="n">CAST</span><span class="p">(</span><span class="o">%</span><span class="p">(</span><span class="n">param_1</span><span class="p">)</span><span class="n">s</span> <span class="n">AS</span> <span class="n">VARCHAR</span><span class="p">)</span> <span class="n">AS</span> <span class="n">INTEGER</span><span class="p">)</span>
<span class="p">{</span><span class="s1">'param_1'</span><span class="p">:</span> <span class="mi">5</span><span class="p">}</span></pre></div>
</div>
<p>Where above, we see that our in-Python value of <code class="docutils literal notranslate"><span class="pre">5</span></code> is CAST first
to a VARCHAR, then back to an INTEGER in SQL; a double CAST which works,
but is nevertheless not what we asked for.</p>
<p>With the change, the <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.type_coerce" title="sqlalchemy.sql.expression.type_coerce"><code class="xref py py-func docutils literal notranslate"><span class="pre">type_coerce()</span></code></a> function maintains a wrapper
even after the column is swapped out for a bound parameter, and the query now
looks like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SELECT</span> <span class="n">pets</span><span class="o">.</span><span class="n">id</span> <span class="n">AS</span> <span class="n">pets_id</span><span class="p">,</span> <span class="n">pets</span><span class="o">.</span><span class="n">person_id</span> <span class="n">AS</span> <span class="n">pets_person_id</span>
<span class="n">FROM</span> <span class="n">pets</span>
<span class="n">WHERE</span> <span class="n">pets</span><span class="o">.</span><span class="n">person_id</span> <span class="o">=</span> <span class="n">CAST</span><span class="p">(</span><span class="o">%</span><span class="p">(</span><span class="n">param_1</span><span class="p">)</span><span class="n">s</span> <span class="n">AS</span> <span class="n">INTEGER</span><span class="p">)</span>
<span class="p">{</span><span class="s1">'param_1'</span><span class="p">:</span> <span class="mi">5</span><span class="p">}</span></pre></div>
</div>
<p>Where our outer CAST that’s in our primaryjoin still takes effect, but the
needless CAST that’s in part of the <code class="docutils literal notranslate"><span class="pre">StringAsInt</span></code> custom type is removed
as intended by the <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.type_coerce" title="sqlalchemy.sql.expression.type_coerce"><code class="xref py py-func docutils literal notranslate"><span class="pre">type_coerce()</span></code></a> function.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3531">#3531</a></p>
</div>
</div>
<div class="section" id="key-behavioral-changes-orm">
<h2>Key Behavioral Changes - ORM<a class="headerlink" href="#key-behavioral-changes-orm" title="Permalink to this headline">¶</a></h2>
<div class="section" id="json-columns-will-not-insert-json-null-if-no-value-is-supplied-and-no-default-is-established">
<span id="behavior-change-3514"></span><h3>JSON Columns will not insert JSON NULL if no value is supplied and no default is established<a class="headerlink" href="#json-columns-will-not-insert-json-null-if-no-value-is-supplied-and-no-default-is-established" title="Permalink to this headline">¶</a></h3>
<p>As detailed in <a class="reference internal" href="#change-3514"><span class="std std-ref">JSON “null” is inserted as expected with ORM operations, omitted when not present</span></a>, <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON" title="sqlalchemy.types.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.JSON</span></code></a> will not render
a JSON “null” value if the value is missing entirely. To prevent SQL NULL,
a default should be set up. Given the following mapping:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyObject</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="c1"># ...</span>
<span class="n">json_value</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">JSON</span><span class="p">(</span><span class="n">none_as_null</span><span class="o">=</span><span class="kc">False</span><span class="p">),</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span></pre></div>
</div>
<p>The following flush operation will fail with an integrity error:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">obj</span> <span class="o">=</span> <span class="n">MyObject</span><span class="p">()</span> <span class="c1"># note no json_value</span>
<span class="n">session</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
<span class="n">session</span><span class="o">.</span><span class="n">commit</span><span class="p">()</span> <span class="c1"># will fail with integrity error</span></pre></div>
</div>
<p>If the default for the column should be JSON NULL, set this on the
Column:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyObject</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="c1"># ...</span>
<span class="n">json_value</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span>
<span class="n">JSON</span><span class="p">(</span><span class="n">none_as_null</span><span class="o">=</span><span class="kc">False</span><span class="p">),</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="n">JSON</span><span class="o">.</span><span class="n">NULL</span><span class="p">)</span></pre></div>
</div>
<p>Or, ensure the value is present on the object:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">obj</span> <span class="o">=</span> <span class="n">MyObject</span><span class="p">(</span><span class="n">json_value</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
<span class="n">session</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
<span class="n">session</span><span class="o">.</span><span class="n">commit</span><span class="p">()</span> <span class="c1"># will insert JSON NULL</span></pre></div>
</div>
<p>Note that setting <code class="docutils literal notranslate"><span class="pre">None</span></code> for the default is the same as omitting it entirely;
the <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON.params.none_as_null" title="sqlalchemy.types.JSON"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">types.JSON.none_as_null</span></code></a> flag does not impact the value of <code class="docutils literal notranslate"><span class="pre">None</span></code>
passed to <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Column.params.default" title="sqlalchemy.schema.Column"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">Column.default</span></code></a> or <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Column.params.server_default" title="sqlalchemy.schema.Column"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">Column.server_default</span></code></a>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># default=None is the same as omitting it entirely, does not apply JSON NULL</span>
<span class="n">json_value</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">JSON</span><span class="p">(</span><span class="n">none_as_null</span><span class="o">=</span><span class="kc">False</span><span class="p">),</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span></pre></div>
</div>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="#change-3514"><span class="std std-ref">JSON “null” is inserted as expected with ORM operations, omitted when not present</span></a></p>
</div>
</div>
<div class="section" id="columns-no-longer-added-redundantly-with-distinct-order-by">
<span id="change-3641"></span><h3>Columns no longer added redundantly with DISTINCT + ORDER BY<a class="headerlink" href="#columns-no-longer-added-redundantly-with-distinct-order-by" title="Permalink to this headline">¶</a></h3>
<p>A query such as the following will now augment only those columns
that are missing from the SELECT list, without duplicates:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">q</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">User</span><span class="o">.</span><span class="n">id</span><span class="p">,</span> <span class="n">User</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">label</span><span class="p">(</span><span class="s1">'name'</span><span class="p">))</span><span class="o">.</span>\
<span class="n">distinct</span><span class="p">()</span><span class="o">.</span>\
<span class="n">order_by</span><span class="p">(</span><span class="n">User</span><span class="o">.</span><span class="n">id</span><span class="p">,</span> <span class="n">User</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">User</span><span class="o">.</span><span class="n">fullname</span><span class="p">)</span></pre></div>
</div>
<p>Produces:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SELECT</span> <span class="n">DISTINCT</span> <span class="n">user</span><span class="o">.</span><span class="n">id</span> <span class="n">AS</span> <span class="n">a_id</span><span class="p">,</span> <span class="n">user</span><span class="o">.</span><span class="n">name</span> <span class="n">AS</span> <span class="n">name</span><span class="p">,</span>
<span class="n">user</span><span class="o">.</span><span class="n">fullname</span> <span class="n">AS</span> <span class="n">a_fullname</span>
<span class="n">FROM</span> <span class="n">a</span> <span class="n">ORDER</span> <span class="n">BY</span> <span class="n">user</span><span class="o">.</span><span class="n">id</span><span class="p">,</span> <span class="n">user</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">user</span><span class="o">.</span><span class="n">fullname</span></pre></div>
</div>
<p>Previously, it would produce:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SELECT</span> <span class="n">DISTINCT</span> <span class="n">user</span><span class="o">.</span><span class="n">id</span> <span class="n">AS</span> <span class="n">a_id</span><span class="p">,</span> <span class="n">user</span><span class="o">.</span><span class="n">name</span> <span class="n">AS</span> <span class="n">name</span><span class="p">,</span> <span class="n">user</span><span class="o">.</span><span class="n">name</span> <span class="n">AS</span> <span class="n">a_name</span><span class="p">,</span>
<span class="n">user</span><span class="o">.</span><span class="n">fullname</span> <span class="n">AS</span> <span class="n">a_fullname</span>
<span class="n">FROM</span> <span class="n">a</span> <span class="n">ORDER</span> <span class="n">BY</span> <span class="n">user</span><span class="o">.</span><span class="n">id</span><span class="p">,</span> <span class="n">user</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">user</span><span class="o">.</span><span class="n">fullname</span></pre></div>
</div>
<p>Where above, the <code class="docutils literal notranslate"><span class="pre">user.name</span></code> column is added unnecessarily. The results
would not be affected, as the additional columns are not included in the
result in any case, but the columns are unnecessary.</p>
<p>Additionally, when the PostgreSQL DISTINCT ON format is used by passing
expressions to <a class="reference internal" href="../orm/query.html#sqlalchemy.orm.query.Query.distinct" title="sqlalchemy.orm.query.Query.distinct"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Query.distinct()</span></code></a>, the above “column adding” logic
is disabled entirely.</p>
<p>When the query is being bundled into a subquery for the purposes of
joined eager loading, the “augment column list” rules are necessarily
more aggressive so that the ORDER BY can still be satisfied, so this case
remains unchanged.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3641">#3641</a></p>
</div>
<div class="section" id="same-named-validates-decorators-will-now-raise-an-exception">
<span id="change-3776"></span><h3>Same-named @validates decorators will now raise an exception<a class="headerlink" href="#same-named-validates-decorators-will-now-raise-an-exception" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../orm/mapped_attributes.html#sqlalchemy.orm.validates" title="sqlalchemy.orm.validates"><code class="xref py py-func docutils literal notranslate"><span class="pre">orm.validates()</span></code></a> decorator is only intended to be created once
per class for a particular attribute name. Creating more than one
now raises an error, whereas previously it would silently pick only the
last defined validator:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">A</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'a'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">String</span><span class="p">)</span>
<span class="nd">@validates</span><span class="p">(</span><span class="s2">"data"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">_validate_data_one</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">assert</span> <span class="s2">"x"</span> <span class="ow">in</span> <span class="n">data</span>
<span class="nd">@validates</span><span class="p">(</span><span class="s2">"data"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">_validate_data_two</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">assert</span> <span class="s2">"y"</span> <span class="ow">in</span> <span class="n">data</span>
<span class="n">configure_mappers</span><span class="p">()</span></pre></div>
</div>
<p>Will raise:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sqlalchemy</span><span class="o">.</span><span class="n">exc</span><span class="o">.</span><span class="n">InvalidRequestError</span><span class="p">:</span> <span class="n">A</span> <span class="n">validation</span> <span class="n">function</span> <span class="k">for</span> <span class="n">mapped</span> <span class="n">attribute</span> <span class="s1">'data'</span> <span class="n">on</span> <span class="n">mapper</span> <span class="n">Mapper</span><span class="o">|</span><span class="n">A</span><span class="o">|</span><span class="n">a</span> <span class="n">already</span> <span class="n">exists</span><span class="o">.</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3776">#3776</a></p>
</div>
</div>
<div class="section" id="key-behavioral-changes-core">
<h2>Key Behavioral Changes - Core<a class="headerlink" href="#key-behavioral-changes-core" title="Permalink to this headline">¶</a></h2>
<div class="section" id="textclause-columns-will-match-columns-positionally-not-by-name-when-passed-positionally">
<span id="behavior-change-3501"></span><h3>TextClause.columns() will match columns positionally, not by name, when passed positionally<a class="headerlink" href="#textclause-columns-will-match-columns-positionally-not-by-name-when-passed-positionally" title="Permalink to this headline">¶</a></h3>
<p>The new behavior of the <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.TextClause.columns" title="sqlalchemy.sql.expression.TextClause.columns"><code class="xref py py-meth docutils literal notranslate"><span class="pre">TextClause.columns()</span></code></a> method, which itself
was recently added as of the 0.9 series, is that when
columns are passed positionally without any additional keyword arguments,
they are linked to the ultimate result set
columns positionally, and no longer on name. It is hoped that the impact
of this change will be low due to the fact that the method has always been documented
illustrating the columns being passed in the same order as that of the
textual SQL statement, as would seem intuitive, even though the internals
weren’t checking for this.</p>
<p>An application that is using this method by passing <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Column" title="sqlalchemy.schema.Column"><code class="xref py py-class docutils literal notranslate"><span class="pre">Column</span></code></a> objects
to it positionally must ensure that the position of those <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Column" title="sqlalchemy.schema.Column"><code class="xref py py-class docutils literal notranslate"><span class="pre">Column</span></code></a>
objects matches the position in which these columns are stated in the
textual SQL.</p>
<p>E.g., code like the following:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">stmt</span> <span class="o">=</span> <span class="n">text</span><span class="p">(</span><span class="s2">"SELECT id, name, description FROM table"</span><span class="p">)</span>
<span class="c1"># no longer matches by name</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">stmt</span><span class="o">.</span><span class="n">columns</span><span class="p">(</span><span class="n">my_table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">my_table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">description</span><span class="p">,</span> <span class="n">my_table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">id</span><span class="p">)</span></pre></div>
</div>
<p>Would no longer work as expected; the order of the columns given is now
significant:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># correct version</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">stmt</span><span class="o">.</span><span class="n">columns</span><span class="p">(</span><span class="n">my_table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">id</span><span class="p">,</span> <span class="n">my_table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">my_table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">description</span><span class="p">)</span></pre></div>
</div>
<p>Possibly more likely, a statement that worked like this:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">stmt</span> <span class="o">=</span> <span class="n">text</span><span class="p">(</span><span class="s2">"SELECT * FROM table"</span><span class="p">)</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">stmt</span><span class="o">.</span><span class="n">columns</span><span class="p">(</span><span class="n">my_table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">id</span><span class="p">,</span> <span class="n">my_table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">my_table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">description</span><span class="p">)</span></pre></div>
</div>
<p>is now slightly risky, as the “*” specification will generally deliver columns
in the order in which they are present in the table itself. If the structure
of the table changes due to schema changes, this ordering may no longer be the same.
Therefore when using <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.TextClause.columns" title="sqlalchemy.sql.expression.TextClause.columns"><code class="xref py py-meth docutils literal notranslate"><span class="pre">TextClause.columns()</span></code></a>, it’s advised to list out
the desired columns explicitly in the textual SQL, though it’s no longer
necessary to worry about the names themselves in the textual SQL.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="#change-3501"><span class="std std-ref">ResultSet column matching enhancements; positional column setup for textual SQL</span></a></p>
</div>
</div>
<div class="section" id="string-server-default-now-literal-quoted">
<span id="change-3809"></span><h3>String server_default now literal quoted<a class="headerlink" href="#string-server-default-now-literal-quoted" title="Permalink to this headline">¶</a></h3>
<p>A server default passed to <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Column.params.server_default" title="sqlalchemy.schema.Column"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">Column.server_default</span></code></a> as a plain
Python string that has quotes embedded is now
passed through the literal quoting system:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy.schema</span> <span class="k">import</span> <span class="n">MetaData</span><span class="p">,</span> <span class="n">Table</span><span class="p">,</span> <span class="n">Column</span><span class="p">,</span> <span class="n">CreateTable</span>
<span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy.types</span> <span class="k">import</span> <span class="n">String</span>
<span class="gp">>>> </span><span class="n">t</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span><span class="s1">'t'</span><span class="p">,</span> <span class="n">MetaData</span><span class="p">(),</span> <span class="n">Column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">String</span><span class="p">(),</span> <span class="n">server_default</span><span class="o">=</span><span class="s2">"hi ' there"</span><span class="p">))</span>
<span class="gp">>>> </span><span class="nb">print</span> <span class="n">CreateTable</span><span class="p">(</span><span class="n">t</span><span class="p">)</span>
<span class="go">CREATE TABLE t (</span>
<span class="go"> x VARCHAR DEFAULT 'hi '' there'</span>
<span class="go">)</span></pre></div>
</div>
<p>Previously the quote would render directly. This change may be backwards
incompatible for applications with such a use case who were working around
the issue.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3809">#3809</a></p>
</div>
<div class="section" id="a-union-or-similar-of-selects-with-limit-offset-order-by-now-parenthesizes-the-embedded-selects">
<span id="change-2528"></span><h3>A UNION or similar of SELECTs with LIMIT/OFFSET/ORDER BY now parenthesizes the embedded selects<a class="headerlink" href="#a-union-or-similar-of-selects-with-limit-offset-order-by-now-parenthesizes-the-embedded-selects" title="Permalink to this headline">¶</a></h3>
<p>An issue that, like others, was long driven by SQLite’s lack of capabilities
has now been enhanced to work on all supporting backends. We refer to a query that
is a UNION of SELECT statements that themselves contain row-limiting or ordering
features which include LIMIT, OFFSET, and/or ORDER BY:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">SELECT</span> <span class="n">x</span> <span class="n">FROM</span> <span class="n">table1</span> <span class="n">ORDER</span> <span class="n">BY</span> <span class="n">y</span> <span class="n">LIMIT</span> <span class="mi">1</span><span class="p">)</span> <span class="n">UNION</span>
<span class="p">(</span><span class="n">SELECT</span> <span class="n">x</span> <span class="n">FROM</span> <span class="n">table2</span> <span class="n">ORDER</span> <span class="n">BY</span> <span class="n">y</span> <span class="n">LIMIT</span> <span class="mi">2</span><span class="p">)</span></pre></div>
</div>
<p>The above query requires parenthesis within each sub-select in order to
group the sub-results correctly. Production of the above statement in
SQLAlchemy Core looks like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">stmt1</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">table1</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">x</span><span class="p">])</span><span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="n">table1</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">y</span><span class="p">)</span><span class="o">.</span><span class="n">limit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="n">stmt2</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">table1</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">x</span><span class="p">])</span><span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="n">table2</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">y</span><span class="p">)</span><span class="o">.</span><span class="n">limit</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">union</span><span class="p">(</span><span class="n">stmt1</span><span class="p">,</span> <span class="n">stmt2</span><span class="p">)</span></pre></div>
</div>
<p>Previously, the above construct would not produce parenthesization for the
inner SELECT statements, producing a query that fails on all backends.</p>
<p>The above formats will <strong>continue to fail on SQLite</strong>; additionally, the format
that includes ORDER BY but no LIMIT/SELECT will <strong>continue to fail on Oracle</strong>.
This is not a backwards-incompatible change, because the queries fail without
the parentheses as well; with the fix, the queries at least work on all other
databases.</p>
<p>In all cases, in order to produce a UNION of limited SELECT statements that
also works on SQLite and in all cases on Oracle, the
subqueries must be a SELECT of an ALIAS:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">stmt1</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">table1</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">x</span><span class="p">])</span><span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="n">table1</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">y</span><span class="p">)</span><span class="o">.</span><span class="n">limit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">()</span><span class="o">.</span><span class="n">select</span><span class="p">()</span>
<span class="n">stmt2</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">table2</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">x</span><span class="p">])</span><span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="n">table2</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">y</span><span class="p">)</span><span class="o">.</span><span class="n">limit</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">()</span><span class="o">.</span><span class="n">select</span><span class="p">()</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">union</span><span class="p">(</span><span class="n">stmt1</span><span class="p">,</span> <span class="n">stmt2</span><span class="p">)</span></pre></div>
</div>
<p>This workaround works on all SQLAlchemy versions. In the ORM, it looks like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">stmt1</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">Model1</span><span class="p">)</span><span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="n">Model1</span><span class="o">.</span><span class="n">y</span><span class="p">)</span><span class="o">.</span><span class="n">limit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">subquery</span><span class="p">()</span><span class="o">.</span><span class="n">select</span><span class="p">()</span>
<span class="n">stmt2</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">Model2</span><span class="p">)</span><span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="n">Model2</span><span class="o">.</span><span class="n">y</span><span class="p">)</span><span class="o">.</span><span class="n">limit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">subquery</span><span class="p">()</span><span class="o">.</span><span class="n">select</span><span class="p">()</span>
<span class="n">stmt</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">query</span><span class="p">(</span><span class="n">Model1</span><span class="p">)</span><span class="o">.</span><span class="n">from_statement</span><span class="p">(</span><span class="n">stmt1</span><span class="o">.</span><span class="n">union</span><span class="p">(</span><span class="n">stmt2</span><span class="p">))</span></pre></div>
</div>
<p>The behavior here has many parallels to the “join rewriting” behavior
introduced in SQLAlchemy 0.9 in <a class="reference internal" href="migration_09.html#feature-joins-09"><span class="std std-ref">Many JOIN and LEFT OUTER JOIN expressions will no longer be wrapped in (SELECT * FROM ..) AS ANON_1</span></a>; however in this case
we have opted not to add new rewriting behavior to accommodate this
case for SQLite.
The existing rewriting behavior is very complicated already, and the case of
UNIONs with parenthesized SELECT statements is much less common than the
“right-nested-join” use case of that feature.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/2528">#2528</a></p>
</div>
</div>
<div class="section" id="dialect-improvements-and-changes-postgresql">
<h2>Dialect Improvements and Changes - PostgreSQL<a class="headerlink" href="#dialect-improvements-and-changes-postgresql" title="Permalink to this headline">¶</a></h2>
<div class="section" id="support-for-insert-on-conflict-do-update-do-nothing">
<span id="change-3529"></span><h3>Support for INSERT..ON CONFLICT (DO UPDATE | DO NOTHING)<a class="headerlink" href="#support-for-insert-on-conflict-do-update-do-nothing" title="Permalink to this headline">¶</a></h3>
<p>The <code class="docutils literal notranslate"><span class="pre">ON</span> <span class="pre">CONFLICT</span></code> clause of <code class="docutils literal notranslate"><span class="pre">INSERT</span></code> added to PostgreSQL as of
version 9.5 is now supported using a PostgreSQL-specific version of the
<a class="reference internal" href="../core/dml.html#sqlalchemy.sql.expression.Insert" title="sqlalchemy.sql.expression.Insert"><code class="xref py py-class docutils literal notranslate"><span class="pre">Insert</span></code></a> object, via <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.dml.insert" title="sqlalchemy.dialects.postgresql.dml.insert"><code class="xref py py-func docutils literal notranslate"><span class="pre">sqlalchemy.dialects.postgresql.dml.insert()</span></code></a>.
This <a class="reference internal" href="../core/dml.html#sqlalchemy.sql.expression.Insert" title="sqlalchemy.sql.expression.Insert"><code class="xref py py-class docutils literal notranslate"><span class="pre">Insert</span></code></a> subclass adds two new methods <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.dml.Insert.on_conflict_do_update" title="sqlalchemy.dialects.postgresql.dml.Insert.on_conflict_do_update"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Insert.on_conflict_do_update()</span></code></a>
and <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.dml.Insert.on_conflict_do_nothing" title="sqlalchemy.dialects.postgresql.dml.Insert.on_conflict_do_nothing"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Insert.on_conflict_do_nothing()</span></code></a> which implement the full syntax
supported by PostgreSQL 9.5 in this area:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy.dialects.postgresql</span> <span class="k">import</span> <span class="n">insert</span>
<span class="n">insert_stmt</span> <span class="o">=</span> <span class="n">insert</span><span class="p">(</span><span class="n">my_table</span><span class="p">)</span><span class="o">.</span> \\
<span class="n">values</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="s1">'some_id'</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="s1">'some data to insert'</span><span class="p">)</span>
<span class="n">do_update_stmt</span> <span class="o">=</span> <span class="n">insert_stmt</span><span class="o">.</span><span class="n">on_conflict_do_update</span><span class="p">(</span>
<span class="n">index_elements</span><span class="o">=</span><span class="p">[</span><span class="n">my_table</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">id</span><span class="p">],</span>
<span class="n">set_</span><span class="o">=</span><span class="nb">dict</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="s1">'some data to update'</span><span class="p">)</span>
<span class="p">)</span>
<span class="n">conn</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">do_update_stmt</span><span class="p">)</span></pre></div>
</div>
<p>The above will render:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">INSERT</span> <span class="n">INTO</span> <span class="n">my_table</span> <span class="p">(</span><span class="nb">id</span><span class="p">,</span> <span class="n">data</span><span class="p">)</span>
<span class="n">VALUES</span> <span class="p">(:</span><span class="nb">id</span><span class="p">,</span> <span class="p">:</span><span class="n">data</span><span class="p">)</span>
<span class="n">ON</span> <span class="n">CONFLICT</span> <span class="nb">id</span> <span class="n">DO</span> <span class="n">UPDATE</span> <span class="n">SET</span> <span class="n">data</span><span class="o">=</span><span class="p">:</span><span class="n">data_2</span></pre></div>
</div>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../dialects/postgresql.html#postgresql-insert-on-conflict"><span class="std std-ref">INSERT…ON CONFLICT (Upsert)</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3529">#3529</a></p>
</div>
<div class="section" id="array-and-json-types-now-correctly-specify-unhashable">
<span id="change-3499-postgresql"></span><h3>ARRAY and JSON types now correctly specify “unhashable”<a class="headerlink" href="#array-and-json-types-now-correctly-specify-unhashable" title="Permalink to this headline">¶</a></h3>
<p>As described in <a class="reference internal" href="#change-3499"><span class="std std-ref">Changes regarding “unhashable” types, impacts deduping of ORM rows</span></a>, the ORM relies upon being able to
produce a hash function for column values when a query’s selected entities
mixes full ORM entities with column expressions. The <code class="docutils literal notranslate"><span class="pre">hashable=False</span></code>
flag is now correctly set on all of PG’s “data structure” types, including
<a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.ARRAY</span></code></a> and <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON" title="sqlalchemy.dialects.postgresql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.JSON</span></code></a>.
The <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSONB" title="sqlalchemy.dialects.postgresql.JSONB"><code class="xref py py-class docutils literal notranslate"><span class="pre">JSONB</span></code></a> and <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.HSTORE" title="sqlalchemy.dialects.postgresql.HSTORE"><code class="xref py py-class docutils literal notranslate"><span class="pre">HSTORE</span></code></a>
types already included this flag. For <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.ARRAY</span></code></a>,
this is conditional based on the <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY.params.as_tuple" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">postgresql.ARRAY.as_tuple</span></code></a>
flag, however it should no longer be necessary to set this flag
in order to have an array value present in a composed ORM row.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="#change-3499"><span class="std std-ref">Changes regarding “unhashable” types, impacts deduping of ORM rows</span></a></p>
<p><a class="reference internal" href="#change-3503"><span class="std std-ref">Correct SQL Types are Established from Indexed Access of ARRAY, JSON, HSTORE</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3499">#3499</a></p>
</div>
<div class="section" id="correct-sql-types-are-established-from-indexed-access-of-array-json-hstore">
<span id="change-3503"></span><h3>Correct SQL Types are Established from Indexed Access of ARRAY, JSON, HSTORE<a class="headerlink" href="#correct-sql-types-are-established-from-indexed-access-of-array-json-hstore" title="Permalink to this headline">¶</a></h3>
<p>For all three of <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">ARRAY</span></code></a>, <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON" title="sqlalchemy.dialects.postgresql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">JSON</span></code></a> and <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.HSTORE" title="sqlalchemy.dialects.postgresql.HSTORE"><code class="xref py py-class docutils literal notranslate"><span class="pre">HSTORE</span></code></a>,
the SQL type assigned to the expression returned by indexed access, e.g.
<code class="docutils literal notranslate"><span class="pre">col[someindex]</span></code>, should be correct in all cases.</p>
<p>This includes:</p>
<ul>
<li><p>The SQL type assigned to indexed access of an <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">ARRAY</span></code></a> takes into
account the number of dimensions configured. An <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">ARRAY</span></code></a> with three
dimensions will return a SQL expression with a type of <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">ARRAY</span></code></a> of
one less dimension. Given a column with type <code class="docutils literal notranslate"><span class="pre">ARRAY(Integer,</span> <span class="pre">dimensions=3)</span></code>,
we can now perform this expression:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">int_expr</span> <span class="o">=</span> <span class="n">col</span><span class="p">[</span><span class="mi">5</span><span class="p">][</span><span class="mi">6</span><span class="p">][</span><span class="mi">7</span><span class="p">]</span> <span class="c1"># returns an Integer expression object</span></pre></div>
</div>
<p>Previously, the indexed access to <code class="docutils literal notranslate"><span class="pre">col[5]</span></code> would return an expression of
type <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Integer" title="sqlalchemy.types.Integer"><code class="xref py py-class docutils literal notranslate"><span class="pre">Integer</span></code></a> where we could no longer perform indexed access
for the remaining dimensions, unless we used <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.cast" title="sqlalchemy.sql.expression.cast"><code class="xref py py-func docutils literal notranslate"><span class="pre">cast()</span></code></a> or <a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.type_coerce" title="sqlalchemy.sql.expression.type_coerce"><code class="xref py py-func docutils literal notranslate"><span class="pre">type_coerce()</span></code></a>.</p>
</li>
<li><p>The <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON" title="sqlalchemy.dialects.postgresql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">JSON</span></code></a> and <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSONB" title="sqlalchemy.dialects.postgresql.JSONB"><code class="xref py py-class docutils literal notranslate"><span class="pre">JSONB</span></code></a> types now mirror what PostgreSQL
itself does for indexed access. This means that all indexed access for
a <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON" title="sqlalchemy.dialects.postgresql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">JSON</span></code></a> or <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSONB" title="sqlalchemy.dialects.postgresql.JSONB"><code class="xref py py-class docutils literal notranslate"><span class="pre">JSONB</span></code></a> type returns an expression that itself
is <em>always</em> <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON" title="sqlalchemy.dialects.postgresql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">JSON</span></code></a> or <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSONB" title="sqlalchemy.dialects.postgresql.JSONB"><code class="xref py py-class docutils literal notranslate"><span class="pre">JSONB</span></code></a> itself, unless the
<a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON.Comparator.astext" title="sqlalchemy.dialects.postgresql.JSON.Comparator.astext"><code class="xref py py-attr docutils literal notranslate"><span class="pre">astext</span></code></a> modifier is used. This means that whether
the indexed access of the JSON structure ultimately refers to a string,
list, number, or other JSON structure, PostgreSQL always considers it
to be JSON itself unless it is explicitly cast differently. Like
the <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.ARRAY" title="sqlalchemy.dialects.postgresql.ARRAY"><code class="xref py py-class docutils literal notranslate"><span class="pre">ARRAY</span></code></a> type, this means that it is now straightforward
to produce JSON expressions with multiple levels of indexed access:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">json_expr</span> <span class="o">=</span> <span class="n">json_col</span><span class="p">[</span><span class="s1">'key1'</span><span class="p">][</span><span class="s1">'attr1'</span><span class="p">][</span><span class="mi">5</span><span class="p">]</span></pre></div>
</div>
</li>
<li><p>The “textual” type that is returned by indexed access of <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.HSTORE" title="sqlalchemy.dialects.postgresql.HSTORE"><code class="xref py py-class docutils literal notranslate"><span class="pre">HSTORE</span></code></a>
as well as the “textual” type that is returned by indexed access of
<a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON" title="sqlalchemy.dialects.postgresql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">JSON</span></code></a> and <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSONB" title="sqlalchemy.dialects.postgresql.JSONB"><code class="xref py py-class docutils literal notranslate"><span class="pre">JSONB</span></code></a> in conjunction with the
<a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON.Comparator.astext" title="sqlalchemy.dialects.postgresql.JSON.Comparator.astext"><code class="xref py py-attr docutils literal notranslate"><span class="pre">astext</span></code></a> modifier is now configurable; it defaults
to <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Text" title="sqlalchemy.types.Text"><code class="xref py py-class docutils literal notranslate"><span class="pre">Text</span></code></a> in both cases but can be set to a user-defined
type using the <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON.params.astext_type" title="sqlalchemy.dialects.postgresql.JSON"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">postgresql.JSON.astext_type</span></code></a> or
<a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.HSTORE.params.text_type" title="sqlalchemy.dialects.postgresql.HSTORE"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">postgresql.HSTORE.text_type</span></code></a> parameters.</p></li>
</ul>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="#change-3503-cast"><span class="std std-ref">The JSON cast() operation now requires .astext is called explicitly</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3499">#3499</a>
<a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3487">#3487</a></p>
</div>
<div class="section" id="the-json-cast-operation-now-requires-astext-is-called-explicitly">
<span id="change-3503-cast"></span><h3>The JSON cast() operation now requires <code class="docutils literal notranslate"><span class="pre">.astext</span></code> is called explicitly<a class="headerlink" href="#the-json-cast-operation-now-requires-astext-is-called-explicitly" title="Permalink to this headline">¶</a></h3>
<p>As part of the changes in <a class="reference internal" href="#change-3503"><span class="std std-ref">Correct SQL Types are Established from Indexed Access of ARRAY, JSON, HSTORE</span></a>, the workings of the
<a class="reference internal" href="../core/sqlelement.html#sqlalchemy.sql.expression.ColumnElement.cast" title="sqlalchemy.sql.expression.ColumnElement.cast"><code class="xref py py-meth docutils literal notranslate"><span class="pre">ColumnElement.cast()</span></code></a> operator on <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON" title="sqlalchemy.dialects.postgresql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.JSON</span></code></a> and
<a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSONB" title="sqlalchemy.dialects.postgresql.JSONB"><code class="xref py py-class docutils literal notranslate"><span class="pre">postgresql.JSONB</span></code></a> no longer implicitly invoke the
<a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.JSON.Comparator.astext" title="sqlalchemy.dialects.postgresql.JSON.Comparator.astext"><code class="xref py py-attr docutils literal notranslate"><span class="pre">postgresql.JSON.Comparator.astext</span></code></a> modifier; PostgreSQL’s JSON/JSONB types
support CAST operations to each other without the “astext” aspect.</p>
<p>This means that in most cases, an application that was doing this:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">expr</span> <span class="o">=</span> <span class="n">json_col</span><span class="p">[</span><span class="s1">'somekey'</span><span class="p">]</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">Integer</span><span class="p">)</span></pre></div>
</div>
<p>Will now need to change to this:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">expr</span> <span class="o">=</span> <span class="n">json_col</span><span class="p">[</span><span class="s1">'somekey'</span><span class="p">]</span><span class="o">.</span><span class="n">astext</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">Integer</span><span class="p">)</span></pre></div>
</div>
</div>
<div class="section" id="array-with-enum-will-now-emit-create-type-for-the-enum">
<span id="change-2729"></span><h3>ARRAY with ENUM will now emit CREATE TYPE for the ENUM<a class="headerlink" href="#array-with-enum-will-now-emit-create-type-for-the-enum" title="Permalink to this headline">¶</a></h3>
<p>A table definition like the following will now emit CREATE TYPE
as expected:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">enum</span> <span class="o">=</span> <span class="n">Enum</span><span class="p">(</span>
<span class="s1">'manager'</span><span class="p">,</span> <span class="s1">'place_admin'</span><span class="p">,</span> <span class="s1">'carwash_admin'</span><span class="p">,</span>
<span class="s1">'parking_admin'</span><span class="p">,</span> <span class="s1">'service_admin'</span><span class="p">,</span> <span class="s1">'tire_admin'</span><span class="p">,</span>
<span class="s1">'mechanic'</span><span class="p">,</span> <span class="s1">'carwasher'</span><span class="p">,</span> <span class="s1">'tire_mechanic'</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s2">"work_place_roles"</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">WorkPlacement</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span>
<span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'work_placement'</span>
<span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">roles</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">ARRAY</span><span class="p">(</span><span class="n">enum</span><span class="p">))</span>
<span class="n">e</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s2">"postgresql://scott:tiger@localhost/test"</span><span class="p">,</span> <span class="n">echo</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">Base</span><span class="o">.</span><span class="n">metadata</span><span class="o">.</span><span class="n">create_all</span><span class="p">(</span><span class="n">e</span><span class="p">)</span></pre></div>
</div>
<p>emits:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">CREATE</span> <span class="n">TYPE</span> <span class="n">work_place_roles</span> <span class="n">AS</span> <span class="n">ENUM</span> <span class="p">(</span>
<span class="s1">'manager'</span><span class="p">,</span> <span class="s1">'place_admin'</span><span class="p">,</span> <span class="s1">'carwash_admin'</span><span class="p">,</span> <span class="s1">'parking_admin'</span><span class="p">,</span>
<span class="s1">'service_admin'</span><span class="p">,</span> <span class="s1">'tire_admin'</span><span class="p">,</span> <span class="s1">'mechanic'</span><span class="p">,</span> <span class="s1">'carwasher'</span><span class="p">,</span>
<span class="s1">'tire_mechanic'</span><span class="p">)</span>
<span class="n">CREATE</span> <span class="n">TABLE</span> <span class="n">work_placement</span> <span class="p">(</span>
<span class="nb">id</span> <span class="n">SERIAL</span> <span class="n">NOT</span> <span class="n">NULL</span><span class="p">,</span>
<span class="n">roles</span> <span class="n">work_place_roles</span><span class="p">[],</span>
<span class="n">PRIMARY</span> <span class="n">KEY</span> <span class="p">(</span><span class="nb">id</span><span class="p">)</span>
<span class="p">)</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/2729">#2729</a></p>
</div>
<div class="section" id="check-constraints-now-reflect">
<h3>Check constraints now reflect<a class="headerlink" href="#check-constraints-now-reflect" title="Permalink to this headline">¶</a></h3>
<p>The PostgreSQL dialect now supports reflection of CHECK constraints
both within the method <a class="reference internal" href="../core/reflection.html#sqlalchemy.engine.reflection.Inspector.get_check_constraints" title="sqlalchemy.engine.reflection.Inspector.get_check_constraints"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Inspector.get_check_constraints()</span></code></a> as well
as within <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Table" title="sqlalchemy.schema.Table"><code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code></a> reflection within the <code class="xref py py-attr docutils literal notranslate"><span class="pre">Table.constraints</span></code>
collection.</p>
</div>
<div class="section" id="plain-and-materialized-views-can-be-inspected-separately">
<h3>“Plain” and “Materialized” views can be inspected separately<a class="headerlink" href="#plain-and-materialized-views-can-be-inspected-separately" title="Permalink to this headline">¶</a></h3>
<p>The new argument <a class="reference internal" href="../dialects/postgresql.html#sqlalchemy.dialects.postgresql.base.PGInspector.get_view_names.params.include" title="sqlalchemy.dialects.postgresql.base.PGInspector.get_view_names"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">PGInspector.get_view_names.include</span></code></a>
allows specification of which sub-types of views should be returned:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">inspect</span>
<span class="n">insp</span> <span class="o">=</span> <span class="n">inspect</span><span class="p">(</span><span class="n">engine</span><span class="p">)</span>
<span class="n">plain_views</span> <span class="o">=</span> <span class="n">insp</span><span class="o">.</span><span class="n">get_view_names</span><span class="p">(</span><span class="n">include</span><span class="o">=</span><span class="s1">'plain'</span><span class="p">)</span>
<span class="n">all_views</span> <span class="o">=</span> <span class="n">insp</span><span class="o">.</span><span class="n">get_view_names</span><span class="p">(</span><span class="n">include</span><span class="o">=</span><span class="p">(</span><span class="s1">'plain'</span><span class="p">,</span> <span class="s1">'materialized'</span><span class="p">))</span></pre></div>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3588">#3588</a></p>
</div>
<div class="section" id="added-tablespace-option-to-index">
<h3>Added tablespace option to Index<a class="headerlink" href="#added-tablespace-option-to-index" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../core/constraints.html#sqlalchemy.schema.Index" title="sqlalchemy.schema.Index"><code class="xref py py-class docutils literal notranslate"><span class="pre">Index</span></code></a> object now accepts the argument <code class="docutils literal notranslate"><span class="pre">postgresql_tablespace</span></code>
in order to specify TABLESPACE, the same way as accepted by the
<a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Table" title="sqlalchemy.schema.Table"><code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code></a> object.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../dialects/postgresql.html#postgresql-index-storage"><span class="std std-ref">Index Storage Parameters</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3720">#3720</a></p>
</div>
<div class="section" id="support-for-pygresql">
<h3>Support for PyGreSQL<a class="headerlink" href="#support-for-pygresql" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference external" href="https://pypi.python.org/pypi/PyGreSQL">PyGreSQL</a> DBAPI is now supported.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../dialects/postgresql.html#dialect-postgresql-pygresql"><span class="std std-ref">pygresql</span></a></p>
</div>
</div>
<div class="section" id="the-postgres-module-is-removed">
<h3>The “postgres” module is removed<a class="headerlink" href="#the-postgres-module-is-removed" title="Permalink to this headline">¶</a></h3>
<p>The <code class="docutils literal notranslate"><span class="pre">sqlalchemy.dialects.postgres</span></code> module, long deprecated, is
removed; this has emitted a warning for many years and projects
should be calling upon <code class="docutils literal notranslate"><span class="pre">sqlalchemy.dialects.postgresql</span></code>.
Engine URLs of the form <code class="docutils literal notranslate"><span class="pre">postgres://</span></code> will still continue to function,
however.</p>
</div>
<div class="section" id="support-for-for-update-skip-locked-for-no-key-update-for-key-share">
<h3>Support for FOR UPDATE SKIP LOCKED / FOR NO KEY UPDATE / FOR KEY SHARE<a class="headerlink" href="#support-for-for-update-skip-locked-for-no-key-update-for-key-share" title="Permalink to this headline">¶</a></h3>
<p>The new parameters <a class="reference internal" href="../core/selectable.html#sqlalchemy.sql.expression.GenerativeSelect.with_for_update.params.skip_locked" title="sqlalchemy.sql.expression.GenerativeSelect.with_for_update"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">GenerativeSelect.with_for_update.skip_locked</span></code></a>
and <a class="reference internal" href="../core/selectable.html#sqlalchemy.sql.expression.GenerativeSelect.with_for_update.params.key_share" title="sqlalchemy.sql.expression.GenerativeSelect.with_for_update"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">GenerativeSelect.with_for_update.key_share</span></code></a>
in both Core and ORM apply a modification to a “SELECT…FOR UPDATE”
or “SELECT…FOR SHARE” query on the PostgreSQL backend:</p>
<ul>
<li><p>SELECT FOR NO KEY UPDATE:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">stmt</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">table</span><span class="p">])</span><span class="o">.</span><span class="n">with_for_update</span><span class="p">(</span><span class="n">key_share</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span></pre></div>
</div>
</li>
<li><p>SELECT FOR UPDATE SKIP LOCKED:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">stmt</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">table</span><span class="p">])</span><span class="o">.</span><span class="n">with_for_update</span><span class="p">(</span><span class="n">skip_locked</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span></pre></div>
</div>
</li>
<li><p>SELECT FOR KEY SHARE:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">stmt</span> <span class="o">=</span> <span class="n">select</span><span class="p">([</span><span class="n">table</span><span class="p">])</span><span class="o">.</span><span class="n">with_for_update</span><span class="p">(</span><span class="n">read</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">key_share</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span></pre></div>
</div>
</li>
</ul>
</div>
</div>
<div class="section" id="dialect-improvements-and-changes-mysql">
<h2>Dialect Improvements and Changes - MySQL<a class="headerlink" href="#dialect-improvements-and-changes-mysql" title="Permalink to this headline">¶</a></h2>
<div class="section" id="mysql-json-support">
<span id="change-3547"></span><h3>MySQL JSON Support<a class="headerlink" href="#mysql-json-support" title="Permalink to this headline">¶</a></h3>
<p>A new type <a class="reference internal" href="../dialects/mysql.html#sqlalchemy.dialects.mysql.JSON" title="sqlalchemy.dialects.mysql.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">mysql.JSON</span></code></a> is added to the MySQL dialect supporting
the JSON type newly added to MySQL 5.7. This type provides both persistence
of JSON as well as rudimentary indexed-access using the <code class="docutils literal notranslate"><span class="pre">JSON_EXTRACT</span></code>
function internally. An indexable JSON column that works across MySQL
and PostgreSQL can be achieved by using the <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.JSON" title="sqlalchemy.types.JSON"><code class="xref py py-class docutils literal notranslate"><span class="pre">types.JSON</span></code></a> datatype
common to both MySQL and PostgreSQL.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="#change-3619"><span class="std std-ref">JSON support added to Core</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3547">#3547</a></p>
</div>
<div class="section" id="added-support-for-autocommit-isolation-level">
<span id="change-3332"></span><h3>Added support for AUTOCOMMIT “isolation level”<a class="headerlink" href="#added-support-for-autocommit-isolation-level" title="Permalink to this headline">¶</a></h3>
<p>The MySQL dialect now accepts the value “AUTOCOMMIT” for the
<a class="reference internal" href="../core/engines.html#sqlalchemy.create_engine.params.isolation_level" title="sqlalchemy.create_engine"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">create_engine.isolation_level</span></code></a> and
<a class="reference internal" href="../core/connections.html#sqlalchemy.engine.Connection.execution_options.params.isolation_level" title="sqlalchemy.engine.Connection.execution_options"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">Connection.execution_options.isolation_level</span></code></a>
parameters:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">connection</span> <span class="o">=</span> <span class="n">engine</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span>
<span class="n">connection</span> <span class="o">=</span> <span class="n">connection</span><span class="o">.</span><span class="n">execution_options</span><span class="p">(</span>
<span class="n">isolation_level</span><span class="o">=</span><span class="s2">"AUTOCOMMIT"</span>
<span class="p">)</span></pre></div>
</div>
<p>The isolation level makes use of the various “autocommit” attributes
provided by most MySQL DBAPIs.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3332">#3332</a></p>
</div>
<div class="section" id="no-more-generation-of-an-implicit-key-for-composite-primary-key-w-auto-increment">
<span id="change-mysql-3216"></span><h3>No more generation of an implicit KEY for composite primary key w/ AUTO_INCREMENT<a class="headerlink" href="#no-more-generation-of-an-implicit-key-for-composite-primary-key-w-auto-increment" title="Permalink to this headline">¶</a></h3>
<p>The MySQL dialect had the behavior such that if a composite primary key
on an InnoDB table featured AUTO_INCREMENT on one of its columns which was
not the first column, e.g.:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">t</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span>
<span class="s1">'some_table'</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">autoincrement</span><span class="o">=</span><span class="kc">False</span><span class="p">),</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'y'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">autoincrement</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
<span class="n">mysql_engine</span><span class="o">=</span><span class="s1">'InnoDB'</span>
<span class="p">)</span></pre></div>
</div>
<p>DDL such as the following would be generated:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">CREATE</span> <span class="n">TABLE</span> <span class="n">some_table</span> <span class="p">(</span>
<span class="n">x</span> <span class="n">INTEGER</span> <span class="n">NOT</span> <span class="n">NULL</span><span class="p">,</span>
<span class="n">y</span> <span class="n">INTEGER</span> <span class="n">NOT</span> <span class="n">NULL</span> <span class="n">AUTO_INCREMENT</span><span class="p">,</span>
<span class="n">PRIMARY</span> <span class="n">KEY</span> <span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">),</span>
<span class="n">KEY</span> <span class="n">idx_autoinc_y</span> <span class="p">(</span><span class="n">y</span><span class="p">)</span>
<span class="p">)</span><span class="n">ENGINE</span><span class="o">=</span><span class="n">InnoDB</span></pre></div>
</div>
<p>Note the above “KEY” with an auto-generated name; this is a change that
found its way into the dialect many years ago in response to the issue that
the AUTO_INCREMENT would otherwise fail on InnoDB without this additional KEY.</p>
<p>This workaround has been removed and replaced with the much better system
of just stating the AUTO_INCREMENT column <em>first</em> within the primary key:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">CREATE</span> <span class="n">TABLE</span> <span class="n">some_table</span> <span class="p">(</span>
<span class="n">x</span> <span class="n">INTEGER</span> <span class="n">NOT</span> <span class="n">NULL</span><span class="p">,</span>
<span class="n">y</span> <span class="n">INTEGER</span> <span class="n">NOT</span> <span class="n">NULL</span> <span class="n">AUTO_INCREMENT</span><span class="p">,</span>
<span class="n">PRIMARY</span> <span class="n">KEY</span> <span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span>
<span class="p">)</span><span class="n">ENGINE</span><span class="o">=</span><span class="n">InnoDB</span></pre></div>
</div>
<p>To maintain explicit control of the ordering of primary key columns,
use the <a class="reference internal" href="../core/constraints.html#sqlalchemy.schema.PrimaryKeyConstraint" title="sqlalchemy.schema.PrimaryKeyConstraint"><code class="xref py py-class docutils literal notranslate"><span class="pre">PrimaryKeyConstraint</span></code></a> construct explicitly (1.1.0b2)
(along with a KEY for the autoincrement column as required by MySQL), e.g.:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">t</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span>
<span class="s1">'some_table'</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'y'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">autoincrement</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
<span class="n">PrimaryKeyConstraint</span><span class="p">(</span><span class="s1">'x'</span><span class="p">,</span> <span class="s1">'y'</span><span class="p">),</span>
<span class="n">UniqueConstraint</span><span class="p">(</span><span class="s1">'y'</span><span class="p">),</span>
<span class="n">mysql_engine</span><span class="o">=</span><span class="s1">'InnoDB'</span>
<span class="p">)</span></pre></div>
</div>
<p>Along with the change <a class="reference internal" href="#change-3216"><span class="std std-ref">The .autoincrement directive is no longer implicitly enabled for a composite primary key column</span></a>, composite primary keys with
or without auto increment are now easier to specify;
<a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Column.params.autoincrement" title="sqlalchemy.schema.Column"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">Column.autoincrement</span></code></a>
now defaults to the value <code class="docutils literal notranslate"><span class="pre">"auto"</span></code> and the <code class="docutils literal notranslate"><span class="pre">autoincrement=False</span></code>
directives are no longer needed:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">t</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span>
<span class="s1">'some_table'</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'y'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">autoincrement</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
<span class="n">mysql_engine</span><span class="o">=</span><span class="s1">'InnoDB'</span>
<span class="p">)</span></pre></div>
</div>
</div>
</div>
<div class="section" id="dialect-improvements-and-changes-sqlite">
<h2>Dialect Improvements and Changes - SQLite<a class="headerlink" href="#dialect-improvements-and-changes-sqlite" title="Permalink to this headline">¶</a></h2>
<div class="section" id="right-nested-join-workaround-lifted-for-sqlite-version-3-7-16">
<span id="change-3634"></span><h3>Right-nested join workaround lifted for SQLite version 3.7.16<a class="headerlink" href="#right-nested-join-workaround-lifted-for-sqlite-version-3-7-16" title="Permalink to this headline">¶</a></h3>
<p>In version 0.9, the feature introduced by <a class="reference internal" href="migration_09.html#feature-joins-09"><span class="std std-ref">Many JOIN and LEFT OUTER JOIN expressions will no longer be wrapped in (SELECT * FROM ..) AS ANON_1</span></a> went
through lots of effort to support rewriting of joins on SQLite to always
use subqueries in order to achieve a “right-nested-join” effect, as
SQLite has not supported this syntax for many years. Ironically,
the version of SQLite noted in that migration note, 3.7.15.2, was the <em>last</em>
version of SQLite to actually have this limitation! The next release was
3.7.16 and support for right nested joins was quietly added. In 1.1, the work
to identify the specific SQLite version and source commit where this change
was made was done (SQLite’s changelog refers to it with the cryptic phrase “Enhance
the query optimizer to exploit transitive join constraints” without linking
to any issue number, change number, or further explanation), and the workarounds
present in this change are now lifted for SQLite when the DBAPI reports
that version 3.7.16 or greater is in effect.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3634">#3634</a></p>
</div>
<div class="section" id="dotted-column-names-workaround-lifted-for-sqlite-version-3-10-0">
<span id="change-3633"></span><h3>Dotted column names workaround lifted for SQLite version 3.10.0<a class="headerlink" href="#dotted-column-names-workaround-lifted-for-sqlite-version-3-10-0" title="Permalink to this headline">¶</a></h3>
<p>The SQLite dialect has long had a workaround for an issue where the database
driver does not report the correct column names for some SQL result sets, in
particular when UNION is used. The workaround is detailed at
<a class="reference internal" href="../dialects/sqlite.html#sqlite-dotted-column-names"><span class="std std-ref">Dotted Column Names</span></a>, and requires that SQLAlchemy assume that any
column name with a dot in it is actually a <code class="docutils literal notranslate"><span class="pre">tablename.columnname</span></code> combination
delivered via this buggy behavior, with an option to turn it off via the
<code class="docutils literal notranslate"><span class="pre">sqlite_raw_colnames</span></code> execution option.</p>
<p>As of SQLite version 3.10.0, the bug in UNION and other queries has been fixed;
like the change described in <a class="reference internal" href="#change-3634"><span class="std std-ref">Right-nested join workaround lifted for SQLite version 3.7.16</span></a>, SQLite’s changelog only
identifies it cryptically as “Added the colUsed field to sqlite3_index_info for
use by the sqlite3_module.xBestIndex method”, however SQLAlchemy’s translation
of these dotted column names is no longer required with this version, so is
turned off when version 3.10.0 or greater is detected.</p>
<p>Overall, the SQLAlchemy <a class="reference internal" href="../core/connections.html#sqlalchemy.engine.ResultProxy" title="sqlalchemy.engine.ResultProxy"><code class="xref py py-class docutils literal notranslate"><span class="pre">ResultProxy</span></code></a> as of the 1.0 series relies much
less on column names in result sets when delivering results for Core and ORM
SQL constructs, so the importance of this issue was already lessened in any
case.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3633">#3633</a></p>
</div>
<div class="section" id="improved-support-for-remote-schemas">
<span id="change-sqlite-schemas"></span><h3>Improved Support for Remote Schemas<a class="headerlink" href="#improved-support-for-remote-schemas" title="Permalink to this headline">¶</a></h3>
<p>The SQLite dialect now implements <a class="reference internal" href="../core/reflection.html#sqlalchemy.engine.reflection.Inspector.get_schema_names" title="sqlalchemy.engine.reflection.Inspector.get_schema_names"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Inspector.get_schema_names()</span></code></a>
and additionally has improved support for tables and indexes that are
created and reflected from a remote schema, which in SQLite is a
database that is assigned a name via the <code class="docutils literal notranslate"><span class="pre">ATTACH</span></code> statement; previously,
the``CREATE INDEX`` DDL didn’t work correctly for a schema-bound table
and the <a class="reference internal" href="../core/reflection.html#sqlalchemy.engine.reflection.Inspector.get_foreign_keys" title="sqlalchemy.engine.reflection.Inspector.get_foreign_keys"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Inspector.get_foreign_keys()</span></code></a> method will now indicate the
given schema in the results. Cross-schema foreign keys aren’t supported.</p>
</div>
<div class="section" id="reflection-of-the-name-of-primary-key-constraints">
<span id="change-3629"></span><h3>Reflection of the name of PRIMARY KEY constraints<a class="headerlink" href="#reflection-of-the-name-of-primary-key-constraints" title="Permalink to this headline">¶</a></h3>
<p>The SQLite backend now takes advantage of the “sqlite_master” view
of SQLite in order to extract the name of the primary key constraint
of a table from the original DDL, in the same way that is achieved for
foreign key constraints in recent SQLAlchemy versions.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3629">#3629</a></p>
</div>
<div class="section" id="id1">
<h3>Check constraints now reflect<a class="headerlink" href="#id1" title="Permalink to this headline">¶</a></h3>
<p>The SQLite dialect now supports reflection of CHECK constraints
both within the method <a class="reference internal" href="../core/reflection.html#sqlalchemy.engine.reflection.Inspector.get_check_constraints" title="sqlalchemy.engine.reflection.Inspector.get_check_constraints"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Inspector.get_check_constraints()</span></code></a> as well
as within <a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Table" title="sqlalchemy.schema.Table"><code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code></a> reflection within the <code class="xref py py-attr docutils literal notranslate"><span class="pre">Table.constraints</span></code>
collection.</p>
</div>
<div class="section" id="on-delete-and-on-update-foreign-key-phrases-now-reflect">
<h3>ON DELETE and ON UPDATE foreign key phrases now reflect<a class="headerlink" href="#on-delete-and-on-update-foreign-key-phrases-now-reflect" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="../core/reflection.html#sqlalchemy.engine.reflection.Inspector" title="sqlalchemy.engine.reflection.Inspector"><code class="xref py py-class docutils literal notranslate"><span class="pre">Inspector</span></code></a> will now include ON DELETE and ON UPDATE
phrases from foreign key constraints on the SQLite dialect, and the
<a class="reference internal" href="../core/constraints.html#sqlalchemy.schema.ForeignKeyConstraint" title="sqlalchemy.schema.ForeignKeyConstraint"><code class="xref py py-class docutils literal notranslate"><span class="pre">ForeignKeyConstraint</span></code></a> object as reflected as part of a
<a class="reference internal" href="../core/metadata.html#sqlalchemy.schema.Table" title="sqlalchemy.schema.Table"><code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code></a> will also indicate these phrases.</p>
</div>
</div>
<div class="section" id="dialect-improvements-and-changes-sql-server">
<h2>Dialect Improvements and Changes - SQL Server<a class="headerlink" href="#dialect-improvements-and-changes-sql-server" title="Permalink to this headline">¶</a></h2>
<div class="section" id="added-transaction-isolation-level-support-for-sql-server">
<span id="change-3534"></span><h3>Added transaction isolation level support for SQL Server<a class="headerlink" href="#added-transaction-isolation-level-support-for-sql-server" title="Permalink to this headline">¶</a></h3>
<p>All SQL Server dialects support transaction isolation level settings
via the <a class="reference internal" href="../core/engines.html#sqlalchemy.create_engine.params.isolation_level" title="sqlalchemy.create_engine"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">create_engine.isolation_level</span></code></a> and
<a class="reference internal" href="../core/connections.html#sqlalchemy.engine.Connection.execution_options.params.isolation_level" title="sqlalchemy.engine.Connection.execution_options"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">Connection.execution_options.isolation_level</span></code></a>
parameters. The four standard levels are supported as well as
<code class="docutils literal notranslate"><span class="pre">SNAPSHOT</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">engine</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span>
<span class="s2">"mssql+pyodbc://scott:tiger@ms_2008"</span><span class="p">,</span>
<span class="n">isolation_level</span><span class="o">=</span><span class="s2">"REPEATABLE READ"</span>
<span class="p">)</span></pre></div>
</div>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../dialects/mssql.html#mssql-isolation-level"><span class="std std-ref">Transaction Isolation Level</span></a></p>
</div>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3534">#3534</a></p>
</div>
<div class="section" id="string-varlength-types-no-longer-represent-max-explicitly-on-reflection">
<span id="change-3504"></span><h3>String / varlength types no longer represent “max” explicitly on reflection<a class="headerlink" href="#string-varlength-types-no-longer-represent-max-explicitly-on-reflection" title="Permalink to this headline">¶</a></h3>
<p>When reflecting a type such as <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.String" title="sqlalchemy.types.String"><code class="xref py py-class docutils literal notranslate"><span class="pre">String</span></code></a>, <a class="reference internal" href="../core/type_basics.html#sqlalchemy.types.Text" title="sqlalchemy.types.Text"><code class="xref py py-class docutils literal notranslate"><span class="pre">Text</span></code></a>, etc.
which includes a length, an “un-lengthed” type under SQL Server would
copy the “length” parameter as the value <code class="docutils literal notranslate"><span class="pre">"max"</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="k">import</span> <span class="n">create_engine</span><span class="p">,</span> <span class="n">inspect</span>
<span class="gp">>>> </span><span class="n">engine</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s1">'mssql+pyodbc://scott:tiger@ms_2008'</span><span class="p">,</span> <span class="n">echo</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">engine</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"create table s (x varchar(max), y varbinary(max))"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">insp</span> <span class="o">=</span> <span class="n">inspect</span><span class="p">(</span><span class="n">engine</span><span class="p">)</span>
<span class="gp">>>> </span><span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">insp</span><span class="o">.</span><span class="n">get_columns</span><span class="p">(</span><span class="s2">"s"</span><span class="p">):</span>
<span class="gp">... </span> <span class="nb">print</span><span class="p">(</span><span class="n">col</span><span class="p">[</span><span class="s1">'type'</span><span class="p">]</span><span class="o">.</span><span class="vm">__class__</span><span class="p">,</span> <span class="n">col</span><span class="p">[</span><span class="s1">'type'</span><span class="p">]</span><span class="o">.</span><span class="n">length</span><span class="p">)</span>
<span class="gp">...</span>
<span class="go"><class 'sqlalchemy.sql.sqltypes.VARCHAR'> max</span>
<span class="go"><class 'sqlalchemy.dialects.mssql.base.VARBINARY'> max</span></pre></div>
</div>
<p>The “length” parameter in the base types is expected to be an integer value
or None only; None indicates unbounded length which the SQL Server dialect
interprets as “max”. The fix then is so that these lengths come
out as None, so that the type objects work in non-SQL Server contexts:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">insp</span><span class="o">.</span><span class="n">get_columns</span><span class="p">(</span><span class="s2">"s"</span><span class="p">):</span>
<span class="gp">... </span> <span class="nb">print</span><span class="p">(</span><span class="n">col</span><span class="p">[</span><span class="s1">'type'</span><span class="p">]</span><span class="o">.</span><span class="vm">__class__</span><span class="p">,</span> <span class="n">col</span><span class="p">[</span><span class="s1">'type'</span><span class="p">]</span><span class="o">.</span><span class="n">length</span><span class="p">)</span>
<span class="gp">...</span>
<span class="go"><class 'sqlalchemy.sql.sqltypes.VARCHAR'> None</span>
<span class="go"><class 'sqlalchemy.dialects.mssql.base.VARBINARY'> None</span></pre></div>
</div>
<p>Applications which may have been relying on a direct comparison of the “length”
value to the string “max” should consider the value of <code class="docutils literal notranslate"><span class="pre">None</span></code> to mean
the same thing.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3504">#3504</a></p>
</div>
<div class="section" id="support-for-non-clustered-on-primary-key-to-allow-clustered-elsewhere">
<h3>Support for “non clustered” on primary key to allow clustered elsewhere<a class="headerlink" href="#support-for-non-clustered-on-primary-key-to-allow-clustered-elsewhere" title="Permalink to this headline">¶</a></h3>
<p>The <code class="docutils literal notranslate"><span class="pre">mssql_clustered</span></code> flag available on <a class="reference internal" href="../core/constraints.html#sqlalchemy.schema.UniqueConstraint" title="sqlalchemy.schema.UniqueConstraint"><code class="xref py py-class docutils literal notranslate"><span class="pre">UniqueConstraint</span></code></a>,
<a class="reference internal" href="../core/constraints.html#sqlalchemy.schema.PrimaryKeyConstraint" title="sqlalchemy.schema.PrimaryKeyConstraint"><code class="xref py py-class docutils literal notranslate"><span class="pre">PrimaryKeyConstraint</span></code></a>, <a class="reference internal" href="../core/constraints.html#sqlalchemy.schema.Index" title="sqlalchemy.schema.Index"><code class="xref py py-class docutils literal notranslate"><span class="pre">Index</span></code></a> now defaults to <code class="docutils literal notranslate"><span class="pre">None</span></code>, and
can be set to False which will render the NONCLUSTERED keyword in particular
for a primary key, allowing a different index to be used as “clustered”.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="../dialects/mssql.html#mssql-indexes"><span class="std std-ref">Clustered Index Support</span></a></p>
</div>
</div>
<div class="section" id="the-legacy-schema-aliasing-flag-is-now-set-to-false">
<span id="change-3434"></span><h3>The legacy_schema_aliasing flag is now set to False<a class="headerlink" href="#the-legacy-schema-aliasing-flag-is-now-set-to-false" title="Permalink to this headline">¶</a></h3>
<p>SQLAlchemy 1.0.5 introduced the <code class="docutils literal notranslate"><span class="pre">legacy_schema_aliasing</span></code> flag to the
MSSQL dialect, allowing so-called “legacy mode” aliasing to be turned off.
This aliasing attempts to turn schema-qualified tables into aliases;
given a table such as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">account_table</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span>
<span class="s1">'account'</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'id'</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
<span class="n">Column</span><span class="p">(</span><span class="s1">'info'</span><span class="p">,</span> <span class="n">String</span><span class="p">(</span><span class="mi">100</span><span class="p">)),</span>
<span class="n">schema</span><span class="o">=</span><span class="s2">"customer_schema"</span>
<span class="p">)</span></pre></div>
</div>
<p>The legacy mode of behavior will attempt to turn a schema-qualified table
name into an alias:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">eng</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s2">"mssql+pymssql://mydsn"</span><span class="p">,</span> <span class="n">legacy_schema_aliasing</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="gp">>>> </span><span class="nb">print</span><span class="p">(</span><span class="n">account_table</span><span class="o">.</span><span class="n">select</span><span class="p">()</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="n">eng</span><span class="p">))</span>
<span class="go">SELECT account_1.id, account_1.info</span>
<span class="go">FROM customer_schema.account AS account_1</span></pre></div>
</div>
<p>However, this aliasing has been shown to be unnecessary and in many cases
produces incorrect SQL.</p>
<p>In SQLAlchemy 1.1, the <code class="docutils literal notranslate"><span class="pre">legacy_schema_aliasing</span></code> flag now defaults to
False, disabling this mode of behavior and allowing the MSSQL dialect to behave
normally with schema-qualified tables. For applications which may rely
on this behavior, set the flag back to True.</p>
<p><a class="reference external" href="http://www.sqlalchemy.org/trac/ticket/3434">#3434</a></p>
</div>
</div>
<div class="section" id="dialect-improvements-and-changes-oracle">
<h2>Dialect Improvements and Changes - Oracle<a class="headerlink" href="#dialect-improvements-and-changes-oracle" title="Permalink to this headline">¶</a></h2>
<div class="section" id="support-for-skip-locked">
<h3>Support for SKIP LOCKED<a class="headerlink" href="#support-for-skip-locked" title="Permalink to this headline">¶</a></h3>
<p>The new parameter <a class="reference internal" href="../core/selectable.html#sqlalchemy.sql.expression.GenerativeSelect.with_for_update.params.skip_locked" title="sqlalchemy.sql.expression.GenerativeSelect.with_for_update"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">GenerativeSelect.with_for_update.skip_locked</span></code></a>
in both Core and ORM will generate the “SKIP LOCKED” suffix for a
“SELECT…FOR UPDATE” or “SELECT.. FOR SHARE” query.</p>
</div>
</div>
</div>
</div>
</div>
<div id="docs-bottom-navigation" class="docs-navigation-links, withsidebar">
Previous:
<a href="changelog_01.html" title="previous chapter">0.1 Changelog</a>
Next:
<a href="migration_10.html" title="next chapter">What’s New in SQLAlchemy 1.0?</a>
<div id="docs-copyright">
© <a href="../copyright.html">Copyright</a> 2007-2019, the SQLAlchemy authors and contributors.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 2.0.1.
</div>
</div>
</div>
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '1.2.19',
COLLAPSE_MODINDEX: false,
FILE_SUFFIX: '.html'
};
</script>
<script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
<!-- begin iterate through sphinx environment script_files -->
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<script type="text/javascript" src="../_static/language_data.js"></script>
<!-- end iterate through sphinx environment script_files -->
<script type="text/javascript" src="../_static/detectmobile.js"></script>
<script type="text/javascript" src="../_static/init.js"></script>
</body>
</html>
distrib
>
Mageia
>
7
>
armv7hl
>
media
>
core-updates
>
by-pkgid
>
b0b6ffab06cbeede296e36ce94734bf8
>
files
>
750
