<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML
><HEAD
><TITLE
>WITH Queries (Common Table Expressions)</TITLE
><META
NAME="GENERATOR"
CONTENT="Modular DocBook HTML Stylesheet Version 1.79"><LINK
REV="MADE"
HREF="mailto:pgsql-docs@postgresql.org"><LINK
REL="HOME"
TITLE="PostgreSQL 9.3.9 Documentation"
HREF="index.html"><LINK
REL="UP"
TITLE="Queries"
HREF="queries.html"><LINK
REL="PREVIOUS"
TITLE="VALUES Lists"
HREF="queries-values.html"><LINK
REL="NEXT"
TITLE="Data Types"
HREF="datatype.html"><LINK
REL="STYLESHEET"
TYPE="text/css"
HREF="stylesheet.css"><META
HTTP-EQUIV="Content-Type"
CONTENT="text/html; charset=ISO-8859-1"><META
NAME="creation"
CONTENT="2015-06-13T20:07:22"></HEAD
><BODY
CLASS="SECT1"
><DIV
CLASS="NAVHEADER"
><TABLE
SUMMARY="Header navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TH
COLSPAN="5"
ALIGN="center"
VALIGN="bottom"
><A
HREF="index.html"
>PostgreSQL 9.3.9 Documentation</A
></TH
></TR
><TR
><TD
WIDTH="10%"
ALIGN="left"
VALIGN="top"
><A
TITLE="VALUES Lists"
HREF="queries-values.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="10%"
ALIGN="left"
VALIGN="top"
><A
HREF="queries.html"
ACCESSKEY="U"
>Up</A
></TD
><TD
WIDTH="60%"
ALIGN="center"
VALIGN="bottom"
>Chapter 7. Queries</TD
><TD
WIDTH="20%"
ALIGN="right"
VALIGN="top"
><A
TITLE="Data Types"
HREF="datatype.html"
ACCESSKEY="N"
>Next</A
></TD
></TR
></TABLE
><HR
ALIGN="LEFT"
WIDTH="100%"></DIV
><DIV
CLASS="SECT1"
><H1
CLASS="SECT1"
><A
NAME="QUERIES-WITH"
>7.8. <TT
CLASS="LITERAL"
>WITH</TT
> Queries (Common Table Expressions)</A
></H1
><P
> <TT
CLASS="LITERAL"
>WITH</TT
> provides a way to write auxiliary statements for use in a
larger query. These statements, which are often referred to as Common
Table Expressions or <ACRONYM
CLASS="ACRONYM"
>CTE</ACRONYM
>s, can be thought of as defining
temporary tables that exist just for one query. Each auxiliary statement
in a <TT
CLASS="LITERAL"
>WITH</TT
> clause can be a <TT
CLASS="COMMAND"
>SELECT</TT
>,
<TT
CLASS="COMMAND"
>INSERT</TT
>, <TT
CLASS="COMMAND"
>UPDATE</TT
>, or <TT
CLASS="COMMAND"
>DELETE</TT
>; and the
<TT
CLASS="LITERAL"
>WITH</TT
> clause itself is attached to a primary statement that can
also be a <TT
CLASS="COMMAND"
>SELECT</TT
>, <TT
CLASS="COMMAND"
>INSERT</TT
>, <TT
CLASS="COMMAND"
>UPDATE</TT
>, or
<TT
CLASS="COMMAND"
>DELETE</TT
>.
</P
><DIV
CLASS="SECT2"
><H2
CLASS="SECT2"
><A
NAME="QUERIES-WITH-SELECT"
>7.8.1. <TT
CLASS="COMMAND"
>SELECT</TT
> in <TT
CLASS="LITERAL"
>WITH</TT
></A
></H2
><P
> The basic value of <TT
CLASS="COMMAND"
>SELECT</TT
> in <TT
CLASS="LITERAL"
>WITH</TT
> is to
break down complicated queries into simpler parts. An example is:
</P><PRE
CLASS="PROGRAMLISTING"
>WITH regional_sales AS (
SELECT region, SUM(amount) AS total_sales
FROM orders
GROUP BY region
), top_regions AS (
SELECT region
FROM regional_sales
WHERE total_sales > (SELECT SUM(total_sales)/10 FROM regional_sales)
)
SELECT region,
product,
SUM(quantity) AS product_units,
SUM(amount) AS product_sales
FROM orders
WHERE region IN (SELECT region FROM top_regions)
GROUP BY region, product;</PRE
><P>
which displays per-product sales totals in only the top sales regions.
The <TT
CLASS="LITERAL"
>WITH</TT
> clause defines two auxiliary statements named
<TT
CLASS="STRUCTNAME"
>regional_sales</TT
> and <TT
CLASS="STRUCTNAME"
>top_regions</TT
>,
where the output of <TT
CLASS="STRUCTNAME"
>regional_sales</TT
> is used in
<TT
CLASS="STRUCTNAME"
>top_regions</TT
> and the output of <TT
CLASS="STRUCTNAME"
>top_regions</TT
>
is used in the primary <TT
CLASS="COMMAND"
>SELECT</TT
> query.
This example could have been written without <TT
CLASS="LITERAL"
>WITH</TT
>,
but we'd have needed two levels of nested sub-<TT
CLASS="COMMAND"
>SELECT</TT
>s. It's a bit
easier to follow this way.
</P
><P
> The optional <TT
CLASS="LITERAL"
>RECURSIVE</TT
> modifier changes <TT
CLASS="LITERAL"
>WITH</TT
>
from a mere syntactic convenience into a feature that accomplishes
things not otherwise possible in standard SQL. Using
<TT
CLASS="LITERAL"
>RECURSIVE</TT
>, a <TT
CLASS="LITERAL"
>WITH</TT
> query can refer to its own
output. A very simple example is this query to sum the integers from 1
through 100:
</P><PRE
CLASS="PROGRAMLISTING"
>WITH RECURSIVE t(n) AS (
VALUES (1)
UNION ALL
SELECT n+1 FROM t WHERE n < 100
)
SELECT sum(n) FROM t;</PRE
><P>
The general form of a recursive <TT
CLASS="LITERAL"
>WITH</TT
> query is always a
<I
CLASS="FIRSTTERM"
>non-recursive term</I
>, then <TT
CLASS="LITERAL"
>UNION</TT
> (or
<TT
CLASS="LITERAL"
>UNION ALL</TT
>), then a
<I
CLASS="FIRSTTERM"
>recursive term</I
>, where only the recursive term can contain
a reference to the query's own output. Such a query is executed as
follows:
</P
><DIV
CLASS="PROCEDURE"
><P
><B
>Recursive Query Evaluation</B
></P
><OL
TYPE="1"
><LI
CLASS="STEP"
><P
> Evaluate the non-recursive term. For <TT
CLASS="LITERAL"
>UNION</TT
> (but not
<TT
CLASS="LITERAL"
>UNION ALL</TT
>), discard duplicate rows. Include all remaining
rows in the result of the recursive query, and also place them in a
temporary <I
CLASS="FIRSTTERM"
>working table</I
>.
</P
></LI
><LI
CLASS="STEP"
><P
> So long as the working table is not empty, repeat these steps:
</P
><OL
CLASS="SUBSTEPS"
TYPE="a"
><LI
CLASS="STEP"
><P
> Evaluate the recursive term, substituting the current contents of
the working table for the recursive self-reference.
For <TT
CLASS="LITERAL"
>UNION</TT
> (but not <TT
CLASS="LITERAL"
>UNION ALL</TT
>), discard
duplicate rows and rows that duplicate any previous result row.
Include all remaining rows in the result of the recursive query, and
also place them in a temporary <I
CLASS="FIRSTTERM"
>intermediate table</I
>.
</P
></LI
><LI
CLASS="STEP"
><P
> Replace the contents of the working table with the contents of the
intermediate table, then empty the intermediate table.
</P
></LI
></OL
></LI
></OL
></DIV
><DIV
CLASS="NOTE"
><BLOCKQUOTE
CLASS="NOTE"
><P
><B
>Note: </B
> Strictly speaking, this process is iteration not recursion, but
<TT
CLASS="LITERAL"
>RECURSIVE</TT
> is the terminology chosen by the SQL standards
committee.
</P
></BLOCKQUOTE
></DIV
><P
> In the example above, the working table has just a single row in each step,
and it takes on the values from 1 through 100 in successive steps. In
the 100th step, there is no output because of the <TT
CLASS="LITERAL"
>WHERE</TT
>
clause, and so the query terminates.
</P
><P
> Recursive queries are typically used to deal with hierarchical or
tree-structured data. A useful example is this query to find all the
direct and indirect sub-parts of a product, given only a table that
shows immediate inclusions:
</P><PRE
CLASS="PROGRAMLISTING"
>WITH RECURSIVE included_parts(sub_part, part, quantity) AS (
SELECT sub_part, part, quantity FROM parts WHERE part = 'our_product'
UNION ALL
SELECT p.sub_part, p.part, p.quantity
FROM included_parts pr, parts p
WHERE p.part = pr.sub_part
)
SELECT sub_part, SUM(quantity) as total_quantity
FROM included_parts
GROUP BY sub_part</PRE
><P>
</P
><P
> When working with recursive queries it is important to be sure that
the recursive part of the query will eventually return no tuples,
or else the query will loop indefinitely. Sometimes, using
<TT
CLASS="LITERAL"
>UNION</TT
> instead of <TT
CLASS="LITERAL"
>UNION ALL</TT
> can accomplish this
by discarding rows that duplicate previous output rows. However, often a
cycle does not involve output rows that are completely duplicate: it may be
necessary to check just one or a few fields to see if the same point has
been reached before. The standard method for handling such situations is
to compute an array of the already-visited values. For example, consider
the following query that searches a table <TT
CLASS="STRUCTNAME"
>graph</TT
> using a
<TT
CLASS="STRUCTFIELD"
>link</TT
> field:
</P><PRE
CLASS="PROGRAMLISTING"
>WITH RECURSIVE search_graph(id, link, data, depth) AS (
SELECT g.id, g.link, g.data, 1
FROM graph g
UNION ALL
SELECT g.id, g.link, g.data, sg.depth + 1
FROM graph g, search_graph sg
WHERE g.id = sg.link
)
SELECT * FROM search_graph;</PRE
><P>
This query will loop if the <TT
CLASS="STRUCTFIELD"
>link</TT
> relationships contain
cycles. Because we require a <SPAN
CLASS="QUOTE"
>"depth"</SPAN
> output, just changing
<TT
CLASS="LITERAL"
>UNION ALL</TT
> to <TT
CLASS="LITERAL"
>UNION</TT
> would not eliminate the looping.
Instead we need to recognize whether we have reached the same row again
while following a particular path of links. We add two columns
<TT
CLASS="STRUCTFIELD"
>path</TT
> and <TT
CLASS="STRUCTFIELD"
>cycle</TT
> to the loop-prone query:
</P><PRE
CLASS="PROGRAMLISTING"
>WITH RECURSIVE search_graph(id, link, data, depth, path, cycle) AS (
SELECT g.id, g.link, g.data, 1,
ARRAY[g.id],
false
FROM graph g
UNION ALL
SELECT g.id, g.link, g.data, sg.depth + 1,
path || g.id,
g.id = ANY(path)
FROM graph g, search_graph sg
WHERE g.id = sg.link AND NOT cycle
)
SELECT * FROM search_graph;</PRE
><P>
Aside from preventing cycles, the array value is often useful in its own
right as representing the <SPAN
CLASS="QUOTE"
>"path"</SPAN
> taken to reach any particular row.
</P
><P
> In the general case where more than one field needs to be checked to
recognize a cycle, use an array of rows. For example, if we needed to
compare fields <TT
CLASS="STRUCTFIELD"
>f1</TT
> and <TT
CLASS="STRUCTFIELD"
>f2</TT
>:
</P><PRE
CLASS="PROGRAMLISTING"
>WITH RECURSIVE search_graph(id, link, data, depth, path, cycle) AS (
SELECT g.id, g.link, g.data, 1,
ARRAY[ROW(g.f1, g.f2)],
false
FROM graph g
UNION ALL
SELECT g.id, g.link, g.data, sg.depth + 1,
path || ROW(g.f1, g.f2),
ROW(g.f1, g.f2) = ANY(path)
FROM graph g, search_graph sg
WHERE g.id = sg.link AND NOT cycle
)
SELECT * FROM search_graph;</PRE
><P>
</P
><DIV
CLASS="TIP"
><BLOCKQUOTE
CLASS="TIP"
><P
><B
>Tip: </B
> Omit the <TT
CLASS="LITERAL"
>ROW()</TT
> syntax in the common case where only one field
needs to be checked to recognize a cycle. This allows a simple array
rather than a composite-type array to be used, gaining efficiency.
</P
></BLOCKQUOTE
></DIV
><DIV
CLASS="TIP"
><BLOCKQUOTE
CLASS="TIP"
><P
><B
>Tip: </B
> The recursive query evaluation algorithm produces its output in
breadth-first search order. You can display the results in depth-first
search order by making the outer query <TT
CLASS="LITERAL"
>ORDER BY</TT
> a
<SPAN
CLASS="QUOTE"
>"path"</SPAN
> column constructed in this way.
</P
></BLOCKQUOTE
></DIV
><P
> A helpful trick for testing queries
when you are not certain if they might loop is to place a <TT
CLASS="LITERAL"
>LIMIT</TT
>
in the parent query. For example, this query would loop forever without
the <TT
CLASS="LITERAL"
>LIMIT</TT
>:
</P><PRE
CLASS="PROGRAMLISTING"
>WITH RECURSIVE t(n) AS (
SELECT 1
UNION ALL
SELECT n+1 FROM t
)
SELECT n FROM t LIMIT 100;</PRE
><P>
This works because <SPAN
CLASS="PRODUCTNAME"
>PostgreSQL</SPAN
>'s implementation
evaluates only as many rows of a <TT
CLASS="LITERAL"
>WITH</TT
> query as are actually
fetched by the parent query. Using this trick in production is not
recommended, because other systems might work differently. Also, it
usually won't work if you make the outer query sort the recursive query's
results or join them to some other table, because in such cases the
outer query will usually try to fetch all of the <TT
CLASS="LITERAL"
>WITH</TT
> query's
output anyway.
</P
><P
> A useful property of <TT
CLASS="LITERAL"
>WITH</TT
> queries is that they are evaluated
only once per execution of the parent query, even if they are referred to
more than once by the parent query or sibling <TT
CLASS="LITERAL"
>WITH</TT
> queries.
Thus, expensive calculations that are needed in multiple places can be
placed within a <TT
CLASS="LITERAL"
>WITH</TT
> query to avoid redundant work. Another
possible application is to prevent unwanted multiple evaluations of
functions with side-effects.
However, the other side of this coin is that the optimizer is less able to
push restrictions from the parent query down into a <TT
CLASS="LITERAL"
>WITH</TT
> query
than an ordinary sub-query. The <TT
CLASS="LITERAL"
>WITH</TT
> query will generally be
evaluated as written, without suppression of rows that the parent query
might discard afterwards. (But, as mentioned above, evaluation might stop
early if the reference(s) to the query demand only a limited number of
rows.)
</P
><P
> The examples above only show <TT
CLASS="LITERAL"
>WITH</TT
> being used with
<TT
CLASS="COMMAND"
>SELECT</TT
>, but it can be attached in the same way to
<TT
CLASS="COMMAND"
>INSERT</TT
>, <TT
CLASS="COMMAND"
>UPDATE</TT
>, or <TT
CLASS="COMMAND"
>DELETE</TT
>.
In each case it effectively provides temporary table(s) that can
be referred to in the main command.
</P
></DIV
><DIV
CLASS="SECT2"
><H2
CLASS="SECT2"
><A
NAME="QUERIES-WITH-MODIFYING"
>7.8.2. Data-Modifying Statements in <TT
CLASS="LITERAL"
>WITH</TT
></A
></H2
><P
> You can use data-modifying statements (<TT
CLASS="COMMAND"
>INSERT</TT
>,
<TT
CLASS="COMMAND"
>UPDATE</TT
>, or <TT
CLASS="COMMAND"
>DELETE</TT
>) in <TT
CLASS="LITERAL"
>WITH</TT
>. This
allows you to perform several different operations in the same query.
An example is:
</P><PRE
CLASS="PROGRAMLISTING"
>WITH moved_rows AS (
DELETE FROM products
WHERE
"date" >= '2010-10-01' AND
"date" < '2010-11-01'
RETURNING *
)
INSERT INTO products_log
SELECT * FROM moved_rows;</PRE
><P>
This query effectively moves rows from <TT
CLASS="STRUCTNAME"
>products</TT
> to
<TT
CLASS="STRUCTNAME"
>products_log</TT
>. The <TT
CLASS="COMMAND"
>DELETE</TT
> in <TT
CLASS="LITERAL"
>WITH</TT
>
deletes the specified rows from <TT
CLASS="STRUCTNAME"
>products</TT
>, returning their
contents by means of its <TT
CLASS="LITERAL"
>RETURNING</TT
> clause; and then the
primary query reads that output and inserts it into
<TT
CLASS="STRUCTNAME"
>products_log</TT
>.
</P
><P
> A fine point of the above example is that the <TT
CLASS="LITERAL"
>WITH</TT
> clause is
attached to the <TT
CLASS="COMMAND"
>INSERT</TT
>, not the sub-<TT
CLASS="COMMAND"
>SELECT</TT
> within
the <TT
CLASS="COMMAND"
>INSERT</TT
>. This is necessary because data-modifying
statements are only allowed in <TT
CLASS="LITERAL"
>WITH</TT
> clauses that are attached
to the top-level statement. However, normal <TT
CLASS="LITERAL"
>WITH</TT
> visibility
rules apply, so it is possible to refer to the <TT
CLASS="LITERAL"
>WITH</TT
>
statement's output from the sub-<TT
CLASS="COMMAND"
>SELECT</TT
>.
</P
><P
> Data-modifying statements in <TT
CLASS="LITERAL"
>WITH</TT
> usually have
<TT
CLASS="LITERAL"
>RETURNING</TT
> clauses, as seen in the example above.
It is the output of the <TT
CLASS="LITERAL"
>RETURNING</TT
> clause, <SPAN
CLASS="emphasis"
><I
CLASS="EMPHASIS"
>not</I
></SPAN
> the
target table of the data-modifying statement, that forms the temporary
table that can be referred to by the rest of the query. If a
data-modifying statement in <TT
CLASS="LITERAL"
>WITH</TT
> lacks a <TT
CLASS="LITERAL"
>RETURNING</TT
>
clause, then it forms no temporary table and cannot be referred to in
the rest of the query. Such a statement will be executed nonetheless.
A not-particularly-useful example is:
</P><PRE
CLASS="PROGRAMLISTING"
>WITH t AS (
DELETE FROM foo
)
DELETE FROM bar;</PRE
><P>
This example would remove all rows from tables <TT
CLASS="STRUCTNAME"
>foo</TT
> and
<TT
CLASS="STRUCTNAME"
>bar</TT
>. The number of affected rows reported to the client
would only include rows removed from <TT
CLASS="STRUCTNAME"
>bar</TT
>.
</P
><P
> Recursive self-references in data-modifying statements are not
allowed. In some cases it is possible to work around this limitation by
referring to the output of a recursive <TT
CLASS="LITERAL"
>WITH</TT
>, for example:
</P><PRE
CLASS="PROGRAMLISTING"
>WITH RECURSIVE included_parts(sub_part, part) AS (
SELECT sub_part, part FROM parts WHERE part = 'our_product'
UNION ALL
SELECT p.sub_part, p.part
FROM included_parts pr, parts p
WHERE p.part = pr.sub_part
)
DELETE FROM parts
WHERE part IN (SELECT part FROM included_parts);</PRE
><P>
This query would remove all direct and indirect subparts of a product.
</P
><P
> Data-modifying statements in <TT
CLASS="LITERAL"
>WITH</TT
> are executed exactly once,
and always to completion, independently of whether the primary query
reads all (or indeed any) of their output. Notice that this is different
from the rule for <TT
CLASS="COMMAND"
>SELECT</TT
> in <TT
CLASS="LITERAL"
>WITH</TT
>: as stated in the
previous section, execution of a <TT
CLASS="COMMAND"
>SELECT</TT
> is carried only as far
as the primary query demands its output.
</P
><P
> The sub-statements in <TT
CLASS="LITERAL"
>WITH</TT
> are executed concurrently with
each other and with the main query. Therefore, when using data-modifying
statements in <TT
CLASS="LITERAL"
>WITH</TT
>, the order in which the specified updates
actually happen is unpredictable. All the statements are executed with
the same <I
CLASS="FIRSTTERM"
>snapshot</I
> (see <A
HREF="mvcc.html"
>Chapter 13</A
>), so they
cannot <SPAN
CLASS="QUOTE"
>"see"</SPAN
> one another's effects on the target tables. This
alleviates the effects of the unpredictability of the actual order of row
updates, and means that <TT
CLASS="LITERAL"
>RETURNING</TT
> data is the only way to
communicate changes between different <TT
CLASS="LITERAL"
>WITH</TT
> sub-statements and
the main query. An example of this is that in
</P><PRE
CLASS="PROGRAMLISTING"
>WITH t AS (
UPDATE products SET price = price * 1.05
RETURNING *
)
SELECT * FROM products;</PRE
><P>
the outer <TT
CLASS="COMMAND"
>SELECT</TT
> would return the original prices before the
action of the <TT
CLASS="COMMAND"
>UPDATE</TT
>, while in
</P><PRE
CLASS="PROGRAMLISTING"
>WITH t AS (
UPDATE products SET price = price * 1.05
RETURNING *
)
SELECT * FROM t;</PRE
><P>
the outer <TT
CLASS="COMMAND"
>SELECT</TT
> would return the updated data.
</P
><P
> Trying to update the same row twice in a single statement is not
supported. Only one of the modifications takes place, but it is not easy
(and sometimes not possible) to reliably predict which one. This also
applies to deleting a row that was already updated in the same statement:
only the update is performed. Therefore you should generally avoid trying
to modify a single row twice in a single statement. In particular avoid
writing <TT
CLASS="LITERAL"
>WITH</TT
> sub-statements that could affect the same rows
changed by the main statement or a sibling sub-statement. The effects
of such a statement will not be predictable.
</P
><P
> At present, any table used as the target of a data-modifying statement in
<TT
CLASS="LITERAL"
>WITH</TT
> must not have a conditional rule, nor an <TT
CLASS="LITERAL"
>ALSO</TT
>
rule, nor an <TT
CLASS="LITERAL"
>INSTEAD</TT
> rule that expands to multiple statements.
</P
></DIV
></DIV
><DIV
CLASS="NAVFOOTER"
><HR
ALIGN="LEFT"
WIDTH="100%"><TABLE
SUMMARY="Footer navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
><A
HREF="queries-values.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
><A
HREF="index.html"
ACCESSKEY="H"
>Home</A
></TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
><A
HREF="datatype.html"
ACCESSKEY="N"
>Next</A
></TD
></TR
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
><TT
CLASS="LITERAL"
>VALUES</TT
> Lists</TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
><A
HREF="queries.html"
ACCESSKEY="U"
>Up</A
></TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
>Data Types</TD
></TR
></TABLE
></DIV
></BODY
></HTML
>
