Allow user control of CTE materialization, and change the default behavior

Enterprise / PostgreSQL - Tom Lane [sss.pgh.pa.us] - 16 February 2019 21:11 EST

Historically we've always materialized the full output of a CTE query, treating WITH as an optimization fence (so that, for example, restrictions from the outer query cannot be pushed into it). This is appropriate when the CTE query is INSERT/UPDATE/DELETE, or is recursive; but when the CTE query is non-recursive and side-effect-free, there's no hazard of changing the query results by pushing restrictions down.

Another argument for materialization is that it can avoid duplicate computation of an expensive WITH query --- but that only applies if the WITH query is called more than once in the outer query. Even then it could still be a net loss, if each call has restrictions that would allow just a small part of the WITH query to be computed.

Hence, let's change the behavior for WITH queries that are non-recursive and side-effect-free. By default, we will inline them into the outer query (removing the optimization fence) if they are called just once. If they are called more than once, we will keep the old behavior by default, but the user can override this and force inlining by specifying NOT MATERIALIZED. Lastly, the user can force the old behavior by specifying MATERIALIZED; this would mainly be useful when the query had deliberately been employing WITH as an optimization fence to prevent a poor choice of plan.

Andreas Karlsson, Andrew Gierth, David Fetter

Discussion: https://postgr.es/m/87sh48ffhb.fsf@news-spur.riddles.org.uk

608b167f9f Allow user control of CTE materialization, and change the default behavior.
contrib/pg_stat_statements/pg_stat_statements.c | 1 +
contrib/postgres_fdw/expected/postgres_fdw.out | 4 +-
contrib/postgres_fdw/sql/postgres_fdw.sql | 4 +-
doc/src/sgml/queries.sgml | 84 ++++++++++-
doc/src/sgml/ref/select.sgml | 57 ++++++--
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/nodes/readfuncs.c | 1 +
src/backend/optimizer/plan/planner.c | 4 +-
src/backend/optimizer/plan/subselect.c | 178 +++++++++++++++++++++++-
src/backend/parser/gram.y | 14 +-
src/backend/utils/adt/ruleutils.c | 14 +-
src/include/catalog/catversion.h | 2 +-
src/include/nodes/parsenodes.h | 8 ++
src/test/regress/expected/rowsecurity.out | 9 +-
src/test/regress/expected/rowtypes.out | 4 +-
src/test/regress/expected/rules.out | 5 +-
src/test/regress/expected/subselect.out | 156 +++++++++++++++++++++
src/test/regress/sql/rowsecurity.sql | 9 +-
src/test/regress/sql/rowtypes.sql | 4 +-
src/test/regress/sql/rules.sql | 5 +-
src/test/regress/sql/subselect.sql | 58 ++++++++
23 files changed, 580 insertions(+), 44 deletions(-)

Upstream: git.postgresql.org


  • Share