Support GROUPING SETS, CUBE and ROLLUP.

Enterprise / PostgreSQL - Andres Freund [anarazel.de] - 15 May 2015 20:46 UTC

This SQL standard functionality allows to aggregate data by different GROUP BY clauses at once. Each grouping set returns rows with columns grouped by in other sets set to NULL.

This could previously be achieved by doing each grouping as a separate query, conjoined by UNION ALLs. Besides being considerably more concise, grouping sets will in many cases be faster, requiring only one scan over the underlying data.

The current implementation of grouping sets only supports using sorting for input. Individual sets that share a sort order are computed in one pass. If there are sets that don't share a sort order, additional sort & aggregation steps are performed. These additional passes are sourced by the previous sort step; thus avoiding repeated scans of the source data.

The code is structured in a way that adding support for purely using hash aggregation or a mix of hashing and sorting is possible. Sorting was chosen to be supported first, as it is the most generic method of implementation.

Instead of, as in an earlier versions of the patch, representing the chain of sort and aggregation steps as full blown planner and executor nodes, all but the first sort are performed inside the aggregation node itself. This avoids the need to do some unusual gymnastics to handle having to return aggregated and non-aggregated tuples from underlying nodes, as well as having to shut down underlying nodes early to limit memory usage. The optimizer still builds Sort/Agg node to describe each phase, but they're not part of the plan tree, but instead additional data for the aggregation node. They're a convenient and preexisting way to describe aggregation and sorting. The first (and possibly only) sort step is still performed as a separate execution step. That retains similarity with existing group by plans, makes rescans fairly simple, avoids very deep plans (leading to slow explains) and easily allows to avoid the sorting step if the underlying data is sorted by other means.

A somewhat ugly side of this patch is having to deal with a grammar ambiguity between the new CUBE keyword and the cube extension/functions named cube (and rollup). To avoid breaking existing deployments of the cube extension it has not been renamed, neither has cube been made a reserved keyword. Instead precedence hacking is used to make GROUP BY cube(..) refer to the CUBE grouping sets feature, and not the function cube(). To actually group by a function cube(), unlikely as that might be, the function name has to be quoted.

Needs a catversion bump because stored rules may change.

Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund

f3d3118 Support GROUPING SETS, CUBE and ROLLUP.
contrib/pg_stat_statements/pg_stat_statements.c | 21 +
doc/src/sgml/func.sgml | 70 +-
doc/src/sgml/queries.sgml | 175 +++
doc/src/sgml/ref/select.sgml | 33 +-
src/backend/catalog/sql_features.txt | 6 +-
src/backend/commands/explain.c | 160 ++-
src/backend/executor/execQual.c | 63 ++
src/backend/executor/execUtils.c | 5 +-
src/backend/executor/nodeAgg.c | 1315 +++++++++++++++++------
src/backend/lib/Makefile | 3 +-
src/backend/lib/bipartite_match.c | 161 +++
src/backend/nodes/copyfuncs.c | 38 +
src/backend/nodes/equalfuncs.c | 32 +
src/backend/nodes/list.c | 26 +
src/backend/nodes/makefuncs.c | 15 +
src/backend/nodes/nodeFuncs.c | 51 +
src/backend/nodes/outfuncs.c | 32 +
src/backend/nodes/readfuncs.c | 37 +
src/backend/optimizer/path/allpaths.c | 3 +-
src/backend/optimizer/path/indxpath.c | 3 +-
src/backend/optimizer/plan/analyzejoins.c | 28 +-
src/backend/optimizer/plan/createplan.c | 7 +-
src/backend/optimizer/plan/planagg.c | 2 +-
src/backend/optimizer/plan/planner.c | 829 ++++++++++++--
src/backend/optimizer/plan/setrefs.c | 30 +-
src/backend/optimizer/plan/subselect.c | 57 +-
src/backend/optimizer/prep/prepjointree.c | 1 +
src/backend/optimizer/prep/prepunion.c | 7 +-
src/backend/optimizer/util/clauses.c | 1 +
src/backend/optimizer/util/pathnode.c | 3 +-
src/backend/optimizer/util/tlist.c | 22 +
src/backend/optimizer/util/var.c | 24 +
src/backend/parser/analyze.c | 5 +-
src/backend/parser/gram.y | 123 ++-
src/backend/parser/parse_agg.c | 723 ++++++++++++-
src/backend/parser/parse_clause.c | 504 ++++++++-
src/backend/parser/parse_expr.c | 5 +
src/backend/parser/parse_target.c | 4 +
src/backend/rewrite/rewriteHandler.c | 2 +-
src/backend/rewrite/rewriteManip.c | 23 +
src/backend/utils/adt/ruleutils.c | 217 +++-
src/backend/utils/adt/selfuncs.c | 13 +-
src/include/catalog/catversion.h | 2 +-
src/include/commands/explain.h | 2 +
src/include/lib/bipartite_match.h | 44 +
src/include/nodes/execnodes.h | 35 +-
src/include/nodes/makefuncs.h | 2 +
src/include/nodes/nodes.h | 3 +
src/include/nodes/parsenodes.h | 69 ++
src/include/nodes/pg_list.h | 3 +-
src/include/nodes/plannodes.h | 2 +
src/include/nodes/primnodes.h | 35 +
src/include/nodes/relation.h | 3 +
src/include/optimizer/planmain.h | 1 +
src/include/optimizer/tlist.h | 3 +
src/include/parser/kwlist.h | 4 +
src/include/parser/parse_agg.h | 5 +
src/include/parser/parse_clause.h | 1 +
src/include/utils/selfuncs.h | 2 +-
src/test/regress/expected/groupingsets.out | 590 ++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/groupingsets.sql | 165 +++
63 files changed, 5245 insertions(+), 608 deletions(-)

Upstream: git.postgresql.org

  • Share