Add support for asynchronous execution

Enterprise / PostgreSQL - Etsuro Fujita [postgresql.org] - 31 March 2021 09:45 UTC

This implements asynchronous execution, which runs multiple parts of a non-parallel-aware Append concurrently rather than serially to improve performance when possible. Currently, the only node type that can be run concurrently is a ForeignScan that is an immediate child of such an Append. In the case where such ForeignScans access data on different remote servers, this would run those ForeignScans concurrently, and overlap the remote operations to be performed simultaneously, so it'll improve the performance especially when the operations involve time-consuming ones such as remote join and remote aggregation.

We may extend this to other node types such as joins or aggregates over ForeignScans in the future.

This also adds the support for postgres_fdw, which is enabled by the table-level/server-level option "async_capable". The default is false.

Robert Haas, Kyotaro Horiguchi, Thomas Munro, and myself. This commit is mostly based on the patch proposed by Robert Haas, but also uses stuff from the patch proposed by Kyotaro Horiguchi and from the patch proposed by Thomas Munro. Reviewed by Kyotaro Horiguchi, Konstantin Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro, Justin Pryzby, and others.

Discussion: https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com Discussion: https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com Discussion: https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com

27e1f14563 Add support for asynchronous execution.
contrib/postgres_fdw/connection.c | 26 +-
contrib/postgres_fdw/expected/postgres_fdw.out | 509 ++++++++++++++++++++++++-
contrib/postgres_fdw/option.c | 6 +-
contrib/postgres_fdw/postgres_fdw.c | 374 ++++++++++++++++--
contrib/postgres_fdw/postgres_fdw.h | 17 +-
contrib/postgres_fdw/sql/postgres_fdw.sql | 195 ++++++++++
doc/src/sgml/config.sgml | 14 +
doc/src/sgml/fdwhandler.sgml | 90 +++++
doc/src/sgml/monitoring.sgml | 5 +
doc/src/sgml/postgres-fdw.sgml | 28 ++
src/backend/commands/explain.c | 3 +
src/backend/executor/Makefile | 1 +
src/backend/executor/README | 40 ++
src/backend/executor/execAmi.c | 4 +
src/backend/executor/execAsync.c | 124 ++++++
src/backend/executor/nodeAppend.c | 461 +++++++++++++++++++++-
src/backend/executor/nodeForeignscan.c | 48 +++
src/backend/nodes/copyfuncs.c | 2 +
src/backend/nodes/outfuncs.c | 2 +
src/backend/nodes/readfuncs.c | 2 +
src/backend/optimizer/path/costsize.c | 1 +
src/backend/optimizer/plan/createplan.c | 41 ++
src/backend/postmaster/pgstat.c | 3 +
src/backend/storage/ipc/latch.c | 9 +
src/backend/utils/misc/guc.c | 10 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/executor/execAsync.h | 25 ++
src/include/executor/nodeAppend.h | 2 +
src/include/executor/nodeForeignscan.h | 4 +
src/include/foreign/fdwapi.h | 14 +
src/include/nodes/execnodes.h | 37 +-
src/include/nodes/plannodes.h | 6 +
src/include/optimizer/cost.h | 1 +
src/include/pgstat.h | 3 +-
src/include/storage/latch.h | 1 +
src/test/regress/expected/explain.out | 7 +
src/test/regress/expected/incremental_sort.out | 2 +
src/test/regress/expected/insert_conflict.out | 4 +-
src/test/regress/expected/sysviews.out | 3 +-
39 files changed, 2068 insertions(+), 57 deletions(-)

Upstream: git.postgresql.org


  • Share