Modify pg_basebackup to use a new COPY subprotocol for base backups

Enterprise / PostgreSQL - Robert Haas [postgresql.org] - 18 January 2022 18:47 UTC

In the new approach, all files across all tablespaces are sent in a single COPY OUT operation. The CopyData messages are no longer raw archive content; rather, each message is prefixed with a type byte that describes its purpose, e.g. 'n' signifies the start of a new archive and 'd' signifies archive or manifest data. This protocol is significantly more extensible than the old approach, since we can later create more message types, though not without concern for backward compatibility.

The new protocol sends a few things to the client that the old one did not. First, it sends the name of each archive explicitly, instead of letting the client compute it. This is intended to make it easier to write future patches that might send archives in a format other that tar (e.g. cpio, pax, tar.gz). Second, it sends explicit progress messages rather than allowing the client to assume that progress is defined by the number of bytes received. This will help with future features where the server compresses the data, or sends it someplace directly rather than transmitting it to the client.

The old protocol is still supported for compatibility with previous releases. The new protocol is selected by means of a new TARGET option to the BASE_BACKUP command. Currently, the only supported target is 'client'. Support for additional targets will be added in a later commit.

Patch by me. The patch set of which this is a part has had review and/or testing from Jeevan Ladhe, Tushar Ahuja, Suraj Kharage, Dipesh Pandit, and Mark Dilger.

Discussion: http://postgr.es/m/CA+TgmoaYZbz0=Yk797aOJwkGJC-LK3iXn+wzzMx7KdwNpZhS5g@mail.gmail.com

cc333f3233 Modify pg_basebackup to use a new COPY subprotocol for base backups.
doc/src/sgml/protocol.sgml | 130 +++++++++-
src/backend/replication/basebackup.c | 36 ++-
src/backend/replication/basebackup_copy.c | 277 +++++++++++++++++++-
src/bin/pg_basebackup/pg_basebackup.c | 410 +++++++++++++++++++++++++++---
src/include/replication/basebackup_sink.h | 1 +
5 files changed, 806 insertions(+), 48 deletions(-)

Upstream: git.postgresql.org


  • Share