Add GUC to enable compression of full page images stored in WAL.

Enterprise / PostgreSQL - Fujii Masao [postgresql.org] - 11 March 2015 01:52 UTC

When newly-added GUC parameter, wal_compression, is on, the PostgreSQL server compresses a full page image written to WAL when full_page_writes is on or during a base backup. A compressed page image will be decompressed during WAL replay. Turning this parameter on can reduce the WAL volume without increasing the risk of unrecoverable data corruption, but at the cost of some extra CPU spent on the compression during WAL logging and on the decompression during WAL replay.

This commit changes the WAL format (so bumping WAL version number) so that the one-byte flag indicating whether a full page image is compressed or not is included in its header information. This means that the commit increases the WAL volume one-byte per a full page image even if WAL compression is not used at all. We can save that one-byte by borrowing one-bit from the existing field like hole_offset in the header and using it as the flag, for example. But which would reduce the code readability and the extensibility of the feature. Per discussion, it's not worth paying those prices to save only one-byte, so we decided to add the one-byte flag to the header.

This commit doesn't introduce any new compression algorithm like lz4. Currently a full page image is compressed using the existing PGLZ algorithm. Per discussion, we decided to use it at least in the first version of the feature because there were no performance reports showing that its compression ratio is unacceptably lower than that of other algorithm. Of course, in the future, it's worth considering the support of other compression algorithm for the better compression.

Rahila Syed and Michael Paquier, reviewed in various versions by myself, Andres Freund, Robert Haas, Abhijit Menon-Sen and many others.

57aa5b2 Add GUC to enable compression of full page images stored in WAL.
contrib/pg_xlogdump/pg_xlogdump.c | 28 +++--
doc/src/sgml/config.sgml | 24 +++++
src/backend/access/transam/xlog.c | 1 +
src/backend/access/transam/xloginsert.c | 141 +++++++++++++++++++++----
src/backend/access/transam/xlogreader.c | 101 ++++++++++++++++--
src/backend/utils/misc/guc.c | 10 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/xlog.h | 1 +
src/include/access/xlog_internal.h | 2 +-
src/include/access/xlogreader.h | 2 +
src/include/access/xlogrecord.h | 48 ++++++++-
11 files changed, 320 insertions(+), 39 deletions(-)

Upstream: git.postgresql.org

  • Share