Optimize partial TOAST decompression

Enterprise / PostgreSQL - Tomas Vondra [postgresql.org] - 1 October 2019 12:28 UTC

Commit 4d0e994eed added support for partial TOAST decompression, so the decompression is interrupted after producing the requested prefix. For prefix and slices near the beginning of the entry, this may saves a lot of decompression work.

That however only deals with decompression - the whole compressed entry was still fetched and re-assembled, even though the compression used only a small fraction of it. This commit improves that by computing how much compressed data may be needed to decompress the requested prefix, and then fetches only the necessary part.

We always need to fetch a bit more compressed data than the requested (uncompressed) prefix, because the prefix may not be compressible at all and pglz itself adds a bit of overhead. That means this optimization is most effective when the requested prefix is much smaller than the whole compressed entry.

Author: Binguo Bao

11a078cf87 Optimize partial TOAST decompression
src/backend/access/common/detoast.c | 54 ++++++++++++++++++++++++++++++------
src/common/pg_lzcompress.c | 37 ++++++++++++++++++++++++
src/include/access/toast_internals.h | 1 +
src/include/common/pg_lzcompress.h | 2 ++
4 files changed, 86 insertions(+), 8 deletions(-)

Upstream: git.postgresql.org

  • Share