Change the way pre-reading in external sort's merge phase works

Enterprise / PostgreSQL - Heikki Linnakangas [iki.fi] - 3 October 2016 05:37 UTC

Don't pre-read tuples into SortTuple slots during merge. Instead, use the memory for larger read buffers in logtape.c. We're doing the same number of READTUP() calls either way, but managing the pre-read SortTuple slots is much more complicated. Also, the on-tape representation is more compact than SortTuples, so we can fit more pre-read tuples into the same amount of memory this way. And we have better cache-locality, when we use just a small number of SortTuple slots.

Now that we only hold one tuple from each tape in the SortTuple slots, we can greatly simplify the "batch memory" management. We now maintain a small set of fixed-sized slots, to hold the tuples, and fall back to palloc() for larger tuples. We use this method during all merge phases, not just the final merge, and also when randomAccess is requested, and also in the TSS_SORTEDONTAPE case. In other words, it's used whenever we do an external sort.

Reviewed by Peter Geoghegan and Claudio Freire.

Discussion:

e94568e Change the way pre-reading in external sort's merge phase works.
src/backend/utils/sort/logtape.c | 153 ++++-
src/backend/utils/sort/tuplesort.c | 1216 +++++++++++++-----------------------
src/include/utils/logtape.h | 2 +
3 files changed, 574 insertions(+), 797 deletions(-)

Upstream: git.postgresql.org

  • Share