Presently, the archive_status directory was scanned for each file to archive. When there are many status files, say because archive_command has been failing for a long time, these directory scans can get very slow. With this change, the archiver remembers several files to archive during each directory scan, speeding things up.
To ensure timeline history files are archived as quickly as possible, XLogArchiveNotify() forces the archiver to do a new directory scan as soon as the .ready file for one is created.
Nathan Bossart, per a long discussion involving many people. It is not clear to me exactly who out of all those people reviewed this particular patch.
Discussion: http://postgr.es/m/CA+TgmobhAbs2yabTuTRkJTq_kkC80-+jw=pfpypdOJ7+gAbQbw@mail.gmail.com Discussion: http://postgr.es/m/620F3CE1-0255-4D66-9D87-0EADE866985A@amazon.com
beb4e9ba16 Improve performance of pgarch_readyXlog() with many status files.
src/backend/access/transam/xlogarchive.c | 14 +++
src/backend/postmaster/pgarch.c | 193 ++++++++++++++++++++++++++-----
src/include/postmaster/pgarch.h | 1 +
3 files changed, 182 insertions(+), 26 deletions(-)