s3:smbprofile: Replace sysv shmem with tdb

Enterprise / Samba - Volker Lendecke [samba.org] - 6 March 2015 05:31 UTC

What?

This patch gets rid of the central shared memory segment referenced by "profile_p". Instead, every smbd gets a static profile_area where it collects profiling data. Once a second, every smbd writes this profiling data into a record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this database and sums up what it finds.

Why?

At least in my perception sysv IPC has not the best reputation on earth. The code before this patch uses shmat(). Samba ages ago has developed a good abstraction of shared memory: It's called tdb.

The main reason why I started this is that I have a request to become more flexible with profiling data. Samba should be able to collect data per share or per user, something which is almost impossible to do with a fixed structure. My idea is to for example install a profile area per share and every second marshall this into one tdb record indexed by share name. smbstatus -P would then also collect the data and either aggregate them or put them into individual per-share statistics. This flexibility in the data model is not really possible with one fixed structure.

But isn't it slow?

Well, I don't think so. I can't really prove it, but I do believe that on large boxes atomically incrementing a shared memory value for every SMB does show up due to NUMA effects. With this patch the hot code path is completely process-local. Once a second every smbd writes into a central tdb, this of course does atomic operations. But it's once a second, not on every SMB2 read.

There's two places where I would like to improve things: With the current code all smbds wake up once a second. With 10,000 potentially idle smbds this will become noticable. That's why the current only starts the timer when something has changed.

The second place is the tdb traverse: Right now traverse is blocking in the sense that when it has to switch hash chains it will block. With mutexes, this means a syscall. I have a traverse light in mind that works as follows: It assumes a locked hash chain and then walks the complete chain in one run without unlocking in between. This way the caller can do nonblocking locks in the first round and only do blocking locks in a second round. Also, a lot of syscall overhead will vanish. This way smbstatus -P will have almost zero impact on normal operations.

Pair-Programmed-With: Stefan Metzmacher

74a16a1 s3:smbprofile: Replace sysv shmem with tdb
source3/include/smbprofile.h | 116 ++++++++++--
source3/profile/profile.c | 381 +++++++++++++++++++++++++++++++---------
source3/smbd/process.c | 25 +++
source3/smbd/server.c | 2 +
source3/smbd/server_exit.c | 2 +
source3/smbd/smb2_server.c | 4 +-
source3/utils/status_profile.c | 26 +--
7 files changed, 446 insertions(+), 110 deletions(-)

Upstream: gitweb.samba.org

  • Share