SSSD consists of multiple processes, namely:
- The monitor
This is the main
There is one process for each responder with distinguishable name, for example
There is one process for each configured SSSD domain. Each process is named
sssd_beand the domain is determined by the
- Child processes
These are special processes that are usually invoked from backends to perform individual tasks such as Kerberos authentication or reading GPO data. They are located in
/usr/libexec/sssdand suffixed with
_child, for example
The following snippet shows a list of running SSSD process with three responders (nss, pam and pac) and three different domains (ldap.test, samba.test and ipa.test).
# ps aux | grep sssd root 44 0.4 0.0 29248 11308 ? Ss 10:51 0:00 /usr/sbin/sssd -i --logger=files root 57 0.2 0.0 24728 11412 ? S 10:51 0:00 /usr/libexec/sssd/sssd_be --domain ldap.test --uid 0 --gid 0 --logger=files root 58 0.3 0.1 88920 20748 ? S 10:51 0:00 /usr/libexec/sssd/sssd_be --domain samba.test --uid 0 --gid 0 --logger=files root 59 0.3 0.1 73300 19872 ? S 10:51 0:00 /usr/libexec/sssd/sssd_be --domain ipa.test --uid 0 --gid 0 --logger=files root 60 0.3 0.2 54408 37984 ? S 10:51 0:00 /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --logger=files root 61 0.0 0.0 28416 9864 ? S 10:51 0:00 /usr/libexec/sssd/sssd_pam --uid 0 --gid 0 --logger=files root 62 0.1 0.0 74416 16220 ? S 10:51 0:00 /usr/libexec/sssd/sssd_pac --uid 0 --gid 0 --logger=files # ll /usr/libexec/sssd -rwxr-xr-x. 1 root root 32584 Jan 25 12:03 gpo_child -rwxr-xr-x. 1 root root 71 Mar 10 12:03 krb5_child -rwxr-x---. 1 root root 119344 Jan 25 12:03 krb5_child.exe -rwxr-x---. 1 root root 53136 Jan 25 12:03 ldap_child -rwxr-xr-x. 1 root root 69416 Jan 25 12:03 p11_child -rwxr-x---. 1 root root 28424 Jan 25 12:03 proxy_child -rwxr-x---. 1 root root 32544 Jan 25 12:03 selinux_child -rwxr-xr-x. 1 root root 72 Jan 25 10:44 sss_analyze -rwxr-xr-x. 1 root root 15960 Jan 25 12:03 sss_signal -rwxr-xr-x. 1 root root 208088 Jan 25 12:03 sssd_autofs -rwxr-xr-x. 1 root root 245728 Jan 25 12:03 sssd_be -rwxr-xr-x. 1 root root 15992 Jan 25 12:03 sssd_check_socket_activated_responders -rwxr-xr-x. 1 root root 303048 Jan 25 12:03 sssd_ifp -rwxr-xr-x. 1 root root 212160 Jan 25 12:03 sssd_kcm -rwxr-xr-x. 1 root root 287696 Jan 25 12:03 sssd_nss -rwxr-xr-x. 1 root root 203848 Jan 25 12:03 sssd_pac -rwxr-xr-x. 1 root root 278368 Jan 25 12:03 sssd_pam -rwxr-xr-x. 1 root root 216096 Jan 25 12:03 sssd_ssh -rwxr-xr-x. 1 root root 216336 Jan 25 12:03 sssd_sudo
See the SSSD Architecture for more information about the different processes that makes SSSD.
Disabling the watchdog
There is a special deadlock guard called the watchdog that runs inside each SSSD process. Its purpose is to make sure that the process is still responsive and kill it if it is not so it can be restarted again by the monitor.
The watchdog uses timers and signals mechanisms therefore it interrupts the process every now and then which makes it difficult to debug the process in the debugger, not to mention that the process will be killed if you keep it hanging on a breakpoint for a longer time.
You could tell the debugger to don’t react on the
SIG34 signal so it is
passed directly to the process.
$ sudo gdb program $(pidof sssd_nss) -ex 'handle SIG34 nostop noprint'
This however will not solve the problem when the process is waiting on a
breakpoint and therefore it eventually gets terminated by the watchdog. To solve
this, it is recommended to set a long watchdog interval using the
sssd.conf, for example to 30000 seconds.
[sssd] config_file_version = 2 services = nss, pam domains = ldap.test, samba.test, ipa.test user = root [nss] timeout=30000 ... [pam] timeout=30000 ... [domain/ldap.test] timeout=30000 ... [domain/samba.test] timeout=30000 ... [domain/ipa.test] timeout=30000 ...
Attaching debugger to a running process
There is only one process for each responder that can be distinguished by name, therefore it is simple to attach a debugger to the running process. For example:
$ sudo gdb program `pgrep sssd_nss`
There can be multiple backend
sssd_be processes and we need to use the
--domain parameter to distinguish between them. Therefore we want to use the
-f/--full option of the
pgrep command to make it match the whole command
line and not only the process name. The following snippet shows how to attach
the debugger to
$ sudo gdb program `pgrep -f "sssd_be.+ldap.test"`
We created set of gdb extensions for SSSD that provides pretty printers to some difficult SSSD structures.
Debugging forked process from its start
Usually, you will suffice with attaching debugger to a running process. But
sometimes, you want to debug the process from the very beginning. This applies
especially to the different child processes that are forked from
perform various stuff like Kerberos authentication.
We can use gdbserver for that which provides a remote access for
can either listen on given device or a TCP connection which we will use in our
examples. To install it, run:
$ sudo dnf install -y gdb-gdbserver
$ sudo yum install -y gdb-gdbserver
$ sudo apt-get install -y gdbserver
Now, we need to create a wrapper that would execute the process in the gdbserver that will listen on a specific port.
$ process=/usr/libexec/sssd/krb5_child $ sudo mv "$process" "$process.exe" $ sudo cat << 'EOF' > $process #!/bin/bash exec gdbserver :1234 /usr/libexec/sssd/krb5_child.exe "$@" EOF $ sudo chmod +x "$process"
Now we need to increase timeout of the child process in order to avoid its
termination during the debugging session. We’ll use the
[domain/ipa.test] timeout=30000 krb5_auth_timeout=30000
Now you can start SSSD and let it get to the process that you want to debug, it
krb5_child in our example so we can try authenticate as some user,
su email@example.com. Then start gdb and attach it to the server:
$ sudo gdb /usr/libexec/sssd/krb5_child.exe -ex "target remote :1234" -ex "b main" -ex "c"
Another way of debugging a child process is to use
child when debugging the parent process. It will tell
gdb to start
debugging the child once it is forked off the parent. See gdb manual for more