Log Analyzer

The SSSD 2.6.0 release includes a new log parsing tool for SSSD debug log analysis. This analyzer tool can be called using the sssctl analyze command, the log analysis tool primarily acts as a grep front-end.

Identifying SSSD failures can be a difficult task without knowledge of SSSD internal components. If an administrator or SSSD user doesn’t know what to look for, it may become a very slow and time consuming process. The sssctl analyze tool improves the overall troubleshooting workflow for administrators, users, and anyone needing to review SSSD debug logs. The initial use case is to extract and print SSSD logs pertaining only to certain client requests across responder, backend, and child sssd processes.

  • debug_level should be set to at least 7 in the [$responder] section, and [domain/$domain] section to enable analyze log parsing functionality.

  • Logs to analyze must be from compatible SSSD version built with tevent chain ID support.

The sssctl analyze command uses git-like subcommand invocation.

# sssctl analyze MODULE [ARGS]

The only supported module currently is the request module. The request module is used to print logs associated with client requests made to SSSD.

Note

Additional modules to be added contingent on future SSSD development plans

# sssctl analyze request
Usage: sss_analyze.py request [OPTIONS] COMMAND [ARGS]...

Request module

Options:
--help  Show this message and exit.

Commands:
list
show

sssctl analyze request operates in two different primary modes shown in the table below. list mode is intended to use first, to find the client ID which can then be passed to the show command. The client ID can also be found in the log files (search for [CID #] tag).

mode

mode functionality

additional options

list

Output list of recent client requests made to sssd

-v, --verbose

Enables verbose output

--pam

Filter only PAM requests

show

Print logs pertaining to a provided client ID number

--merge

Merge logs together sorted by timestamp (requires debug_microseconds = True)

--child

Include child process logs

--pam

Track only PAM requests

Warning

Requests which return from the SSSD memory cache will not be logged, and therefore not tracked by the analyzer

Print client command request list, NSS (default), or PAM

# sssctl analyze request list
# sssctl analyze request list --pam

Note

SSSD tracks identity user/group information (id, getent) in the NSS separately from PAM responder user authentication(su, ssh). The CID in the NSS responder is independent of the CID in the PAM responder, you will see overlapping numbers when analyzing NSS and PAM requests. Use –pam option when necessary.

Verbose list output

# sssctl analyze request list -v

Track individual NSS request id number 20

# sssctl analyze request show 20

Track PAM request including child process logs

# sssctl analyze request show 13 --child --pam

Track individual PAM request

# sssctl analyze request show 20 --pam

Supports --logger=journald configurations

# sssctl analyze --source=journald request list

Analyze logs extracted, or sent from another user.

# sssctl analyze --logdir=/path/to/var/log/sssd request list

SSSD development would appreciate any positive, or negative, feedback on the log analyzer tool. One reason the log analyzer tool is written in python is to encourage contributions. Improvement suggestions and Pull Requests are welcome!