dlpt.log module

Common wrappers and helper functions to simplify most common use cases of builtin ‘logging’ module.

  1. Create logger and (optionally) set it as a default dlpt logger.
    Note:

    ‘Default’ logger means that (once initialized), any dlpt log functions, such as info() and warning() will log to this default logger. Once initialized as default logger, any other file can simply use dlpt log functions without setting up logger.

    Example:
    >>> #file1.py
>>> logger = dlpt.log.create_logger("my_logger")
>>> dlpt.log.add_file_hdlr(logger, "dlpt_example.log")
>>> dlpt.log.info("Log from file1.")
>>>
>>> #file2.py
>>> dlpt.log.info("Log from file2.")
  2. Use dlpt.log.add_*() functions to add common handlers to any

    logging.Logger instance: console (terminal) handler, file handler, rotating file handler, server socket handler (for logs when multiple processes are used).

## Logging server To unify logs from multiple processes, user can create logging server via function create_log_server_proc(). This process will create a custom logger with file handler and open a socket connection on a designated port. Any logger (from any process) that has configured logging server handler (via add_logging_server_hdlr()) will push logs to this logging server and therefore to a shared log file. Note that log statements order might not be exactly the same as this is OS-dependant.

dlpt.log.create_logger(name: str | None = None, set_as_default: bool = True, level: int | None = 10) → Logger

Create new logger instance with the given ‘name’ and optionally set it as a default logger whenever dlpt.log.* log functions are invoked.

Parameters:

  • name – Optional name of the new logger instance or root by default.

  • set_as_default – If True, created logger instance will be set as a default logger whenewer dlpt.log.* log functions are invoked.

  • level – log level for this specific logger. If None, everything is logged (logging.DEBUG level).

dlpt.log.add_console_hdlr(logger: Logger | str, fmt: Formatter | None = None, level: int = 10) → StreamHandler

Add console handler to logger instance.

Note

Create custom formatter with: logging.Formatter(<formatter>, datefmt=<time formatter>)

Parameters:

  • logger – logger instance or logger name.

  • fmt – Optional custom formatter for created handler. By default, DEFAULT_FORMATTER and DEFAULT_FORMATTER_TIME is used.

  • level – set log level for this specific handler. By default, everything is logged (DEBUG level).

Returns:

Created console (stream) handler object.

dlpt.log.add_file_hdlr(logger: Logger | str, file_name: str | None = None, dir_path: str | None = None, fmt: Formatter | None = None, level: int = 10, mode: str = 'w') → Tuple[FileHandler, str]

Add file handler to logger instance.

Parameters:

  • logger – logger instance or logger name.

  • file_name – name of a log file. If there is no file extension, default DEFAULT_LOG_FILE_EXT is appended. If None, logger name is used as a file name.

  • dir_path – path to a directory where logs will be stored. If None, path is fetched with get_default_log_dir().If log directory does not exist, it is created.

  • fmt – Optional custom formatter for created handler. By default, DEFAULT_FORMATTER and DEFAULT_FORMATTER_TIME is used.

  • level – Log level for this specific handler. By default, everything is logged (DEBUG level).

  • mode – file open mode (“w”, “a”, … See logging docs.).

Returns:

(created file handler, file path).

Return type:

A tuple

dlpt.log.add_rotating_file_hdlr(logger: Logger | str, file_name: str | None = None, dir_path: str | None = None, fmt: Formatter | None = None, level: int = 10, max_size_kb: int = 100, backup_count: int = 1) → Tuple[RotatingFileHandler, str]

Add rotating file handler to logger instance.

Parameters:

  • logger – logger instance or logger name.

  • file_name – name of a log file. If there is no file extension, default DEFAULT_LOG_FILE_EXT is appended. If None, logger name is used as a file name.

  • dir_path – path to a directory where logs will be stored. If None, path is fetched with get_default_log_dir(). If log directory does not exist, it is created.

  • max_size_kb – number of KB at which rollover is performed on a current log file.

  • backup_count – number of files to store (if file with given name already exists).

  • fmt – Optional custom formatter for created handler. By default, DEFAULT_FORMATTER and DEFAULT_FORMATTER_TIME is used.

  • level – Log level for this specific handler. By default, everything is logged (DEBUG level).

Returns:

(created rotating file handler, file path).

Return type:

A tuple

dlpt.log.add_logging_server_hdlr(logger: Logger | str, port: int = 9020, fmt: Formatter | None = None, level: int = 10) → _SocketHandler

Add log socket handler to this logger instance. This function assume that log socket server is already initialized.

Parameters:

  • logger – logger instance or logger name.

  • port – socket port where logger writes data to.

  • fmt – Optional custom formatter for created handler. By default, DEFAULT_FORMATTER and DEFAULT_FORMATTER_TIME is used.

  • level – Log level for this specific handler. By default, everything is logged (DEBUG level).

dlpt.log.get_log_file_paths(logger: Logger | str) → List[str]

Return log file paths of logging.FileHandler(s) of a given logger instance.

Parameters:

logger – logger instance or logger name.

Returns:

List of loggers file handlers file paths.

dlpt.log.get_rotating_log_file_paths(logger: Logger | str) → List[str]

Return log file paths of logging.RotatingFileHandler(s) of a given logger instance.

Parameters:

logger – logger instance or logger name.

Returns:

List of loggers rotating file handlers file paths.

dlpt.log.get_default_logger() → Logger | None

Get default logger instance object (if set).

Returns:

Current logger instance when dlpt.log.* log functions are invoked.

dlpt.log.set_default_logger(logger: Logger)

Set default logger instance.

Parameters:

logger – logger instance or logger name.

Returns:

Current logger instance object.

dlpt.log.get_file_name(logger: Logger | str, file_name: str | None = None) → str

Determine log file name, based on a current logger name or given file_name.

Parameters:

  • logger – logger instance or logger name.

  • file_name – if given, this name is checked (for extension) or default file name is created from logger name.

Returns:

File name with extension.

dlpt.log.get_default_log_dir() → str

Get default log directory path: <cwd>/DEFAULT_LOG_DIR_NAME.

Returns:

Path to a directory where logs are usually created.

class dlpt.log.ReleaseFileLock(logger: Logger | str, file_path: str)

Bases: object

__init__(logger: Logger | str, file_path: str)

Temporary release file handler of logging file streams to be able to copy file (for example, log file is locked by logger on Windows.) Use as a context manager.

Parameters:

  • logger – logger instance or logger name.

  • file_path – logging file path

Example

>>> with dlpt.log.ReleaseFileLock(logger, file_path):
>>>     shutil.move(file_path, tmp_path)
class dlpt.log.LoggingServer(port: int = 9020)

Bases: ThreadingTCPServer

Simple TCP socket-based logging server (receiver). All messages are pickled at client TX side and un-pickled here. Any received message is than further handled by handleLogRecord().

Sets Allow address reuse.

allow_reuse_address = True
daemon_threads = True
__init__(port: int = 9020)

Init log socket server. By default, this server logs all received messages to a root logger as configured.

Note

Use add_logging_server_hdlr() function to add socket handler to any logger instance and logs will be automatically pushed to the created LoggingServer process. Note that ports must be properly configured for logging to work.

Parameters:

port – port where socket server reads data from.

handle_error(request, client_address)

Handle an error gracefully. May be overridden.

The default is to print a traceback and continue.

dlpt.log.create_log_server_proc(file_path: str, port: int = 9020) → int

Create socket server logger subprocess that logs all received messages on a given port socket to a log file handler.

Parameters:

  • file_path – absolute path to a log file, including extension.

  • port – port where socket server will listen.

Note

port number must be unique - this means that the default create_log_server_proc() can be called only once. Further socket servers (of any process) and logger handlers must set ports manually.

Returns:

PID of created socket server subprocess.

dlpt.log.log_server_shutdown_request(logger: Logger | str, pid: int, timeout_sec: int = 3) → bool

Send ‘unique’ log message that indicates to server to perform shutdown - close all connections and finish process.

Parameters:

  • logger – logger instance or logger name. Note that logging server handler must be available in this logger.

  • pid – logging server PID.

  • timeout_sec – time to wait until logging server PID is checked for status. In case of 0, function return value is not relevant.

Returns:

True if server was successfully stopped (PID not alive anymore), False otherwise.

dlpt.log.debug(msg: str, logger: Logger | str | None = None, *args, **kwargs)

Log to a given or default logger (if available) with ‘DEBUG’ level.

Parameters:

  • msg – message to log.

  • logger – logger instance or logger name. If not set, default logger is used (as set by create_logger()).

dlpt.log.info(msg: str, logger: Logger | str | None = None, *args, **kwargs)

Log to a given or default logger (if available) with ‘INFO’ level.

Parameters:

  • msg – message to log.

  • logger – logger instance or logger name. If not set, default logger is used (as set by create_logger()).

dlpt.log.warning(msg: str, logger: Logger | str | None = None, *args, **kwargs)

Log to a given or default logger (if available) with ‘WARNING’ level.

Parameters:

  • msg – message to log.

  • logger – logger instance or logger name. If not set, default logger is used (as set by create_logger()).

dlpt.log.warning_with_traceback(msg: str, logger: Logger | str | None = None, *args, **kwargs)

Log to a given or default logger (if available) with ‘WARNING’ level and append exception info traceback at the end (if available).

Parameters:

  • msg – message to log.

  • logger – logger instance or logger name. If not set, default logger is used (as set by create_logger()).

dlpt.log.error(msg: str, logger: Logger | str | None = None, *args, **kwargs)

Log to a given or default logger (if available) with ‘ERROR’ level.

Parameters:

  • msg – message to log.

  • logger – logger instance or logger name. If not set, default logger is used (as set by create_logger()).

dlpt.log.error_with_traceback(msg: str, logger: Logger | str | None = None, *args, **kwargs)

Log to a given or default logger (if available) with ‘ERROR’ level and append exception info traceback at the end (if available).

Parameters:

  • msg – message to log.

  • logger – logger instance or logger name. If not set, default logger is used (as set by create_logger()).

dlpt.log.critical(msg: str, logger: Logger | str | None = None, *args, **kwargs)

Log to a given or default logger (if available) with ‘CRITICAL’ level.

Parameters:

  • msg – message to log.

  • logger – logger instance or logger name. If not set, default logger is used (as set by create_logger()).

dlpt.log.critical_with_traceback(msg: str, logger: Logger | str | None = None, *args, **kwargs)

Log to a given or default logger (if available) with ‘CRITICAL’ level and append exception info traceback at the end (if available).

Parameters:

  • msg – message to log.

  • logger – logger instance or logger name. If not set, default logger is used (as set by create_logger()).