Basic usage¶
CombinedLogParser
¶
This processor will parse events formatted in Apache Combined log format into Elastic common schema fields.
Example¶
This is an example for configuring gunicorn to emit json logs.
gunicorn.conf.py
import structlog
import structlog_extensions
# --- Structlog logging initialisation code
pre_chain = [
# Add the log level and a timestamp to the event_dict if the log entry
# is not from structlog.
structlog.stdlib.add_log_level,
structlog.stdlib.add_logger_name,
structlog_extensions.processors.CombinedLogParser("gunicorn.access")
]
logconfig_dict = {
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"json_formatter": {
"()": structlog.stdlib.ProcessorFormatter,
"processor": structlog.processors.JSONRenderer(),
"foreign_pre_chain": pre_chain,
}
},
"handlers": {
"error_console": {
"class": "logging.StreamHandler",
"formatter": "json_formatter",
},
"console": {
"class": "logging.StreamHandler",
"formatter": "json_formatter",
}
},
}
NestedDictJSONRenderer
¶
This processor will convert key names using a specified separator into nested dictionaries prior to rendering the
output as JSON using the structlog JSONRenderer. This processor can for example convert Elastic Common Schema namespaced
keynames produced by the CombinedLogParser
into the nested JSON format that ECS specifies. This processor must be the
final processor in a chain because it renders the output as a string instead of passing along an event dictionary.
Example¶
When using this logging initialisation:
# --- std logging initialisation code using structlog rendering import structlog import structlog_extensions pre_chain = [ # Add the log level and a timestamp to the event_dict if the log entry # is not from structlog. structlog.stdlib.add_log_level, structlog.stdlib.add_logger_name, structlog_extensions.processors.CombinedLogParser("gunicorn.access") ] logging.dict_config( { "version": 1, "disable_existing_loggers": False, "formatters": { "json_formatter": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog_extensions.processors.NestedDictJSONRenderer('.'), "foreign_pre_chain": pre_chain, } }, "handlers": { "error_console": { "class": "logging.StreamHandler", "formatter": "json_formatter", }, "console": { "class": "logging.StreamHandler", "formatter": "json_formatter", } }, })These entries (produced by
structlog_extensions.processors.CombinedLogParser
):{ 'http.request.method': 'get', 'http:.request.referrer': 'http://www.example.com', 'http.version': '1.0'}`will be transformed into the following nested json structure:
{ 'http': { 'version': '1.0', 'request': { 'method': 'get', 'referrer': 'http://www.example.com'} } }