The LogNormalizer custom transformer is designed to read and normalize log files.

It attempts to extract and standardize key information such as the timestamp and severity level, producing structured attributes that can be used for further processing in FME.

Originally developed to handle FME Flow log files, it can also be applied to log files from other applications.

How It Works

Each input feature triggers the parsing of a log file.

Typically, it is combined with the Directory and File Pathnames reader to list all log files within a directory and its subdirectories.

The transformer uses different regular expressions to detect:

The severity level of each log line

The timestamp format

When a timestamp is found, it is converted and normalized into both FME and ISO 8601 formats.

If no timestamp is detected for a line, the transformer attempts to reuse the timestamp of the previous line and marks it as "unknown".

Parameters

Log File(s) to Parse

Type: File path

Description: Path(s) to the log file(s) to be parsed.

Output Attributes

Each log line is output as a feature with the following attributes:

Attribute

Description

_log.timestampFME

Timestamp converted to FME datetime format

_log.timestampISO

Timestamp converted to ISO 8601 format

_log.isDateTimeKnown

yes / no – indicates whether a timestamp was detected for this line. If no, the timestamp is inherited from the previous line.

_log.severity

Detected severity level: INFO, WARN, ERROR, or UNKNOWN

_log.message

The log message text, without the timestamp

_log.dataset

The source log file from which the line originates

Typical Usage

Normalize FME Flow log files for further analysis

Parse third-party application logs and extract structured information

Combine with readers like Directory and File Pathnames to batch-process multiple log files