Question

How can I extract certain lines from a logfile (txt)

  • 13 December 2019
  • 4 replies
  • 16 views
I

Hi, I'm new at FME and want to do the following:

there's a process running at our server which is logged every night. We would like to get a warning when certain errorcodes pop up in the logfile.

For those (= [ERROR] and [WARN ]) of that night (filter on date of today) the whole line should be placed in another file.

The file looks like this:

2019-12-10 12:45:58,644 [WARN ] TrfAgiv.TrfCeviErrorHelper terreinobject 7654048 - CRCODE 'aardTerreinobject' - 'grbAdmPerceel' (6004) wordt niet verwerkt (NAARGEM = 0)

2019-12-10 12:45:58,646 [ERROR] TrfAgiv.TrfCeviErrorHelper terreinobject-huisnummer 20903264 - Niet alle noodzakelijke bovenliggende gegevens werden gevonden.

2019-12-10 12:45:58,647 [DEBUG] TrfAgiv.TrfCeviHelperEf terreinobject-huisnummer id: 20903322

2019-12-10 12:45:58,874 [DEBUG] TrfAgiv.TrfCeviHelperEf --> straatnaam id: 55794 opslaan, synchroniseren: False

 

There's an exemple of the file attached.

What would be the best way to go?

Thanks!

Isa

logfile.txt

1 Attachment

4 replies

R
Badge

You can have a process that runs after the workspace from your log is finished, that reads this log as textfile and use a tester to look for lines with the Errors and Warns (Using "Contains") that you want to filter then save it.

 

Or a LogMessageStreamer in your workspace, then select the TAG that you want and save it. You can be more specific using a tester using "Contains" in the log message after the TAG selection.

erik_jan
Rank
Userlevel 2
Badge +12

FME has the option to filter log messages.

Have a look at the options menu in FME Desktop:

warrendev
Rank
Badge +16

Using the python shutdown script is the way I go about parsing the log files.

 

 

Here is some good information going over how to set it up.

 

https://knowledge.safe.com/questions/60186/capturing-errors-in-fme-server-log-files.html

 

I

Thanks for the quick reply everybody!

@rodrigo_ferrao I think I will go with your first option. I was trying with the stringsearcher before but I kept getting stuck building the regular exporession.

@erik_jan Thanks for the answer. But the process is not runing in FME (server). It's a totally different process and by scanning that log we hope to get a better view on things when it goes wrong. (kind of a black box right now)

@warrengis thanks for the answer: I'm almost there using the tester transformer but this info can get in handy another time!

 

 

Reply


Community Stats

28,666
Posts
109,340
Replies
36,535
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.