Skip to main content
Solved

Querying log folders - writing all logs to one folder

  • January 5, 2024
  • 3 replies
  • 50 views
sam_appleton
Contributor
Forum|alt.badge.img+7

Hi,

I'm trying to ingest log files into Azure monitor so I can send out organisation wide notifications but i've run into an issue.

 

When I look into the machine that flow is installed on, I can see that it is storing logs in folders in batches of 1000 and it will create new folder each time it hits a multiple of 1000. I can process files into Azure, but this doesn't seem to work with a wildcard in the /current/ folder - it needs a folder that already exists.

 

Is there a way to write these logs all to the same folder?

The file pattern/query i'm using:

 C:\\ProgramData......logs\\engine\\current\\jobs\\*\\job_*.txt

Many thanks,

 

Sam

Best answer by david_r

Thank you for the response, David. Unfortunately the cloud team has dictated that this all must go through Azure. I'll try the double asterisk and see if that helps at all.

If you're using software supplied by Azure to ingest the log files you may have better luck asking on an Azure specific forum. I don't think it's possible or even desirable to have FME keep all the (potentially tens of thousands) job logs in a single directory.

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

3 replies

david_r
Celebrity
  • david_rCelebrity
  • 8392 replies
  • January 5, 2024

If you're using FME to iterate over the log files, you can use the following pattern to look for files in all sub-directories:

 C:\ProgramData......logs\engine\current\jobs\**\job_*.txt

Note the double asterisk to match any directory.

sam_appleton
Contributor
Forum|alt.badge.img+7
  • sam_appletonContributor
  • Author
  • Contributor
  • 21 replies
  • January 5, 2024

If you're using FME to iterate over the log files, you can use the following pattern to look for files in all sub-directories:

 C:\ProgramData......logs\engine\current\jobs\**\job_*.txt

Note the double asterisk to match any directory.

Thank you for the response, David. Unfortunately the cloud team has dictated that this all must go through Azure. I'll try the double asterisk and see if that helps at all.

david_r
Celebrity
  • david_rCelebrity
  • 8392 replies
  • Best Answer
  • January 5, 2024

Thank you for the response, David. Unfortunately the cloud team has dictated that this all must go through Azure. I'll try the double asterisk and see if that helps at all.

If you're using software supplied by Azure to ingest the log files you may have better luck asking on an Azure specific forum. I don't think it's possible or even desirable to have FME keep all the (potentially tens of thousands) job logs in a single directory.

Community Stats

32,068
Posts
121,762
Replies
39,693
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.