Unfortunately I'm not aware of any such option.
Which FME version is this, by the way? I have the impression that current versions of FME are less "chatty" in this regard.
Unfortunately I'm not aware of any such option.
Which FME version is this, by the way? I have the impression that current versions of FME are less "chatty" in this regard.
Ok - thanks. I'm using FME 2020.1 and it's logging every single SQL statement executed with INFO level set. For now I will have to disable INFO, but will raise an idea to have this either as another setting or change the log level to DEBUG for all SQL statements.
Ok - thanks. I'm using FME 2020.1 and it's logging every single SQL statement executed with INFO level set. For now I will have to disable INFO, but will raise an idea to have this either as another setting or change the log level to DEBUG for all SQL statements.
Sounds like something you could consider flagging to Safe support.
@john_gis4bus We added the logging of the SQL calls at the request of other uses, so they can see what FME is firing off to the database. However, we should only log the SQL once for each instance of SQLExecutor/Creator. There is no need to repeat the logging each time the an initiator feature enters the SQLExecutor and triggers the SQL.
In some cases you'll see something like:
INFORM|Executing custom data query: 'SELECT COUNT(*) FROM "public"."CellSignals"'
INFORM|... Last line repeated 30 times ...
But if other transformers interrupt the flow of data then the logging of the SQL will start again.
Cleaning up the FME logging is actually a cause dear to my heart. You'll see some changes to the presentation and filtering of the logs in FME 2021 and we are also trying to cleanup reputative logging from individual transformers. (we recently fixed Clipper which logged 400000+ messages in one case!)
I've added this request to the list of transformer logging updates we should look into - so that we only log once instance of each SQL statement.
If you have any other logging issues you'd like to see fixed, please log them here
@john_gis4bus We added the logging of the SQL calls at the request of other uses, so they can see what FME is firing off to the database. However, we should only log the SQL once for each instance of SQLExecutor/Creator. There is no need to repeat the logging each time the an initiator feature enters the SQLExecutor and triggers the SQL.
In some cases you'll see something like:
INFORM|Executing custom data query: 'SELECT COUNT(*) FROM "public"."CellSignals"'
INFORM|... Last line repeated 30 times ...
But if other transformers interrupt the flow of data then the logging of the SQL will start again.
Cleaning up the FME logging is actually a cause dear to my heart. You'll see some changes to the presentation and filtering of the logs in FME 2021 and we are also trying to cleanup reputative logging from individual transformers. (we recently fixed Clipper which logged 400000+ messages in one case!)
I've added this request to the list of transformer logging updates we should look into - so that we only log once instance of each SQL statement.
If you have any other logging issues you'd like to see fixed, please log them here
I'm all for the ability to log SQL statements for debugging purposes, but I can't see a valid use case where you would need that level of granularity turned on permanently. Hence my suggestion to make these SQL statements DEBUG level messages, which would solve the problem for now.
In our case, with the standard INFO level set the logs are running into 100s of MB for an execution that processes a few thousand features, but makes multiple SQLExecutor queries per feature.
FME is not doing the thing you mention of just logging the first feature or first n features. It is definitely logging each and every SQL statement executed.
Also, If we turn off INFO level logging, then we lose all sorts of other useful information, including the summary info when the workbench completes!
