I have developed some scripts to massage data from one SQL db, spatialize features, then write to SDE db. They have been running for 2 yrs on FME Server (2017.1) fine for about a year. Then suddenly, all of them started to run real slow. There are about 20 scripts running serially (a master script kicks them off using fmeserverjobsubmitter). It used to run from 10pm until 1:30am, but now it has climbed from 3am (when the issue was first spotted) to 5am (latest run).
We've looked at resources and saw that the memory was hitting the ceiling (we have 16GB on the virtual server). 8GB more were added 2 days ago and the memory was no longer hitting the ceiling, but scripts are still running until 5am.
So now, we want to maybe check the connection to the database to see maybe if the read or the write is the issue. This is where I need help. I'm not sure how to go about that via fme workbench or even if it's possible? I created a workspace that only has the reader and a logger. But I would need to get the run time of it and write to a database (but I don't want the added overhead of the writing the time, just the actual read).
When I run an individual script during the day, it doesn't take as long as when it's run by the schedule at night which makes me think it might be something with the reading/writing to the dbs. So I want to run a simple script that just looks at the reading of the db every hour for like a week or two just to see when there could be an issue with reading the data as this wasn't a problem until about two months ago.
I would like to have a chart of read time comparing hourly and between days.
Any help would be greatly appreciated!
