Question

Notification Service - Send Notification Data to Reader - Distributed engines


Badge

Basically we have some fmw files that write data, and then we have a second fmw file that loads the job status info into an oracle table for logging. The first fmw, runs and then calls the DataLoadingStatus topic

Then the called mw, which populates an Oracle table with job status info from the first fmw runs, it is registered with the notification service with the “Send Notification Data to Reader” populated

We are running with engines distributed across a number of different boxes.

The process is working correctly for what it is, it is passing the correct path to the second fmw to find the temp file.

e:\\FMEServer\\Utilities\\tomcat\\temp\\603774853810044_gpas_status.txt - That file does exist on the core host.

The problem is the Engines are distributed, so when the second fmw runs on a distributed engine and goes to read that file, it is looking for that path on the local box’s E drive and not the Core host box, and that file does not exist on the local box, so it errors with:

ERROR |Text File Reader: Source dataset 'e:\\FMEServer\\Utilities\\tomcat\\temp\\605131760468036_gpas_status.txt' does not exist

Something somewhere is telling FME server where to write these temp files to this path:

e:\\FMEServer\\Utilities\\tomcat\\temp\\603774853810044_gpas_status.txt

It used to write them to the resource path, which the distributed engines could find:

resources\\system\\temp\\requestdata\\4072616362113062_gpas_status.txt'

Any Ideas where this setting might be ?


17 replies

Badge +2

Hi @mferwerda111, in the first workspace are you specifying where to write the data? If so, what does that look like?

I can’t think of a setting off the top of my head, but as a workaround could you replace the drive path with a unc path and then use a featurereader that will always read the data in from a text file from the core machine.

Userlevel 5

Based on your wording it seems like you're running distributed engines without a common server system share, is that right?

If so, I suspect the solution is to rethink your installation so that your engines all connect to the same system share, see https://docs.safe.com/fme/html/FME_Server_Documentation/Content/AdminGuide/Install-Core-and-Engines-3-Tier.htm

Badge

So when the fmw runs, it creates a file, this file is automatically created with ta guid name like “33358624659273_gpas_status.txt”

 

Example

{

"workspace": "GPAS_WSR.fmw",

"timeRequested": "Tue-05-Dec-2017 01:00:00 AM",

"NumFeaturesOutput": "250",

"requestKeyword": "SERVER_SCHEDULER",

"timeStarted": "Tue-05-Dec-2017 01:00:00 AM",

"repository": "ETL_PROD",

"jobsuccess_topic": "gPasDataLoadingStatus",

"LogFileName": "job_603.log",

"StatusMessage": "Translation Successful",

"jobfailure_topic": "gPasDataLoadingStatus",

"StatusNumber": "0",

"id": "603",

"timeFinished": "Tue-05-Dec-2017 01:00:21 AM",

"logHome": "//ETL/FME16Repository///resources/logs",

"logUrl": "http://Nz1320/fmerest/v3/transformations/jobs/id/603/log"

}

We have FME server set up so it uses a common drive, all the repository’s, resources, etc, are on the same drive where all the engines can see the same data like \\\\my.net\\Reference\\ETL\\FME16Repository\\

In our old set up, these files were automatically created in this path, and all the engines can get to them.

\\\\myfolder\\Reference\\GeoTool\\EDW\\FME16Repository\\resources\\system\\temp\\requestdata\\4072616362113062_gpas_status.txt'

In our new install, for some reason, the files are being created here on the host core box, so the fmw run on distributed engines cannot see it. The process is passing he correct path the the called fmw, so that part is technically working.

e:\\FMEServer\\Utilities\\tomcat\\temp\\33358624659273_gpas_status.txt

VDC

Somewhere there is something that is telling FME server where to create these files

Userlevel 5

I would start by looking into the configuration files, notably if you find any of these in your engine installation:

  • fmeCommonConfig.txt
  • fmeEngineConfig.txt
  • fmeServerConfig.txt

In particular, see if you can find a reference to your tomcat temp folder somewhere there.

Userlevel 5

I would start by looking into the configuration files, notably if you find any of these in your engine installation:

  • fmeCommonConfig.txt
  • fmeEngineConfig.txt
  • fmeServerConfig.txt

In particular, see if you can find a reference to your tomcat temp folder somewhere there.

In fmeEngineConfig.txt, check your definition of the FME_SHAREDRESOURCE_* macros (probably on lines 18-23), they should all point at your shared server repository.
Userlevel 5

Also check the following values in <FMEServer>\\Server\\config\\subscribers\\workspace.properties

  • TOPIC_DATA_FOLDER
  • RESULT_DATA_FOLDER

For some reason they are both relative paths on my (single engine) installation, which is always a bit suspicious when this type of error occurs. You could always try to set them to absolute paths and see if that makes a difference. Just don't forget to restart your core+engine service if you change anything.

Badge +2
Can you confirm that the instructions for https://docs.safe.com/fme/html/FME_Server_Documentation/Content/AdminGuide/Adding_FME_Engines_on_Separate_Machine.htm were followed?

 

I'm wondering if you look at step 10, what the REQUEST_DATA_DIR parameter is set to for each service?

 

 

Badge

Also check the following values in <FMEServer>\\Server\\config\\subscribers\\workspace.properties

  • TOPIC_DATA_FOLDER
  • RESULT_DATA_FOLDER

For some reason they are both relative paths on my (single engine) installation, which is always a bit suspicious when this type of error occurs. You could always try to set them to absolute paths and see if that makes a difference. Just don't forget to restart your core+engine service if you change anything.

I have looked over the config files and the workspace.properties and do not see anything obvious yet. We do have a install of FME server in our DEV environment, that is supposed to mimic the prod install, and this process is working correctly in that install, so I have been comparing the config files in that install to the prod config files, but still see nothing different other than what one would expect

 

 

Badge

 

So we do have a install of FME server in our DEV environment, that is supposed to mimic the prod install, and this process is working correctly in that install.

 

 

I have looked at the E:/FMEServer\\Utilities\\tomcat\\webapps\\fmejobsubmitter\\WEB-INF\\conf\\propertiesFile.properties

 

 

and see this, which looks OK to me

 

fmejobsubmitter

 

#REQUEST_DATA_DIR - The directory which stores request data from HTTP POST body

 

REQUEST_DATA_DIR=//usda.net/fs/FS/Reference/ETL/FME16Repository//resources/system/temp/requestdata

 

 

fmenotification

 

#REQUEST_DATA_DIR - The directory which stores request data from HTTP POST body

 

REQUEST_DATA_DIR=//usda.net/fs/FS/Reference/ETL/FME16Repository//resources/system/temp/requestdata

 

 

Badge
Status update, so far I have not found anything obvious in config files, it is probably in there someplace. I have implemented a work around and modified the Target URL for the subscription to have a job tag so it only runs on the core, and then passes the json text to another fmw that will do the writing (we do not have oracle client on the core host)

 

I think it might be good to add in a parameter for Job tag when registering with the notification service

 

 

Badge +11
Hi @mferwerda111– I'm really sorry to read about this! I reviewed this Q&A; posting (incl. its comments) and I couldn't find explicit mention, so to confirm: both FME Server installs (Dev/Prod) are the same build?

 

 

I wonder if there might be a system environment variable affecting the Prod installation? Can you confirm if this might be the issue or not?

 

Badge
Hi @mferwerda111– I'm really sorry to read about this! I reviewed this Q&A; posting (incl. its comments) and I couldn't find explicit mention, so to confirm: both FME Server installs (Dev/Prod) are the same build?

 

 

I wonder if there might be a system environment variable affecting the Prod installation? Can you confirm if this might be the issue or not?

 

Yes, both are FME Server 2016.1.2.1 - Build 16674 - win64

 

So this worked in our old environment, works in a new dev environment, but not the prod install, both the Dev FME Server and Prod FME server are in the same "environment" or at least it is supposed to be. I use the back up and restore process to move all the content over from the Old environment to both the Dev FME Server and prod FME Server in the new environment. The PATH Environment Variables for FME server look similar in both, slightly different paths,but basically the same.

 

 

Is there another Env Var I should look at ?

 

 

Badge +11
Yes, both are FME Server 2016.1.2.1 - Build 16674 - win64

 

So this worked in our old environment, works in a new dev environment, but not the prod install, both the Dev FME Server and Prod FME server are in the same "environment" or at least it is supposed to be. I use the back up and restore process to move all the content over from the Old environment to both the Dev FME Server and prod FME Server in the new environment. The PATH Environment Variables for FME server look similar in both, slightly different paths,but basically the same.

 

 

Is there another Env Var I should look at ?

 

 

With respect to environment variables, there isn't any one in particular I had in mind – essentially just a diff between the two machines. This issue has me puzzled.

 

 

Badge
With respect to environment variables, there isn't any one in particular I had in mind – essentially just a diff between the two machines. This issue has me puzzled.

 

 

I think Jennifer might have been on the right track, I am looking at the Utilities\\tomcat\\webapps\\fmenotification\\WEB-INF\\conf\\propertiesFile.properties

 

file and I see this path:

 

#REQUEST_DATA_DIR - The directory which stores request data from HTTP POST body

 

REQUEST_DATA_DIR=//my.net/Ref/ETL/FME16Repository//resources/system/temp/requestdata

 

 

Not sure why there are a // in there, and I can see that the requestdata folder exists in our old install, and in our dev install, but that folder does not exist in our prod install.

 

 

I have a feeling that // is the problem

 

 

 

Badge
I think Jennifer might have been on the right track, I am looking at the Utilities\\tomcat\\webapps\\fmenotification\\WEB-INF\\conf\\propertiesFile.properties

 

file and I see this path:

 

#REQUEST_DATA_DIR - The directory which stores request data from HTTP POST body

 

REQUEST_DATA_DIR=//my.net/Ref/ETL/FME16Repository//resources/system/temp/requestdata

 

 

Not sure why there are a // in there, and I can see that the requestdata folder exists in our old install, and in our dev install, but that folder does not exist in our prod install.

 

 

I have a feeling that // is the problem

 

 

 

or I think that was david_r suggested looking at those files

 

 

Userlevel 5
or I think that was david_r suggested looking at those files

 

 

I would be surprised if a double slash was the issue here. They're usually ignored by the file system/OS.
Badge
I would be surprised if a double slash was the issue here. They're usually ignored by the file system/OS.
Yes, the // is not the issue. We manually added the requestdata folder and now it is working as expected, not sure why that folder did not get created. And thanks for the help pointing to those config files which eventually got me to look to see if the folder was actually there.

 

 

Reply