Creating files in Network Shared Folders using FME Flow

Right, it does seem like the user which FME Flow is running as only has read access to those data. 

You can look at the services to see which user FME Flow Services as. In this case they are running as the LocalSystem. 

The specific doc about changing the user is here: https://docs.safe.com/fme/html/FME-Flow/AdminGuide/Running_System_Services_Under_Different_Accounts.htm
 

Hi @virtualcitymatt my server is running on Debian 11, which is why I’m having a harder time pinpointing which user is doing what.

Any idea how to check the same in Deb 11?.

Thanks again.

Randall Z.-

@rzel24, Linux. That’s interesting question that I do not have a precise answer for… or tested in this way before that I can clearly remember. I’ve only ever run jobs that connected with mounted shares.

I’m wondering if Tomcat (FME Flow Application Server) is using a user account that is not ‘fmeflow’ on Linux, that doesn’t have access to the Share… but for sure the Core & Engines are using ‘fmeflow’ user so it will need to be given access to RW (if it doesn’t already have).  Agree, it is interesting that you can can ‘see’, ‘read’ and ‘download’ folders but not upload them.

Can you test that the FME Flow Engine can create a file in the same location? So publish a workspace that is using the share and see if it can create the file, if it can’t then the ‘fmeflow’ user has read access but no write access and once this is resolved it will likely resolve the Web UI issue too. 

If that test is successful, then I am suspect of the user behind the FME Flow Application Server… and we will need to confirm that user.

Hi @steveatsafe, the reason I found out we weren’t able to write into the shared network drives was because I have a workspace schedule in an automation that came back with errors:
 

The workspace reads from a shared network folder (added as a resources), then after processing the information from the file it writes 3 different files in a temporary folder in the FME server (the temp folder is created using the TempPathnameCreator transformer), after the files are created we copy the files from the temp folder to the final destination in a different shared network folder (added as a separate resource from where the file is read).

I decided to try and see if I could write directly to the shared network folder (I tried in 2 different locations, one at a time) so I removed the fileCopy transformers as well as the TempPathnameCreator to read the file from the shared network folder and write the 3 x CSV files directly to the shared network folder. This time the workspace came back without the error above (I guess because I am not using the FileCopy writer) and in fact no error was reported for the CSV writer (I’m using a featurewriter configured for CSV files), but when I checked the shared network folders I cannot see any of the files there (nothing was created there).

I did this with 2 different locations on the shared network folders and despite not getting any errors in the log from the job (in fact the log shows the files are created):

*********************************************************************

**********************************************************************

The files were not created in any of the tests I did.

I tried to create the file in the FME Server folder and that worked without problems.

Any thoughts?

RZ.-

@rzel24 

Could you turn on Log Debug in the Workspace? Might get some more information?
Could you test with the CSV Writer vs the FeatureWriter?
I believe we will have to look into this more. 

Are you able to create a ticket? That way we can ask you a few more detailed questions and if necessary connect via a screen share. 

In any case, we’ll need to test this so we can give you conclusive answer - could be a bug - seems kind of odd that the CSV Writer isn’t being more helpful and reporting an error… or is it writing somewhere else?

​
@steveatsafe 

I’m new at FME and I can’t seem to find this Log Debug option or setting, how do I turn this on?.

I tested the workspace with a CSV writer and I get an error indicating that it cannot write in the destination:
******************************************************************************************************

024-06-19 07:48:46|  15.4|  0.0|INFORM|CSV writer: Opening dataset '/mnt/share/AMP/Roster/Archive/ac_all_Test.csv'
2024-06-19 07:48:46|  15.4|  0.0|ERROR |CSV writer: Failed to open file '/mnt/share/ AMP/Roster/Archive/ac_all_Test.csv' for writing.  Please ensure that you have sufficient privileges to write this file.  If the file already exists, please ensure it is not open in another application
2024-06-19 07:48:46|  15.4|  0.0|ERROR |CSV2 writer: An error has occurred. Check the logfile above for details
2024-06-19 07:48:46|  15.4|  0.0|INFORM|Excel Reader: Closing dataset '/tmp/user/997/fmeengines/localhost_Engine1/TempFS_1718783309939_2216839/AMP All Candidates.xlsx'...
2024-06-19 07:48:46|  15.4|  0.0|INFORM|Excel Writer: Workbook contains 129 fonts and 375 cell formats after writing
2024-06-19 07:48:47|  15.8|  0.4|INFORM|Excel Reader: Closing dataset '/tmp/user/997/fmeengines/localhost_Engine1/TempFS_1718783309939_2216839/AMP All Candidates.xlsx'...
2024-06-19 07:48:47|  15.8|  0.0|INFORM|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2024-06-19 07:48:47|  15.8|  0.0|INFORM|Feature output statistics for `CSV2' writer using keyword `CSV2_1':
2024-06-19 07:48:47|  15.8|  0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2024-06-19 07:48:47|  15.8|  0.0|STATS |                               Features Written
2024-06-19 07:48:47|  15.8|  0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2024-06-19 07:48:47|  15.8|  0.0|STATS |ac_all_Test                                                                232
2024-06-19 07:48:47|  15.8|  0.0|STATS |==============================================================================
2024-06-19 07:48:47|  15.8|  0.0|STATS |Total Features Written                                                     232
2024-06-19 07:48:47|  15.8|  0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2024-06-19 07:48:47|  15.8|  0.0|INFORM|Translation FAILED with 2 error(s) and 5 warning(s) (232 feature(s) output)
2024-06-19 07:48:47|  15.8|  0.0|INFORM|FME Session Duration: 18.1 seconds. (CPU: 14.4s user, 1.4s system)
2024-06-19 07:48:47|  15.8|  0.0|INFORM|END - ProcessID: 2216839, peak process heap usage: 1392194 kB, current process heap usage: 28355 kB
 

******************************************************************************************************

I’ll ask the account to see if they can open a ticket, not sure how long will that take but in the mean time if you have any other inputs it would be great.

Thanks again.

RZ.-

​