Creating files in Network Shared Folders using FME Flow
Hello,
I’m trying to write files into a Shared Network Folders via an automation in FME Flow running a workspace.
The Shared Network Folders were mounted onto the VM Server and then added to the Resources in the FME Flow app with the credentials from “user1”.
After adding the shared network folders to the Resources in FME Flow I can see the files and folders inside them and I can download and read the files and folders via the FME Flow interface without problems. But when I’m trying to upload files/folders or create a folder directly in the share it gives me an error indicating issues with access rights. I find it weird that I can see, read and download folders but not upload them, but I guess it is because the credentials used by FME Flow to access the shared network folders are not the ones from “user1” but from the service user account that FME Flow uses, like “fmflow” user or something like that.
Question is how can I find out what is the user used by FME Flow to access the shared network drives?.
Thanks in advance.
RZ.-
Page 1 / 1
Right, it does seem like the user which FME Flow is running as only has read access to those data.
You can look at the services to see which user FME Flow Services as. In this case they are running as the LocalSystem.
Hi @virtualcitymatt my server is running on Debian 11, which is why I’m having a harder time pinpointing which user is doing what.
Any idea how to check the same in Deb 11?.
Thanks again.
Randall Z.-
@rzel24, Linux. That’s interesting question that I do not have a precise answer for… or tested in this way before that I can clearly remember. I’ve only ever run jobs that connected with mounted shares.
I’m wondering if Tomcat (FME Flow Application Server) is using a user account that is not ‘fmeflow’ on Linux, that doesn’t have access to the Share… but for sure the Core & Engines are using ‘fmeflow’ user so it will need to be given access to RW (if it doesn’t already have). Agree, it is interesting that you can can ‘see’, ‘read’ and ‘download’ folders but not upload them.
Can you test that the FME Flow Engine can create a file in the same location? So publish a workspace that is using the share and see if it can create the file, if it can’t then the ‘fmeflow’ user has read access but no write access and once this is resolved it will likely resolve the Web UI issue too.
If that test is successful, then I am suspect of the user behind the FME Flow Application Server… and we will need to confirm that user.
Hi @steveatsafe, the reason I found out we weren’t able to write into the shared network drives was because I have a workspace schedule in an automation that came back with errors:
146
2024-6-14 13:23:11 | File Copy Writer: Error attempting to copy from '/tmp/user/997/fmeengines/localhost_Engine1/1718360564692_1757186_d/tor400/ac_qual.csv' to '/mnt/share4/SANDBOX/ALE/AMP/CSV/ac_qual.csv'. Please check for sufficient permissions on the source and destination. Alternatively, if Overwrite Existing File is set to 'No' please ensure output file does not already exist
The workspace reads from a shared network folder (added as a resources), then after processing the information from the file it writes 3 different files in a temporary folder in the FME server (the temp folder is created using the TempPathnameCreator transformer), after the files are created we copy the files from the temp folder to the final destination in a different shared network folder (added as a separate resource from where the file is read).
I decided to try and see if I could write directly to the shared network folder (I tried in 2 different locations, one at a time) so I removed the fileCopy transformers as well as the TempPathnameCreator to read the file from the shared network folder and write the 3 x CSV files directly to the shared network folder. This time the workspace came back without the error above (I guess because I am not using the FileCopy writer) and in fact no error was reported for the CSV writer (I’m using a featurewriter configured for CSV files), but when I checked the shared network folders I cannot see any of the files there (nothing was created there).
I did this with 2 different locations on the shared network folders and despite not getting any errors in the log from the job (in fact the log shows the files are created):
2024-6-18 11:01:56 | Creator_CREATED Brancher -1 4 (BranchingFactory): Branched 1 input feature -- 1 feature routed to the target factory, and 0 features routed to the fallback factory.
2024-6-18 11:01:56 | Destination Feature Type Routing Correlator (RoutingFactory): Tested 0 input feature(s), wrote 0 output feature(s): 0 matched merge filters, 0 were routed to output, 0 could not be routed.
136
2024-6-18 11:01:56 | Final Output Nuker (TeeFactory): Cloned 0 input feature(s) into 0 output feature(s)
The files were not created in any of the tests I did.
I tried to create the file in the FME Server folder and that worked without problems.
Any thoughts?
RZ.-
@rzel24
Could you turn on Log Debug in the Workspace? Might get some more information? Could you test with the CSV Writer vs the FeatureWriter? I believe we will have to look into this more.
Are you able to create a ticket? That way we can ask you a few more detailed questions and if necessary connect via a screen share.
In any case, we’ll need to test this so we can give you conclusive answer - could be a bug - seems kind of odd that the CSV Writer isn’t being more helpful and reporting an error… or is it writing somewhere else?
@steveatsafe
I’m new at FME and I can’t seem to find this Log Debug option or setting, how do I turn this on?.
I tested the workspace with a CSV writer and I get an error indicating that it cannot write in the destination: ******************************************************************************************************
024-06-19 07:48:46| 15.4| 0.0|INFORM|CSV writer: Opening dataset '/mnt/share/AMP/Roster/Archive/ac_all_Test.csv' 2024-06-19 07:48:46| 15.4| 0.0|ERROR |CSV writer: Failed to open file '/mnt/share/ AMP/Roster/Archive/ac_all_Test.csv' for writing. Please ensure that you have sufficient privileges to write this file. If the file already exists, please ensure it is not open in another application 2024-06-19 07:48:46| 15.4| 0.0|ERROR |CSV2 writer: An error has occurred. Check the logfile above for details 2024-06-19 07:48:46| 15.4| 0.0|INFORM|Excel Reader: Closing dataset '/tmp/user/997/fmeengines/localhost_Engine1/TempFS_1718783309939_2216839/AMP All Candidates.xlsx'... 2024-06-19 07:48:46| 15.4| 0.0|INFORM|Excel Writer: Workbook contains 129 fonts and 375 cell formats after writing 2024-06-19 07:48:47| 15.8| 0.4|INFORM|Excel Reader: Closing dataset '/tmp/user/997/fmeengines/localhost_Engine1/TempFS_1718783309939_2216839/AMP All Candidates.xlsx'... 2024-06-19 07:48:47| 15.8| 0.0|INFORM|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 2024-06-19 07:48:47| 15.8| 0.0|INFORM|Feature output statistics for `CSV2' writer using keyword `CSV2_1': 2024-06-19 07:48:47| 15.8| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 2024-06-19 07:48:47| 15.8| 0.0|STATS | Features Written 2024-06-19 07:48:47| 15.8| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 2024-06-19 07:48:47| 15.8| 0.0|STATS |ac_all_Test 232 2024-06-19 07:48:47| 15.8| 0.0|STATS |============================================================================== 2024-06-19 07:48:47| 15.8| 0.0|STATS |Total Features Written 232 2024-06-19 07:48:47| 15.8| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 2024-06-19 07:48:47| 15.8| 0.0|INFORM|Translation FAILED with 2 error(s) and 5 warning(s) (232 feature(s) output) 2024-06-19 07:48:47| 15.8| 0.0|INFORM|FME Session Duration: 18.1 seconds. (CPU: 14.4s user, 1.4s system) 2024-06-19 07:48:47| 15.8| 0.0|INFORM|END - ProcessID: 2216839, peak process heap usage: 1392194 kB, current process heap usage: 28355 kB
I’ll ask the account to see if they can open a ticket, not sure how long will that take but in the mean time if you have any other inputs it would be great.
Thanks again.
RZ.-
Hello,
So I wasn’t able to get the ticket with safe created for this issue. Not sure why but it just didn’t happen.
In any case my issue with creating the files is that the fmeflow user doesn’t have any access to the Share Network Drives.
Once that is fixed I expect the problem to be solved. For now I will mark this as best answer.
