I have a workbench script that makes multiple ggplots (graphs) for end-user reports. The Rcaller works great in FME Server without error, but it will not export the plots. The output excel files export, but not the plots.
Page 1 / 1
Hi @wisegis , I haven't used ggplot myself, but you'll want to make sure that FME Server has access to the location you're trying to export your plots to. Instead of trying to export to "C:\\\\TEMP\\\\OUTPUT", perhaps try exporting to FME Server's Shared Resource Data folder (In FME Server, this is found under Files & Connections > Resources > Data). The macro you can use to specify that in your R code is $(FME_SHAREDRESOURCE_DATA).
If you get an error when running the workspace on FME Server, can you provide the job log?
Hi @wisegis , I haven't used ggplot myself, but you'll want to make sure that FME Server has access to the location you're trying to export your plots to. Instead of trying to export to "C:\\\\TEMP\\\\OUTPUT", perhaps try exporting to FME Server's Shared Resource Data folder (In FME Server, this is found under Files & Connections > Resources > Data). The macro you can use to specify that in your R code is $(FME_SHAREDRESOURCE_DATA).
If you get an error when running the workspace on FME Server, can you provide the job log?
Sorry for the delay. I attempted just to comment out the ggsave command I have at the end of my Rcaller_3 in the workbench file.
# ggsave("C:\\\\TEMP\\\\OUTPUT\\\\RANGE_BIGTEST.png")
ggsave($(FME_SHAREDRESOURCE_DATA))
Here is the log after I posted it to FME Server. The commented code above (#) works. I didn't add the name for the png as I assumed it would take whatever the dataframe was called.
Hi @wisegis , I haven't used ggplot myself, but you'll want to make sure that FME Server has access to the location you're trying to export your plots to. Instead of trying to export to "C:\\\\TEMP\\\\OUTPUT", perhaps try exporting to FME Server's Shared Resource Data folder (In FME Server, this is found under Files & Connections > Resources > Data). The macro you can use to specify that in your R code is $(FME_SHAREDRESOURCE_DATA).
If you get an error when running the workspace on FME Server, can you provide the job log?
Can you clarify where you think I should include $(FME_SHAREDRESOURCE_DATA)? If I put it in the output of the excel spreadsheets, I just pointed to the exact folder (C:\\\\ProgramData\\\\Safe Software\\\\FME Server\\\\resources\\\\data\\\\OUTPUT\\\\). In the Rcaller I attempted a couple of different ways to slice it. The first attempt was to put the FME_SHAREDRESOURCE_DATA macro in the ggsave portion of the code. That didn't work. I also tried to set the working directory at the beginning of the code, and it didn't like that either. Any guidance would be appreciated.
Hi @wisegis , the $(FME_SHAREDRESOURCE_DATA) macro will only return the path to the Shared Resources Data folder, which is C:\ProgramData\Safe Software\FME Server\resources\data by default. You will still need to append your filename. I admittedly don't use R, but it appears you can concatenate the filename using the code below, and then provide the path variable to your ggplot() function:
path <- paste($(FME_SHAREDRESOURCE_DATA), "RANGE_BIGTEST.png", sep="")If you have an OUTPUT folder in your Data folder on FME Server, then you could try
path <- paste($(FME_SHAREDRESOURCE_DATA), "OUTPUT\\RANGE_BIGTEST.png", sep="")With regards to where to put this, the error in the log file is coming from RCaller_3, so I would start there, but this approach should be used wherever FME Server is trying to write out in your R scripts.
Hi @wisegis , the $(FME_SHAREDRESOURCE_DATA) macro will only return the path to the Shared Resources Data folder, which is C:\ProgramData\Safe Software\FME Server\resources\data by default. You will still need to append your filename. I admittedly don't use R, but it appears you can concatenate the filename using the code below, and then provide the path variable to your ggplot() function:
path <- paste($(FME_SHAREDRESOURCE_DATA), "RANGE_BIGTEST.png", sep="")If you have an OUTPUT folder in your Data folder on FME Server, then you could try
path <- paste($(FME_SHAREDRESOURCE_DATA), "OUTPUT\\RANGE_BIGTEST.png", sep="")With regards to where to put this, the error in the log file is coming from RCaller_3, so I would start there, but this approach should be used wherever FME Server is trying to write out in your R scripts.
It looks like it is choking on the ggsave command that works correctly in workbench through the Rcaller. This was in my first example.
"Error: unexpected '$' in "ggsave($"
I'll keep working through it, and I appreciate your help. It is tricky because the little bit of code does two different things.
1) It creates a graphical plot of the data (ggplot).
2) It is passing the cleaned-up data on in the process to be a new excel spreadsheet.
The excel part works well. It grabs the graph and attempts to send it out as a .png file that server is just a little different with how to export.
It looks like it is choking on the ggsave command that works correctly in workbench through the Rcaller. This was in my first example.
"Error: unexpected '$' in "ggsave($"
I'll keep working through it, and I appreciate your help. It is tricky because the little bit of code does two different things.
1) It creates a graphical plot of the data (ggplot).
2) It is passing the cleaned-up data on in the process to be a new excel spreadsheet.
The excel part works well. It grabs the graph and attempts to send it out as a .png file that server is just a little different with how to export.
Hi @wisegis , if you're still getting the error, perhaps try to create a path attribute that concatenates the $(FME_SHAREDRESOURCE_DATA) and the filename before your R caller, use a StringReplacer to replace '\\' with '\\\\', and then pass that attribute to the ggsave() function.
Edit: also, you can test this on workbench by setting the FME_SHAREDRESOURCE_DATA to a local path, like C:\\Data, in Navigator under FME Server Parameters.
Thank you for helping me with this problem. So I got the script to run successfully in FME Server. I have one last bump that probably isn't nearly as hard as what we have progressed through. To fix the ggplot output issue I did two things:
1) I set the working to the FME Server output you showed me:
setwd("C:\\\\ProgramData\\\\Safe Software\\\\FME Server\\\\resources\\\\data\\\\OUTPUT\\\\")
2) I changed how the graphs were exported from R.
png("RANGE_BIGTEST.png")
print(OUTPUT_plot)
dev.off()
The last bump I alluded to is that when you run the data download service in FME Server, it runs successfully but only provides the end user the CSV output files, not the graphs. They are exported to the correct folder, just not included in the output zip that FME Server provides. I played around with creating a second workbench file that gets fired off, grabs all of it, and zips it up, but I ideally would like it to happen in the original workspace if that makes sense.
Hi @wisegis, glad to hear you almost have it working!
Files need to be written using a writer or feature writer for the data download service to include them in the output.
This article provides an overview. In your case, since the files are being created by the R Caller, you will want to use a FeatureReader afterward with the Format set to 'Directory and File Pathnames' and the Dataset set using the using FME Server Parameters, like $(FME_SHAREDRESOURCE_DATA)\\OUTPUT\\RANGE_BIGTEST.png
You should then be able to pass the feature created by that FeatureReader to a File Copy writer that is linked to your XLSX writer, as per the article.
I will get reading, and thank you very much for your help.
Hi @wisegis, glad to hear you almost have it working!
Files need to be written using a writer or feature writer for the data download service to include them in the output.
This article provides an overview. In your case, since the files are being created by the R Caller, you will want to use a FeatureReader afterward with the Format set to 'Directory and File Pathnames' and the Dataset set using the using FME Server Parameters, like $(FME_SHAREDRESOURCE_DATA)\\OUTPUT\\RANGE_BIGTEST.png
You should then be able to pass the feature created by that FeatureReader to a File Copy writer that is linked to your XLSX writer, as per the article.
So I was able to PATH reader and the Writer to copy the OUTPUT folder. What I am finding in my testing is that you have to run it twice to get it to pick up the new plots (if that makes sense). It copies the FME Server output folder before the new data is written. I found that if I run it again, it grabs the file that was written the first time that it now sees in the folder. I played around with the reading and writing order in workbench, hoping that would take care of it, but it did not.
So I was able to PATH reader and the Writer to copy the OUTPUT folder. What I am finding in my testing is that you have to run it twice to get it to pick up the new plots (if that makes sense). It copies the FME Server output folder before the new data is written. I found that if I run it again, it grabs the file that was written the first time that it now sees in the folder. I played around with the reading and writing order in workbench, hoping that would take care of it, but it did not.
Do you use a writer and a reader? In that case replace the writer and reader with a FeatureWriter and a FeatureReader, daisychaining the writing and reading. This way you avoid having to run the workspace twice.
So I was able to PATH reader and the Writer to copy the OUTPUT folder. What I am finding in my testing is that you have to run it twice to get it to pick up the new plots (if that makes sense). It copies the FME Server output folder before the new data is written. I found that if I run it again, it grabs the file that was written the first time that it now sees in the folder. I played around with the reading and writing order in workbench, hoping that would take care of it, but it did not.
I am not sure I completely follow. I will have to see how to set up that daisy-chaining. I have attached the export that I set up based on your recommendation. It has a bookmark called EXPORT. I am writing excel spreadsheets to the FME Server folder and, with the RCaller creating multiple graphs. Ideally, I would like to grab all of it and present that to the end user as a download service .zip file in FME Server.
So I was able to PATH reader and the Writer to copy the OUTPUT folder. What I am finding in my testing is that you have to run it twice to get it to pick up the new plots (if that makes sense). It copies the FME Server output folder before the new data is written. I found that if I run it again, it grabs the file that was written the first time that it now sees in the folder. I played around with the reading and writing order in workbench, hoping that would take care of it, but it did not.
I could not study your entire workspace, as I am stuck with an older version of FME, but you have several readers and several writers in your workspace, and I expect one of the readers reads the results of one of the writers. Replace that writer with a FeatureWriter, replace that reader with a Feature Reader, and connect them.
