Creating new CSV files to upload to Azure Blob Storage, via FME Server
Hello,
I was hoping to be able to create an FME workspace to be uploaded to FME server, that would run an SQL script (via SQL Creator) and create a CSV file that would then be uploaded directly to Azure Blob storage.
I tried the 'AzureBlobStorageConnector' (using the 'File' upload option), but it appears that the file needs to be stored locally to use this option, which seems to be no good if I want to publish and automate this script via server. I also tried 'FeatureWriter', but couldn't see an option to write to a web service.
Any help would be greatly appreciated!
Thanks
Chris
Page 1 / 1
I did some work with Azure Blob Storage and I think you can't create files there like on a drive, you need to upload them, using the AzureBlobStorageConnector. This is not an FME limitation but how Blob storage works.
And it can work well. If you use a TempPathnameCreator, you can create a file, with a FeatureWriter to a temp directory, upload it using the Connector and it will get cleaned up when the workspace is run.
Attached 2019 sample workspace template. FeatureReader is just to show you can dynamically read the file you wrote to a temp directory. Replace the FeatureReader with the AzureBlobStorageConnector and you're good to go.
I did some work with Azure Blob Storage and I think you can't create files there like on a drive, you need to upload them, using the AzureBlobStorageConnector. This is not an FME limitation but how Blob storage works.
And it can work well. If you use a TempPathnameCreator, you can create a file, with a FeatureWriter to a temp directory, upload it using the Connector and it will get cleaned up when the workspace is run.
Attached 2019 sample workspace template. FeatureReader is just to show you can dynamically read the file you wrote to a temp directory. Replace the FeatureReader with the AzureBlobStorageConnector and you're good to go.
Thanks, the principal makes sense, however when I set the 'File to Upload' path in the AzureBlobStorageConnector using the same filepath up applied in the 'FeatureReader', it gets rejected. The 'fme_rejection_code' is 'INVALID_FILE' and 'fme_rejection_message' is 'Source file does not exist'. Any ideas how I can overcome this?
Thanks, the principal makes sense, however when I set the 'File to Upload' path in the AzureBlobStorageConnector using the same filepath up applied in the 'FeatureReader', it gets rejected. The 'fme_rejection_code' is 'INVALID_FILE' and 'fme_rejection_message' is 'Source file does not exist'. Any ideas how I can overcome this?
I managed to figure it out, and appears to be just a minor thing; I changed the 'File to Upload' path to , and it now seems to be working fine. Thank you so much for your help!
I managed to figure it out, and appears to be just a minor thing; I changed the 'File to Upload' path to , and it now seems to be working fine. Thank you so much for your help!
Great, thanks for your feedback :-)
Hello!
I try to upload a shapefile to an Azure blob Storage. My settings are the same as in your FME Workbench. But I get the error message "Cannot upload folder as file". Do you have any suggestions to what I can change? Beneath here are the images of the transformers I used in chronological order.
