There shouldn't be a problem with the CSV files format. FME’s AzureBlobStorageConnector allows you to access an Azure Data Lake Storage Gen2 directory to upload, download, delete files and folders, or list file/folder information with FME Desktop. Once you download the file using the AzureBlobStorageConnector, you can read it into your workspace using the FeatureReader transformer. I'm not too familiar with Data Lake Storage Gen2 so I can't give you my format preference, but you should be able to go with the format you like! This Blog may be helpful to check out too.
Yes, you should be able to work in some logic when designing your workflow to read in the newest file after it is downloaded. If there is a date attribute that may be easier.
It is possible to only use FME Desktop to connect with Azure Storage. Using FME Server allows for more capabilities such as Azure Event Grid as a trigger and an external action in FME Server Automations, which allows you to integrate your Azure environment with complex automated workflows. Please see Connecting FME Server to Azure File Share and How to Connect to Microsoft Azure in FME for more details.
It is possible to only use FME Desktop to connect with Azure Storage. Using FME Server allows for more capabilities such as Azure Event Grid as a trigger and an external action in FME Server Automations, which allows you to integrate your Azure environment with complex automated workflows. Please see Connecting FME Server to Azure File Share and How to Connect to Microsoft Azure in FME for more details.
There shouldn't be a problem with the CSV files format. FME’s AzureBlobStorageConnector allows you to access an Azure Data Lake Storage Gen2 directory to upload, download, delete files and folders, or list file/folder information with FME Desktop. Once you download the file using the AzureBlobStorageConnector, you can read it into your workspace using the FeatureReader transformer. I'm not too familiar with Data Lake Storage Gen2 so I can't give you my format preference, but you should be able to go with the format you like! This Blog may be helpful to check out too.
Yes, you should be able to work in some logic when designing your workflow to read in the newest file after it is downloaded. If there is a date attribute that may be easier.
I have a question regarding connection to Azure Data Lake gen2 as well. Is it possible to connect to ADL gen2 using FME Desktop (my version is 2020.2) using service principle?
I couldn't find any complete tutorial on this.
I was thinking of some tricky way through API calls if that's possible. Do you guys have any experience with such connection and if yes could you please advise the direction?
I have a question regarding connection to Azure Data Lake gen2 as well. Is it possible to connect to ADL gen2 using FME Desktop (my version is 2020.2) using service principle?
I couldn't find any complete tutorial on this.
I was thinking of some tricky way through API calls if that's possible. Do you guys have any experience with such connection and if yes could you please advise the direction?
Thank you in advance!
Regards!
@siriosus it is possible .i did it before but I left the company since few months .so It will be hard to remember how it works .Could u please , create new question and sure u will find who could help u .
We use 3 different kinds of cookies. You can choose which cookies you want to accept. We need basic cookies to make this site work, therefore these are the minimum you can select. Learn more about our cookies.
