when trying to use the select file from web - azure blob storage option on a reader. I can get to the container and i can select files that are at the top level in that container but if i want to go into a folder to select a file in the folder i get an error. ( see screenshots )
Any help appreciated
Hi @baznewman07
I just tested this in FME with version 1.0.6 of the Azure Storage package and it lets me select it and read the data in fine.
Can you try using the AzureBlobStorageConnector transformer to bring in the CSV? Or try embedding the credentials instead of using a web connection?
If you're not on the latest version of the package, I would also recommend downloading it and trying with that.
Hi @baznewman07
I just tested this in FME with version 1.0.6 of the Azure Storage package and it lets me select it and read the data in fine.
Can you try using the AzureBlobStorageConnector transformer to bring in the CSV? Or try embedding the credentials instead of using a web connection?
If you're not on the latest version of the package, I would also recommend downloading it and trying with that.
very strange, i downloaded that version to test it and it still didn't work on mine.
very strange, i downloaded that version to test it and it still didn't work on mine.
I have just realised this is specific to a certain way of handling the data. If i write out a csv file to blob storage using azure databricks then it creates a couple of other files along with the csv. I think this is what is causing the error when i try and access those files inside of the folder. If i write simple csv files inside a folder structure then it works fine.
Hi @baznewman07
I just tested this in FME with version 1.0.6 of the Azure Storage package and it lets me select it and read the data in fine.
Can you try using the AzureBlobStorageConnector transformer to bring in the CSV? Or try embedding the credentials instead of using a web connection?
If you're not on the latest version of the package, I would also recommend downloading it and trying with that.
I got the exact same error. Anything directly under the container can be read without any issues. Opening any folder under the container will result in the error ("It is not a valid operation to get a list of items in '%s' (ID '%s')", '<folder name', '<folder name>/').
Any progress so far @baznewman07 ?
@jlutherthomas My Azure Storage package is 1.3.1 by default. Still not work.
I got the exact same error. Anything directly under the container can be read without any issues. Opening any folder under the container will result in the error ("It is not a valid operation to get a list of items in '%s' (ID '%s')", '<folder name', '<folder name>/').
Any progress so far @baznewman07 ?
@jlutherthomas My Azure Storage package is 1.3.1 by default. Still not work.
@kevint I have been able to do this for a while now.
I am using
safe.azurestorage.AzureBlobStorageConnector Version 2
&
Azure Storage - safe.azurestorage u1.0.6]
I got the exact same error. Anything directly under the container can be read without any issues. Opening any folder under the container will result in the error ("It is not a valid operation to get a list of items in '%s' (ID '%s')", '<folder name', '<folder name>/').
Any progress so far @baznewman07 ?
@jlutherthomas My Azure Storage package is 1.3.1 by default. Still not work.
Thanks @baznewman07 .
I downgraded my safe.azurestorage to 1.0.6 but still got the same error. I think you got the same issue from your comment a year ago? For safe.azurestorage.AzureBlobStorageConnector, could you let me know how to check its version, please?
I managed to overcome this issue by tweaking the url that fetches the data. Hopes it will help someone else.
For example: assume I can read a csv called model.csv file directly under the container. Its url (in dataset parameter) will look like this:
fme://safe.azurestorage.microsoft-azure-blob-storage/model.csv?id=model.csv&module=....
To read another csv called test.csv under the test folder inside the same container, I just need to modify the the table name and the id parameter in the url to:
fme://safe.azurestorage.microsoft-azure-blob-storage/test.csv?id=test/test.csv&module=.....
I got the exact same error. Anything directly under the container can be read without any issues. Opening any folder under the container will result in the error ("It is not a valid operation to get a list of items in '%s' (ID '%s')", '<folder name', '<folder name>/').
Any progress so far @baznewman07 ?
@jlutherthomas My Azure Storage package is 1.3.1 by default. Still not work.
@kevint right click on the transformer and click on 'show summary annotation'
I got the exact same error. Anything directly under the container can be read without any issues. Opening any folder under the container will result in the error ("It is not a valid operation to get a list of items in '%s' (ID '%s')", '<folder name', '<folder name>/').
Any progress so far @baznewman07 ?
@jlutherthomas My Azure Storage package is 1.3.1 by default. Still not work.
I'm not sure if this was resolved for @kevint , but I tried the steps for show summary annotation and do not see the version for safe.azurestorage.AzureBlobStorageConnector
I got the exact same error. Anything directly under the container can be read without any issues. Opening any folder under the container will result in the error ("It is not a valid operation to get a list of items in '%s' (ID '%s')", '<folder name', '<folder name>/').
Any progress so far @baznewman07 ?
@jlutherthomas My Azure Storage package is 1.3.1 by default. Still not work.
That is strange, as you can see i get the version when i do it.
I got the exact same error. Anything directly under the container can be read without any issues. Opening any folder under the container will result in the error ("It is not a valid operation to get a list of items in '%s' (ID '%s')", '<folder name', '<folder name>/').
Any progress so far @baznewman07 ?
@jlutherthomas My Azure Storage package is 1.3.1 by default. Still not work.
That is really weird - I'm running FME 2020.2 and am wondering if versions are only reported for newer builds (?)
I got the exact same error. Anything directly under the container can be read without any issues. Opening any folder under the container will result in the error ("It is not a valid operation to get a list of items in '%s' (ID '%s')", '<folder name', '<folder name>/').
Any progress so far @baznewman07 ?
@jlutherthomas My Azure Storage package is 1.3.1 by default. Still not work.
Not sure, i am using
FME(R) 2021.2.3.0 (20220131 - Build 21812 - macosx)
@geographicus97 You can enable transformer version in FME Options > Workbench:
FME Option > Packages will also show you the package version.