FME Blob Storage in FME 2022.2

import fme
import fmeobjects
import sys, os

buildstring = fmeobjects.FME_BUILD_STRING[17:].lower().replace(" ","")
pkgPath = fr"{os.getenv('APPDATA')}\Safe Software\FME\Packages\{buildstring}\python\safe.azurestorage"
sys.path.remove(pkgPath)

from azure.storage.blob import BlobServiceClient

<your Script here>

# Add package path back to PATH
sys.path.insert(0,pkgPath)

Greaaat!

Hi @danilo_fme I tried using your approach but the issue is once you append delete the path from sys.path and then again insert it back at the same position doesn’t work if there is AzureBlobStorageConnector  transformer in the same workbench. I don’t know why but FME tries to import BlockBlobService then. I printed the updated sys.path, the path was added correctly but don’t know it didn’t try to import from that location again. If I run the workbench in one go it fails but if I run step by step means one transformer after another then it works without any issue

Hello @rishi1804 

FME shares one Python interpreter in a workspace. Once you have changed sys.path and the newer Python library is imported, the reference to the non-FME package location remains in sys.modules even if you re-add the removed package path. You can either remove modules loaded with del sys.modules[“<moduleName>”]. Since Azure is a complex library with many individual modules ~300), it is best to do this programmatically.

import fme
import fmeobjects
import sys, os

buildstring = fmeobjects.FME_BUILD_STRING[17:].lower().replace(" ","")
pkgPath = fr"{os.getenv('APPDATA')}\Safe Software\FME\Packages\{buildstring}\python\safe.azurestorage"
sys.path.remove(pkgPath)

from azure.storage.blob import BlobServiceClient

class FeatureProcessor(object):
    def __init__(self):
        pass

    def has_support_for(self, support_type: int):
        return support_type == fmeobjects.FME_SUPPORT_FEATURE_TABLE_SHIM

    def input(self, feature: fmeobjects.FMEFeature):
        self.pyoutput(feature)

    def close(self):
        pass

    def process_group(self):
        pass

#Find all modules starting with 'azure' and delete
az = [k for k in sys.modules if k.startswith('azure')]
for m in az:
    del sys.modules[m]        

sys.path.insert(0,pkgPath)

Alternatively, you can separate the portion that creates the container and the AzureBlobStorageConnector into separate workspaces and chain them together with a WorkspaceRunner. This should work since the changes to sys.path made in the workspace creating the container does not affect the workspace running the AzureBlobStorageConnector.

Partial runs of the workspace succeed because the Python interpreter is restarted for each run of the workspace.