Yes, sort of, I think.
If you have the custom transformers laid out in your workspace in parallel streams you can route features to them by testing on a user parameter.
But you have to have the processing designed and in your workspace for this to work (either directly in the workspace or in custom transformers). If you want something truly dynamic you could attempt to generate mapping files on the fly, store them in a temp location and call them using a WorkspaceRunner (I'm not even sure about how this would work on FME Server, I think you have to register the mapping files with a repository for them to become available)
I think @redgeographics idea of routing via a published parameter test is the best one if you have a set of well known ahead of time transformations you want to apply.
Though I'd be surprised if it would be truly dynamic -- that is, you want the user to supply a custom transformer and you'll apply it -- you could accomplish such a thing by copying the custom transformer on top of whatever one the workspace was authored to use. I also played around and the FMEFunctionCaller allows you to inject an arbitrary line into the resulting workspace mapping file so a super duper advanced old timer (say, a SpatialDirect veteran) could use that to INCLUDE a mapping file fragment and then apply hand-coded transformation dynamically.
Not sure I'd recommend it though.
If you're not intimidated by some Python programming, your can accomplish a lot using the fmeobjects API inside a PythonCaller. It's really easy to analyse features based on attributes, geometry types, etc and to act accordingly. Starting with FME 2017 even raster features are supported.
The fmeobjects documentation gives a pretty good overview of the capabilities:
http://docs.safe.com/fme/html/FME_Objects_Python_API/index.html
I think @redgeographics idea of routing via a published parameter test is the best one if you have a set of well known ahead of time transformations you want to apply.
Though I'd be surprised if it would be truly dynamic -- that is, you want the user to supply a custom transformer and you'll apply it -- you could accomplish such a thing by copying the custom transformer on top of whatever one the workspace was authored to use. I also played around and the FMEFunctionCaller allows you to inject an arbitrary line into the resulting workspace mapping file so a super duper advanced old timer (say, a SpatialDirect veteran) could use that to INCLUDE a mapping file fragment and then apply hand-coded transformation dynamically.
Not sure I'd recommend it though.
Peter Laulund could probably do it :)
I think @redgeographics idea of routing via a published parameter test is the best one if you have a set of well known ahead of time transformations you want to apply.
Though I'd be surprised if it would be truly dynamic -- that is, you want the user to supply a custom transformer and you'll apply it -- you could accomplish such a thing by copying the custom transformer on top of whatever one the workspace was authored to use. I also played around and the FMEFunctionCaller allows you to inject an arbitrary line into the resulting workspace mapping file so a super duper advanced old timer (say, a SpatialDirect veteran) could use that to INCLUDE a mapping file fragment and then apply hand-coded transformation dynamically.
Not sure I'd recommend it though.
Thanks Dale, I like this idea. I don't want to be restricted to pre-defined transformers in a workspace because this would ideally be driven by information coming from database tables and everything would happen at runtime.
I'll play around with the FMEFunctionCaller to see if I can get it to do what I need but I think the idea of copying the required transformer over a pre-defined transformer would probably be the simplest solution.
It would be really nice if Safe could implement a "CustomTransformerCaller" transformer. :)
I think @redgeographics idea of routing via a published parameter test is the best one if you have a set of well known ahead of time transformations you want to apply.
Though I'd be surprised if it would be truly dynamic -- that is, you want the user to supply a custom transformer and you'll apply it -- you could accomplish such a thing by copying the custom transformer on top of whatever one the workspace was authored to use. I also played around and the FMEFunctionCaller allows you to inject an arbitrary line into the resulting workspace mapping file so a super duper advanced old timer (say, a SpatialDirect veteran) could use that to INCLUDE a mapping file fragment and then apply hand-coded transformation dynamically.
Not sure I'd recommend it though.
On second thoughts, not sure the "copying over" approach will be the best solution as I won't be able to guarantee the workspace is run in isolation (ie. multiple requests may come in at the same time, each requiring a different custom transformer). Perhaps the FMEFunctionCaller is the best approach after all...
Time to start playing!
I think @redgeographics idea of routing via a published parameter test is the best one if you have a set of well known ahead of time transformations you want to apply.
Though I'd be surprised if it would be truly dynamic -- that is, you want the user to supply a custom transformer and you'll apply it -- you could accomplish such a thing by copying the custom transformer on top of whatever one the workspace was authored to use. I also played around and the FMEFunctionCaller allows you to inject an arbitrary line into the resulting workspace mapping file so a super duper advanced old timer (say, a SpatialDirect veteran) could use that to INCLUDE a mapping file fragment and then apply hand-coded transformation dynamically.
Not sure I'd recommend it though.
For what it's worth, I've implemented a successful test case for this on FME Desktop.
I created a Python scripted parameter which copies the required transformer over at run time. I initially tried this with a startup Python script but there are obviously some subtle differences in runtime between the startup scripts and the scripted parameters. The startup script runs too late in the workspace initialisation for the copied custom transformer to get used.
Next test... implement this on FME Server.
I'll post my results once done.
If you're not intimidated by some Python programming, your can accomplish a lot using the fmeobjects API inside a PythonCaller. It's really easy to analyse features based on attributes, geometry types, etc and to act accordingly. Starting with FME 2017 even raster features are supported.
The fmeobjects documentation gives a pretty good overview of the capabilities:
http://docs.safe.com/fme/html/FME_Objects_Python_API/index.html
David, thanks for this suggestion, however, I want a solution that will be easy to implement and maintain in the future. I'm personally not averse to using custom Python but it's not very transparent what's going on and I want other developers to be able use whatever solution gets implemented in a "plug and play" manner. Being able to have them reference a custom FME transformer seems the simplest solution.
Yes, sort of, I think.
If you have the custom transformers laid out in your workspace in parallel streams you can route features to them by testing on a user parameter.
But you have to have the processing designed and in your workspace for this to work (either directly in the workspace or in custom transformers). If you want something truly dynamic you could attempt to generate mapping files on the fly, store them in a temp location and call them using a WorkspaceRunner (I'm not even sure about how this would work on FME Server, I think you have to register the mapping files with a repository for them to become available)
Thanks redgeo. Parallel streams won't work because I want the solution to be generic and not hard-coded into the workspace. Using a workspace runner won't work because I want the solution to run against every feature in the controlling workspace. A workspace runner would fire up a new FME for each feature and would be very slow.
For what it's worth, I've implemented a successful test case for this that works on both FME Desktop and FME Server, with some minor limitations.
The attached zip file contains a very simple workspace and some custom transformers for testing the solution.
dynamic-transformer-caller.zip
General Solution
I created a Python scripted parameter which copies the required transformer over from a custom resources folder to the FME transformers folder at run time. I initially tried this with a startup Python script but had to switch to a scripted parameter. There are obviously some subtle differences in runtime between the startup scripts and the scripted parameters. The startup script runs too late in the workspace initialisation for the copied custom transformer to get used.
Limitations
The solution works well but has some limitations that I'm aware of:
- each custom transformer instance must have the exact same name defined for the "Name" value in the Transformer Properties, although the actual file names can (and should) be different.
- each custom transformer must also expose the same input parameters. If different parameters are required, then they can be passed in as feature attributes and exposed using the AttributeExposer inside the custom transformer.
Strengths
One of my main concerns was how FME Server would behave in the event of a conflict. Due to the way the solution is implemented, the custom transformer has to be copied over to a shared resources directory at run time. This directory is shared by all users of FME Server and there is no way to prevent other users from running the same workspace at the same time, using a different custom transformer. My brief testing indicates that the file copy is very fast and occurs at the time the workspace is first loaded. Immediately after the file copy has occurred the transformer is loaded into memory for the workspace to reference. Copying a different transformer into the shared directory after the copy has occurred does not change the internal reference of the transformer being used by the workspace and each workspace uses the correct version of transformer.
