Skip to main content

So the scenario is: I have an SDE featureclass with attributes A,B,C. User has shapefile with attributes A,bob,C,D. I want to create a workspace to be run on FME server that will allow the user to map one, many, or none of the fields. I'm not sure how to go about this. I also want to map the matched ones so the users will only have to match the remaining, but still give them the option to change the mapped one if they so choose. Thanks.

Hi @tnarladni,

I believe that use the publish parameter option. Link Publish Parameter

Once configurated your Publish Parameter in FME Workbench and publish it to FME Server.

On FME Server the user can to choose to map field to run your workspace.

Thanks,

Danilo


I'm unclear what "to map attributes" means in the context. Could you please elaborate a bit more?

 


I have a picture:) mapattributeatruntime.png

 

 


From your example I would say: if the attributes do not match but the writer is set to automatic, it is not a problem that some attributes might be missing?


Do you need to allow the user to configure columns in the destination table at each run?

 

For example, a user can create a table consisting of "RoadName", "SurveyCo", and "SurfveyDate" columns; another user can create a table consisting of "RoadName" and "SurfaceMat" columns; and so on.

 

 


From your example I would say: if the attributes do not match but the writer is set to automatic, it is not a problem that some attributes might be missing?

nothing-matched.png except the shapefiles are more like this. :(

 

 


Do you need to allow the user to configure columns in the destination table at each run?

 

For example, a user can create a table consisting of "RoadName", "SurveyCo", and "SurfveyDate" columns; another user can create a table consisting of "RoadName" and "SurfaceMat" columns; and so on.

 

 

no. Users just need to pick a column from the shapefile they are trying to load that matches the existing column in the destination table.

 

 


@takashi any ideas? The featureReader and see the fields through the generic port but I'm not sure how to display them so users can match them up to the destination attributes.
@takashi any ideas? The featureReader and see the fields through the generic port but I'm not sure how to display them so users can match them up to the destination attributes.
Sorry, I'm still unclear what your goal is.

 

 

Assume the destination feature type has the attributes A, B, and C.

 

  • When a source data has attributes A, B, C, 😨 A, B, C will be mapped to the destination attributes and D will be just ignored.
  • When a source data has attributes A, C: A, C will be mapped and B will be missing.
Then, what you want?

 

 


Sorry, I'm still unclear what your goal is.

 

 

Assume the destination feature type has the attributes A, B, and C.

 

  • When a source data has attributes A, B, C, 😨 A, B, C will be mapped to the destination attributes and D will be just ignored.
  • When a source data has attributes A, C: A, C will be mapped and B will be missing.
Then, what you want?

 

 

Hi @takashi,

 

@tnarladni visited live chat, and gave some clarification on the scenario. Let me see if I can explain:

 

Say the destination SDE Schema is A, B, C, D, E

 

User shapefile schema could be different from destination, and it also varies from time to time. e.g. it could be A, bob, C, D, or A, B, cat, dog.

 

We need the matching attribute names to be automatically mapped, meanwhile allow user to manually map from bob -> B, cat ->C, dog->D, at run time.

 

When source schema doesn't have any attribute to map to a particular attribute on the destination, it's ok to write missing value to that attribute. (E could be missing)

 

I'm also try to brainstorm some ideas with colleagues, but any suggestions from anyone on the community would be much appreciated.

 


Hi @tnarladni,

If your user is able to edit a local CSV file at each run in order to match whichever attribute names vary between the source and destination schemas, you may be able to set up a SchemaMapper in your translation. Take a look at this article for some ideas on how to set this up. In its simplest form the CSV is two columns, one for the Source attribute name, and one for the Destination attribute name.

Here's an example:

SourceDestinationAAbobBCCDogD

The SchemaMapper reads this file at each run, so it would need to be edited and saved before running, and will likely need to be edited each time as long as the schema varies each time.

Hope this helps spark a solution!

Thanks,

 

Nathan
@takashi any ideas? The featureReader and see the fields through the generic port but I'm not sure how to display them so users can match them up to the destination attributes.
Thanks @XiaomengAtSafe! That is exactly right.

 


Hi @tnarladni,

If your user is able to edit a local CSV file at each run in order to match whichever attribute names vary between the source and destination schemas, you may be able to set up a SchemaMapper in your translation. Take a look at this article for some ideas on how to set this up. In its simplest form the CSV is two columns, one for the Source attribute name, and one for the Destination attribute name.

Here's an example:

SourceDestinationAAbobBCCDogD

The SchemaMapper reads this file at each run, so it would need to be edited and saved before running, and will likely need to be edited each time as long as the schema varies each time.

Hope this helps spark a solution!

Thanks,

 

Nathan
@NathanAtSafe...That was the idea I have except the users will not have a software package to read a shapefile to update this csv. I thought about doing it for them, via a featureReader to get the list of attributes, a featureWriter to write it out to the csv. But then I'm still stuck on how to match the user provided attributes with the SDE's attributes and also display something to allow the users to match those attributes that I can't match automatically.

 


Hi @tnarladni,

One of the suggestions I got from the team is along the same lines of @NathanAtSafe's thought, but using a separate workspace to scan for the input file's schema, and present it in a web form (that you'll need to create) for users to select desired mapping. and this generates a csv file that will be used in a SchemaMapper in a second workspace.

Our Easy Geocoder example uses a very similar idea. And we have detailed information, including some sample workspace and source code for the web form in this link: https://knowledge.safe.com/articles/1116/easygeocoder-data-driven-self-serve-data-transform.html

Hopefully this gives you a good starting point.


If the number of destination attributes was not so large, I would create published parameters corresponding to each destination attribute, so that the user can specify source attribute names to be mapped to the destination attributes for each, like this.

0684Q00000ArKvTQAV.png

Then, in the workspace, rename the source attribute names based on the parameter values immediately after reading from the source dataset. Maybe writing a Python script would be an easy way to perform renaming.

# PythonCaller Script Example
import fme
class AttributeMapper(object):
    def __init__(self):
        # Destination attribute names = published parameter names
        destAttributes = Â
            'AccessRdId',
            'NewExist',
            'PermTemp',
            'PublicPriv',
            'RoadName',
        ]
        self.map = {}
        for dest in destAttributes:
            src = fme.macroValuesidest]
            if src and src != dest:
                self.map src] = dest
        
    def input(self, feature):
        for src, dest in self.map.items():
            null, missing, type = feature.getAttributeNullMissingAndType(src)
            if null:
                feature.setAttributeNullWithType(dest, type)
            elif not missing:
                feature.setAttribute(dest, feature.getAttribute(src))
            feature.removeAttribute(src)
        self.pyoutput(feature)

Hi @tnarladni,

One of the suggestions I got from the team is along the same lines of @NathanAtSafe's thought, but using a separate workspace to scan for the input file's schema, and present it in a web form (that you'll need to create) for users to select desired mapping. and this generates a csv file that will be used in a SchemaMapper in a second workspace.

Our Easy Geocoder example uses a very similar idea. And we have detailed information, including some sample workspace and source code for the web form in this link: https://knowledge.safe.com/articles/1116/easygeocoder-data-driven-self-serve-data-transform.html

Hopefully this gives you a good starting point.

That was the part I was missing. I kept thinking somehow I could do the mapping within 1 or 2 workspaces, but it needed to be handled by an outside application. In this case, a web form. Thanks for the link, that is exactly what I'm looking for.

 

 


Reply