Solved

How do I parameterise the layer name of a ArcGIS Portal Feature Service reader?


I am trying to figure out how to parameterise the layer name for a Portal Feature Service reader.

 

In the screenshot below you can see the highlighted Layer Name field. However it does not seem to have the ability to set it as a user parameter.

 

You can see it is possible to set other layer variables as parameters, eg Input Geometry Type, etc. Just wondering how I might do the same for the layer name?

fme

icon

Best answer by sparki 23 August 2021, 10:37

View original

14 replies

Userlevel 6
Badge +32

You can do this in the Navigator pane on the left. Select the Reader, fold out, double click on Portal URL:.., fold out Parameters.

Or you can use a FeatureReader, which more intuitive in my opinion. You can initiate a FeatureReader with a Creator.

You can do this in the Navigator pane on the left. Select the Reader, fold out, double click on Portal URL:.., fold out Parameters.

Or you can use a FeatureReader, which more intuitive in my opinion. You can initiate a FeatureReader with a Creator.

fmeThanks for the reply.

Unfortunately, the layer name is not one of the parameters in the parameters dialog. The feature service name is but I need the name of the layer within that service to be parameterised.

I need to be able to choose the feature service layer at runtime.

If you have some time, I'm wondering if you can elaborate on the FeatureReader option please.

 

Userlevel 6
Badge +32

fmeThanks for the reply.

Unfortunately, the layer name is not one of the parameters in the parameters dialog. The feature service name is but I need the name of the layer within that service to be parameterised.

I need to be able to choose the feature service layer at runtime.

If you have some time, I'm wondering if you can elaborate on the FeatureReader option please.

 

Ah I see what you mean. This is doable with the FeatureReader:

2021-08-11_06h55_24

fmeThanks for the reply.

Unfortunately, the layer name is not one of the parameters in the parameters dialog. The feature service name is but I need the name of the layer within that service to be parameterised.

I need to be able to choose the feature service layer at runtime.

If you have some time, I'm wondering if you can elaborate on the FeatureReader option please.

 

Interestingly, the Portal feature service writer does allow you parameterise the layer, just not the reader.

I think I understand the FeatureReader solution. You make a user parameter of Feature Types but I wonder which output do you grab from? Is it the Initiator, Generic or Schema?

fme

Userlevel 6
Badge +32

fmeThanks for the reply.

Unfortunately, the layer name is not one of the parameters in the parameters dialog. The feature service name is but I need the name of the layer within that service to be parameterised.

I need to be able to choose the feature service layer at runtime.

If you have some time, I'm wondering if you can elaborate on the FeatureReader option please.

 

To be honest, I quit using the classic readers and writers when the FeatureReader and FeatureWriter were introduced. Somehow I always need to daisy chain readers and writers and these feel more intuitive for me.

 

If "Feature Types to Read" is dynamic, you choose Single Output Port and get the features from Generic. You then have to expose the attributes using the AttributeExposer because the input can, and probably will, vary. Because you configured it dynamic. Please note you can simply import the attributes from a sample file.

 

If you need to write the data you read, outputting the same attributes, you can configure that dynamic as well. This is where you use the Schema Feature.

fmeThanks for the reply.

Unfortunately, the layer name is not one of the parameters in the parameters dialog. The feature service name is but I need the name of the layer within that service to be parameterised.

I need to be able to choose the feature service layer at runtime.

If you have some time, I'm wondering if you can elaborate on the FeatureReader option please.

 

Thanks for your help. It is much appreciated. I have managed to get the Creator and FeatureReader to work together to allow me to dynamically use different feature service layers.

However, I'm struggling to understand how to use AttributeExposer to dynamically expose the attributes of the feature service layer. AttributeExposer allows me to set the layer parameters from a user parameter but the parameter doesn't appear to work.

 

Ideally, there would be a way to dynamically detect the attributes. I see AttributeExploder does something like this but the output is not in a format that I can use readily.

 

I'm wondering if you can expand on the AttributeExposer and how it might be to expose field dynamically?

Userlevel 6
Badge +32

fmeThanks for the reply.

Unfortunately, the layer name is not one of the parameters in the parameters dialog. The feature service name is but I need the name of the layer within that service to be parameterised.

I need to be able to choose the feature service layer at runtime.

If you have some time, I'm wondering if you can elaborate on the FeatureReader option please.

 

I don't think there is a way of dynamic exposing attributes. Transformers can't be setup with "to be determined" attributes. If you want to write dynamic you can simply use the schema of the input data. But I understand the struggle as it took me a while to understand how this works.

Can you explain what you try to achieve? Maybe we can achieve this another way.

fmeThanks for the reply.

Unfortunately, the layer name is not one of the parameters in the parameters dialog. The feature service name is but I need the name of the layer within that service to be parameterised.

I need to be able to choose the feature service layer at runtime.

If you have some time, I'm wondering if you can elaborate on the FeatureReader option please.

 

I'm trying to create a generic workbench that can be dynamic in what inputs/outputs it is dealing with. Specifically, I would like to be able to nominate a particular feature service layer and a shapefile (with the same schema) for comparison using change detector. There is more down stream processing but I'm focusing on this bit for now.

So, ideally, I'd be able to pass in:

  • Portal url
  • Portal creds
  • Feature service
  • Feature service layer
  • Shapefile

fmeThanks for the reply.

Unfortunately, the layer name is not one of the parameters in the parameters dialog. The feature service name is but I need the name of the layer within that service to be parameterised.

I need to be able to choose the feature service layer at runtime.

If you have some time, I'm wondering if you can elaborate on the FeatureReader option please.

 

I did take a look at the following post for inspiration:

 

https://community.safe.com/s/question/0D54Q000080hOlB/featurereader-wont-accept-my-attributes-to-expose-input

 

It does generate a CSV file with the field names of the feature service layer (I swap out Creator for FeatureReader - schema only) but not sure how to use that CSV file to dynamically assign attributes. Assigning it to another FeatureReader with schema and data doesn't seem to work.

Userlevel 6
Badge +32

fmeThanks for the reply.

Unfortunately, the layer name is not one of the parameters in the parameters dialog. The feature service name is but I need the name of the layer within that service to be parameterised.

I need to be able to choose the feature service layer at runtime.

If you have some time, I'm wondering if you can elaborate on the FeatureReader option please.

 

What I would do:

  • Generic FeatureReaders for portal feature service layer and shape.
  • AttributeExposer for attributes you know you will get and differ, like fme basename.
  • ChangeDetector to check for changes. Attribute Matching Strategy: "Match All Except Selected Attributes". Select attributes that you don't want to check, like fme_basename.

fmeThanks for the reply.

Unfortunately, the layer name is not one of the parameters in the parameters dialog. The feature service name is but I need the name of the layer within that service to be parameterised.

I need to be able to choose the feature service layer at runtime.

If you have some time, I'm wondering if you can elaborate on the FeatureReader option please.

 

Thanks again for thinking about this.

I did get something working. Luckily I have a Python wrapper that will be calling this workbench. The Python wrapper is able to get a list of fields to expose, which it passes to the workbench as a parameter. That parameter is assigned to the feature reader's 'attributes to expose' field. That seems to work.

Downstream I am now trying to write output using a FeatureWriter to a feature class in a file geodatabase. I have set the FeatureWriter to have a dynamic schema defnition, ie with Schema Sources set to "Schema From Schema Feature". However, FME does not write to feature class, producing an error:

"Geodatabase Writer: Feature Type Definition 'LocalGovernmentAreas-updated-test' was requested, but not defined."

If I do not set the FeatureWriter to have a dynamic schema then it works fine. This one has me stumped.

Userlevel 6
Badge +32

fmeThanks for the reply.

Unfortunately, the layer name is not one of the parameters in the parameters dialog. The feature service name is but I need the name of the layer within that service to be parameterised.

I need to be able to choose the feature service layer at runtime.

If you have some time, I'm wondering if you can elaborate on the FeatureReader option please.

 

The fme_feature_type (in the features) and the fme_feature_type_name (in the schema) might not match? Maybe you can overrule fme_feature_type with an AttributeCreator?

 

I did not think of the "Attributes to Expose parameter." Good find.

Userlevel 6
Badge +32

fmeThanks for the reply.

Unfortunately, the layer name is not one of the parameters in the parameters dialog. The feature service name is but I need the name of the layer within that service to be parameterised.

I need to be able to choose the feature service layer at runtime.

If you have some time, I'm wondering if you can elaborate on the FeatureReader option please.

 

@sparki​  "If I do not set the FeatureWriter to have a dynamic schema then it works fine. This one has me stumped."

 

If I'm correct, when you write to an existing featureclass or table, the writer will write all columns where the attributename is columnname, even when they are not exposed and these attributenames are not defined in the writer.

This solution works. Important points:

  • I pass in the attributes to expose via a parameter. I can do this because the workbench is being called from a Python wrapper. There is probably a way of using the Schema output from the FeatureReader to nominate which attributes to expose but I wanted to keep the workbench less cluttered and do the attribute discovery in Python.
  • It is necessary limit the features per request, particularly for complex features, such as some polygons. The default number of features sent is 1000, which can be too big if the features are complex. Also, no error is reported when the request fails, which is hard to diagnose.

 

WorkingDynamicChangeDetectorParametersThis is the solution I got working 

Reply