When using a Generic Reader with Guess format from extension, I know it is possible to create a "dummy" resource in the workbench to pass configuration parameters, but how do you set the preferred order of readers that is applied? Say for argument's sake I need to read TAB files using MITAB (not MAPINFO) or SHP files using SHAPEFILE (not ESRISHAPE). I'm not making a judgement on which reader is preferred, rather if you are required to use one that isn't the default, how do you do that?
Solved
Force Generic reader to choose MITAB over MAPINFO
+1Best answer by virtualcitymatt
I think it's just alphabetical I'm not actually sure if you can choose the reader type. One workaround cooould be to create a custom format with the preffered readers inside and then start the format names with 'A' - pretty hacky and the Generic Reader may ignore custom formats. If there isn't an idea already for this you should post one. It's kind of annoying.
+34- Celebrity
- Best Answer
I think it's just alphabetical I'm not actually sure if you can choose the reader type. One workaround cooould be to create a custom format with the preffered readers inside and then start the format names with 'A' - pretty hacky and the Generic Reader may ignore custom formats. If there isn't an idea already for this you should post one. It's kind of annoying.
+34- Celebrity
virtualcitymatt wrote:
I think it's just alphabetical I'm not actually sure if you can choose the reader type. One workaround cooould be to create a custom format with the preffered readers inside and then start the format names with 'A' - pretty hacky and the Generic Reader may ignore custom formats. If there isn't an idea already for this you should post one. It's kind of annoying.
The other option would be to use a PathReader then set up your own preferences for the generic reader based on the extensions. You can then specify the format in the FeatureReader. Also annoying but doable
+1virtualcitymatt wrote:
The other option would be to use a PathReader then set up your own preferences for the generic reader based on the extensions. You can then specify the format in the FeatureReader. Also annoying but doable
Thanks for the comments. In normal circumstances I'd definitely do something logical like this, however I am required to read from a set of formats defined in a JSON file so I can't hardcode any format selection without invalidating the JSON file being used to add new formats in the future.
+34- Celebrity
jstanger wrote:
Thanks for the comments. In normal circumstances I'd definitely do something logical like this, however I am required to read from a set of formats defined in a JSON file so I can't hardcode any format selection without invalidating the JSON file being used to add new formats in the future.
Ahh interesting challenge. So you have a JSON file which lists several formats and could be changed to add more? You need to be able to support these formats. You can't hard code anything like using reader as resource because it's not future proof?
I would try to set up a process which parses the formats in the JSON and then based on the files extension give the format from the json as an attribute which you can then use in the Generic Feature Reader
+17- Contributor
Not sure if it will work, but what you could try is adding a MITAB reader (no feature types) to force TAB files in the Generic Reader to use MITAB.
That is usually the only way to add settings for readers in the Generic Reader.
Maybe it also works forcing a specific reader for a file extension.
Worth the try.
+1erik_jan wrote:
Not sure if it will work, but what you could try is adding a MITAB reader (no feature types) to force TAB files in the Generic Reader to use MITAB.
That is usually the only way to add settings for readers in the Generic Reader.
Maybe it also works forcing a specific reader for a file extension.
Worth the try.
I have tried before using a "dummy" resource as the MITAB reader and the Generic Reader still reported the format as MAPINFO for the fme_format_short descriptor. Just tried adding a MITAB reader with no feature types and the same result. Good idea though, was worth a try.
+1jstanger wrote:
Thanks for the comments. In normal circumstances I'd definitely do something logical like this, however I am required to read from a set of formats defined in a JSON file so I can't hardcode any format selection without invalidating the JSON file being used to add new formats in the future.
I can add a reader resource, just not a transformer dedicated to a specific reader format. It simply is impractical and unfair to expect FME to be able to magically know how to handle newly added formats to the config file perfectly, however, it is required that if a new format is added then FME will take its best guess. If the best guess isn't good enough then additions to the workbench will be requested to control the reader. The current list is about 8 different formats. I have the same issue on the writer side but in that case I simply have to make the call that if you want to add a new writer format, you need to edit the workbench. It is impossible otherwise.
But, what has been bothering me, is that even if I add a reader resource, FME chooses to use a different reader for that format and I can't set a preference to get it to use the resource (specifically MAPINFO vs MITAB as an example).
+34- Celebrity
jstanger wrote:
I can add a reader resource, just not a transformer dedicated to a specific reader format. It simply is impractical and unfair to expect FME to be able to magically know how to handle newly added formats to the config file perfectly, however, it is required that if a new format is added then FME will take its best guess. If the best guess isn't good enough then additions to the workbench will be requested to control the reader. The current list is about 8 different formats. I have the same issue on the writer side but in that case I simply have to make the call that if you want to add a new writer format, you need to edit the workbench. It is impossible otherwise.
But, what has been bothering me, is that even if I add a reader resource, FME chooses to use a different reader for that format and I can't set a preference to get it to use the resource (specifically MAPINFO vs MITAB as an example).
This is how I work around the problem
The only thing that needs to ever be updated here is the attribute value mapper.
If you have customers using this workspace you can actually wrap the AttributeValueMapper up as a custom transformer and publish it to the hub. Customers can the easily update the transformer to the new version to get the new mappings which you have defined.
If instead you are running this on the FME Server you could set up a notification if an unrecognised format is detected so you can quickly update it.
I agree that Safe could perhaps improve the way they define the 'best guess' or even create a WorkspaceParameter where you can define a mapping.
+1virtualcitymatt wrote:
Ahh interesting challenge. So you have a JSON file which lists several formats and could be changed to add more? You need to be able to support these formats. You can't hard code anything like using reader as resource because it's not future proof?
I would try to set up a process which parses the formats in the JSON and then based on the files extension give the format from the json as an attribute which you can then use in the Generic Feature Reader
So I had already setup my config JSON file to store exactly this data. When I pass in the feature with the path to the file to read, that same feature has the expected reader name as FME would refer to it. So with almost no modification this worked very well. I had completely missed that a generic reader has a parameter that can be filled by attributes that is the format name.
Thanks!
Reply
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.