Solved

Export 1 feature per PostGIS-table


Badge

Hello fellow, GISers.

Simple yet complex problem. I want to export 1 feature per table for a schema in my PostGIS-database for test purposes.

I want to output the data to shapefiles with filename=table name and output folder=schema name.

I want the whole setup to be dynamic since I want to run it later on all my schemas (a schema list).

This seems more tricky than I thought it would be.

Any ideas?

icon

Best answer by nielsgerrits 20 December 2016, 11:33

View original

10 replies

Userlevel 5
Badge +25

In the Navigator you can go into the reader parameters, under "Features To Read" there's a "Max Features To Read Per Feature Type" parameter, set that to 1 and it'll read just 1 feature per table.

Hope this helps.

Badge

Thanks, redgeographics, I know. But I can't seem to get that into a dynamic workflow. Guess I'll have to try harder.

Userlevel 5
Badge +25

Oh wait, you want it dynamic... yes, that would make it a bit trickier. I thought setting it to Single Merged Feature Type on the reader would work but that limits the output to just one feature (works as designed I suppose...)

You could remove that Max Features To Read Per Feature Type parameter, merge everything into a single feature type and route it through a Sampler set to group by fme_feature_type and sample the first feature. This will effectively do what you want but I think it'll still read all of your records so on large databases that's a lot of overhead.

Badge

Oh wait, you want it dynamic... yes, that would make it a bit trickier. I thought setting it to Single Merged Feature Type on the reader would work but that limits the output to just one feature (works as designed I suppose...)

You could remove that Max Features To Read Per Feature Type parameter, merge everything into a single feature type and route it through a Sampler set to group by fme_feature_type and sample the first feature. This will effectively do what you want but I think it'll still read all of your records so on large databases that's a lot of overhead.

Thanks again. Yes, I tried that too and it could work although there will be som overhead. I can live with that.

 

 

When read from a Postgis dataset the feature type is schema_tablename. Do you know how I can get that to be only tablename but still keep the schema?

 

 

Userlevel 5
Badge +25
Thanks again. Yes, I tried that too and it could work although there will be som overhead. I can live with that.

 

 

When read from a Postgis dataset the feature type is schema_tablename. Do you know how I can get that to be only tablename but still keep the schema?

 

 

A StringReplacer to replace the regex ^[A-Za-z]+_ with an empty string. That's off the top of my head and untested by the way, so you may need to experiment a bit with it.

 

 

I've been trying to see if there's maybe a magic SQL statement that can do this. The problem is that in a dynamic workflow you don't know what tables you'll be running on.

 

 

Badge
A StringReplacer to replace the regex ^[A-Za-z]+_ with an empty string. That's off the top of my head and untested by the way, so you may need to experiment a bit with it.

 

 

I've been trying to see if there's maybe a magic SQL statement that can do this. The problem is that in a dynamic workflow you don't know what tables you'll be running on.

 

 

I tried SQL too, but couldn't find a suitable way. Let me know if you discover some goodies.

 

 

If I change the feature type name I loose the schema. Can that be solved with some smart setting or will that need to be fixed manually...?

 

 

 

Userlevel 6
Badge +32

Totally doable (in desktop 2016.1.2.0)

Badge

Totally doable (in desktop 2016.1.2.0)

Thanks, nielsgerrits! I like this solution very much indeed. I hade to make som arrangements to fanout on schema name, but nothing serious.

 

 

Only one thing left now. I want the output as shapefile=table_name. Is there an easy way to manipulate the fme_feature_type without loosing the schema?

 

 

Userlevel 6
Badge +32
Thanks, nielsgerrits! I like this solution very much indeed. I hade to make som arrangements to fanout on schema name, but nothing serious.

 

 

Only one thing left now. I want the output as shapefile=table_name. Is there an easy way to manipulate the fme_feature_type without loosing the schema?

 

 

Happy to help @peteralstorp

 

The trick is to edit fme_feature_type_name in the SchemaFeature and fme_feature_type in the GenericFeature. You can easy overwrite these attributes with AttributeCreators. They need to be exactly the same to let the FeatureWriter work this way.

 

 

Another helpful hint is the attribute handling in the PostGIS FeatureReader Parameters. This way you can preserve the table_schema and table_name attributes.

 

 

I would prefix these attributes to prevent problems with duplicate attributes and have the same attributes in both SchemaFeature and GenericFeature. (The prefix option in the FeatureReader only works for the GenericFeature, not the SchemaFeature and I prefer a 1 to 1 situation.)

 

 

Badge
Happy to help @peteralstorp

 

The trick is to edit fme_feature_type_name in the SchemaFeature and fme_feature_type in the GenericFeature. You can easy overwrite these attributes with AttributeCreators. They need to be exactly the same to let the FeatureWriter work this way.

 

 

Another helpful hint is the attribute handling in the PostGIS FeatureReader Parameters. This way you can preserve the table_schema and table_name attributes.

 

 

I would prefix these attributes to prevent problems with duplicate attributes and have the same attributes in both SchemaFeature and GenericFeature. (The prefix option in the FeatureReader only works for the GenericFeature, not the SchemaFeature and I prefer a 1 to 1 situation.)

 

 

Again - THANKS! I have wanted to do this many times but failed somehow. This will too be very helpful for me in the future (and in this particular case).

 

 

Reply