Solved

Hi, I'm struggling with FeatureReader whilst reading Geopackage to postgis

a_ericsson
Rank
Badge +3

The workflow looks like this: I'm downloading a .zip file via an API and unzipping it using the ZipExtractor. Connecting the ZipExtractor 'File' output to a Feature reader, I was hoping to be able to upload the gpkg-file from 5 different output ports to 5 tables in a postgis database. This works fine if I specify the geopackage file manually in the 'Output' section of the feature reader. Is there a way to use a transformer or to edit the FeatureReader to make it read the output ports directly/automatically from the gpkg-file in the folder the ZipExtractor has unzipped?

 

Thanks!

 

/Alex

icon

Best answer by caracadrian 20 May 2022, 13:26

View original

4 replies

caracadrian
Rank
Badge +20

If the tables inside the .gpkg files are always the same you can hard code Feature Types to Read and Output Ports -> Specified. See picture below.

You can even bypass ZipExtractor and send the path of the .zip file directly into FeatureReader.

GPKG FeatureReader from path

a_ericsson
Rank
Badge +3

If the tables inside the .gpkg files are always the same you can hard code Feature Types to Read and Output Ports -> Specified. See picture below.

You can even bypass ZipExtractor and send the path of the .zip file directly into FeatureReader.

GPKG FeatureReader from path

Hey, thanks! My problem, if I have understood this correctly, would be that I would have to specify the .gpkg file many times over. As I will run the script probably on a weekly basis a new file with a new folder name will be downloaded each time, which would make the specified path obsolete. The total output is around 80 tables in the postgis database, so specifying the files manually would be very time-consuming, which is what I'm trying to avoid :) I might be missing something obvious..

caracadrian
Rank
Badge +20

Hey, thanks! My problem, if I have understood this correctly, would be that I would have to specify the .gpkg file many times over. As I will run the script probably on a weekly basis a new file with a new folder name will be downloaded each time, which would make the specified path obsolete. The total output is around 80 tables in the postgis database, so specifying the files manually would be very time-consuming, which is what I'm trying to avoid :) I might be missing something obvious..

If the 80 or so table names are the same each week you can set Feature Types to Read and Output Ports only when you first set up FeatureReader (by selecting your .gpkg file). Most of the time API's don't change over time 😀

The path should come in as an attribute (_response_file_path attribute from HTTPCaller or path_windows from ZipExtractor) so you don't have to worry about that.

Or, another way to explain it, the first time you set FeatureReader you use a .gpkg file downloaded from the API that is already stored on your disk, then you change Dataset to an attribute that contains the file path. The settings should stay the same by default. If not, use that .gpkg file to set Feature Types to Read and Output Ports -> Specified after you changed the Dataset parameter to an attribute value.

If this is not the case, please give us some more info about the API response and the schema(s) of the .gpkg file(s).

a_ericsson
Rank
Badge +3

Hey, thanks! My problem, if I have understood this correctly, would be that I would have to specify the .gpkg file many times over. As I will run the script probably on a weekly basis a new file with a new folder name will be downloaded each time, which would make the specified path obsolete. The total output is around 80 tables in the postgis database, so specifying the files manually would be very time-consuming, which is what I'm trying to avoid :) I might be missing something obvious..

Yes, the tables are the same every week, luckily :) I think I understand now, the file I use to specify the output ports do not affect which .gpkg-file is read to the postgis database? If that is the case, I'll give it a go and hope for the best!

 

Thank you so much for your help and patience :) Still figuring this out! Have a nice weekend!

Reply


Community Stats

28,741
Posts
109,578
Replies
36,583
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.