Skip to main content
Solved

Hi, I'm struggling with FeatureReader whilst reading Geopackage to postgis


a_ericsson
Contributor
Forum|alt.badge.img+3

The workflow looks like this: I'm downloading a .zip file via an API and unzipping it using the ZipExtractor. Connecting the ZipExtractor 'File' output to a Feature reader, I was hoping to be able to upload the gpkg-file from 5 different output ports to 5 tables in a postgis database. This works fine if I specify the geopackage file manually in the 'Output' section of the feature reader. Is there a way to use a transformer or to edit the FeatureReader to make it read the output ports directly/automatically from the gpkg-file in the folder the ZipExtractor has unzipped?

 

Thanks!

 

/Alex

Best answer by caracadrian

a_ericsson wrote:

Hey, thanks! My problem, if I have understood this correctly, would be that I would have to specify the .gpkg file many times over. As I will run the script probably on a weekly basis a new file with a new folder name will be downloaded each time, which would make the specified path obsolete. The total output is around 80 tables in the postgis database, so specifying the files manually would be very time-consuming, which is what I'm trying to avoid :) I might be missing something obvious..

If the 80 or so table names are the same each week you can set Feature Types to Read and Output Ports only when you first set up FeatureReader (by selecting your .gpkg file). Most of the time API's don't change over time 😀

The path should come in as an attribute (_response_file_path attribute from HTTPCaller or path_windows from ZipExtractor) so you don't have to worry about that.

Or, another way to explain it, the first time you set FeatureReader you use a .gpkg file downloaded from the API that is already stored on your disk, then you change Dataset to an attribute that contains the file path. The settings should stay the same by default. If not, use that .gpkg file to set Feature Types to Read and Output Ports -> Specified after you changed the Dataset parameter to an attribute value.

If this is not the case, please give us some more info about the API response and the schema(s) of the .gpkg file(s).

View original
Did this help you find an answer to your question?

4 replies

caracadrian
Contributor
Forum|alt.badge.img+22
  • Contributor
  • May 20, 2022

If the tables inside the .gpkg files are always the same you can hard code Feature Types to Read and Output Ports -> Specified. See picture below.

You can even bypass ZipExtractor and send the path of the .zip file directly into FeatureReader.

GPKG FeatureReader from path


a_ericsson
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • May 20, 2022
caracadrian wrote:

If the tables inside the .gpkg files are always the same you can hard code Feature Types to Read and Output Ports -> Specified. See picture below.

You can even bypass ZipExtractor and send the path of the .zip file directly into FeatureReader.

GPKG FeatureReader from path

Hey, thanks! My problem, if I have understood this correctly, would be that I would have to specify the .gpkg file many times over. As I will run the script probably on a weekly basis a new file with a new folder name will be downloaded each time, which would make the specified path obsolete. The total output is around 80 tables in the postgis database, so specifying the files manually would be very time-consuming, which is what I'm trying to avoid :) I might be missing something obvious..


caracadrian
Contributor
Forum|alt.badge.img+22
  • Contributor
  • Best Answer
  • May 20, 2022
a_ericsson wrote:

Hey, thanks! My problem, if I have understood this correctly, would be that I would have to specify the .gpkg file many times over. As I will run the script probably on a weekly basis a new file with a new folder name will be downloaded each time, which would make the specified path obsolete. The total output is around 80 tables in the postgis database, so specifying the files manually would be very time-consuming, which is what I'm trying to avoid :) I might be missing something obvious..

If the 80 or so table names are the same each week you can set Feature Types to Read and Output Ports only when you first set up FeatureReader (by selecting your .gpkg file). Most of the time API's don't change over time 😀

The path should come in as an attribute (_response_file_path attribute from HTTPCaller or path_windows from ZipExtractor) so you don't have to worry about that.

Or, another way to explain it, the first time you set FeatureReader you use a .gpkg file downloaded from the API that is already stored on your disk, then you change Dataset to an attribute that contains the file path. The settings should stay the same by default. If not, use that .gpkg file to set Feature Types to Read and Output Ports -> Specified after you changed the Dataset parameter to an attribute value.

If this is not the case, please give us some more info about the API response and the schema(s) of the .gpkg file(s).


a_ericsson
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • May 20, 2022
a_ericsson wrote:

Hey, thanks! My problem, if I have understood this correctly, would be that I would have to specify the .gpkg file many times over. As I will run the script probably on a weekly basis a new file with a new folder name will be downloaded each time, which would make the specified path obsolete. The total output is around 80 tables in the postgis database, so specifying the files manually would be very time-consuming, which is what I'm trying to avoid :) I might be missing something obvious..

Yes, the tables are the same every week, luckily :) I think I understand now, the file I use to specify the output ports do not affect which .gpkg-file is read to the postgis database? If that is the case, I'll give it a go and hope for the best!

 

Thank you so much for your help and patience :) Still figuring this out! Have a nice weekend!


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings