Skip to main content
Solved

Read shapefile with GDrive connector

geo-x
Contributor
Forum|alt.badge.img+6
  • geo-xContributor
  • Contributor

Dear FME community.

I've Shapefiles in GDrive with folders and subfolders. This is the workflow :

1/ I read them with GoogleDriveConnector, this is the result :

2/ After I use a Tester for keep the *.shp files

3/ I use a FeatureReader for read the files but that doesn't works.

This is the complete workflow :

Do you know how can I do that ?

Thank you.

Best answer by lars_de_vries

Hi @takashi and @lalandexavier,

 

I got it to work, using the previously described method!

View original
Did this help you find an answer to your question?

18 replies

lars_de_vries
Forum|alt.badge.img+10

I am wondering towards the error you receive. I presume that you used the GDrive webconnection in the FeatureReader to access the shapefile.

I would expect that the problem would be that it can't get multiple files, as is needed in the case of shapefiles. Is it possible to store the file as a ZIP-file? Possible with .SHZ extension to make it recognizable as a shapefile?

geo-x
Contributor
Forum|alt.badge.img+6
  • geo-xContributor
  • Author
  • Contributor
  • July 9, 2018
lars_de_vries wrote:

I am wondering towards the error you receive. I presume that you used the GDrive webconnection in the FeatureReader to access the shapefile.

I would expect that the problem would be that it can't get multiple files, as is needed in the case of shapefiles. Is it possible to store the file as a ZIP-file? Possible with .SHZ extension to make it recognizable as a shapefile?

Thank you @lars_de_vries. In this case I can't modify the datasource. Can I do that directly in the workflow ? (for example change shapefile 2 Geojson ?)

 

lars_de_vries
Forum|alt.badge.img+10
geo-x wrote:
Thank you @lars_de_vries. In this case I can't modify the datasource. Can I do that directly in the workflow ? (for example change shapefile 2 Geojson ?)

 

What do you exactly mean with "I can't change the datasource", isn't it possible to change the workspace or isn't possible to change the filetype on GDrive?

 

I mean that if you would want to read GeoJSON you'll still have to change both.

 

 

geo-x
Contributor
Forum|alt.badge.img+6
  • geo-xContributor
  • Author
  • Contributor
  • July 9, 2018
lars_de_vries wrote:
What do you exactly mean with "I can't change the datasource", isn't it possible to change the workspace or isn't possible to change the filetype on GDrive?

 

I mean that if you would want to read GeoJSON you'll still have to change both.

 

 

I would say : "isn't possible to change the filetype on GDrive"

 

 

takashi
Influencer
  • takashiInfluencer
  • July 9, 2018

Hi @lalandexavier, I don't think FME readers can read any dataset directly from GoogleDrive shared link. After downloading all the files into a local folder with the GooleDriveConnector (Goole Drive Action: Download), you can then read them.

[Edited]

Creator -> GoogleDriveConnector 1 (List all files) -> GoogleDriveConnector 2 (Download all target files into a local folder) -> FeatureHolder (Wait for download to complete) -> Tester (Filter *.shp files) -> FeatureReader

lars_de_vries
Forum|alt.badge.img+10
geo-x wrote:
I would say : "isn't possible to change the filetype on GDrive"

 

 

I did a test with my own GDrive and set the same GDrive in the Connector as in the FeatureReader. I selected the file manually and got a comma seperated list. Perhaps something like that is possible. 

 

The list started with "fme://google-drive/" and than mentioned the filename and a list of properties containing 'id', the webconnection name in 'connection' and a property 'sidecars'. The sidecars is a comma-seperated list of all ids and subfiles.

 

 

fme://google-drive/file.shp?id=123&connection;=ConnectionName&sidecars;=345,file.shx,456,file.dbf,789,file.shx

 

Maybe you can built that string using a AttributeManager and an Aggregator.

 

 

lars_de_vries
Forum|alt.badge.img+10

Hi @takashi and @lalandexavier,

 

I got it to work, using the previously described method!

takashi
Influencer
  • takashiInfluencer
  • July 9, 2018
lars_de_vries wrote:

Hi @takashi and @lalandexavier,

 

I got it to work, using the previously described method!

Great. Thanks for sharing the technique.

 

geo-x
Contributor
Forum|alt.badge.img+6
  • geo-xContributor
  • Author
  • Contributor
  • July 9, 2018
takashi wrote:

Hi @lalandexavier, I don't think FME readers can read any dataset directly from GoogleDrive shared link. After downloading all the files into a local folder with the GooleDriveConnector (Goole Drive Action: Download), you can then read them.

[Edited]

Creator -> GoogleDriveConnector 1 (List all files) -> GoogleDriveConnector 2 (Download all target files into a local folder) -> FeatureHolder (Wait for download to complete) -> Tester (Filter *.shp files) -> FeatureReader

Thant you very much @takashi

 

geo-x
Contributor
Forum|alt.badge.img+6
  • geo-xContributor
  • Author
  • Contributor
  • July 9, 2018
lars_de_vries wrote:

Hi @takashi and @lalandexavier,

 

I got it to work, using the previously described method!

Thank you very much @lars_de_vries

 

rahulsharma
Safer
Forum|alt.badge.img+10

Hi @lalandexavier @lars_de_vries

 

There is another way to read Shape Files directly from Google Drive using WebAsFileFormat. You can select Google Drive and browse to folder (similar to GoogleDrive Connector) and be able to select either a Single or Multiple Shape files.

 

This should also work in FeatureReader and any other transformer that support reading data

lars_de_vries
Forum|alt.badge.img+10
rahulsharma wrote:

Hi @lalandexavier @lars_de_vries

 

There is another way to read Shape Files directly from Google Drive using WebAsFileFormat. You can select Google Drive and browse to folder (similar to GoogleDrive Connector) and be able to select either a Single or Multiple Shape files.

 

This should also work in FeatureReader and any other transformer that support reading data

The methode described workshop as well, but IT does not container the dynamic approach @lalandexavier was looking for, i.e. reading from alist of files

 

 

geo-x
Contributor
Forum|alt.badge.img+6
  • geo-xContributor
  • Author
  • Contributor
  • July 10, 2018
rahulsharma wrote:

Hi @lalandexavier @lars_de_vries

 

There is another way to read Shape Files directly from Google Drive using WebAsFileFormat. You can select Google Drive and browse to folder (similar to GoogleDrive Connector) and be able to select either a Single or Multiple Shape files.

 

This should also work in FeatureReader and any other transformer that support reading data

Dear @rahulsharma, the problem in this case is to load many files in many folders, so it's not exactly what I need. Thank you :-)

 

geo-x
Contributor
Forum|alt.badge.img+6
  • geo-xContributor
  • Author
  • Contributor
  • July 11, 2018
takashi wrote:

Hi @lalandexavier, I don't think FME readers can read any dataset directly from GoogleDrive shared link. After downloading all the files into a local folder with the GooleDriveConnector (Goole Drive Action: Download), you can then read them.

[Edited]

Creator -> GoogleDriveConnector 1 (List all files) -> GoogleDriveConnector 2 (Download all target files into a local folder) -> FeatureHolder (Wait for download to complete) -> Tester (Filter *.shp files) -> FeatureReader

Dear @takashi, I don't understand how to configure the FeatureReader in this case. I'm downloading each Shapefile (11 files), and the FeatureReader read each files 11 times.

takashi
Influencer
  • takashiInfluencer
  • July 11, 2018
takashi wrote:

Hi @lalandexavier, I don't think FME readers can read any dataset directly from GoogleDrive shared link. After downloading all the files into a local folder with the GooleDriveConnector (Goole Drive Action: Download), you can then read them.

[Edited]

Creator -> GoogleDriveConnector 1 (List all files) -> GoogleDriveConnector 2 (Download all target files into a local folder) -> FeatureHolder (Wait for download to complete) -> Tester (Filter *.shp files) -> FeatureReader

This screenshot illustrates my intention. Supposing the user parameter "FME_TARGET_FOLDER" gives an existing folder path in the local disk.

 

 

 

geo-x
Contributor
Forum|alt.badge.img+6
  • geo-xContributor
  • Author
  • Contributor
  • July 11, 2018
takashi wrote:
This screenshot illustrates my intention. Supposing the user parameter "FME_TARGET_FOLDER" gives an existing folder path in the local disk.

 

 

 

Thanks @takashi in fact, this is more simple if I group the results :-) How can I do that and preserve the originals attributes of shapefile ?

 

takashi
Influencer
  • takashiInfluencer
  • July 11, 2018
takashi wrote:
This screenshot illustrates my intention. Supposing the user parameter "FME_TARGET_FOLDER" gives an existing folder path in the local disk.

 

 

 

All the attributes will be preserved but FME Workbench won't expose them automatically, since the schema of the source datasets cannot be retrieved when creating the workspace. If you know attribute names in the source datasets, you can expose them manually through the Attributes to Expose parameter in the FeatureReader or a subsequent AttributeExposer transformer.

 

geo-x
Contributor
Forum|alt.badge.img+6
  • geo-xContributor
  • Author
  • Contributor
  • July 11, 2018
takashi wrote:
All the attributes will be preserved but FME Workbench won't expose them automatically, since the schema of the source datasets cannot be retrieved when creating the workspace. If you know attribute names in the source datasets, you can expose them manually through the Attributes to Expose parameter in the FeatureReader or a subsequent AttributeExposer transformer.

 

It's perfect @takashi thank you very much !

Reply


Helpful Members This Week

Community Stats

31,358
Posts
118,956
Replies
38,994
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.