Skip to main content
Question

FME Server: problems when reading multiple shapefiles with generic reader.


davtorgh
Contributor
Forum|alt.badge.img+11

Hi,

 

I have a workspace with a generic reader that reads multiple shapefiles. Everything works fine as long as I run the workspace with FME Desktop, but when I run the workspace with FME Server (Debian Linux) I get an error message like this:

2022-1-5 12:43:30 | Shapefile Reader: The .dbf for '/data/fmeshare/resources/system/temp/upload/Tests/crs_fmt_converter.fmw/<username>/060101_BOSCO.shp' could not be found. Reading will continue, but features will not have any attributes

I have investigated a bit further and I have discovered that if I upload a single file, everything works fine on FME Server too.

 

Could it be a bug of the reader used by FME Server?

 

I'm using FME Server 2021.2.1 (Build 21797 - linux-x64 )

 

Thanks!

8 replies

virtualcitymatt
Celebrity
Forum|alt.badge.img+36

Are you uploading all parts of the shapefile/zip file or just the *.shp file?


davtorgh
Contributor
Forum|alt.badge.img+11
  • Author
  • Contributor
  • January 11, 2022
virtualcitymatt wrote:

Are you uploading all parts of the shapefile/zip file or just the *.shp file?

Hi @virtualcitymatt​ ,

 

I've tried both ways, but I reached the conclusion that the only way is (naturally) to upload the .shp file, because FME Server takes care of uploading all the related files. If you upload all parts ot the shapefile FME treats each file as .shp).

 


davtorgh
Contributor
Forum|alt.badge.img+11
  • Author
  • Contributor
  • January 11, 2022

Maybe any suggestion from @lauraatsafe​ ?

Thank you in advance, Laura!


laurawatsafe
Safer
Forum|alt.badge.img+11
davtorgh wrote:

Maybe any suggestion from @lauraatsafe​ ?

Thank you in advance, Laura!

Hi @davtorgh​, Right, the combination of using the Generic reader with a format like shapefile that requires sidecar files in addition to the main .shp file can get a little awkward to work with when uploading the files to process on FME Server. When uploading the shapefile, you need to make sure to include all the additional files as well (.dbf, .prj, .shx, etc). However, if all the uploaded files are selected in the source dataset parameter (like this image):

image.png you'll likely see warning messages about the Generic reader not being able to find a reader to read each individual sidecar file as if they were separate files.

To avoid that, you need to only select the .shp file(s) for the datasets you are working with, but uploading only the .shp files will cause the error you are seeing where it complains about the missing .dbf etc. The easiest way to handle this is to upload all the files for each shapefile, then only select the .shp files in the upload dialog in FME Server. The end result might look something like this, with all the files shown as uploaded on the left part of the dialog and only the .shp files shown under selected:

imageDoes that help? If that doesn't match the behaviour you're seeing, could you please upload a full copy of the log file from when you run this? I can help take a closer look into what might be going wrong.


davtorgh
Contributor
Forum|alt.badge.img+11
  • Author
  • Contributor
  • January 13, 2022
lauraatsafe wrote:

Hi @davtorgh​, Right, the combination of using the Generic reader with a format like shapefile that requires sidecar files in addition to the main .shp file can get a little awkward to work with when uploading the files to process on FME Server. When uploading the shapefile, you need to make sure to include all the additional files as well (.dbf, .prj, .shx, etc). However, if all the uploaded files are selected in the source dataset parameter (like this image):

image.png you'll likely see warning messages about the Generic reader not being able to find a reader to read each individual sidecar file as if they were separate files.

To avoid that, you need to only select the .shp file(s) for the datasets you are working with, but uploading only the .shp files will cause the error you are seeing where it complains about the missing .dbf etc. The easiest way to handle this is to upload all the files for each shapefile, then only select the .shp files in the upload dialog in FME Server. The end result might look something like this, with all the files shown as uploaded on the left part of the dialog and only the .shp files shown under selected:

imageDoes that help? If that doesn't match the behaviour you're seeing, could you please upload a full copy of the log file from when you run this? I can help take a closer look into what might be going wrong.

Hi @lauraatsafe​ ,

 

the suggested workaround works, but it's a bit awkward. My goal is to make available a self-service process for non-FME users and this way they will (probably) make errors and think the process doesn't work.

Do you think this problem will be resolved in a future (I hope near) version of FME Server?

 

Thanks!


davtorgh
Contributor
Forum|alt.badge.img+11
  • Author
  • Contributor
  • January 24, 2022
lauraatsafe wrote:

Hi @davtorgh​, Right, the combination of using the Generic reader with a format like shapefile that requires sidecar files in addition to the main .shp file can get a little awkward to work with when uploading the files to process on FME Server. When uploading the shapefile, you need to make sure to include all the additional files as well (.dbf, .prj, .shx, etc). However, if all the uploaded files are selected in the source dataset parameter (like this image):

image.png you'll likely see warning messages about the Generic reader not being able to find a reader to read each individual sidecar file as if they were separate files.

To avoid that, you need to only select the .shp file(s) for the datasets you are working with, but uploading only the .shp files will cause the error you are seeing where it complains about the missing .dbf etc. The easiest way to handle this is to upload all the files for each shapefile, then only select the .shp files in the upload dialog in FME Server. The end result might look something like this, with all the files shown as uploaded on the left part of the dialog and only the .shp files shown under selected:

imageDoes that help? If that doesn't match the behaviour you're seeing, could you please upload a full copy of the log file from when you run this? I can help take a closer look into what might be going wrong.

Hi @lauraatsafe​ ,

 

I tried a simple workspace with a Shapefile reader setup as "Merge feature type", but the incorrect behaviour is still there on FME Server. So, it doesn't seem to be a problem related to a Generic reader when reads shapefiles and I'm getting convinced it's a bug.

 

I'm running FME Server 2021.2.2 (Build 21806 - linux-x64 ) at the moment.


Forum|alt.badge.img
  • April 29, 2022
lauraatsafe wrote:

Hi @davtorgh​, Right, the combination of using the Generic reader with a format like shapefile that requires sidecar files in addition to the main .shp file can get a little awkward to work with when uploading the files to process on FME Server. When uploading the shapefile, you need to make sure to include all the additional files as well (.dbf, .prj, .shx, etc). However, if all the uploaded files are selected in the source dataset parameter (like this image):

image.png you'll likely see warning messages about the Generic reader not being able to find a reader to read each individual sidecar file as if they were separate files.

To avoid that, you need to only select the .shp file(s) for the datasets you are working with, but uploading only the .shp files will cause the error you are seeing where it complains about the missing .dbf etc. The easiest way to handle this is to upload all the files for each shapefile, then only select the .shp files in the upload dialog in FME Server. The end result might look something like this, with all the files shown as uploaded on the left part of the dialog and only the .shp files shown under selected:

imageDoes that help? If that doesn't match the behaviour you're seeing, could you please upload a full copy of the log file from when you run this? I can help take a closer look into what might be going wrong.

@davtorgh​  - did you ever figure out a solution or workaround to this? I am in the same boat here. Kind of annoying. A lot of people work in shapefiles now. There should be a decent/easy way to upload these to FME Server


davtorgh
Contributor
Forum|alt.badge.img+11
  • Author
  • Contributor
  • May 10, 2022
lauraatsafe wrote:

Hi @davtorgh​, Right, the combination of using the Generic reader with a format like shapefile that requires sidecar files in addition to the main .shp file can get a little awkward to work with when uploading the files to process on FME Server. When uploading the shapefile, you need to make sure to include all the additional files as well (.dbf, .prj, .shx, etc). However, if all the uploaded files are selected in the source dataset parameter (like this image):

image.png you'll likely see warning messages about the Generic reader not being able to find a reader to read each individual sidecar file as if they were separate files.

To avoid that, you need to only select the .shp file(s) for the datasets you are working with, but uploading only the .shp files will cause the error you are seeing where it complains about the missing .dbf etc. The easiest way to handle this is to upload all the files for each shapefile, then only select the .shp files in the upload dialog in FME Server. The end result might look something like this, with all the files shown as uploaded on the left part of the dialog and only the .shp files shown under selected:

imageDoes that help? If that doesn't match the behaviour you're seeing, could you please upload a full copy of the log file from when you run this? I can help take a closer look into what might be going wrong.

Hi @candimk​,

 

I've used a workaround: I've created a workspace app on FME Server and inserted a message to instruct users to upload all the additional files (at least .dbf, .prj and .shx) alongside the .shp.

 

It's not obviously a solution.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings