Skip to main content

I am getting an error when run a workbench on FME Flow. The workbench supposed to extract shapefile from S3 bucket and write it to the database.

The shapefile has very complicated and big area in it. Not sure if it matters. The total size is 300Mb.

Log file has just this error:

image.pngimage 

Job doesn't fail, it looks like it is stuck. It lasts for hours, more than 12 hours. I had to cancel it. 

 

I thought that the problem is explicit with shp.xml file. But I checked previous uploads and there were successful ones. Even with this file.  Please advice.

So this ERROR here is because FME is (probably) using the wrong reader to read the file. Here it's using the ADAC reader: https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_ReadersWriters/adac/adac.htm ADAC is an xml based format. Perhaps you are using the 'Generic' reader? How are you choosing which files to read?

 

 

 


Thank you for your reply.

We are using S3objectlister and S3downloader transformers to extract file from S3. It is not me who created this workbench, I just inherited it. I am not sure which transformer considered as reader in this case, because in Navigator I have just Writer - POSTGIS type.

 

imageIn the log files I have found the mentioning of ADAC reader as well as generic. I hope it is visible on the sreenshot

 

CaptureCreating reader for format: Generic (Any Format)

Trying to find a DYNAMIC plugin for reader named `GENERIC'

FME API version of module 'GENERIC' matches current internal version (3.8 20200115)

Performing query against GENERIC dataset `D:\\Temp\\Bucket\\XXX.shp.xml'

Creating reader for format: Generic (Any Format)

Trying to find a DYNAMIC plugin for reader named `GENERIC'

FME API version of module 'GENERIC' matches current internal version (3.8 20200115)

Generic Reader: No valid reader name specified, guessing reader name from dataset 'D:\\Temp\\Bucket\\XXX.shp.xml'

Generic Reader: Reader name 'ADAC' guessed from dataset 'D:\\Temp\\Bucket\\XXX.xml'

Generic Reader: No format specific settings have been found for the ADAC Reader -- default values as documented in the Reader/Writer manual will be used

Creating reader for format: Australian Asset Design and As Constructed (ADAC)

Trying to find a DYNAMIC plugin for reader named `ADAC'

FME API version of module 'ADAC' matches current internal version (3.8 20200115)

Opening the GENERIC Reader on dataset `D:\\Temp\\Bucket\\XXX.shp.xml'

Opening the ADAC reader with source dataset 'D:\\Temp\\BucketXXX.shp.xml'

Parsing XML document 'D:\\Temp\\Bucket\\XXX.shp.xml'

Unable to recognize the source dataset as an ADAC file, 'D:\\Temp\\Bucket\\XXX.shp.xml'

A fatal error has occurred. Check the logfile above for details

A fatal error has occurred. Check the logfile above for details

Closing the GENERIC Reader


ok yep, so the reader is a Generic reader here so that's where your ERROR is coming from in this case. FME spots an XML file and tried to read it with the Generic reader - the generic reader (I think) just picks the first format in the list which supports XML files.

 

You can see that the workspace also reads the shapefile as well with the actual shapefile reader so that's good.

 

From your initial post you mentioned that the job doesn't fail but gets stuck. I think this error is not the cause of the problem however. I assume that the workspace just reports the ERROR and then keeps working on the shapefile data. You'd need to share more of the logfile and/or data for us to figure out what is going on.

 

 


Reply