Skip to main content
  • EN
Hi,

 

I have a problem workspace that runs fine for most datasets but for one dataset it just stops, with no error indicated in the output log.

 

 

I've tried turning on the "Log debugging information" option in Workbench. I've also tried adding "FME_DEBUG BADNEWS FME_STACK_TRACE" to the workspace header but I'm still not getting any error output at the point the translation hangs, it just stops.

 

 

I've let the translation sit overnight, so it's not just a matter of me being impatient!

 

 

I assume it's some bad geometry but it's hard to know where to start looking, there's a fiar amount of data and the translation runs for 1/2 hour before dying.

 

 

Originally the translation looked like it was dying on a FeatureMerger so I exported the inputs to that transformer and then ran them through in a seperate translation but they ran through fine. I replaced the FeatureMerger with a Matcher in my original workspace and now it looks like it's failing on the Matcher. I'm not convinced that's where the problem lies, that just seems to be where the translation stops.

 

 

Anyone have any ideas on how to get any log data out of this, or how to best troubleshoot it?

 

 

I'm on the latest FME 2013 sp2 beta but also have the same problem on earlier versions of FME.

 

 

thanks,

 

Nic
Hi Nic,

 

 

could it perhaps be a corrupted dataset? What happens if you try to visualize the input data using the FME Data Inspector?

 

 

David
I expect it is bad data but the data inspector has no problem with it, so the problem is in finding which data is bad and also in finding where FME is blowing up....
Hi Nic,

 

 

Have a go with the new GeometryValidator. Principally the test for 'Degenerate or Corrupt Geometries' and the 'Contains NaNs or Infinities'. Those two rules help with corrupt data. Failing that you'll need to sift through the file bit by bit. The most practical way to do this is to use the advanced parameter on the reader 'max features to read', set this to read half the data, then if that works use the 'Start Feature' parameter too and read from halfway to the end. When you find the crash or hang then do it again so read the first part of the last half etc. After a few quick runs you should get to a point where you have the problem feature in amongst only a few hundred features and then you can start to use the option to 'Run with Inspection' to test features and try to track down the culpret. Of course you should try to do all of this without any transformers in the way to begin with just in case you are corrupting the data with one of your processes inline.

 

 

Good luck hunting...
Thanks SpatialDave, good advice. I've already started filtering out the input dataset and am getting closer to isolating the problem feature. I'll give the GeometryValidator a go too.
Brief update for anyone who's interested.

 

 

It turns out that I have a very complex line feature in my source data. It passes "Basic Geometry Integrity" and "OGC Valid" testing using the GeometryValidator. It takes an hour to run the "Self Intersections in 2D" geometry validation but, unfortunately, I didn't trap the output so I don't know if it failed that test. (I would have to assume that it did.

 

 

I'm attempting to buffer this feature and FME is failing without warning.

 

 

I understand that I have "bad" geometry but will be submitting a support ticket with Safe for better handling of this situation. Just FYI for anyone following this thread.

Reply


Community Stats

30,157
Posts
114,473
Replies
37,802
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.