Skip to main content
Question

WFS reader in 2017 running slower than 2016

  • October 10, 2017
  • 7 replies
  • 64 views
fmelizard
Safer
Forum|alt.badge.img+22

I am noticing WFS reader times that are more than 5 times slower on FME 2017 when compared to FME 2016. A workbench that extracted data in 7 minutes in 2016 is taking over 40 minutes in 2017. In addition there are differences in the log files

2017: all of the "HTTP response - so far received: ##.## MB" messages were written to the log before any "Reading Source Feature # xxxx" messages appeared.

2016: the read messages start appearing within the "HTTP response - so far received: ##.## MB" messages.

2017: The display count holds at one until all features have been read

2016: display counts continuously increase as features are being processed through transformers, while data is still being read in.

 

Are there settings in FME 2017 that can be changed to get it to perform the same way it did in 2016?

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

7 replies

redgeographics
VIP
Forum|alt.badge.img+62

Just curious, but is this a workspace that was developed in 2016 and you're now running it in 2017? If so, try removing the reader and adding it again to make sure it's the latest version.

Still that shouldn't happen (although with WFS there might be countless other factors involved) so if you have a reproducible case I would urge you to send it in to support.

FME rocks! \m/
redgeographics
VIP
Forum|alt.badge.img+62

Just curious, but is this a workspace that was developed in 2016 and you're now running it in 2017? If so, try removing the reader and adding it again to make sure it's the latest version.

Still that shouldn't happen (although with WFS there might be countless other factors involved) so if you have a reproducible case I would urge you to send it in to support.

Just tried it with a small (28000 features) request and FME 2016 and 2017 report nearly the same processing time, 2017 even being a few tenths of a second faster.

 

 

FME rocks! \m/
fmelizard
Safer
Forum|alt.badge.img+22
  • fmelizardSafer
  • Author
  • Safer
  • October 11, 2017

Definitely sounds like something has changed for the worse. Sorry to hear this @angelam -- thanks for letting us know.

Can you supply the log files you've got, and the workspace to support@safe.com . Your description makes me think we are now waiting for an entire download before we can start reading. There may have been a reason for this, but obviously it worked the other way in the past so we should be able to find a way to have best of both worlds. Please do send things in to support and we'll be on it. Thanks.

deanatsafe
Safer
Forum|alt.badge.img+6
  • deanatsafeSafer
  • Safer
  • October 11, 2017
Hi,
 
One of our customers has customized the S57 Reader as explained in https://knowledge.safe.com/articles/570/s-57-enc-hydrographic-data-format.html.
 
He is concerned by the fact that all customized files are replaced whenever a new FME version is installed, or that the customization should be done again if a new FME version is installed at another location on disk.
This might be acceptable for one user, but this causes much more trouble as the customization has to be performed on all workstations used in the company.
I suggested having FME (or the GDAL S57 format used ultimately) check some location (for example in FME shared directories) for a customization of the format and @danatsafe has filed the FMEENGINE-61034 change request.

Anybody who finds this is a good idea, please vote or suggest any other improvement to the S57 customization process. Thanks.

Fred
fmelizard
Safer
Forum|alt.badge.img+22
  • fmelizardSafer
  • Author
  • Safer
  • October 11, 2017

What version of FME are you using? If its 2017.0 try upgrading to the latest release: 2017.1.1 which has a number of performance improvements over 2017.0. Just as a test, you can also try the WFS/GML 'ignore schema' mode which will skip the reading and parsing of the schema files (this would likely require you to add a new reader). Try testing from Data Inspector as well to see if there may be something happening specific to your workspace. If you are still having trouble, send us the WFS url so we can test it here. If you don't want to post it you can open a case with support at https://www.safe.com/support/report-a-problem/

I am using FME 2017.1 Build 17539. Unfortunately the WFS is on a restricted access system so I cannot post that here. The same results are seen in inspector. It is also seen on an existing workbench created in FME2016 and in one that was created in 2017.

 

 

fmelizard
Safer
Forum|alt.badge.img+22
  • fmelizardSafer
  • Author
  • Safer
  • October 11, 2017

Definitely sounds like something has changed for the worse. Sorry to hear this @angelam -- thanks for letting us know.

Can you supply the log files you've got, and the workspace to support@safe.com . Your description makes me think we are now waiting for an entire download before we can start reading. There may have been a reason for this, but obviously it worked the other way in the past so we should be able to find a way to have best of both worlds. Please do send things in to support and we'll be on it. Thanks.

I've supplied the log files to safe support, but have not be able to replicate the issue on a WFS outside of the affected network.

 

deanatsafe
Safer
Forum|alt.badge.img+6
  • deanatsafeSafer
  • Safer
  • October 14, 2017
I am using FME 2017.1 Build 17539. Unfortunately the WFS is on a restricted access system so I cannot post that here. The same results are seen in inspector. It is also seen on an existing workbench created in FME2016 and in one that was created in 2017.

 

 

We are currently running some tests and will post an update here once we have conclusive results.

Community Stats

32,647
Posts
123,913
Replies
41,042
Members
Terms & ConditionsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.