Skip to main content
  • EN


Hello,


We
have a database (SQL-server) where we have stored water pipes with connected
jpg-pictures (1 pipe:many pictures). The data is collected with "Collector
for ArcGIS" and the pipes are stored in one table and each pipe has its
unique guid. The Pictures are stored in a separate table and the Connection
between the pipes and the pictures is handled by a "REL_GUID"
attribute.


The
thing I have to do is to extract the pictures from the database and store them
as "normal files" on a disc and write the URL for the pictures to an
attribute on the waterpipe object.


Just
before the summer vacation I wrote a FME-script that read the two tables with
an ArcSDE Geodb-reader. I then joined the water pipes and the pictures with
FeatureMerger and the result was a water pipe object with a list containing all
relevant information from the picture table, including the actual data blob. The
next step was to write the picture to a relevant disc using
AttributeFileWriter. Everything worked fine while testing so I went on
vacation. But the field staff didn’t and they were busy filling the database
and now due to the amount of data FeatureMerger dies the out-of-memory-death.


I tried to use the
Joiner transformer instead but now the pictures get corrupted when written to
disc. With my limited knowledge it seems like there is a bug in the Joiner. Any
ideas for an alternative solution anyone ?


I’m running the
script on a 32-BIT FME 2016.1


Regards


Thomas Norlin

Are you using the Suppliers First = Yes option on the FeatureMerger? You have guarantee that all the Supplier features arrive at the transformer before any Requestor features. You can do this using some tricks such as two Readers connecting to the same database and using the order in the Navigator to control which is read first (assumes you are reading with standard Reader rather than FeatureReader).

However the image BLOB could be pushing up the memory also. There could be another approach where you make the join without the BLOB, then refetch the BLOB after the join.

Uploading the Workspace would help.

Tanks,

I tried the Supplier First option but the result was still "out of memory". Then I tried to upgrade to 2017.1 and to use the transformer DatabaseJoiner in the same way as I used Joiner but with limited success. Finally I upgraded to 2017.1 64 BIT with FeatureMerger and then everyting worked fine.

Regards

Thomas

Tanks,

I tried the Supplier First option but the result was still "out of memory". Then I tried to upgrade to 2017.1 and to use the transformer DatabaseJoiner in the same way as I used Joiner but with limited success. Finally I upgraded to 2017.1 64 BIT with FeatureMerger and then everyting worked fine.

Regards

Thomas

 

Great! Was going to suggest 64 bit but thought you may have been restricto 32 bit.

collectorpipes-safe17.fmw

Here's the script

Another solution (in case somebody else reads this) is to read the table with an inner join in SQLCreator. But I'm afraid that I still will have problems in the future. FME 2017 64 Bit is the best solution

Reply


Community Stats

30,186
Posts
114,572
Replies
37,850
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.