Solved

fme did not read all features(strange)

  • 7 December 2020
  • 6 replies
  • 49 views
f.kemminje
Badge +3

I used fme desktop 2019

my source data is fgdb with 104,073,607 (104 million +)features

org table shows 104073607 but only 80,900,000 records processed.

there is no transformers between , reader is fgdb and writer is oracle spatial data.

 

dont know , where was the issue. This is a silent failure of fme.

no error msg. after writing the checked the counts, it was less.

 

then i splitted the fgdb files into two part(using arcmap)

then im fine.

 

how to fix this issue? Please

 

icon

Best answer by halmar 27 September 2021, 08:46

View original

6 replies

rahulsharma
Rank
Badge +8

Hi @f.kemminje​ 

Can you elaborate on this more i.e. are you trying to read system table?

Do you have "Ignore Network Info" check in Parameter?

Does your table contain Relationship?

f.kemminje
Badge +3

No, not a system table.

Source is fgdb file with 104 million records. And no relationship.

This is just one to one translation. Fgdb to oracle spatial table. But fme read only 80 million records , and wrote only those 80 million records .​

H

Hi @fkemminje​ 

I have the same problem, arcsde to arcsde, 2 million records, but only read 200.000. Can't find a solution yet. Did you?

Thanks

JennaKAtSafe
Rank
Badge +6

Hi @fkemminje​ 

I have the same problem, arcsde to arcsde, 2 million records, but only read 200.000. Can't find a solution yet. Did you?

Thanks

Hi @halmar​ Do you have any warnings/errors that show up in your translation log?

Any connection errors with ArcSDE?

I would also suggest posting a new question with a copy of your translation log to get better input from the community!

f.kemminje
Badge +3

Hi @fkemminje​ 

I have the same problem, arcsde to arcsde, 2 million records, but only read 200.000. Can't find a solution yet. Did you?

Thanks

This is an old question from me. I completed that task Long back.

But my suggestion would be...

 

increase your system memory.

close all fme sessions before the run

keep the only fme open or run from the command line

remove/avoid waiting transformers

or (for ex: set suppliers first, if the option is available(i am saying feature merger transformer's setting. you have a different name

depends on which transformer you are using) ) so that features will not stop, FME keep on processing...

Create environment variable FME_temp (d:\\FME_TEMP or where disk space available)

Run batchwise(if you can create a batch_numbers to source dataset)

hope this helps

Thank you!

 

H

Hi @jennaatsafe​ and @f.kemminje​ !

Thanks for your replys, and sorry for my late response. I have found my problem. My source ArcSDE was originally only an SQL database, and I was enabled geodatabase, when it was full loaded. In SQL, geometry was valid for all objects, but in ArcSDE, it finds geometry errors (the number of points is less than required..) and when FME find this object, finished the table.. (arcpy search cursor finished as well..) So i have to find theese arcmap geometry errors (it's difficult, because in file geodb it was not an error...). After I deleted theese rows in SQL server management studio, FME works well!

Reply


Community Stats

28,729
Posts
109,527
Replies
36,565
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.