Skip to main content
Solved

Translating large dataset failing to output

thomastupuivao
Contributor
Forum|alt.badge.img+6

I have a esri geodatabase that I'm trying to translate to a "Mapinfo" table. It is successful if I limit the max features to a million records, but the geodatabase is 2 million records and I keep getting the following error (see image).

 

I'm running FME on a Windows Server 2008 machine. It has 2 XEON (E5-2670 v2) processors and has 10GB of RAM installed.

I've set the temp folders to the Secondary "D" drive. Not sure why this is happening, as this wasn't an issue when I was running FME 2013. Since upgrading to FME 2018 this has become an issue.

 

 

Best answer by markatsafe

@thomastupuivao I think MapInfo has a 2Gbyte file size limit. You might have to divide your data into parts. Also, ensure that you are not 'wasting' space by having string data types that are longer than really needed. Using Extended Tab would possibly solve this issue.

View original
Did this help you find an answer to your question?

4 replies

daveatsafe
Safer
Forum|alt.badge.img+19
  • daveatsafeSafer
  • Safer
  • January 11, 2019

Hi @thomastupuivao,

Please try your workspace with the Mapinfo (MITAB) to see if you still get the error. This writer uses a different library to write the TAB data, and if it works, will help us pinpoint the source of the error you are seeing.

You may also want to try using the new Mapinfo Extended TAB writer. If you set the Table Type in the writer parameters to Native instead of NativeX, it will write a standard TAB file. This writer uses the latest library supplied by Mapinfo, and should provide the best results.

M
Forum|alt.badge.img+2
  • markatsafe
  • Best Answer
  • January 11, 2019

@thomastupuivao I think MapInfo has a 2Gbyte file size limit. You might have to divide your data into parts. Also, ensure that you are not 'wasting' space by having string data types that are longer than really needed. Using Extended Tab would possibly solve this issue.

thomastupuivao
Contributor
Forum|alt.badge.img+6
  • thomastupuivaoContributor
  • Author
  • Contributor
  • January 13, 2019
markatsafe wrote:

@thomastupuivao I think MapInfo has a 2Gbyte file size limit. You might have to divide your data into parts. Also, ensure that you are not 'wasting' space by having string data types that are longer than really needed. Using Extended Tab would possibly solve this issue.

I've reduced the lengths of my string fields and changed all the fields to float for my "ID" fields. Seems to have done the trick!! Thanks again

thomastupuivao
Contributor
Forum|alt.badge.img+6
  • thomastupuivaoContributor
  • Author
  • Contributor
  • January 13, 2019
daveatsafe wrote:

Hi @thomastupuivao,

Please try your workspace with the Mapinfo (MITAB) to see if you still get the error. This writer uses a different library to write the TAB data, and if it works, will help us pinpoint the source of the error you are seeing.

You may also want to try using the new Mapinfo Extended TAB writer. If you set the Table Type in the writer parameters to Native instead of NativeX, it will write a standard TAB file. This writer uses the latest library supplied by Mapinfo, and should provide the best results.

Ended up reducing the sizes of my string fields and changing my ID fields to "Float" and it worked??

Reply


Helpful Members This Week

Community Stats

31,345
Posts
118,888
Replies
38,960
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.