Do you know if there is difference between transformer from 32-bit to 64-bit? or transformer are all the same between 32 and 64?
Thanks!
Hi, I'm doing some test between FME 32-bit on a 32-bit machine and FME 64-bit on a 64-bit machine.
Page 1 / 1
Hi there Francis. The actual Transformers themselves are the same, though depending on the bit version of FME and the OS will determine their relative performance in terms of memory.
The general rules are as follows:
- 32Bit FME on a 32Bit OS can utilise 2GB of system RAM (and whatever virtual memory you have assigned).
- Unless on an OS like WinXP or similar you have turned on the /3G switch. In this case FME can get access to 3GB of addressable memory. This is a boot.ini hack. See here:
- 32Bit FME on a 64Bit OS can utilise 4GB of RAM...
- 64Bit FME on a 64Bit OS, can address all the RAM you have access to.
Beware though, 9 times out of 10 the best option is 32Bit FME on a 64Bit OS, because the 64Bit FME does not support all of the formats you might need. As you haven't tried that option yet, give it a go. Check here first to ensure your formats are supported by the 64Bit version:
http://www.safe.com/fme/format-search/
...sort by Platform on the page.
A good test though is run your workspace and check to see what your actual peak memory usage gets up to in the log at the end. If its not even 2GB then you don't need to worry. Certainly unless you are doing heavy raster or point cloud operations 32bit FME on a 64bit OS is usually ok for most scenarios.
Good luck.
Be aware that 64-bit FME doesn't support a small subset of transformers such as the MRFCleaner.
However, as David pointed out, the transformers themselves are the same.
Be aware that you can copy a transformer into a text editor to see its actual contents!
One of the anoying problems is Excel. If you have office in 32 bit format then it will fail to write to Excel with 64 bit FME, not sure if it is the other way around as well.
Thank you for the reply.
But can you explain that :
I run a small script that load 3201069 point and 7584 polygon.
I do a PointOnAreaOverlayer and some basic Tester.
On a 32bits : 23 minutes
On a 64bits : 1hour and 8minutes.
I was thinking that on a 64bits the script would have run much faster.
What's your peak memory usage, listed at the end of the log? If it doesn't peak too high then you're not getting the benefit from 64bit. Also its not clear if your timings are 32bit FME on 32bit OS and 64 on 64... Try 32bit FME on a 64bit OS.
As David says, for a real comparison run both 32/64-bit versions on the same 64-bit machine. Using different machines may be responsible for the difference.
This FMEpedia article gives a little advice on performance tuning a machine and a translation.
RE: PointOnAreaOverlayer -- in FME 2013 SP1 we're introducing a new mode "AreasComeFirst" which greatly optimizes this transformers behaviour and memory usage. This may help it get closer in performance between 64 and 32 bit.
I would add that overall I've been surprized that 64 bit rarely seems to give a performance benefit, except in cases which require enormous memory footprints. As others have said, running FME 32 bit on a 64 bit OS for most situations is a good way to go.
Well a lot has been said in the discussion here already and I may have overlooked some details. Let me add some points from my experience and tests I did with mass data (traditional vector type) over the last year.
I did tests on a 64-Bit Win-7 machine with 16GB memory and on another Win 64-Bit server with 32GB memory. All test with direct comparison of FME 32-Bit vs. FME 64-Bit on the same machine.
1) FME 64-Bit allows you to address all memory you can get on todays machines, but Windows 64-Bit recommends to use only about 70% of available memory. From 16GB FME will use 12GB, from 32GB FME will use 24GB before it starts to swap.
2) FME 64-Bit uses more ressources for the same processing than the 32-Bit version. e.g. it uses ~20% more memory in a FeatureMerger
My guess is due to longer addresses (8byte vs. 4byte) in internal processing, etc.
3) The amount of data I processed required 60GB+ swap with a 32-Bit FME and 80GB+ with a 64-Bit FME
--> even good Windows machines will not give you such amounts of memory
--> only with all done in memory and NO swap involved will the 64-Bit give you better performance
4) My impression was, that if FME finds it needs to start swapping out on disk it will ALWAYS swap out everything and not only that which exceeds available memory. I may be wrong, but my experience pointed in this direction.
5) As for restrictions on some readers/writers I needed to install both FME variants (64/32-Bit) in parallel and used some sophisticated BATCH programming to call the right FME variant as needed by the reader/writer.
6) Self implemented extensions programmed with C++, like transformers, readers, writers need to be compiled extra for the 64-Bit variant of FME and logistics to install and handle this grow more complex.
So in the end my conclusion for best performance was:
- use 32-Bit FME on 64-Bit OS
- run it in parallel depending on the number of processors available
- and very important: use SSD disk for swap area
- SSD can handle max 4 sessions in parallel without performance loss
- classic HDD will kill most parallelization efforts if processes swap to the same disk