Skip to main content

Open ideas have been reviewed by our Product Management and are open for commenting and voting.

Filter by idea status

Filter by product area

4625 Ideas

mhab
Contributor
mhabContributor

Ressource handling and virtual memory usageOpen

I want to make a point on FME handling of memory ressources and why I think it is not optimum. I am referring to Windows environment (as I have no experience with FME on other OS)In the frist part of any FME log file some remarks can be found about how much physical and virtual memory it available and what limits will be used by FME for start/stop of requiring/freeing memory.As described here(https://community.safe.com/s/article/Performance-Tuning-FME-FME-Memory-Management) FME will decide ressource usage depending on virtual memory available.I think this is counter-productive in situations when more than one large FME process will fight for the same limited amount of memory.In practise I see windows machines with 64Gb physical memory and 264Gb virtual memory quite often. In such an environment we run 3-4 very large FME processes in parallel and each one thinks 264Gb is available, which is not true of course. We run into serious performance issues in this way, with physical memory completely used (99%) and hanging fme processes, as this relies on OS swapping, which is very bad on Windows.This does not occur, if the virtual memory is reconfigured to be only a little larger than physical memory. So we have to advice our power customers to reconfigure their Windows System for small extra virtual memory only because FME is following a "strange" memory policy.This is an undesirable situation, which cannot be followed in every customer situation as other applications may need the large extra virtual memory configuration.So I want to really question the fact, why FME is optimizing memory usage based on virtual memory and not on physical memory. Maybe a suggestion could be to influence the memroy policy to stay closer to the physical than the virtual limit by some parameter/environment variable or whatever.Repeating myself: What I want to avoid is to tell our customers that they need to reconfigure their Windows OS setting only due to FME reasons.

geographix
Contributor
geographixContributor

Decode Option in GeometryReplacer for 6 decimal places polylineOpen

GeometryReplacer has an option for "EncodedPolyline". This supports polylines encoded to Google's documented 5 decimal places. But I have a polyline encoded to 6 decimal places, and so had to use PythonCaller and some ugly hacks to get a WKT formatted polyline.Why am I using 6 decimal places? I'm using the Valhalla open-source routing engine for OpenStreetMap (https://valhalla.github.io/valhalla/).Valhalla's reasoning for using 6 decimal places is documented at https://valhalla.github.io/valhalla/decoding/I used the Python script in FME to get my decoded polylines, but would prefer an option in the GeometryReplacer for "Encoded Polyline 6dp" below "Encoded Polyline"Their demo tool at https://valhalla.github.io/demos/polyline/ is useful in debugging.Here's a random 6dp encoded polyline:zsyo}@{cobaH??_KtDiHn@{HjHaMtDyOlHmI_AyL}HbGyt@jDwTHsc@tCsBw^pMgl@dFi_@j@}UlDw_@rCaKrQy[PmFdU{p@fZqx@pFuOfCiIVeCBkBAyBOkA{AcB}CyB}AaAgEyAqr@a]wdAmf@aJcCoHa@oIrAqIzDkGpDiF~DuB`B{B`As@~A CJnDDhDKxFN|EBfHeRjyCeIhjAiCfWmCdNk`mA}Jfa@cFrb@sHniA_EhU}EzReMre@}Lhe@kDpRmAbPmC`i@q@jO{IhqBaAlU_ClWkKvs@qAxOWdO`@`K`D|[hHzn@rO||AxHzx@tDlb@hX`nDbAhKnAfLzQhnAtEb]RrRFbF_A|Nc@`HGj@cCpW_AtIoFfh@mHtr@mCrLaD|JuFvIkHhImMnO_LrLiAlAw]|Z{FbF}}@|t@qwCr~BeGvEwSjPkRhOob@x^qm@fl@wObL_JtCqHpAuGZmQqAsDj@}Bf@gDpAEj@FzEXvFbClNrX~|@zDxVzA`N`BdTp@jWg@|kA~SgBbUwErr@kDld@cD~b@e@nFsEzp@sOlsAQbH~BvQNj@Hn@Bn@?n@Cl@In@Oj@Qh@Wf@[`@c@b@N`DmBpK]hEa@nH{DrdAg@bUcAh}@t]eDl}@}@~WqBxh@yJnwDF~Lh@tOf@~S|Af KhrAnGjw@hNnzA~Axa@d@xq@Anh@A`T`A|tCFlTlBb|DXryDr@viBHly@tBd`CX|[`AndAa@~f@]tTYpOy@z]wEz{@iIbaAoJbfAkh@rcGi^l{DeCpNcEdYgGpd@wD`r@sBjq@o`@|lEqNtaBe@xXf@`V~Eb]zGnUvGrPbVvZ|NtLtStKdm@xMd_BdXlr@zLv^dIpOxFfUlO|TdWlJfQfJ|UhPfcArFz_@nJfc@bKvWhNrWvQdV`UdTh|A|sAtk@re@jhYhZ`XfMfLjWp[`Wbc@t`~@bGtdFrYnK`v@tl@zyGhGpr@vDnd@rJ~iAbMjkAbK`y@fe@ntDtPdrAhGnc@zQrfAny@xfD~Rvq@xObl@n[nmA`Orq@hLtn@jPtkAbKf}@~Fdv@xTdsDhInn@jFfZ~Mli@jzAf{Djd@|gCfh@diC|Utq@~b@fz@jh@xz@|h@zz@zWvd@vV|r@zIda@rH|i@zOvmDvAr]xG`aBh[~oGpMncD^pS_AvUuAbTgEbXcGlVoJ~VuLxSwNvQoSnTap@nn@{d@`d@cMvLmm@`m@y[p]eOlTuTx_@av@ttAqX|f@{HnQsGrR{ClO}CtTaCrUaKrfAaLnjA}JxbAe@|EiK`jAsGzw@iDhj@wIvwAiAdW[hPp@fTtCvXnE~TzJzZrLbT`KzMtJzJbWpPzs@~YnP|GxKlHtJjInLdMpLhQhH~MzGzPfUhw@lPdk@hGbV`DlTtAjQZ|ToAv^g_@|uDcIt}@qBdVcAdJ}AnJ}@xH_AtDYxAMpBQfLJXPt@Fx@?x@Iv@Qt@Yn@GLyEhh@_NrdBmErj@mBpVo@be@M~JUvO{@bp@Zld@jBr`@zB|WdEtYjFtZz@|Cn@fBdA~A~GzGn@ @f@h@l@x@fA|@jBn@jBfBRbBFlAAxBOxA]nAe@dAq@~@q@n@s@`@_Ak@JaAFe@?}EzEaCpBiBtBiCxDsE|Dia@xt@qu@vtAax@lwAiMxUeLxVuJnWmI`X_eFn}Ps~DbsLuKz[ge@~yAeUvs@{x@j`C_]zaA_kEpfMyc@xaBoNf`BeC~xBu@`oBdAx]RlEl@jFzAjIt@vAb@`BPfB?hBQfBc@~As@tAaAfAWRmGxr@_BvxByAnkCj_Itj@eVtr@ulAluBq]pu@wN|c@qTvzAgHzsAmZnbGmPj`Cmf@j~Ic@|j@VxbIoYvuMcU~yFgEffAeFtfDs@jmAc@fw@lAx^dHlg@bZthAnY~eBlHdaAv@l[Kb_AuAr|Al@fs@dC`s@xe@~vHlG|p@lJjj@|Mjf@pPhc@di@reAni@vfAxHhS|BdJ~AxJnBrSVpUkAjSmCvSiOlq@kGxVsN`e@iIzPwI`MgL|K}OjJuPrFmSnCcRCwPgBaZmGyYuHkTuEgTiDiQiBsQ}@oS[{SNgOr@oOtAqOpBaOpCcQbE_QrF{MbFuMbG}w@he@ew@pe@koBplAeQjLeOnMaIlIsGdJaP`YaMpoI~^iDvYwEfk@wHlfAycA`uLoHzu@cAjK{g@xbGeAfLoJ|eAQtQKpE@vCBfIBLLr@Dr@At@Ir@Op@Wj@]f@c@^IFYrB]nBiA|Eo@nFgB~Re@fDw@`FUzAG|AFbBd@pCCnDHLbDPdC`ArDnAvDbArEd@zTvB|h@zFlf@rHfrBne@hf@`Mvf@jInsBtX|j@nGpGv@fgExr@rgClp@dd@nRjc@tTvb@xUd`Alq@h{@zw@zt@`x@rs@ro@r`BzfBj`An}Ari@|iAhi@dbB~P~p@zFfX~FnXjDxQtHhc@nLrt@tJvi@~Ido@rM|w@dArFxO`z@x@vD|o@`~CjQpz@z@xFfJtm@|@~H|C|XnBjQb@`E~@nIrGhDpCtE`AbENpGo@|E}DpHwHjDiI]yDsB}CiEaAwCk@mGLkD_BuR_BeScAoGsUesA{Nw{@ke@cwDigoBG[

FME Job BoardArchived

I wanted to throw around the idea of adding a job board on this site. With knowledge.safe.com being the premier location for FME analysts, developers, and decision makers to find anything related to questions why not add a help wanted/help needed page. The quality of people here is one of the highest pedigree and most active of many technical forums I've been a part of in my career. I have found myself recently on both sides of the fence... needing to find work, short-term/long-term consulting and at times needing to hire a short-term expert to compliment or supplement my own abilities. I've used sites like Hired, Upwork, elance, etc... to find work, but by having an FME centric job board, both contractors and employers would know exactly what they are getting. Maybe include something like a "Twitter Verified Account" checkbox next to their name for FME Certified Professionals and Trainers. With the 'tag' system in place, I would be able to find a specific type of FME person I need to contract. For example, I need and FME Certified Professional with SQL and DBA experience. With other sites I've had various success, but the FME community is quite tight-knit. The trust-factor is already built in through interactions with the community and I propose and opportunity to take it one step further. Safe would act as the broker with no liability, essentially it is just making the connection with FME being the common factor. How the deal occurs offline is up to the two parties communicating. @daleatsafe @donatsafe