Skip to main content
  • EN

FME Form

A space to discuss FME Form

10000 Posts
H
hayden
asked in Data
Alternative methods for feature cache of large datasets

Feature cache works great as a method to inspect all transformers after a workbench has ran, but causes a few issues for me.it hurts performance to cache each and every transformerI lose all the cached data if FME crashes or I close the workbench.I am looking for advice on ways to cache or otherwise save output from only a few transformers. My only idea is to write features out to a file, and later bring back in with a writer. are there other transformers I may not be aware of that can save/recall data after restarting the workbench? 

142
hkingsbury
Celebrity
H
hayden
asked in Data
Restore Cached Data after crash

Is it possible to restore the cached data from a ran workbench that has been closed or crashed? I am running a heavy dataset (10s of millions of features) and it takes several hours to run. As I troubleshoot the workbench after, I am bouncing around between the cached inspectors for each transformer. My workbench has been crashing often in this process, and I wish I could re-open it and get back to the full cached state without having to re-run it.

130
hkingsbury
Celebrity
H
hayden
asked in Data
Best file type to read/write locally for performance

I would like to write features to a local file to read back in later and looking for the best file type for performance.My local machine does not have any DBs to use for this purpose (Oracle, SQL, etc)so I am limited to files I can save to a PC folder.Does FME have it's own DB file type that I could be utilizing? I tried FME Feature Store (FFS) but it has limited functionality, such as not being able to set truncate or table drop on runs. And it's main purpose is to add encryption which I don't need and seems like it hurts performance.I have read on here that CSV is very high performance but I don't want to lose metadata such as data type, len, etc.Excel has a feature limit that does not work for me. 

160
david_r
Celebrity
fmeuser_gc
Contributor
fmeuser_gcContributor
asked in General
How to visualize PostGIS table with two different geometry types - Point & Polygon?

I've PostGIS table with two geometry types - Point & Polygon. When I read table, I'm only able to see Polygons. But point geometry column is only shown in table and unable to visualize. Is there any way to visualize point geometry instead? How do we explicitly mention for which geometry columns to be visualized? Thanks!

1312
fmeuser_gc
Contributor
dataman
Contributor
datamanContributor
asked in Transformers
Outputs FME Form - FME Flow

Hi there,we have a workspace where we check with a HTTPCaller, if an URL has data to download. This workspace works perfectly in FME Form.  If we notice, that there are no data available, we want to become an E-Mail in order to react on time. We have created an automation in FME Flow with an E-Mail-block.  Now we do not know how can we make the match: No data to download (FME Form) -> Action failed way in FME Flow in order to send the mailData to download (FME Form) -> Action successful in FME in order to log a message how can we make it? Thanks a lot in advance

131
dataman
Contributor
dsign-world
Contributor
dsign-worldContributor
asked in General
PythonCaller is not working

My python code is working fine outside of FME but it fails inside of a PythonCaller. The documentation doesn't give any information about the scope or execution context in which the code inside a PythonCaller is run. What are the restrictions when running python code inside a PythonCaller? Why does it not run the same code that you can run outside of FME? (while using the same installation of python in FME and on the command line)I'm trying to use concurrent.futures.ProcessPoolExecutor for parallel processing in Python inside a PythonCaller but I'm getting this error:from concurrent.futures import ProcessPoolExecutordef test(x): return xwith ProcessPoolExecutor(16) as executor: result = executor.map(test, range(0,10)) next(result)Python Exception <PicklingError>: Can't pickle <function test at 0x000000000F6A8F70>: attribute lookup test on __main__ failedconcurrent.futures.process._RemoteTraceback: """Traceback (most recent call last): File "multiprocessing\queues.

3373
D
buckrogers
Contributor
buckrogersContributor
asked in Transformers
How to use a generic reader with published user parameter for the dataset

I am creating the desktop version first but it will eventually be published to FME Server so the specific file choice would need to be done using a user parameter.Ideally, I would like the generic reader to be able to accept any of the following file formats: feature class, shapefile, GeoJSON and CSV. I have created a published user parameter for File/URL which would allow the user to locate the file they wanted to run through the workbench.Currently, when I try to add this user parameter in as the Dataset for a generic reader, I get the following error:Workspace generation failed. Parameter file was C:\\Users\\********\\AppData\\Local\\Temp\\FME_1697209596703_9204. I have also set up a workbench where I tried filtering to feature readers for specific formats e.g. feature class would have it's own Feature Reader where the Format is: Esri Geodatabase(File Geodb) and the Dataset is still my user parameter. However, in this example, it needs to know the name of the feature class(Feat

2387
nielsgerrits
VIP
lazarlubomir
Contributor
lazarlubomirContributor
asked in Transformers
How to generate next letter of alphabet?

Hello everyone, is there any way how to generate next letter of alphabet? E.g I have feature with value "A" and I wanna generate next letter in alphabet, so letter "B". Is there any function for that? Thanks a lot! Lubo

436
david_r
Celebrity
candacel
candacel
asked in Integration
ArcGIS portal feature service reader - type 4, Number: 1, Message: Failed to get auth...error

I've followed this article https://community.safe.com/s/article/How-to-create-an-FME-Web-Connection-for-Esri-ArcGIS-Portal-Feature-Service-OAuth-2-0Initially I was able to read features. I've come back into the workspace after a week and I'm getting this error Type: 4, Number: 1, Message: Failed to get authorization query string.Updating access token for 'WebServiceSanDiego'JSON Error: refresh_token expiredUnable to update access token for 'WebServiceSanDiego'I've re-tested by webservice by going to preferences, manage service and test (requires you to login), but it's still not working. I would assume I'd only have to set this up once and it would refresh itself going forward, or is this something configuration wise on the portal end? Note, this is something setup for us to connect to by a customer. Is there a more ideal suggestion for ongoing integration? Read/Write 

7090
chadprop
Participant
billybob
Contributor
billybobContributor
asked in Data
read only the newest file from a One Drive directory

Does any one know how to read the latest file from a One Drive folder. I'm able to select a file from the One Drive folder using a Microsoft OneDrive (Graph) web connection, but I can't find a way to read all the files in the folder and then find the latest. I have read the question regarding reading the newest file for a folder https://knowledge.safe.com/questions/65203/read-only-the-newest-file-from-a-directory.html. It's very helpful condeptually. I would appreciate an thoughts or insight. Thanks,Frank

997
chadprop
Participant
sam_appleton
Contributor
sam_appletonContributor
asked in Transformers
Upload entire folder from local PC to Flow Shared resources, via server app

Hi, Is it possible to upload an entire folder from a local PC to Flow Shared resources, via server/automation app? I know that single files are able to be uploaded, but I was hoping my workflow would go like this: User uploads file via web app. It is stored in Shared resourcesSeparate workbench uses a Path reader on the file, gets all .shp files and performs ETLI can't see any other way to do it, other than the user having to upload 20 different shape files via the app interface. I get this warning when using FME copy writer, with s published parameter of 'filecopy_source_dataset':File Copy Writer: Feature contains an invalid source dataset 'C:\\Users\\myuser\\Documents\\Development\\FME\\UPLOADFOLDER'. No file operations will be performed for this feature Many thanks, Sam

490
hkingsbury
Celebrity
sami26beniaouf
Contributor
sami26beniaoufContributor
asked in Transformers
Drap a 2D surface or polygone on a 3d Surface

Hello everyone,Is there a way to drap a 2d surface or polygone into a 3d surface (like in the picture joined). If there is in which format do we need the surface and polygone to be? Thank you in advance

351
kailinatsafe
Safer
aron
Supporter
aronSupporter
asked in Transformers
Write to 'Primary Key' and 'geometry_column' in a Dynamic PostgisWriter?

I am using a SchemaScanner to prepare a dataset for dynamic writing to a PostGISWriter. How do I write to the columns for PrimaryKey and Geometry in an dynamic environment?

751
nielsgerrits
VIP
timboberoosky
Contributor
timboberooskyContributor
asked in General
Is there a way of disabling the Workspace Startup and Shutdown scripts?

I haven't been able to figure out a way of disabling the Startup and Shutdown Python scripts I have setup in my workspace. When running my workspace for testing, I don't want the Startup or Shutdown scripts to run since they relate to moving and renaming files and folders that just fail when I'm running tests. Does anyone know if this is possible? Thanks!

340
M
S
sophieko
asked in Transformers
Hi, I have an issue about the RasterPolygonCoercer transfrormer.

I want to turn my vector layer into a raster layer and then back into a vector layer to simplify it. When I used NumericRasterizer, I grouped my layer by category, but when I convert it back to a vector layer, it's not working. When I tried to rasterize it without grouping it, it worked fine, but I lost the attribute I was interested in.

90
S
dataman
Contributor
datamanContributor
asked in General
Process a table with a loop

Hi there,We have an excel file with an easy table. Just 2 columns: Position and Id. The file has about 100 lines.We want to read the file and process each line. We have tried with a custom transformer but we are not able to process the lines individually. If we read the Excelfile we get the 100 lines and all lines will be process at the same time. If we convert the lines into a List, we get the error "Transformer/Factory test_loop_2_ListBuilder is blocking and cannot be used in a loop, but was found between 'Loop to test_loop_2_Looper1697190238 Input Splitter' and 'test_loop_2_Looper1697190238 Input Splitter'".  Has anyone an idea to make it?I have found this link (https://docs.safe.com/fme/html/FME-Form-Documentation/FME-Form/Workbench/transformers_custom_looping.htm) but it helps me only with the loop. Not how I can process the ids separatelly. Many thanks in advance!

777
nielsgerrits
VIP
A
arnoldatk
asked in Transformers
GDB Annotation to DWG Text, lots of labels missing

UPDATE: Managed to figure out what went wrong in my comment below the main post (TLDR: Used the wrong transformer and field value from my dataset) Hi, so I'm trying to convert my annotation layer in my GDB files along side the point, polygon and line layers into DWG. In my screenshot of my ArcGIS Pro instance, you can see the various labels above my polygons.But it looks like only 2 of them passed through to DWG when converted through Workbench.In my current workspace I'm using a LayerTypeField field in the attribute table of all of my layers I wish to converted to sort them into the correct flows (Points, Polygon, Lines, Annotation).Those values are exposed via an AttributeExposer parameter as you can see here:The annotation layers are very basic ones that are created from running Convert Label To Annotations within ArcGIS Pro itself and an additional field added that just contains "Annotation" as text for filtering purposes in Workbench (LayerTypeField).  At this ju

670
A
dewrightco
Contributor
dewrightcoContributor
asked in Transformers
Find the nearest line segment to a point with a couple common attributes?

I have a point array of addresses, and a line geometry layer of streets, and I need to be able to get the ID of the nearest line to the point that also has a couple matching attributes. What are the best options to consider?

331
nielsgerrits
VIP
so_much_more
Supporter
so_much_moreSupporter
asked in Data
Uppercase characters for spatial column in feature writer not accepted

I am using the feature writer to write new column into a geopackage database. Instead of using the default geom I need to change the name to Geometry, but the writer does not seem to accept the spatial column to have uppercase characters?When I specify the spatial column to be called Geometry it is being writter as geometry in the table. Any ideas how it can be solved? 

190
so_much_more
Supporter
mattwilkie
Supporter
mattwilkieSupporter
asked in General
Python 3.12 compatibility

What, if any, are people's experiences using FME with python v3.12? any new foot guns or tar pits?

750
debbiatsafe
Safer
kschultz
Participant
kschultzParticipant
asked in Transformers
Directory And File Pathnames reader does not read the file properties anymore? Is there a better way to do this or am I missing something?

Under path parameters for the reader it no longer has "retrieve file properties" which means I can't access the create date of files breaking one my workbenches. I tried using the filepropertyextractor transformer but its just too slow for the amount of files I need to read. Is there a way to fix this?

1520
kschultz
Participant
vxn43
Supporter
vxn43Supporter
asked in General
Move file(s) from folder to another folder

I want to move files from one folder to another. I have added the FileCopy Writer to my workbench and set the parameter to Move, not Copy. I am using FME Desktop 2021.  I think I am missing something because when I run the workbench it runs successfully but no file gets moved. I have included screenshots of my workbench.

4371
dustin
Influencer
jaygrg111_
Contributor
jaygrg111_Contributor
asked in General
Why do I keep getting the "Unverified and need to ask admin permission" to establish a Sharepoint web connection on FME Form?

Although my Sharepoint admin team has already given me the permission for Sharepoint Online Connector, I keep getting the "Unverified" warning.

710
jaygrg111_
Contributor
adrianwelsh
Contributor
adrianwelshContributor
asked in General
Batch KML to Layer in ArcGIS Pro, but all append to one layer

I have about 1,000 KMLs in a folder, all with the same schema and only one feature each (line features). I want to bring in each KML into a single feature class in a file geodatabase in ArcGIS Pro and populate a field in that feature class with the unique name of each KML. Is there a way to even do this in FME? Thanks!

1267
nielsgerrits
VIP
bobo
Contributor
boboContributor
asked in General
How to Create FGDB features based on given attribute description table?

Hi, everyone, I need to create a lot of feature classes in an Esri File Geodatabase. Creating them one by one in ArcCatalog takes too much time, I'm sure that there must be a fast way by using FME, I just don't know how, can someone help me out or give me some hints? Thanks very much.Here's the attribute description table:

60
nielsgerrits
VIP

Badge Winners

  • FME Certified Professional
    stuart_misohas earned the badge FME Certified Professional
  • English Speaker
    pwalshhas earned the badge English Speaker
  • Webinar Watcher (Grey)
    pwalshhas earned the badge Webinar Watcher (Grey)
  • Welcome Back
    clewishas earned the badge Welcome Back
  • Archaeologist
    couchhas earned the badge Archaeologist
Show all badges

Community Stats

31,849
Posts
120,957
Replies
39,475
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.