Using transformers to modify data.
Recently active
I've seen similar questions asked here but cannot find real solutions. So here's what I'm trying to do..I would like to get information from this page as an example: https://help.autodesk.com/view/CONNECT/ENU/?guid=GUID-03D59AAD-65B0-45E3-84F2-A12AAA5BB267.If you look at the page source, you'll see there isn't much valuable info. here and it seems to be using javascript to serve up the page (which updates when a new version is released, maintaining the original URL). If I use an HTTP caller with a GET method, I get the same page source. However I'm interested in the text you see in the loaded page. There are lots of sites out there built this way nowadays, which work in a very similar, dynamic way.Is there a solution to get data from this kind of page? The only plausible (and costly) way would be to use some service such as parsehub.com (which I have not used or tested) where you could use their REST API to get data into FME: https://www.parsehub.com/docs/ref/a
Hi,I am trying to delete files using system cleanup but no file is being deleted. I am using FME Server 2016. Any pointers would be much appreciated.Thanks.
Hey folks,Just wanted to alert you all to a new custom badge: the Inventor.This badge is awarded to any user who posts an idea that makes it to "In Beta" or "Released" (Completed) status. I've been through the existing ideas list and the badge has been awarded to the following users:If you too want to earn this badge - which is a bit more prestigious than "Pirate Chief" - then get posting new ideas for FME - and then encourage your FME friends and colleagues to vote up that idea.It's late in the development cycle, but still not inconceivable that you could post an idea that makes it into FME2017! RegardsMarkps - I've been fairly lenient so far in awarding this badge. But from now on you'll need to come up with something really unique - not functionality already on our radar or in the beta!
Hello FME-Experts!I have no experience with reading of html code, but I have problems reading data from a simple html file.If I use the HTML Table reader I get only the first record (witch contain the attribute name of the data) and not the 4 records that I need.I think, the workflow has to be simple but I tried several options without results.I attach an html example file.Thank you very much for the help!
When opening and trying to view qleader arrow types in FME from an AutoCAD file, the qleaders appears just as 2 straight lines, where as it should appear as a spline. Multileaders appear to work fine. The autocad_path_type is autocad_spline_leader See the attached dwg for an example. The 2 large straight lines on the right appear as proper spline leaders in AutoCAD. Is there any workaround to transformer these into the proper splines? I can't even see attributes carried through to be able to do this manually. Thanks in advance,Luke
lat lon to geometry
I have attempted a few different techniques to get my desired output.1.) Reader->AttributeExposer->JSONTemplater2.) Reader->AttributeExposer->ListConcatenator->JSONTemplater3.) Reader->AttributeExposer->JSON Writer4.) Reader->AttributeExposer->ListKeyValuePairExtractor->5.) Reader->AttributeExposer->ListExploder->JSONTemplater6.) OthersI am working with values such as:objid | fme_attrib_info{}.field_name | fme_attrib_info{}.field_value |-------+--------------------------------------+--------------------------------------|1 | SERIAL_NUM | 12345 |2 | TYPE | SOME_TYPE | My desired output would be as follows:{ "SERIAL_NUM": "123456", "TYPE": "SOMETYPE}I've used the tutorial on site here for extracting attributes from CAD blocks. Now, I would like to take these attributes and make a json payload per unique block to be wri
Hi community,I have a very complex challenge. I have a csv with an unknown set of (dynamic) attributes. I was asked to rename the attributes so everything from the dot is removed. That is very easy I would say with the BulkAttributeRenamer (BAR), but then I found out I get duplicate attribute names and this is not allowed in the BAR. I asked what to do with these duplicates and they said I have to sum the attributevalues if there are duplicates. So for instance I have and attribute xxx17.1 and xxx17.2 the result must have an attribute xxx17 and the values have to be summed. This is no problem with a fixed schema, but I have a dynamic one and I have no idea how to do this. I have attached an example csv. All ideas are welcome.Thank you.
Requires functionality in a CAD environment that allows anyone in the * .dwg or *.shp file layers to prioritize what can be common "Trim" the lines (TRIM function) according to the given priorities.
Hello all, Is it possible to add your own python functions to be callable from the Arithmetic/Text editors? I find I keep having to write multiple entries in an AttributeCreator to do things like find the smallest difference between two angles. ie. Situations where you need some state so you cant just use an expression. It would be nice if I could add a python function to a workspace that is then callable like the Math functions that are available in the Arithmetic editor.Let me know if this is not currently possible and I will add it to the ideas section.ThanksMichael Pascoecc: @deanhowell, @luke.giles
I know it's probably something simple, but I'm not seeing it. I thought perhaps inlineQuerier would help, but I am not versed with SQL. Essentially I want to do a "vlookup" from one file into the other based off the key field. thanks in advance!
Hello.I have calibrated route in file geodatabase (ArcGIS). I have line coordinate (kilometre) in the start and the end point. I try export this route with M-value to PostGIS in FME. But in result I have line without M-value. Do you know how it works?If i created MULTILINESTRING M in postgis directly sql it work. So metadata field geom support M-value PS GeometryExtractor have field with M-value
I am looking to merge into a list selected attributes from nearby points and output a single feature for each group of nearby points. PointOnPointOverlayer perfectly finds the nearby points and generates the list I want, but outputs all the input features with a list of all the other features. Thus for say 4 nearby features it will output 4 features each with a list which identifies the other 3 features in the group, but without anything like a group ID with which you could select just the first in each group with a DuplicateFilter.This is almost exactly the example given in the PointOnPointOverlayer manual page which creates lists of crimes at common locations... but doesn't explain how to not have multiple copies of the same list at the same location. Am I missing something obvious? It seems that PointOnPointOverlayer has done all the hard work!The closest I could get to generating a group ID is to give all my input points a unique ID field, then set "Merge attributes"
Hello all, I need some help performing a spatial operation. What I haveA set of coordinates (sample enclosed) What I wantRemove / dissolve coordinate pairs if the are within 2 km radius of each other so that only coordinate pairs remain that have no overlap. What I've triedA Bufferer transformer with a radius of 2km connected to a PointOnAreaOverlayer.The result is a bunch of overlaps but I don't know how to further process these. Cheers,Eduard
Hello,I am a researcher from Slovenia. I'm having huge trouble working with Slovenian datasets, as they are in EPSG:3912 or EPSG:3794. These two systems are not supported by FME. I solved this for shapefiles by using ESRIReprojector. But it does not seem to work with las datasets. I am trying to extrude footprints using LIDAR data and convert that to KML, which needs WGS84 data.Is there a sollution? I am quite surprised that FME does not support all EPSG projections.I even tried to add custom coordinate system and was not successful.Regards!Jernej
Hello, I'm passing a load of lines through an FME workbench and eachline has a project name as an an attribute. I want to add an attribute that counts how many lines are in the project (share the project name). Does anyone know how to do this? I've tried playing around with the counter but have had no luck. what I need is it to separate out into all the different projects, use the counter, then chuck everything back together. I would use a filter but there are hundreds of projects. Any ideas? Thanks
I have fed it into a NullAttributeMapper to make sure all Nulls, Missing and Empty fields are uniform. That's fine, and so is the Date transformation when inspected. However it still rejects any of the Null fields, and when it outputs to the AttributeManager, all of the fields are then Null. Even the ones with a date value. I've checked the "Passthrough Nulls, Empty and Missing" values to "Yes", but still no luck. It seems to apply the null to everything in the DateTimeConverter.I've tried with a new Transformer/rebuilding that section, and have also restarted the workspace, as well as clearing any temp files. Any help? Thanks
The statistics Calculator gives this error when trying to count the sum of an attribute: 2020-08-11 12:49:59|2233.8| 2.3|ERROR |Python Exception <MemoryError>: 2020-08-11 12:49:59|2233.8| 0.0|FATAL |@Python failed to execute function `StatisticsCalculator.recordFeature'2020-08-11 12:49:59|2233.8| 0.0|ERROR |StatisticsCalculator_2_InputProcessor (TeeFactory): StatisticsCalculator_2_InputProcessor: @Python execution failed2020-08-11 12:49:59|2233.9| 0.1|ERROR |StatisticsCalculator_2_InputProcessor: @Python execution failed There was approx 2.5M features passing through each one having a value of 1. So the sum operation should not be exhaustive. What could cause this? A bug?
I use a custom transformer with two inputs, 1 per table.In my custom transformer i have 2 published parameters to get the attribute of:table A - Fieldtable B - Field But both parameters return globales listes of fieldsIt's possible to isolate the fields list for each table on it specific parameter
I have a simple workbench that read data from a table and produce a number of Excel files. Depending on the config, the output can be 2, 3 or more Excel files (via FeatureWriter). At the end, I want to send the result by email. If I use Emailer and put my attachment manually, it works perfecly, but if I want it to be more flexible, I want as many attachment as my workbench produce. Is it possible to concatenate the output dataset produced ? I tried but failed... any Idea ? No zip please!
Hi there, I am defining a data frame at Python caller on __init__ function like below: class FeatureProcessor(object): def __init__(self): df2 = pandas.DataFrame(columns= ['INCIDENT_DATE',INCIDENT_MINS','INCIDENT_VALUE']) later in the program at input function I am storing feature attribute value in data frame something like that: def input(self,feature): self.pyoutput(feature) df2['INCIDENT_DATE'] = feature.getAttribute('DATE') df2['INCIDENT_MINS'] = feature.getAttribute('MINS') df2['INCIDENT_VALUE'] = feature.getAttribute('VALUE_LOST') However, when I ran the Python caller block, it throws an error message that df2 is not defined at the input block. Could anyone point out where am I making the mistake?
I change FME Proxy Settings from "Use system proxy settings" to "No Proxy". But FME is still using the system's settings. So I edited the systems's settings and disabled proxy, and removed proxy settings from environment variables. FME still uses proxy settings. I restarted FME, and it still uses proxy settings. I can't connect to the proxy right now, so the transformation crashes. Snapshot below. How can I get FME to not use any proxy? FME 2020 here. Thanks in advance, Mike
I am not getting the right info when I use the rightleft transformer for the position of the point to the road I check the segment and is has the same direction all Point are getting RIGHT position valueany thought please
Hi,So after lots of testing, it's looks like when I export shapefiles using the FeatureWriter and include the .sbn Spatial Index, the shapefiles do not function properly. When first loading them in ArcMap, they appear to be ok, but when zooming, selecting, panning the map, I get very odd results with certain features suddenly dissapearing, not being selectable, etc. If I modify my FeatureWriter to not include any spatial index, then the shapefiles work fine. I only have this problem when I include the spatial index. I would like to include the spatial index. Is there a fix for this?? Thanks,
Hi - I'm trying to create individual records with attributes from a JSON response from an API. I've been able to call the API using the HTTP Caller and have tried using a JSON Fragmenter to extract attributes. I don't seem able to get attributes from this but as shown in the attached there are a number of attributes in one of the returned rows. This looks as if it is a list called 'array' (there are 49 members in the 'list' which is as expected). I'd have thought I could use a List Exploder but this transformer doesn't pick up any list attributes. I'd really appreciate any advice on how I might explode this structure in to individual rows with attributes. Thanks and all the best. Matt