Using readers and writers to extract and load data.
Recently active
I would like to be able to parse a response from the Bentley OpenGround REST API, and am getting somewhere but not far enough!I can use a mixture of attribute exposer and JSONFragmenter to get some information out, but it never quite looks like the table i want. i will be doing multiple queries over the coming months into this api, and will have a different number of columns each time, so cant parse through each data record based on the number of columns i want in that particular query! I can get to the point where i have a feature for every feature in the database, and then i have a _response_body which i need to parse further. i can keep parsing it through jsonfragmenter, and i end up with a jumbled mess where columns are names wrong and the colunn information i want is in the wrong part of the table. it is a simple structure, such as “DataFields” “heade” for column name and “value” for data itself. i will add some screenshots to aid in query. any help would be greatly appreciated!Th
Starting soon I will receive about 150 XML files each with a different schema but the element name is include in the file name. I have many ways to use a single FME workspace to process these files dynamically but have been unsuccessful. I figured using a Directory File reader and passing the file names to FeatureReader setup as XML then saving to a Dynamic Writer to a database. No joy. What am I missing for something so simple.
Hello everyone. I am working with citygml modeling and I am facing some problems with appearanceSetter. I have a workspace similar to the con terra that I used as a base.Originally everything is fine but when I put an appearance setter before the roofSurface writer (ground or wall too). The geometry traits previously set are automatically modified by the FME. In the image below is the original output.And in this the output with the appearanceSetter.The geometry trait created for polygons was moved to linearRing and a new one was created for polygons.The file is normally opened by the Data Inspector, however, when running Val3dity to perform the validation, the xlink is broken, as it must point to a polygon. EDITED: I forgot to mention that to reproduce the situation above i placed a appearanceRemover just after the reader to remove the original appearance of input data.Does anyone know how to fix it? Thank you in advance for your help. Mcdonnell
Hi, I have this dynamic setup where a postgres FeatureReader reads the schema and the results are written to another database using this schema information to create the new tables. However, a table having colums of type unqualified varchar will be regenerated with varchar(80) and if records are loaded having length >80 the workspace will crash, saying: Error was 'ERROR: value too long for type character varying(80)I attached the workspace. Is this a bug in FME or am I doing something unbelieveable silly?😕 I'm using FME(R) 2020.2.3.0 (20210129 - Build 20820 - WIN64) on a postgres 12.9 (Ubuntu 12.9-1.pgdg18.04+1).
Hi.I have another string replacer question.I have a file of text strings (appellation) similar to LOT 1/2 DP 17528 LOT 1/2 DP 13552 RS 16932 18812 19986/7 PT 18813 20862 BLK IV KAPUNATIKI SDI have split the LOT 1/2 DP 17528 & LOT 1/2 DP 13552 into 4 separate rowsTemp, _result, appellationLOT 1/2 DP 17528, LOT 1 DP 17528, appellationLOT 1/2 DP 17528, LOT 2 DP 17528, appellationLOT 1/2 DP 13552, LOT 1 DP 13552, appellationLOT 1/2 DP 13552, LOT 2 DP 13552, appellation What I want to be able to do is remove the temp part from appellation The resultant appellation in this example would be: RS 16932 18812 19986/7 PT 18813 20862 BLK IV KAPUNATIKI SDI have tried doing this but only ended up removing one part from each instead of both Temp parts. I would be very grateful for any help with this.Regards, Paul
I am reading datasets and can extract the crs (stateplane) from them. but I need to use a spatialrelator for data that is in WGS. I need to be able to determine incoming crs and set a reprojector on the wgs data right before the spatialrelator. In the past i have only been able to accomplish this by stringing together more than 1 workbench.Any ideas on how to get this done in 1?
Hello,I created a model and I want to export it as citygml, but the model does not appear in the FZK gml viewer. How can I solve this problem? Am I making a mistake in the writer? I added my data and FME WB. Could you please check it and share what I am missing?
Hello, I'm sure it's a pretty simple question, but somehow I can't see the wood for the trees to find the answer.I have a list (.xls) of coordinates in the form <name>, <x_coord>, <y_coord>. The coordinates are, for example, in the form N 51° 06.796 E 3° 14.545.Now I want to save the information in an FGDB and would have expected the VertexCreator to help me with this. Unfortunately, all my values are completely rejected. My further attempts - removing all spaces; using the HUB Transformer "DegreeFormatter" or Decimal Degrees Calculator all failed. For this reason, how do I manage to get the coordinate format into a form so that the VertexCreator accepts it? Thank you very much.
Hi there,I have a question about the unit of coordinate in the iFC file, which i wanna clippe with another solid. The coordinates are 1000 times bigger, and the clipper could thus not find overlaps. How can I adjust the coordinate value: using the expression evaluator, and divide all the coordinates with the value 1000? if tho,How can I access all my coordinates because I could not find an attribute that enables that? I also tried the coordinate extractor but it is only another copy of the coordinate values, which doesn't update the geometry coordinate value in the IFC . If anything i understand it wrong pls point it out! thanks in advanceFang
A have one dataset (Dataset A) that has contains fields, one of which is CityName; example value “Sudbury”. I have another dataset (Dataset B) that contains a field made up of comma delimited AltNames; example value “Gran Sudbury,Grand Sudbury,Sudbury, etc.I need for these features to be joined in one dataset if the value in CityName matches any one of the comma delimited names in AltNames, however different records have different numbers of alternate names and it cannot match with a section of a comma delimited value. For example, Sudbury != Grand Sudbury.I think that lists may solve this but I’m not familiar with the transformers that might enable this process.
I would like to visualise data on the inspector data or in the graphic view after I reprojected it. That would be in the middle of a workspace before using other transformers and just to locate some features. I know it’s possible to write the data in a shape and check the features on another software like QGIS or ArcGIS but I would prefer to do that in FME directly if that’s possible, when I’m building my workspace. Can I do that?
I want to convert JPEG file (4 band) to COG with below parameter:I have problem with “Interleaving Type”:When i chooses “BIP”, i received a message log:“COG writer: JPEG compression supports at most 3 bands with BIP interleaving. Please change the interleaving to BSQ, choose another compression method, or reduce the number of bands”When i chooses “BSQ”, i received a message log:“COG writer: The file layout 'Cloud Optimized Tiles' may only be used with BIP interleaving. Please change the interleaving to BIP or select a different file layout”I need to convert JPEG file to COG with compression method “JPEG” and keep alpha band.Can i do that? Am I mistaken about something?
I’m trying to convert DWG’s to SVG’s. It works, but it’s using the AutoCAD colors. I would like to just make the lines all one color, and NOT use the autocad colors. Ideally, I’d like to fill certain layers with another color.
I need to extract TAB or Shape file from S3 and read data from it including schema.I have discovered in this articlehttps://community.safe.com/data-7/how-to-extract-the-actual-shapefile-from-featurereader-geometry-and-attributes-14548?tid=14548&fid=7that FeatureReader apparently can’t read all attributes that are in the file, until I expose them. But what is I don’t know the list of all attributes? I don’t know what exactly will be uploaded by other people to S3 bucket, I just need to read the file. Could you help, please, to identify the correct settings of FeatureReader or another way to extract data?
Hello, I am using a "Directory and File Pathnames" reader to pull shapefiles from a folder. I then walk them through a tester to filter out the files that I do not need. All good and working up to this point. The goal is to work on those shapefiles and have a processed shapefile as an output, in the end. After the tester, I send the "passed" ones to a "FileReader" that outputs the desired features under the <Generic> port. My question is: how can I write those out and keep the geometry + attributes of the shapefiles for future processing? I can see them in the Inspector but I cannot figure out how to move on to the next step as the only attributes I see imported to any transformer I add, are the ones from the "Directory and File Pathnames" reader, not the shapefiles themselves.Long story-short, I want to use the shapefiles that passed the "Tester" as readers.
Hello , i would like to find duplicate of specific column As exampleColumn A contain 256789 - 45674567 - 256789256789 - 4567i would like to find all duplicate values at column A ,consider the values as duplicated in case a-b or b-a i mean , 3 above values consider are matched .the target to ignore order of each cell How could I achieve it ?thanks in advance FME 2021
hello ,i would like to save geometric data into excel file then I read the excel file in arcmap ,It shows me the attribute tables and location of lines .How to achieve it ?fme 2021thanks in advance
I am attaching a sample db3 file that I am using for a project. This db3 file contains many fields of blob type. The table, field of my interest are as belowTable Name: CURRENT_TAGField Name: _TAGOBJ_SUBSTATIONGIDselect _TAGOBJ_SUBSTATIONGID from CURRENT_TAG where _TAGOBJ_GID = '145242397947723855';This particular record in my application shows this value “151433541765593417”But in db3 file it is actually stored as blob. How do I decode the blob as text? I tried to use online tools to convert this binary to text - it doesn’t work. Also, how do I know the data format that is stored as blob - is it either image, xml, json etc.?
Hello, I am trying to create a workbench that will read in all our add/delete tables from our SDE, do a count on the running total of records for each, and export a spreadsheet with the table name, and the counts in each,I’m using a SQL Creator and my SQL statement is something like SELECT COUNT(*) FROM <tablename>,and right now that's not returning me any value even though it was successful? Also, how would I go about coding out to add the next table in?would it be:SELECT COUNT(*) FROM <tablename>unionSELECT COUNT(*) FROM <tablename2>unionand so forth?also for background:We use Microsoft SQL Studio, and I chose the connection as a spatial server.Thanks for any help!
Hello,I want to generate an incrementation from the max value of a column in order to not have duplicate values. The _max value column is not recovered in the counter for the 50 duplicate features : Any idea ? Thank you
Hi, I have a text string that looks something like this…LOT 4-6 DP 287 SEC 2-3 SO 345152 I have a regex “(LOT \d+[\s?&-]\d+ DP \d+ ){1,}” that grabs the LOT 4-6 DP 287 from the text line.How can I split the 4-6 into separate attributes so I end up with Lot 4 DP 287 Lot 5 DP 287 Lot 6 DP 287I could do it with a testfilter but there are unknown combinations. 1-12, 60-62, 78-82 etcAny help is greatly appreciated.Regards, Paul
Hey. Do you have any idea how to solve the sequential processing of input elements on HorizonAngleCalculator? If I have several elements on the input of the Horizontal Transformer, I need one element to be processed and then the next element to follow after that element is processed. After that, I would put a featureholder after the HorizontalAngleCalculator and it might work. I've tried testing the featureholder with counter and filtering features that have already been processed, but somehow I can't get it to work. Thank you very much
Hi.I have a file of text strings that have multiple ‘parts’ that I have split into a _list. Example text string... LOT 9 DP 19723 LOT 12 DP 384274 LOT 8 DP 30287 LOT 9 DP 2402 PT RS 17343 17491 BLK VII X XI TENGAWAI SD-TIMBER NOT ASSESSEDUsing string searcher, I have formed a list of the formatted string to push into the list using regex: (LOT \d+ DP \d+){1,}The result is a list containing the:appellation:LOT 9 DP 19723 LOT 12 DP 384274 LOT 8 DP 30287 LOT 9 DP 2402 PT RS 17343 17491 BLK VII X XI TENGAWAI SD-TIMBER NOT ASSESSED matches: LOT 9 DP 19723 LOT 12 DP 384274 LOT 8 DP 30287 LOT 9 DP 2402 What I want to be able to do is remove the matched part from appellation on the way through leaving me with both the matches as individual features, and a subset of appellation with the matches removed. The resultant appellation in this example would be: PT RS 17343 17491 BLK VII X XI TENGAWAI SD-TIMBER NOT ASSESSED I would be very grateful for any help with this.Regards, Paul
WFS 2.0 services are available on the Geoportal Baden-Württemberg. It is not possible for me to view/download the contents of the objects (BP_Plan) with the WFS Reader using FME 2023.Does anyone have any idea what the problem might be?Example:https://geodienste.komm.one/ows/services/org.165.d5516855-2ef7-4512-92f7-68d208f6f722_wfs/org.165.15f5882e-891a-4ce4-b2fc-000a45978e11?SERVICE=WFS&Request=GetCapabilities
Hi there,I am fairly new to FME. I have around 6000 dwg. files regarding the same industrial drawings and the data is roughly found in the same place in each file. I would like to know how i can extract the attributes that i need out of these files.I have started using a wildcard function to read multiple files. I know this question is fairly vague, still I was wondering if anyone has the time to help me with this, or knows any recomendations for explanation (youtube or similair examples/workspaces) Like I said, FME is still pretty new to me so any advice would help. Thank you