Skip to main content
  • EN

Data

Using readers and writers to extract and load data.

  • 6,708 Posts
  • 24,712 Replies
6708 Posts
fimpe001
Participant
fimpe001Participant
asked in Data
Why does my AutoCad reader turn a single text object into 2 distinct rows if it contains a space character? I would assume that my reader would output just 1 row with autocad_text_string = '1-500 EE'.

Why does my AutoCad reader turn a single text object into 2 distinct rows if it contains a space character? I would assume that my reader would output just 1 row with autocad_text_string = '1-500 EE'.

1
david_r
Influencer
3 years ago
leepinney1972
leepinney1972
asked in Data
Oracle to postgis has a limit of 100000?

I seem to hit a limit of 100,000 features ​when extracting an oracle spatial database of over 150,000 features into PostGIS.The workbench shows only 100,000 and in QGIS the feature count is also 100,000.How can I ensure all features are extracted?Thanks.​

2
leepinney1972
3 years ago
jkr_wrk
Influencer
jkr_wrk
asked in Data
Excel2Excel reader and writer bug/feature-request

I have the following situation. There are multiple excel files with multiple sheets in it. All sheets have the same template. So i use a dynamic FeatureReader to read all the data at ones. And update a single value based on another value in that sheet. There are two problems:1. Is that in cell B1 there is a 'project-name' that is different for every sheet in the dataset.In FME it is not possible to set it to always use the column name as Attribute name so it will use B1 value as column name. Now i have lots of column names that I need to expose and rename. But I don't know on forehand what they will be. A setting that FME will always use column name as attribute name is wanted here. 2. When writing the data I want just one cell to be updated. So I use a dynamic filename and a dynamic sheet name. Then I use the Column Name and xlsx_row_id to identify the cell to update.Then i get the error:Column name 'xlsx_row_id' for sheet '<Generic>' could not be used as a column position.I can

1
danielkroon
3 years ago
K
ksm65
asked in Data
Hello everyone, I want to change the name of the shape file to "asd.shp" rather than "Table1". I need a solution suggestion for this. Also, I don't want to see all the columns in the excel list in my shape file, I need some of them.

Hello everyone, I want to change the name of the shape file to "asd.shp" rather than "Table1". I need a solution suggestion for this. Also, I don't want to see all the columns in the excel list in my shape file, I need some of them.

1
redgeographics
Evangelist
3 years ago
arnovananrooij
Contributor
arnovananrooijContributor
asked in Data
ESRI ArcGIS portal writer doesn't'write all features

Hi there, I'm a experienced FME user but have no experience in writing to ArcGIS portal.I have a hosted feature layer in an ArcGIS portal with three layers. With FME i'am trying to update these layers with new data. I've connected to the portal and imported the feature type definitions from the portal. I've set the layer writer mode to insert and the Truncate first to yes. Tables seems to be truncated and in one layer I see some of the data (670 features instead of 6500). The other layers are empty. There are no errors or warnings in the FME log file. Anybody any idea of what might be wrong?

5
arnovananrooij
Contributor
3 years ago
Z
zhangjinzhou
asked in Data
Elasticsearch writer error

I am trying to write simple features with/without geometry to elasticsearch. I tried different params but always get this errorELASTICSEARCH writer: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Types cannot be provided in put mapping requests, unless the include_type_name parameter is set to true."}],"type":"illegal_argument_exception","reason":"Types cannot be provided in put mapping requests, unless the include_type_name parameter is set to true."},"status":400}Does anyone have a clue?

9
steveatsafe
Safer
3 years ago
C
candimk
asked in Data
Would like to read a DWG and report out (in text, or csv or whatever) a list of all of the layers in that DWG. Any ideas on what transformer to use?

I am creating a workflow to check incoming DWGs for compatibility. Ideally I would like to compare it to a template file and find any additional layers that have been added, plus report out a few other things. We want an easy report to accept or reject a DWG coming in from outside. TIA!

2
C
3 years ago
A
alicja
asked in Data
AutoCAD DWG/DXF dynamic block properties

Hi,I would like to read dynamic block properties (specifically action parameters fx an angle or a flip state). I have the newest 2021.1 beta version of FME Desktop. Do you know where those properties are stored or are they stored at all? I tried both exploded and not exploded block parameters in the reader but with no success.Thanks!

0
A
3 years ago
fneal
Contributor
fnealContributor
asked in Data
Error Message When Writing Lines to SDE: The error number from ArcObjects is: '-2147215922'. The error message from ArcObjects is: {Number of parts in shape is incorrect for its geometry type}

When writing line features onto a poly line SDE feature class I am receiving the following message: “The error number from ArcObjects is: '-2147215922'. The error message from ArcObjects is: {Number of parts in shape is incorrect for its geometry type}.I have not encountered this error leading up to this point. However, there have been a handful of jobs that have failed on this error. I have reviewed the line(s) in question and have found them to have these characteristics:Vertices >= 2Non-duplicated, stacked, or closely consecutive verticesLonger length than SDE toleranceSingle part line features, not aggregatesNot self-intersectingGeometryValidator, GeometryFilter, and GeometryRefiner have been applied with no success. Also, I created a replica file geodatabase and was able to write the features in with out any issue. I verified that it is specific lines and not only the first feature being written causing the error as well. In summary, the jobs are failing on what ap

12
gisbradokla
Supporter
3 years ago
nicholas
Contributor
nicholasContributor
asked in Data
Dynamic workbench to reproject MapInfo tables

I am building a workbench that uses Workspace Runner to reproject a lot of MapInfo tables. My question is about the workbench that does the reprojection. All the MapInfo tables have different schemas (different columns). How do I build dynamic readers and writers that work with any column structure (but still specifically MapInfo format)

1
daveatsafe
Safer
3 years ago
nicholas
Contributor
nicholasContributor
asked in Data
How to filter only native MapInfo tables

I am building a workbench that uses Directory and File Pathnames Reader and Workspace Runner to run a translation on a bunch of MapInfo tables.I want to only run the translation on native MapInfo tables. Not image tables, WMS table or links to oracle datasets.I can do this easily enough in MapBasic, where I can use the TableInfo() function to return if a MapInfo table is of type TAB_TYPE_BASE.Is there a similar transformer in FME? I have already looked at exposing attributes, but none of them are helpful

2
nicholas
Contributor
3 years ago
franco69
Contributor
franco69Contributor
asked in Data
Hi to all, i have to make a test to convert Shape Multipatch to IFC format.....i have already finished the geometry but i didnt fix the problem to transfer some individual attributes to the ifc.....

Also read the IFC examples here on the forum but i didnt come to a solution because here is no really example from SHP Multipatch to IFC

1
jovitaatsafe
Safer
3 years ago
F
fluusha
asked in Data
Esri ArcGIS Online (AGOL) Feature Service Writer Version 2020.2 Problem

I have a Workbench. It has a Esri ArcGIS Online (AGOL) Feature Service Writer. It works perfectly. Workbench was daily running successfully. Now yesterday we upgraded from Version 2019.1 or .2 to Version 2020.2 Now i get errors and the Writer cant find the content of the Group anymore.I dont see anything if i want to choose the Feature Service. Not even Groups. The Writer changed from the two FME Versions because in the new Version at Feature Service the ID gets attached behind. That was not in the old Version. If I use the Esri ArcGIS Online (AGOL) Feature Service Reader i see every Group and everything i should see. If my Collegue uses his Account he can see the Groups and everything in the Writer. I dont. Now the Question is it a FME Problem or ArcGIS Online Problem? With my ArcGIS Online Account and the Groups nothing changed only the FME Version changed and now i cant see any Feature Service and before everything Worked perfectly.

3
andreaatsafe
Safer
3 years ago
W
wmunsell
asked in Data
Issue updating AGOL data?

im trying to use a ChangeDetector and routing the Updated section to an AGOL writer with a writer mode of Update. however, i keep getting the following error and im not sure how to fix it. "ArcGIS Online Feature Service Writer: Encountered a feature for feature type 'LiftStationInspection_Weekly_table' that is missing the ID attribute 'OBJECTID'. This attribute is require"

1
W
3 years ago
mapster
mapster
asked in Data
How can I change a field that contains a data change to an ArcGIS Online compatible Datetime field?

When trying to change the data type in the ArcGIS Online writer, the esriFieldTypeDate(width) is not an option. I want to have this type of field because I want to enable time in my hosted feature service.

1
debbiatsafe
Safer
3 years ago
john.brady
john.brady
asked in Data
Exposing/Accessing AutoCAD System Variables

Hello, we've been working with a client for a couple months to run business rules against dwgs.One of the requirements is to ensure the System Variables BASE and INSBASE X, Y values are set to 0,0 to help ensure that xRef drawings align correctly with the 'base' or currently opened dwgWe have reviewed the following and don't believe what we are looking for is supported with this option and uncertain if what we are trying to unlock is even possible with current 2018.1 release(http://docs.safe.com/fme/2018.0/html/FME_Desktop_Documentation/FME_ReadersWriters/acad/Drawing-System-Variables.htm) Related AutoDesk Links:UCSBASE CommandUSCBASE VariableINSBASE VariableAutodesk Forum Discussion About Thanks for any insight

4
john.brady
3 years ago
A
arnaudfouch
asked in Data
How to get the rotation information of a cell from DGN ?

Hello, I'm trying to get rotation information of a cell which come from a DGN.I found it for text element but not on cell. In microstation the information is present but not in FME, do you have any idea ? In the help here : https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/dgnstyler.htm it say :OpenCell Rotation Specifies rotation of the cell (in degrees counterclockwise from horizontal). This can be set to any floating-point value between -360.0 to 360.0, or be set to a value taken from an attribute. Format attribute set: igds_cell_rotationBut I haven't this attribute in FME : But here in Microstation you can see the information (50 g = 45 deg)

6
A
3 years ago
B
billybuerger
asked in Data
Smallworld export unset values as null

It seems that the FME translator for Smallworld applies the "unset value" to any field where the value is unset. Is it possible to export a null value instead. We would like to preserve the values as is and not have to try to come up with some convoluted process to deal with it. Only thought I have would be to pick a default value for each field that we know doesn't exist in any of the existing data. We would then have to look for those values after exporting the data from smallworld so we could then again put them back to null. That could be a lot of work for all of the different classes we need to export.

4
B
3 years ago
jdh
Contributor
jdhContributor
asked in Data
Count elements in json array

I have a complicated json object that I would like to count the number of elements in one of it's child objects. Given the size and complexity of the original json, I would like to avoid flattening it and using the listElementCounter.

6
salvaleonrp
Enthusiast
3 years ago
L
left65
asked in Data
Hi, I am trying to output the results of an FME workbench to multiple files based on the value of a particular field but also add a specified value in the first row. I

I have worked out that I can use the Fanout feature on both csv and xlsx files, and I need the output as csv files, however outputting to a csv creates separate folders for each file and I need it output to the same file. The second part is that I want the output to start writing on row 2, with each file output to have the same value in cell A1.I know that if i output to excel i get individual files, named based on the attribute I specify, but all the files are in the same folder, which is halfway to what I want.What I cant work out is how to:insert a specified value in cell A1 on every file; andoutput as a csvIs this possible in FME or am i going to have to write a macro for excel that runs through each file, adds the value in cell A1 and then saves the file as a csv?

13
L
3 years ago
E
ekeighan
asked in Data
Hello, my name is Edric Keighan President & CEO at CubeWerx. We are looking at a project that requires the development of a client application using both 3D and 2D geospatial data using CesiumJS or Cesium Ion.

You can reach me at (613) 447-8306.

2
gav
Contributor
3 years ago
O
osdev
asked in Data
Can a build a workflow that will read a kml or kmz file and output a feature service layer to a specified folder in ArcGIS Online?

I'd also like to be able to analyze and set properties for the feature symbology.

2
hkingsbury
Celebrity
3 years ago
D
dkirouac
asked in Data
Can FME output curly brackets instead of square brackets in JSON output? Can I insert text between the first and second open brackets in the JSON output?

I am converting geodatabase data to JSON and using the HTTPCALLER to upload to an API. The JSON output from FME is adding square brackets[] as the first set of brackets but the API is looking for curly brackets{}. Is there a way for FME to output curly brackets instead of square brackets?The API is also looking for some additional text between the first and second open brackets. Can I insert additional text between the brackets in the JSON output?For example the FME JSON Writer is writing [{json text}] but the API is looking for {site:{json text}} Thanks, David

5
D
3 years ago
virtualcitymatt
Evangelist
virtualcitymattEvangelist
asked in Data
GLTF - Output is transparent when it shouldn't (really) be

When converting a 3D model to GLTF (2.0) which has png textures with an alpha band the output looks pretty weird in most viewers. The alpha value in the textures is at the Max (255) so it shouldn't be transparent, however, FME sets the alphamode to 'BLEND'. While I understand the logic in this I think in practice a better default for the alphamode would be 'MASK'. In the image below you can see that the inner cube is rendered even though the outer cube has a texture which has no transparent pixels. In my situation the issue can be solved by simply having jpegs as textures, however, this is often not possible to get.  In other situations, for example billboard trees, you want to have transparency, but in the form of a 'MASK' not a 'BLEND'. I would argue that in 90 % of cases the 'MASK' option would be better than 'BLEND'. I've attached the model for inspection.  Note that this seems to get rendered the same in all the renderers I&

3
virtualcitymatt
Evangelist
3 years ago
A
adamp
asked in Data
MITAB to DWG text sizing issue

I am having an issue with a specific MapInfo TAB file that when I put it through it's different attribute and geometry filters, it is outputting the wrong size text. The last transformer the file is going through is a DWGStyler where I have a specified height. The strange part that is confusing me with all of this is that the single line text is outputting the correct specified height I have in the DWGStyler, but any multiline text is being outputted as a combined text height. For example I have the height set to be 0.4. The single line text displays as 0.4, a two-line multiline text displays each line with a height of 0.2 thus adding up to the specified 0.4. This to is the same if it three lines of text where it displays as 0.13 for each line. I have tried toggleing the Text Entity Type from multi-Text to Text with the same outcome. Any suggestions would be great. I am more Autodesk than FME. I have AutoCAD Map3D 2014 and am aware that I can import these MapInfo files, but the dat

3
gpt_geoinfo
Contributor
3 years ago

Badge winners

Show all badges
29,747
Posts
Terms & ConditionsCookie settings