Skip to main content
Question

Parsing text file with offset field names

  • May 22, 2019
  • 4 replies
  • 48 views

Hello,

Trying to setup a process to parse and load IIS log files to a database to monitor errors on a Geocortex site.

I have a workspace that returns the file path of the most recent log file and that is passed into a FeatureReader with the CSV reader.

The data is space delimited. The issue is that the field name header has one extra field (X). So when the text file is parsed, the field names are offset one column to the right.

 

Header: X |1|2|3|4|5

Data: 1|2|3|4|5|

I need to remove X in the Field name header, and shift the header one cell to the left.

Ideally this could be done in the CSV reader, but I don't see how.

Thanks for the help. Much appreciated.

-R

 

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

4 replies

davideagle
Contributor
Forum|alt.badge.img+22
  • Contributor
  • May 22, 2019

Is it too simplistic to suggest you skip row 1 all together and then just have an AttributeManager straight after to map col0 to appropriate value, col1 etc. Hardcoded admittedly but gets the job done.


  • Author
  • May 22, 2019

I think that may be the only solution. But I was hoping it wasn't :).


jdh
Contributor
Forum|alt.badge.img+40
  • Contributor
  • May 22, 2019

You could do it dynamically with a pythonCreator

import fmeobjects

class FeatureCreator(object):
    def input(self,feature):
        f = open(FME_MacroValues['SourceDataset_TEXTLINE'],'r')
        header = f.readline()
        attributes = header.split()[1:]  #drop first element
        ct = len(attributes)
        for line in f:
            values = line.split()
            newFeature = fmeobjects.FMEFeature()
            for i in range(ct):
                newFeature.setAttribute(attributes[i],values[i])
            self.pyoutput(newFeature)

You would still need to expose the attributes manually to use them on the canvas


fmelizard
Safer
Forum|alt.badge.img+21
  • Safer
  • May 22, 2019

Hi @ogc_ryan I was able to remove the X field from the header when adding a new CSV reader. When you edit the CSV parameters, first change the Attribute Definition to Manual. Un-check X. Under Fields, set "Field Names Line" to "None". Then reset "Data Start Line" to 2.