A colleague wants to read files that have no line ending (either Windows-style CR+LF or Unix-style LF). They have fixed record lengths (100 bytes). Can FME read this type of file (I'm betting it can!)? It needs to process them in 100 byte chunks.
If FME can't do it directly, is there something on the Hub e.g. this?
There may be other solutions, but it's fairly easy to accomplish using a PythonCaller, something like:
class FeatureCreator(object):
def __init__(self):
pass
def input(self,feature):
f = open("input_file.txt", "r")
chunk = 100
while True:
data = f.read(chunk)
if not data:
break
newFeature = fmeobjects.FMEFeature()
newFeature.setAttribute('chunk' data)
self.pyoutput(newFeature)
def close(self):
passWill return one feature with an attribute chunk for each chunk of 100 bytes in the file input_file.txt
i would attempt this with first getting the file into an attribute with AttributeFileReader then making a list from the record layout with AttributeSplitter.
There may be other solutions, but it's fairly easy to accomplish using a PythonCaller, something like:
class FeatureCreator(object):
def __init__(self):
pass
def input(self,feature):
f = open("input_file.txt", "r")
chunk = 100
while True:
data = f.read(chunk)
if not data:
break
newFeature = fmeobjects.FMEFeature()
newFeature.setAttribute('chunk' data)
self.pyoutput(newFeature)
def close(self):
passWill return one feature with an attribute chunk for each chunk of 100 bytes in the file input_file.txt
Here i was avoiding mentioning Python and David has no qualms! The struct module is a candidate too ;-)
Here i was avoiding mentioning Python and David has no qualms! The struct module is a candidate too ;-)
It's late Friday, my qualms have officially left the building ;-)
i would attempt this with first getting the file into an attribute with AttributeFileReader then making a list from the record layout with AttributeSplitter.
This is a good solution, given that the input file isn't very (very) large.
Thanks both for your suggestions. This was partly exploratory to see if FME could be used for this particular job. I'll try make sure I come back and post which solution was used and accept the relevant answer.