Question

I'm trying to use PythonCreator to process and Analyze some data, and my results are in a Pandas DataFrame. There's any way to convert that table to features to use in FME? Thanks for your help

  • 17 February 2021
  • 6 replies
  • 58 views

I'm trying to use PythonCreator to process and Analyze some data, and my results are in a Pandas DataFrame. There's any way to convert that table to features to use in FME? Thanks for your help

6 replies

Userlevel 4

If each record in the DataFrame corresponds to an FME feature, then it should be a simple matter of iterating over the DataFrame, create an instance of FMEFeature and set the attributes (and perhaps even the geometry) before outputting the feature. For example:

# Assuming df is your DataFrame object
for index, row in df.iterrows():
    f = fmeobjects.FMEFeature()
    f.setAttribute('my_attribute_name1', row['attribute1'])
    f.setAttribute('my_attribute_name2', row['attribute2'])
    self.pyoutput(f)

 

If each record in the DataFrame corresponds to an FME feature, then it should be a simple matter of iterating over the DataFrame, create an instance of FMEFeature and set the attributes (and perhaps even the geometry) before outputting the feature. For example:

# Assuming df is your DataFrame object
for index, row in df.iterrows():
    f = fmeobjects.FMEFeature()
    f.setAttribute('my_attribute_name1', row['attribute1'])
    f.setAttribute('my_attribute_name2', row['attribute2'])
    self.pyoutput(f)

 

Thanks for your reply.

I've iterated over the dataframe in every way possible, and I've tried yours, and still have the same error: "TypeError: Could not convert attribute value to a supported attribute type", all my records from the DF are floats. Any Idea what I do wrong?

Badge +22

Thanks for your reply.

I've iterated over the dataframe in every way possible, and I've tried yours, and still have the same error: "TypeError: Could not convert attribute value to a supported attribute type", all my records from the DF are floats. Any Idea what I do wrong?

You can try casting the attribute value to string to see if that works.

I normally encounter that error when the data value is a list.

Thanks for your reply.

I've iterated over the dataframe in every way possible, and I've tried yours, and still have the same error: "TypeError: Could not convert attribute value to a supported attribute type", all my records from the DF are floats. Any Idea what I do wrong?

Yes, that worked.

Thank you all for your replies!

Userlevel 4

Thanks for your reply.

I've iterated over the dataframe in every way possible, and I've tried yours, and still have the same error: "TypeError: Could not convert attribute value to a supported attribute type", all my records from the DF are floats. Any Idea what I do wrong?

Are you sure it is a standard float and not e.g. numpy float32? The fmeobjects API only knows how to work with the basic Python datatypes such as str/unicode, int and float (and Python lists if you specify an FME list attribute).

You could try to simply cast the value to a regular float, I'm suspecting it would work.

Tip: you can find the exact type of any object by doing

print(type(my_object))

Example:

>>> npfloat = numpy.float32(123.0)
>>> print(type(npfloat))
<class 'numpy.float32'>
 
>>> stdfloat = 123.0
>>> print(type(stdfloat))
<class 'float'>

See also https://note.nkmk.me/en/python-type-isinstance/

Thanks for your reply.

I've iterated over the dataframe in every way possible, and I've tried yours, and still have the same error: "TypeError: Could not convert attribute value to a supported attribute type", all my records from the DF are floats. Any Idea what I do wrong?

You are right, it's was numpy.float32

Converting it to standard float worked as well.

Thanks

Reply