I'm trying to migrate some python into FME that involves DataFrames as the end result. Rather than trying to manually handle converting the DataFrame to attributes, I thought I could convert the dataframe to a csv in python memory first, output it as a file on the feature, and then use a CSV reader in the workspace to parse the file. I ran into issues with getting the csv out to an attribute, so if anyone knows of a way to do this I'd appreciate it.
Below is a simplified version of my code (it involves Esri's ArcGIS API for Python), and then I'm using io.StringIO to handle the csv in memory, but I've never used that function before.
When I run the workspace with this as a Python Creator, i get the error:
Python Exception <TypeError>: Could not convert attribute value to a supported attribute type.
Traceback (most recent call last):
File "<string>", line 33, in input
import fme
import fmeobjects
import csv
import io
from arcgis.gis import GIS
import pandas as pd
gis = GIS('https://arcgis.com')
class FeatureProcessor(object):
"""Template Class Interface:
When using this class, make sure its name is set as the value of the 'Class
to Process Features' transformer parameter.
"""
def __init__(self):
"""Base constructor for class members."""
pass
def input(self, feature):
#Search, return data and convert to dataframe
content = gis.content.search(query="transportation",max_items=10)
df = pd.DataFrame(content)
#Convert dataframe to CSV "In python memory"
s_buf = io.StringIO()
df.to_csv(s_buf)
s_buf.seek(0)
#Attach the csv in memory to the FME feature as a new attribute
feature.setAttribute('users_csv_file',s_buf)
self.pyoutput(feature)
def close(self):
"""This method is called once all the FME Features have been processed
from input().
"""
pass
def process_group(self):
"""When 'Group By' attribute(s) are specified, this method is called
once all the FME Features in a current group have been sent to input().
FME Features sent to input() should generally be cached for group-by
processing in this method when knowledge of all Features is required.
The resulting Feature(s) from the group-by processing should be emitted
through self.pyoutput().
FME will continue calling input() a number of times followed
by process_group() for each 'Group By' attribute, so this
implementation should reset any class members for the next group.
"""
pass
