[Megathread] AI Assist in FME Form 2023.1 - Feedback and Discussions

Using the PythonCaller AI Assist in 2023.1

If I ask:

Use arcpy to find all featureclasses and their featurecounts in a file geodatabase.

It returns:

import arcpy
 
# Set the workspace
arcpy.env.workspace = 'C:/data/mygdb.gdb'
 
# Get a list of all feature classes in the geodatabase
feature_classes = arcpy.ListFeatureClasses()
 
# Loop through each feature class and print its name and feature count
for fc in feature_classes:
    count = arcpy.GetCount_management(fc)
    print(f'{fc} has {count} features')

This works, but I need the result on the feature, out of the PythonCaller. So I changed my question to:

Use arcpy to find all featureclasses and their featurecounts in a file geodatabase. Return the result as a list added to the feature.

Now I get:

Invalid AI response.

How we solved this is:

import fme
import fmeobjects
import arcpy
 
class FeatureProcessor(object):
 
    def __init__(self):
        pass
 
    def input(self, feature):
        arcpy.env.workspace = feature.getAttribute('path_windows')
 
        feature_classes = arcpy.ListFeatureClasses()
        record_counts = []
        count = 0
 
        for fc in feature_classes:
            featurecount = arcpy.GetCount_management(fc)[0]
            feature.setAttribute("record_counts" + "{" + str(count) + "}.FeatureClass",fc)
            feature.setAttribute("record_counts" + "{" + str(count) + "}.FeatureCount",featurecount)
            count = count + 1
 
        self.pyoutput(feature)
    def close(self):
        pass
 
    def process_group(self):
        pass

It works, but it feels hacky... It would be great if the AI assistant could generate a complete solution.

Hi all,

This is great feedback, keep it coming!

@nielsgerrits​ @marekszczepan1​ it sounds like you've both had similar experiences with the AI Assist in our PythonCaller, where the AI seems to work well with the more simple, generic requests but can struggle at times with those complex, more specific asks. I'll certainly bring your feedback to our internal teams working on this functionality to see if we can dig into and improve the AI-generated Python.

@marekszczepan1​ Interesting suggestion on the AI integration with our attribute-handling transformers! Would you be able to expand on that idea a bit? We are definitely interested in knowing your thoughts on how we can integrate AI into other areas of the FME Platform!

Is there any way to disable ai assist? I don't want any users accidentally misusing it until such time as we can point it at our own in-house LLM.

@mgg_beca​ That's a great question about disabling the AI Assist.

The service does have it's own unique URL: https://fme-ai-service.safe.com.

If you need to remove access to the FME AI Assist service, the best way to do so would be to add the above URL to a firewall's Do Not Allow list. If there's an alternative form of URL filtering in place, our service URL could also be included on the blocked list of this filter.

Essentially, the AI service requires internet access to function. So long as the customer has some way of restricting internet traffic in place, our service could simply be added to this restriction in the same way that our other in-product URLs can be restricted or whitelisted.

Please let us know if the above doesn't quite address your concerns.

yes, exactly

@natalieatsafe​  I was just told by a client that if the AI URL is blocked, it will lead to an error message shown to the user at startup of Form:

Failed to connect to 'fme-ai-service.safe.com'.

Is it possible to avoid this error message?

I'm not sure about the client, it's a government entity so it could be blocked a number of places and it's going to be complicated to track down. I tried blocking fme-ai-service.safe.com in my router and it worked as expected, i.e. I got no error message and everything except AI Assist worked just fine. But what would happen if calls to fme-ai-service.safe.com gave an unexpected reply, e.g. if a proxy or gateway returned some html page saying that access had been blocked?

@david_r​ If you are still able to pursue more information from the client on how the URL was blocked, as well as a bit on their network config, perhaps a screenshot of the error, I would like to pass these details over to our developers so they can try and reproduce the error and look to correct it. The more info the client can provide, the better! If you'd prefer to provide this info outside of a public forum, a support case would be a much better option for that.

@mgg_beca​ That does indeed look like what we'd expect to see upon blacklisting the FME AI Service URL. That's good news here. Thanks for sharing!

@david_r​ Hi David, I hope I find you well today. I just wanted to provide a little update on the error that you brought forth for your client, when disabling the AI Assist URL.

Our development team has had an opportunity to investigate this one, and unfortunately were not able to reproduce it. It really sounds like the issue is very specific to your client's internal environment. As other users, including yourself, have successfully blocked the AI service URL and not encountered the error, this would likely be an error that is best remedied by the client's own IT team and an exploration of where exactly the AI service URL is being blocked.

Our team has, however, made an adjustment to the error message seen when the AI Service URL is blocked, just to be a little more informative. This adjust will be available in a very near-future release of FME Form 2023.1, and can be tracked on FMEFORM-29415.

If yourself or the client have any lingering questions on this one, please don't hesitate to post those! I'm happy to assist where I can.

@natalieatsafe​ For information, I was able to reproduce the behavior on my laptop when connected through a VPN that blocks all internet access. This message popped up twice in the translation log while running a workspace:

Using build 23619. There were no AI-enabled dialogs open at the time of the message, and no AI-enabled transformers in the workspace.

Again, this is only cosmetic, but could probably confuse inexperienced users into thinking something more serious going wrong.

@natalieatsafe​ It seems the issue has been fixed in build 23631 :-) Thank you!