Is writing python becoming easier than just making a workspace now that we have AI? 

My (very limited) experience with AI thus far is it either doesn’t know the answer or just outright lies, so it doesn’t concern me at the moment.

One of the key principals for me in developing FME workspaces is they should be understandable and supportable. Sticking functionality in pythoncallers is unlikely to meet that criteria. I’d also question whether if you don’t really understand how to do something, whether you can verify that it’s actually doing what you want (You don’t know what you don’t know!)

An AI generated answer to something I googled yesterday suggested the FMEServerEmailGenerator to me which is Deprecated, completely missing that it has been superseded by the FMEFlowEmailGenerator

On the other hand, if AI can learn to perform an FME Flow Upgrade with a fault tolerant setup with https enabled and all the other customisations required then quite frankly I’d be happy to have that job off my hands!

Sympatico ​@virtualcitymatt 

I feel (and hope) this will be a very long thread and with lots of contributions.

1. I think Generative AI in code is great and can only get better. I think there’s been lots of work taking documentation for languages and training LLMs. That’s a big thing. An LLM that has been trained well with all the Python will be a great helper. Which lead me to the next point.
2. I haven’t gone much into it but I am sure it wouldn’t take long to train an AI on FME readers, writers, Transformers, the capabilities of each and connective logic. Take the documentation and load it up. If we can ask an AI “How do I do read the...from the….and do the...and put the results to...” and we can get a step by step transformer list that would be great. And not far off. I have tried that in a few generic AIs and gotten pretty close.
3. I get what you are saying with the AI to PythonCaller and call it a day but that, in my mind, is a one off “temporary personal solution” and in no way would (should) make it to as a multi-user Team Solution or a corporate deployment. The strength in FME is workflow documentation. If we hide 90% of the solution behind a python caller then we have lost that maintenance strength. In my experience the “free” solutions are usually “fast dash” solutions that are never maintained and are guaranteed to break on turn-over. And I say this to point out RISK with that method, not that there is no room for this type of solution.
4. Plus I love connectors. :D And Lizards. :D

Some thoughts for now. Looking forward to discussing more.

Also...looks like AI is now A1 Sauce. And I think that is fun. I will henceforth refer to AI as A1 and or simply The Sauce. :D

I’ve always said, and will continue to say, ‘Everything i’ve done in FME I could’ve done in Python’.

With the AI tools available the generation of that code has become easier. However I do believe in  choosing the best tool (in my tool-belt) for the job; FME, Esri, Python to name a few.

What this means is evaluating the benefits and downsides of each tool. Take FME vs Python for example (list not exhaustive!!):

FME Benefits:

• Visual
• Easy to document
• Scalable
• Easy to migrate between environments
• Can leverage Python with the process

Python Benefits:

• Free to use
• Wider userbase
• Can compile to standalone solutions
• Ability to easily generate solutions using AI

A very important point I'd like to make re. developing a solution myself rather than have AI generate it is that I want to be fully in charge of the process (control freak reporting for duty!).

I don't want to have a black box that works but that I can't fix if it doesn't work, if that makes sense. I don't want AI to make choices on my behalf. I want to make sure I know what's going on in the process and I want to be able to explain every step and every choice made in the development.

So yes, AI can be a very valuable tool but I would still want to be the one ultimately in charge of what's happening.

We also use AI for writing code. The generated code is mostly a quickstart and not the end product. No experience with Visual Studio’s Agent mode and the programming part I need is often minimal.

I do not feel obsolete because the part of FME I like is to organize the bigger picture of a workflow, not fixing the correct syntax of the code. Which is sort of re-inventing the wheel over and over again?

As said, the maintainability and documentation of workspaces is “easier” for non programmers, as most of my team are. I also like the way FME quickly returns results. A lot of the work we do is to analyze a new datasource, to see what it is, test it on quality etc. Before FME I did this using SQL, but it lacked the easy spatial component.

So I think it probably depends on the kind of products an FME user produces. If one can deliver small, Python only, standalone solutions, AI generated Python might be a better solution then a FME workspace. But if I had to choose between AI generating Python for me and FME Form / Flow, I would ditch AI generated Python without a doubt.

Well, I had a bit of an existential crisis over the weekend on all of this stuff. I’ve been following AI development fairly closely, and a few weeks ago, I had my first  success with leveraging AI to help me add a few features to one of my python based custom formats that I use. In that process I was very careful to see and understand what the AI was writing and would cherry pick the bits I wanted and needed. 

But when this agent mode dropped it was really quite a shock. Sure, previously I’d seen some capabilities of things like ChatGPT code interpreter etc etc where you could upload files and perform analysis. That was all well and good, however, it wasn’t exactly at the point where I would actually use it in my job. But now?

I don’t know - you don’t need to know anything really. You just go back and fourth with it until you get the result that you like. I asked for a converter from one 3D format to another with a UI so I could pick the input file and got there in about 2 mins. I then asked it to integrate a viewer, add texture support, add some other features to the application and we got there in a few hours.

Granted I would never ever think about delivering this to anyone, but it’s more that now anyone can do this - no longer does Steve from accounts need me to do some cool thing with his data etc etc, he can just have an AI do that now. With enough back and forth it seems like A LOT is just doable now and it’s only going to get better. I was even able to share screen shots with it when it had messed up the textures and it was able to fix the code. By the time I gave up it have written about 700+ lines of python. 

I really recommend that everyone just at least give a go with these agents when/if you get a chance. it’s worth a few hours even if it’s just to see the limitations.

I still think, as all of you so far have echoed, that my own understanding of data structures, my FME knowledge and my other experience still have a lot of value and FME is still my go-to for most tasks. But now I realise that this will change quite a lot in the next few years and I do not want to get left behind. 
 

To throw my 2c into the ring…

1. I’ve been using AI to help me deploy FME workspaces a lot. It’s been helping me to create JavaScript, HTML, and CSS, which are not my usual strong points. So it’s helping me to expand my FME use, rather than the other way around. 
2. I totally agree with the comments about understanding. Right now, I don’t think AI is smart enough to understand the nuances. If you don’t understand them either, and can’t specifically tell the AI to consider a particular situation, then the results aren’t going to be pretty. I can’t imagine “Steve from Accounts” understands the intricacies of 700 lines of Python code! Or any security implications (I know it generated some insecure blocks of JavaScript that I didn't recognize as problems at first).
3. I think the above counts double for spatial data. What is the AI-generated code going to do with zero-length lines? Or lines that double-back on themselves? Or polygons with a fishtail? Can I trust it to know about these?
4. But I realize that AI is getting better and that we at Safe will have to take all of these considerations on board to stay relevant. We know that people will use the best tools available to them, and we encourage you to do so. Our task is to make sure FME stays the best and easiest-to-use application in your toolbox!

As an aside, AI learns from content on the internet. It seems to me that there are gaps in its knowledge because the information isn’t out there and it cannot make deductions to fill in those gaps. Will it eventually learn to be more creative? Or will there be an AI-roadblock at some point? 

I have not used AI much myself but I have experienced coworkers using it. There are those that do not have the knowledge/background to perform certain duties and rely on AI in order to complete tasks, and then rely on AI again to answer technical questions about those tasks when asked. Sometimes they are successful other times not so much. I completely understand how you feel, we work so hard to gain a level of understanding only for someone else to seemingly output the same using AI. However I have hope that where our level of understanding will shine through is not how to do something but when/why to use a specific method and maybe more importantly when not to do something.

I was recently at the American Association of Geographers annual meeting that was hosted in Detroit and there were many talks that included AI. One talk was tailored to those seeking new jobs in the GIS realm, specifically juniors, and when AI came up they also discussed the implications of its use by those who do not know what they are doing but can now seemingly do the same job. The points made echoed what many of you have already said. That in the short run someone may be able to get the same result without any training or even knowing what they are doing, and maybe that is even enough for some employers. But for good companies and good managers they will learn the difference between those that actually understand the subject at hand, they will start looking for those that understand multiple methods and which is best to use for a given task.

Since AI learns from what is out there on the internet we could all start publishing incorrect solutions to sabotage it and ensure it never catches up. /s

Interesting conversation!

The reason I, and my organization, use FME is to help us build more complex workflows than we could create otherwise in Python. We’ve hit a ceiling with python usage with regard to our skillsets, and we find that we can make much more nuanced and complex scripts with FME. Fortunately, our scripts and analysis in FME are so complex that AI cannot make a python equivalent. 

I am in the place were I think AI can potentially get rid of the things I don’t really want to do and maybe they can also help teach me things quicker than I could pick them up myself. But that might give me more time to take up the challenges/problems that AI can’t do yet. And I do mean “yet”. I will just advance in the things that it can’t do well at the moment.

But like a couple of others mentioned, we often need to at least know exactly what it is doing so we can be sure it meets requirement and that we can update it in future. Sometimes when I do see some AI produced outcome in something I am experienced in, I note that what it has produced will work but may not be the most efficient or cater for differing input.

Plus, I am probably better at doing, than explaining what I want done. It’s going to have to get it to read my mind!  

This, I think has hit the nail on the head a bit when it comes to AI limitation - complexity.

After experimenting more and more I’ve found the limit/roof of the AI capabilities for now it seems. What I’ve found in my testing is that it tends to get stuck at some point and just breaks and fixes the same thing over and over again. I got to the point where just complaining to the AI wasn’t getting me anywhere. 

I had to go in, take a look at the code and make some very specific suggestions to keep moving. as you pointed out ​@mark2atsafe - Steve from accounts can’t do that - I think it wont be long though that Steve from accounts wont need to though.

I’m going to keep experimenting to see where AI can and can’t do my job.

Note, one thing which it continued failing with was trying to install/set up GDAL on Windows….Haha    

According to ChatGPT I am an expert in ArcGIS. I don't know where it found that but I am most definitely not 😄

Here’s another thing I tried today which I thought was worth sharing:

 

“In this folder there is a whole bunch of folders which are named 1km_YYYY_XXX where YYYY and XXX reapresent the lover left cornder grid of the 25832. The folder do not have the 000.

What I want is a shapefile of the grid”

The agent just wrote the code and presented me with a button to run the script it wrote. Did it first pop. 

This is a task which would only have taken me 5 or 10 mins in FME - and asking the AI probably took the same amount of time haha, but still, for others who may have struggled a bit with this task in FME just don’t really need to anymore.