Integration/Interaction FME & Microsoft Copilot

Hi @kirc 

I apologize that no one has reached out about your question yet. I don’t believe we have anything for Copilot but we do have an AzureComputerVisionConnector package on the FME Hub.

If the service you are looking for is not supported by those transformers you can re-use/modify the Web Service that comes with the package and access other Azure based services using the REST or Python API, if available. 

Crystal

Thanks for the info @crystalatsafe, I was also thinking about going via Azure to access the APIs provided by Copilot there. I will also check out the AzureComputerVisionConnector. In case some news/use cases etc. come up along the way you are welcome to let me know.

Thanks,

Christoph

Any idea how to retrieve the URL for the APIs provided by Copilot ?

Hi @philippeb 

You could look into what underlying model Microsoft uses for Copilot and use that in the OpenAIChatGPTConnector.

These links may be helpful:

Tutorial: Getting Started with OpenAI in FME

Unleashing the Power of OpenAI GPT-3 in FME Data Integration Workflows

Alternatively, you could also deploy and fine-tune your own model to try to mimic what Copilot does.

If Copilot Studio allows you to see the required endpoints, you would likely need to create a new custom transformer.  

Let me know if you have any more questions. 

Crystal

Hi @crystalatsafe 

Thanks for the hints!

Unfortunately my employer blocks systemically everything about OpenAI / ChatGPT.

They are forcing us to use only Copilot Pro.

I am about to dig more with the IT where and how I can use Copilot API.

Have a good day!

​@philippeb We are also using MS Copilot now as our exclusive gen AI tool, definitely curious about transformers or if you are able to get this to work.

We are trying to have Copilot auto-classify and summarize text-based requests we get into various categories. I know FME has the Graph integration with Teams and Microsoft does have Teams + Power Automate + Copilot capabilities. Could be a stretch but perhaps getting messages into Teams from FME, then using PA to pickup that message and feed it to Copilot could work, then PA sends message back to FME via webhook. Or have FME communicate directly with Power Automate which then runs Copilot and responds back. Interested in this space.

​@langdonms ​@philippeb ​@kirc Thanks for kick-starting this discussion on Microsoft Copilot. Just to clarify the ask for us, would the expectation here be to have FME somehow ingest the Microsoft Copilot interface, as something of a Bring Your Own assistant?

As a small side note, organizations with a Microsoft enterprise subscription often have access to the Azure OpenAI service as part of their Azure deployment. If your organization has this subscription, they could consider deploying an org Azure OpenAI service of their own, and then allowing FME users to tap into this service via our OpenAI transformers. By adjusting the Service Provider parameter of these transformers over to Azure OpenAI, and providing your Azure OpenAI API key and service base URL, you can leverage the same AI service that powers Microsoft Copilot, just using a different access method than the Copilot interface.

Hi Natalie,

Thanks for the follow-up. I’m not sure if we have Azure OpenAI access, that’s good point that I will need to explore. That will work well if we have access to that using the OpenAI transformers.

I do know that we purchased Copilot licenses specifically for a large number of employees and it shows up now in native apps like Word, Excel, Outlook, for summarizing emails, meetings etc. It works well at those things. It also works well on one-off requests like “derive categories from this table of 311 requests.” 

I’ve experimented with training our own naive Bayesian models using the FME trainer and our sample data, it would be nice if we could tap into those commercial LLMs that are going to be more robust. An example we would want to know if a 311 request of free response text is about a deceased deer or other animals (as these have different responses and we report them differently). The pipeline would be 311 request comes in → webhook with message to FME Flow Hosted → Workbench submits text message to a LLM model to classify the text → updates request category on work order system (Cityworks)

Thanks!

-Langdon Sanders

Sorry I didn’t answer your question directly, no I would not need the Copilot interface like an assistant, this would be more for automatic classifications running on events, in my use case. Perhaps Copilot isn’t built for that though and OpenAI or other solution is a better fit.

For assistant interfaces I would direct people to use the native Microsoft platforms. 

​@lsanders_dublin Ah ok, thanks for clarifying. In that case, if your org does happen to have a Microsoft enterprise / Azure subscription, check out the Microsoft article, Create and deploy an Azure OpenAI Service resource, for details on how to get going with an Azure OpenAI service. If you do give these instructions a try, you might see a red box with an X in its upper left corner at Step 3. If so, this would indicate that your subscription does not include Azure OpenAI access, but there should be a link in that red box which would take you to a form where you / your IT team can request access to Azure OpenAI.

Great, thanks so much for the tips!

-Langdon

Hi ​@natalieatsafe ,

Also I would like to thank you for the follow-up! For me your answer regarding the “access to the Azure OpenAI service as part of their Azure deployment and then allow FME users to tap into this service via our OpenAI transformers” perfectly answered the question that I had.

Thanks and cheers,

Christoph