Skip to main content
Solved

RabbitMQ JMS Client configuration


alex_p
Forum|alt.badge.img+1

Hello FME users,

I am trying to configure the RabbitMQ JMS client for FME Server, without success. I copied the client files and dependencies directly in "FMEServer\\Server\\lib\\jms". Next, I configured a JMS subscriber to send message to my RabbibMQ Server :

- Protocol : JMS

- Provider Type Or Context : com.rabbitmq.jms.admin.RMQObjectFactory

- Provider URL : server-name:5672

- Connection Factory : com.rabbitmq.jms.admin.RMQConnectionFactory

Username and password : OK

Destination: a queue name

When I confirm my configuration the following error message appears : "Unable to find provider named com.rabbitmq.jms.admin.RMQObjectFactory. Ensure that the appropriate libraries are available and the correct provider is selected."

What I am missing?

Any help will be really appreciated

Thank you

Alex

Best answer by johnnygalambos

Nice work, Alex! Your answer would have saved me a bunch of time--I must have missed it by a day or two. Just some additional details, I was able to get both the JMSSender and JMSReceiver working in a local Workspace as well as get FME Server Pub/Sub working. I've posted the .bindings file that works for us. 

For the local workspace, you need to make sure that the Jar containing com.sun.jndi.fscontext.RefFSContextFactory is somewhere in your class path.. I used fscontext-4.5-b25.jar from Maven here: https://mvnrepository.com/artifact/com.sun.messaging.mq/fscontext/4.5-b25. As a quick hack, I put this Jar as well as the RabbitMQ JMS jars into my default FME Java Plugins directory: C:\Users\<USERNAME>\Documents\FME\Plugins\Java. 

 

Configuring log4j on top of the slf4j was useful for debugging--the RabbitMQ client code does output some decent traces. I put a screenshot of all the jars I used to get this working below.

Bindings File  (in c:\RabbitMQBindings\.bindings for our config)

ConnectionFactory/ClassName=javax.jms.ConnectionFactory
ConnectionFactory/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory

#We are using a virtual host
ConnectionFactory/RefAddr/0/Content=fmetest
ConnectionFactory/RefAddr/0/Type=virtualHost
ConnectionFactory/RefAddr/0/Encoding=String
ConnectionFactory/RefAddr/1/Content=127.0.0.1
ConnectionFactory/RefAddr/1/Type=host
ConnectionFactory/RefAddr/1/Encoding=String


# JMS Queue where we have a RabbitMQ queue called "JMS2" boundto an exchange called "jms.durable.queues:.
# I named this very confusinginly, but FME will connect to this using the "JMS"
# destination. See screenshot below.
JMS/ClassName=javax.jms.Queue
JMS/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory
JMS/RefAddr/0/Content=jms/Queue
JMS/RefAddr/0/Type=name
JMS/RefAddr/0/Encoding=String
JMS/RefAddr/1/Content=true
JMS/RefAddr/1/Type=amqp
JMS/RefAddr/1/Encoding=String
JMS/RefAddr/2/Content=JMS
JMS/RefAddr/2/Type=amqpRoutingKey
JMS/RefAddr/2/Encoding=String
JMS/RefAddr/3/Content=com.rabbitmq.jms.admin.RMQDestination
JMS/RefAddr/3/Type=type
JMS/RefAddr/3/Encoding=String
JMS/RefAddr/4/Content=jms.durable.queues
JMS/RefAddr/4/Type=amqpExchangeName
JMS/RefAddr/4/Encoding=String
JMS/RefAddr/5/Content=JMS2
JMS/RefAddr/5/Type=destinationName
JMS/RefAddr/5/Encoding=String
JMS/RefAddr/6/Content=JMS2
JMS/RefAddr/6/Type=amqpQueueName
JMS/RefAddr/6/Encoding=String

FME Workspace Configuration

0684Q00000ArJxAQAV.png

0684Q00000ArJqtQAF.png

0684Q00000ArJnnQAF.png

View original
Did this help you find an answer to your question?

9 replies

Forum|alt.badge.img+2

@alexpillonel FME does not yet support Rabbit AMQP and therefore doesn't support RabbitMQ. I have added your post to the product change request we have to support RabbitMQ. Perhaps consider posting your request to the Ideas page to see if there are other users interested in RabbitMQ support


alex_p
Forum|alt.badge.img+1
  • Author
  • November 28, 2016

I finally
had some success with my JMS RabbitMQ configuration on FME Server. Here the
results of my research :

RabbitMQ

For
information, I send message to the RabbitMQ server via an Exchange (ex-fme-flight-info) binded to a queue (qu-cep-flight-info) 

Configuration
on the server where FME Server is installed:

  1. I copied
         rabbitmq-jms client and its dependencies in the FME Server folder :
         "../Server/lib/jms"
  2. I configured a JNDI bindings
         file with the RabbitMQ configuration : C:\jndi\.bindings

Parts of the .binding file with the important configuration (where I
broke my head ;-)) :

#Factory parameters
ConnectionFactory/ClassName=javax.jms.QueueConnectionFactory
ConnectionFactory/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory
…
…
#Destination queue
SendFlightQueue/ClassName=javax.jms.Queue
SendFlightQueue/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory
…
…
SendFlightQueue/RefAddr/2/Content=ex-fme-flight-info
SendFlightQueue/RefAddr/2/Type=amqpExchangeName
SendFlightQueue/RefAddr/2/Encoding=String
…
… 
FME Server
  • I created a
         "Topic" : IODA_OUT_FlightCurrent
  • I created a
         "Subscription" with the protocol JMS and defined the
         "Destination" name with the name defined in the jndi binding
         files :


ProtocolJMSProvider Typecom.sun.jndi.fscontext.RefFSContextFactoryProvider URLfile:/c:/jndi/Connection FactoryConnectionFactoryDestinationSendFlightQueue
  • I created a FME Workspace
         with a "FMEServerNotifier" transformer configured to send data
         to the Topic previously created :

Unconverted
Try

I tried to send data
with the "JMSSender" transformer, but without success. FME engine
return the following error :

javax.naming.NoInitialContextException: Cannot instantiate class: com.sun.jndi.fscontext.RefFSContextFactory [Root exception is java.lang.ClassNotFoundException: com.sun.jndi.fscontext.RefFSContextFactory]
Failed to connect to JMS provider with url "file:/c:/jndi/" and initial context "com.sun.jndi.fscontext.RefFSContextFactory"

 I hope it will help


mark2atsafe
Safer
Forum|alt.badge.img+44
  • Safer
  • November 28, 2016
alex_p wrote:

I finally
had some success with my JMS RabbitMQ configuration on FME Server. Here the
results of my research :

RabbitMQ

For
information, I send message to the RabbitMQ server via an Exchange (ex-fme-flight-info) binded to a queue (qu-cep-flight-info) 

Configuration
on the server where FME Server is installed:

  1. I copied
         rabbitmq-jms client and its dependencies in the FME Server folder :
         "../Server/lib/jms"
  2. I configured a JNDI bindings
         file with the RabbitMQ configuration : C:\jndi\.bindings

Parts of the .binding file with the important configuration (where I
broke my head ;-)) :

#Factory parameters
ConnectionFactory/ClassName=javax.jms.QueueConnectionFactory
ConnectionFactory/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory
…
…
#Destination queue
SendFlightQueue/ClassName=javax.jms.Queue
SendFlightQueue/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory
…
…
SendFlightQueue/RefAddr/2/Content=ex-fme-flight-info
SendFlightQueue/RefAddr/2/Type=amqpExchangeName
SendFlightQueue/RefAddr/2/Encoding=String
…
… 
FME Server
  • I created a
         "Topic" : IODA_OUT_FlightCurrent
  • I created a
         "Subscription" with the protocol JMS and defined the
         "Destination" name with the name defined in the jndi binding
         files :


ProtocolJMSProvider Typecom.sun.jndi.fscontext.RefFSContextFactoryProvider URLfile:/c:/jndi/Connection FactoryConnectionFactoryDestinationSendFlightQueue
  • I created a FME Workspace
         with a "FMEServerNotifier" transformer configured to send data
         to the Topic previously created :

Unconverted
Try

I tried to send data
with the "JMSSender" transformer, but without success. FME engine
return the following error :

javax.naming.NoInitialContextException: Cannot instantiate class: com.sun.jndi.fscontext.RefFSContextFactory [Root exception is java.lang.ClassNotFoundException: com.sun.jndi.fscontext.RefFSContextFactory]
Failed to connect to JMS provider with url "file:/c:/jndi/" and initial context "com.sun.jndi.fscontext.RefFSContextFactory"

 I hope it will help

Thanks for posting this information. Yes, I hope by having it here some other users will benefit from the knowledge.

 

 


johnnygalambos
Contributor
Forum|alt.badge.img+2
  • Contributor
  • Best Answer
  • December 9, 2016

Nice work, Alex! Your answer would have saved me a bunch of time--I must have missed it by a day or two. Just some additional details, I was able to get both the JMSSender and JMSReceiver working in a local Workspace as well as get FME Server Pub/Sub working. I've posted the .bindings file that works for us. 

For the local workspace, you need to make sure that the Jar containing com.sun.jndi.fscontext.RefFSContextFactory is somewhere in your class path.. I used fscontext-4.5-b25.jar from Maven here: https://mvnrepository.com/artifact/com.sun.messaging.mq/fscontext/4.5-b25. As a quick hack, I put this Jar as well as the RabbitMQ JMS jars into my default FME Java Plugins directory: C:\Users\<USERNAME>\Documents\FME\Plugins\Java. 

 

Configuring log4j on top of the slf4j was useful for debugging--the RabbitMQ client code does output some decent traces. I put a screenshot of all the jars I used to get this working below.

Bindings File  (in c:\RabbitMQBindings\.bindings for our config)

ConnectionFactory/ClassName=javax.jms.ConnectionFactory
ConnectionFactory/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory

#We are using a virtual host
ConnectionFactory/RefAddr/0/Content=fmetest
ConnectionFactory/RefAddr/0/Type=virtualHost
ConnectionFactory/RefAddr/0/Encoding=String
ConnectionFactory/RefAddr/1/Content=127.0.0.1
ConnectionFactory/RefAddr/1/Type=host
ConnectionFactory/RefAddr/1/Encoding=String


# JMS Queue where we have a RabbitMQ queue called "JMS2" boundto an exchange called "jms.durable.queues:.
# I named this very confusinginly, but FME will connect to this using the "JMS"
# destination. See screenshot below.
JMS/ClassName=javax.jms.Queue
JMS/FactoryName=com.rabbitmq.jms.admin.RMQObjectFactory
JMS/RefAddr/0/Content=jms/Queue
JMS/RefAddr/0/Type=name
JMS/RefAddr/0/Encoding=String
JMS/RefAddr/1/Content=true
JMS/RefAddr/1/Type=amqp
JMS/RefAddr/1/Encoding=String
JMS/RefAddr/2/Content=JMS
JMS/RefAddr/2/Type=amqpRoutingKey
JMS/RefAddr/2/Encoding=String
JMS/RefAddr/3/Content=com.rabbitmq.jms.admin.RMQDestination
JMS/RefAddr/3/Type=type
JMS/RefAddr/3/Encoding=String
JMS/RefAddr/4/Content=jms.durable.queues
JMS/RefAddr/4/Type=amqpExchangeName
JMS/RefAddr/4/Encoding=String
JMS/RefAddr/5/Content=JMS2
JMS/RefAddr/5/Type=destinationName
JMS/RefAddr/5/Encoding=String
JMS/RefAddr/6/Content=JMS2
JMS/RefAddr/6/Type=amqpQueueName
JMS/RefAddr/6/Encoding=String

FME Workspace Configuration

0684Q00000ArJxAQAV.png

0684Q00000ArJqtQAF.png

0684Q00000ArJnnQAF.png


peter_s
Contributor
Forum|alt.badge.img+1
  • Contributor
  • August 2, 2017

Thanks for posting this!

I tried it out and had some success.

I found that to get the JMSReceiver-transformer to actually present a jms_content  I had to set the message header "JMSType" to  "TextMessage"

Example using python (pika) to post message with the header:

import pika
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
msg_properties = pika.spec.BasicProperties(headers={'JMSType''TextMessage'})
channel.basic_publish(exchange='jms.durable.queues',
                        routing_key='JMS',
                        properties=msg_properties,
                        body='Hello World!')

I could not find a way to set this header using the JMSSender-transformer though so I don't know how to "eat my own dogfood" yet.

/Peter


steveatsafe
Safer
Forum|alt.badge.img+12
  • Safer
  • November 6, 2019

For the record, I was able to configure JMSSender and JMSReceiver to work in FME Desktop 2019.2 recently.

However, somewhere in the FME Server 2018.1 cycle the support for RabbitMQ and the JMSPublisher stopped working. We are hoping to resolve this in a later release of FME Server. Stay tuned.

Let us know - via an idea - if you are using or looking to use FME with RabbitMQ. Thanks.

 


steveatsafe
Safer
Forum|alt.badge.img+12
  • Safer
  • November 26, 2019
steveatsafe wrote:

For the record, I was able to configure JMSSender and JMSReceiver to work in FME Desktop 2019.2 recently.

However, somewhere in the FME Server 2018.1 cycle the support for RabbitMQ and the JMSPublisher stopped working. We are hoping to resolve this in a later release of FME Server. Stay tuned.

Let us know - via an idea - if you are using or looking to use FME with RabbitMQ. Thanks.

 

I have found I can trigger topics in FME Server 2019 by setting the JMSType=TextMessage. If this isn't set in the header of the message, FME Server will not pass the message and thus won't trigger a related topic.

Here's an example of this in the RabbitMQ Web UI


steveatsafe
Safer
Forum|alt.badge.img+12
markatsafe wrote:

@alexpillonel FME does not yet support Rabbit AMQP and therefore doesn't support RabbitMQ. I have added your post to the product change request we have to support RabbitMQ. Perhaps consider posting your request to the Ideas page to see if there are other users interested in RabbitMQ support

Just a quick update. I recently had a Case related to this subject and it raised questions if FME was able to support RabbitMQ or not.

Back in 2016, this statement was true. However, some users were still able to configure FME and have success. More recently, I have been doing some testing and discovered that for a few releases of FME Server (2018.1, 2019.0, 2019.1) RabbitMQ messages were not triggering topics in FME Server. This was because of a change made.

For FME Server 2019.2 we resolved this. It still requires manual configuration but it is possible to get it to work. You'll need to provide the necessary RabbitMQ client libraries and be able to make sense of the .bindings file.

In summary, I have been able to configure FME Server to support RabbitMQ message types JMSType=TextMessage. I've also been able to use the JMSSender and JMSReciever in FME Desktop as well. This was all made possible with the help of @david_r when he discovered that things were not working in FME Server 2019.0/1 as they were before.

We have made changes to FME Server 2019.2 and even if JMSType is not set to TextMessage a notification topic will be triggered like it used to be in 2018.0 and older. Hope this helps.

If you have questions on this, please do reach out to us at support - add SteveatSafe to the message. I'm working on a KB article to share the configuration (similar to above) that was successful for me.


david_r
Celebrity
  • March 5, 2020
steveatsafe wrote:

Just a quick update. I recently had a Case related to this subject and it raised questions if FME was able to support RabbitMQ or not.

Back in 2016, this statement was true. However, some users were still able to configure FME and have success. More recently, I have been doing some testing and discovered that for a few releases of FME Server (2018.1, 2019.0, 2019.1) RabbitMQ messages were not triggering topics in FME Server. This was because of a change made.

For FME Server 2019.2 we resolved this. It still requires manual configuration but it is possible to get it to work. You'll need to provide the necessary RabbitMQ client libraries and be able to make sense of the .bindings file.

In summary, I have been able to configure FME Server to support RabbitMQ message types JMSType=TextMessage. I've also been able to use the JMSSender and JMSReciever in FME Desktop as well. This was all made possible with the help of @david_r when he discovered that things were not working in FME Server 2019.0/1 as they were before.

We have made changes to FME Server 2019.2 and even if JMSType is not set to TextMessage a notification topic will be triggered like it used to be in 2018.0 and older. Hope this helps.

If you have questions on this, please do reach out to us at support - add SteveatSafe to the message. I'm working on a KB article to share the configuration (similar to above) that was successful for me.

I'm working on a KB article to share the configurationExcellent! :-)

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings