As an FME Desktop Author, I want to be able to create workflows that connect to and process up to 1,000,000 messages per hour from an Apache Kafka stream, so that I can manage and enrich data from my IoT devices that use this protocol.
Released
Apache Kafka Connector
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
+25- Enthusiast
Apache Kafka just came on my radar some weeks ago. Heard it buzzing twice since, in the FME using crowd I meet. Please implement a reader/writer and a publication/subscription model. Not having that in FME kept me back a little form diving into Kafka immediately.
I have been able to get Kafka messages into FME via a PythonCreator running a dotnet exe and capturing its standard output, but a native reader would be much tidier.
+20- Contributor
Curious what kind of throughput you've been able to achieve using the PythonCreator approach? We've got a project that could possibly require up to 1 million messages per minute at peak load...
The KafkaConnector is available as FME Package in FME Desktop 2019.0 and on the FME Hub:
https://hub.safe.com/publishers/safe/packages/kafka
Login to the community
No account yet? Create an account
An FME Account is required to contribute
LoginEnter your E-mail address. We'll send you an e-mail with instructions to reset your password.