Does FME support moving data from an on-premise database (MySQL or SQL Server) to Databricks? We're looking to potentially replace StreamSets.
+12- Safer
Hi @pkandra, at the moment we do not have a Databricks Reader/Writer built into FME. However, you could use the JDBC Reader/Writer alongside the Databricks JDBC Driver to try and connect to the Databricks database. Alternatively, if your Databricks uses a Data Lake on Azure, you could try using the AzureBlobStorageConnector to try and retrieve the files.
I haven't tried either of these methods myself so I cannot confirm if they work. If you do manage to get either of these methods working I'd be interested in hearing how you got that setup.
Hope this provides some insight!
+12- Safer
@pkandra as an update, we now have a Databricks Reader & Writer. Both the Reader & Writer support Databricks hosted on AWS & Azure.
If you're still interested in testing out a workflow with SQL Server to Databricks, take a look at our knowledge base/how-to article series here: Getting Started with Databricks
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.