Skip to main content
  • EN
  • Home
  • Forums
  • FME Form
  • General
  • Does FME support moving data from an on-premise database (MySQL or SQL Server) to Databricks? We're looking to potentially replace StreamSets.
Does FME support moving data from an on-premise database (MySQL or SQL Server) to Databricks? We're looking to potentially replace StreamSets.

Hi @pkandra​, at the moment we do not have a Databricks Reader/Writer built into FME. However, you could use the JDBC Reader/Writer alongside the Databricks JDBC Driver to try and connect to the Databricks database. Alternatively, if your Databricks uses a Data Lake on Azure, you could try using the AzureBlobStorageConnector to try and retrieve the files.

 

I haven't tried either of these methods myself so I cannot confirm if they work. If you do manage to get either of these methods working I'd be interested in hearing how you got that setup.

 

Hope this provides some insight!

@pkandra​ as an update, we now have a Databricks Reader & Writer. Both the Reader & Writer support Databricks hosted on AWS & Azure.

If you're still interested in testing out a workflow with SQL Server to Databricks, take a look at our knowledge base/how-to article series here: Getting Started with Databricks

Reply


Community Stats

30,162
Posts
114,480
Replies
37,808
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.