Best answer by ravenkopelman
View original
Userlevel 4
+13
- Contributor
- 3725 replies
-
19 July 2017
Hi @rfslivajr, FME currently does not support writing to Hive (please check out the table in Using the FME JDBC). I recommend voting on the Hadoop idea on our Ideas page to increase the awareness of the suggestion.
- Participant
- 91 replies
-
26 July 2017
- Best Answer
@TiaAtSafe is correct that FME does not include a Hive reader or writer. However, this is likely achievable through our JDBC format.
In 2015 we performed reader experiments with three Hive JDBC drivers.
Progress/DataDirect: Did not work due to auto-commit conflicts, but between FME improvements since then and the later-discovered TransactionMode driver flag I believe it is viable.
Cloudera: Worked as long as the driver classname was explicitly specified.
Apache: Unable to get working, but this project has advanced since then. There may be hope.
We did not perform writer experiments because Hive at that time did not support standard SQL INSERT statements, so you would have needed to manually send your insert statements through a SqlExecutor transformer. It appears that lack of standard INSERT support may also have improved over the last few years.
In short, if you are willing to put the effort in you can probably get this working with the JDBC tools currently available.
- Participant
- 91 replies
-
4 April 2018
@TiaAtSafe is correct that FME does not include a Hive reader or writer. However, this is likely achievable through our JDBC format.
In 2015 we performed reader experiments with three Hive JDBC drivers.
Progress/DataDirect: Did not work due to auto-commit conflicts, but between FME improvements since then and the later-discovered TransactionMode driver flag I believe it is viable.
Cloudera: Worked as long as the driver classname was explicitly specified.
Apache: Unable to get working, but this project has advanced since then. There may be hope.
We did not perform writer experiments because Hive at that time did not support standard SQL INSERT statements, so you would have needed to manually send your insert statements through a SqlExecutor transformer. It appears that lack of standard INSERT support may also have improved over the last few years.
In short, if you are willing to put the effort in you can probably get this working with the JDBC tools currently available.
Reply
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.