I do not know if this is possible or not, but it would be nice to have the ability for Readers and Transformers that connect to databases (SQLExecutors, Feature Readers, DatabaseJoiner), to have the option to close the connection after the data has been read out. Currently the connections stay open for the entire ETL run, even if the data has been read out, so if the ETL takes 35 minutes to run, and we have a firewall timeout of 30, the ETL will fail even though it has already read out all the data from the source, as it cannot close the source connection, as it has been terminated by the firewall at that point.
Archived
Close Source connection after data has been read out
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
+26- Contributor
This certainly makes sense.
But it will require to reopen the connection if a second transformer or a writer is using the same database.
So, I suggest this as an optional setting.