Skip to main content
  • EN

I encountered something a bit weird. In a bit larger workspace I wanted to apply parallel processing for part of the workspace, so I embedded that part of the workspace (which already contained some custom transformers) in a new custom transformer. Then however I obtained some issues with the parameters, it seemed that they were not passed on correctly.

 

Attached is a workspace where I replicated the issue. It's just a dummy workspace that connects to an Oracle DB, and (aims to) return the static result; 'Greeting' = 'Hello'.

 

Main;

imageCT_0;

imageCT_1;

imageIf I run the workspace with a database connection named 'My_DB_Schema (1-O) @ dev', I obtain an error;

Error connecting to Oracle database: message was `ORA-12154: TNS:could not resolve the connect identifier specified'. (serverType=`<serverType>', serverName=`My_DB_Schema', userName=`', password=`***', dbname=`')

 

Although this issue occurs with a Database connection parameter, in my main workspace it seemed to occur for more parameters.

Does anyone have an idea why I'm having these issues with Parameters in nested Custom Transformers?

 

ps. I notice that if I make some slight deviations, the translation does run again without issues. The workspace does work when;

  • I run the workspace, with the same database configuration, now only named 'OtherDBConnectionName'.
  • I run the workspace, without specifying a 'Group By' attribute for CT_0.
  • I run the workspace, and specify 'My_DB_Schema (1-O) @ dev' as a 'direct value' for the DB Connection parameter to be used in CT_0.
  • I duplicate my database connection, and also store it under the name 'My_DB_Schema' (so that name is stored in my DB connections), and run the workspace with my original DB Connection named 'My_DB_Schema (1-O) @ dev'.

Yeah it looks like it doesn't like the <space> in the database connection name. Seems to be a bug when transferring the parameter down that one extra level.

 

I don't have an oracle database to test against but I put in a PrameterFetcher in using just a dummy connection

 

What I did find to work was to put a parameter fetcher in CT_0 and then use the value of the fetched Parameter in CT_1. Perhaps this could be a workaround.

 

 

Yeah it looks like it doesn't like the <space> in the database connection name. Seems to be a bug when transferring the parameter down that one extra level.

 

I don't have an oracle database to test against but I put in a PrameterFetcher in using just a dummy connection

 

What I did find to work was to put a parameter fetcher in CT_0 and then use the value of the fetched Parameter in CT_1. Perhaps this could be a workaround.

 

 

Thanks again for the feedback. I doubt the <space> in the database connection name is causing the issue, as if I don't use grouped processing (not specify a Group By attribute for CT_0), the workspace runs fine.

 

In this case I'm not really looking for workarounds. I know a ParameterFetcher can often circumvent these kind of issues. However, as mentioned, in my main workspace (which is quite a bit more complex than this) it's not just the DB parameter that seemed to cause issues, but e.g. also part of the dynamic SQL query.

Also I'm afraid that switching to using attributes vs parameters (i.e. using a ParameterFetcher) could cause quite a lot more overhead. Note that in my main workspace, a total of over 1*10^9 features would pass through the workspace/CT (granted, that's after the SQLExecutor, so for just the DB connections and SQL queries it wouldn't be such an issue (at least when you don't accumulate Initiator and Result features)).

 

I would prefer it if someone from Safe could have a look at this. I just would like to be able to trust the functionality as I believe it should work.

Thanks again for the feedback. I doubt the <space> in the database connection name is causing the issue, as if I don't use grouped processing (not specify a Group By attribute for CT_0), the workspace runs fine.

 

In this case I'm not really looking for workarounds. I know a ParameterFetcher can often circumvent these kind of issues. However, as mentioned, in my main workspace (which is quite a bit more complex than this) it's not just the DB parameter that seemed to cause issues, but e.g. also part of the dynamic SQL query.

Also I'm afraid that switching to using attributes vs parameters (i.e. using a ParameterFetcher) could cause quite a lot more overhead. Note that in my main workspace, a total of over 1*10^9 features would pass through the workspace/CT (granted, that's after the SQLExecutor, so for just the DB connections and SQL queries it wouldn't be such an issue (at least when you don't accumulate Initiator and Result features)).

 

I would prefer it if someone from Safe could have a look at this. I just would like to be able to trust the functionality as I believe it should work.

OK well I guess the <space> might not be what is causing the issue, however, it's definitely related. I did find that if I tried a connection without a <space> then the parameter worked. A parameter with a space lost everything after the space. This explains why your first and last points work.

 

But yeah it doesn't really solve the other problems. I tired using a new parameter in the CustomTransformer to test if the issue was limited to the the Database connection type in my tests but I kept getting a Dereferenced Macro Error when trying to run.

 

We can ping a safer here: @andreaatsafe​ . Let's see if Andrea is aware of these issues.

Thanks again for the feedback. I doubt the <space> in the database connection name is causing the issue, as if I don't use grouped processing (not specify a Group By attribute for CT_0), the workspace runs fine.

 

In this case I'm not really looking for workarounds. I know a ParameterFetcher can often circumvent these kind of issues. However, as mentioned, in my main workspace (which is quite a bit more complex than this) it's not just the DB parameter that seemed to cause issues, but e.g. also part of the dynamic SQL query.

Also I'm afraid that switching to using attributes vs parameters (i.e. using a ParameterFetcher) could cause quite a lot more overhead. Note that in my main workspace, a total of over 1*10^9 features would pass through the workspace/CT (granted, that's after the SQLExecutor, so for just the DB connections and SQL queries it wouldn't be such an issue (at least when you don't accumulate Initiator and Result features)).

 

I would prefer it if someone from Safe could have a look at this. I just would like to be able to trust the functionality as I believe it should work.

Tnx again for the feedback. Just as a FYI, I don't think the <space> explains why my last point worked. In my fourth bullit I still ran the workspace with the same settings (i.e. still selected 'My_DB_Schema (1-O) @ dev' for the DB connection). The difference is that I also added a duplicate DB connection (with same configuration) named 'My_DB_Schema' to my stored DB connections.

 

The way I look at it, the <space> is just one of the factors here (explaining why my first bullit worked). Also I agree it seems to break at the space, but I'm not entirely sure if other 'non-word characters' like '()@-' are safe to be used.

That said, next to the used characters, it seems to me that other factors that play a role are;

  • Group By attributes (see second bullit)
  • Other stored DB connections (next to the one you are actually using - see the fourth bullit).

Yeah it looks like it doesn't like the <space> in the database connection name. Seems to be a bug when transferring the parameter down that one extra level.

 

I don't have an oracle database to test against but I put in a PrameterFetcher in using just a dummy connection

 

What I did find to work was to put a parameter fetcher in CT_0 and then use the value of the fetched Parameter in CT_1. Perhaps this could be a workaround.

 

 

FYI, I created a case for this issue, so that hopefully we can get some feedback from Safe on this. The case number is C693132.

FYI, I created a case for this issue, so that hopefully we can get some feedback from Safe on this. The case number is C693132.

Hey @thijsknapen​, looks like your case was just assigned to one of our support specialists. They'll reach out to you soon with a response. Thanks for your patience!

FYI, I created a case for this issue, so that hopefully we can get some feedback from Safe on this. The case number is C693132.

Hi @danminneyatsaf​ , thanks for the heads up! :)

Reply


Community Stats

30,110
Posts
114,303
Replies
37,745
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.