Skip to main content

Applies to FME Workbench 2017.1

If you apply a select statement on a DataReader (in this example MS SQL Non Spatial), it misreads the SQL if there are embedded comments, treating everything after the first comment as a comment, hence producing wrong result.

My Sample Workspace

Here is my workspace (a simplified version to illustrate the problem). I used a DataReader and a SQLExecutor to show the inconsistency in number of records read, resulting from incorrect parsing of SQL statement on the DataReader. Both the DataReader and SQLExecutor have the exact same select SQL. The correct number of records should be 98,792.

DataReader select statement configuration:

After I press OK and go back into it, looks like this:

And now the SQLExecutor configuration:

The SQLExecutor has the same SQL as above, yet produces the correct result. The Editor dialog automatically detects that it is SQL and gives the appropriate syntax highlighting (whereas the DataReader dialog doesn't).

Conclusions:

  • The DataReader doesn't interpret the select SQL correctly where there are comments.
  • Removing the comments makes the DataReader produce the correct result again.
  • Surprisingly the SQLExecutor does not suffer from this problem and produces the correct result, regardless if there are comments or not.
  • The SQLExecutor also preserves the original formatting of the SQL (carriage/returns, indents, etc.), whereas the DataReader doesn't and turns everything into a single line of text.

SQL Execute does not handle 'GO' in SQL as when using it in the database manager (SQL Server)


Hi @peterz, the SELECT statement on the Reader Feature Type was designed to help limit the number of attributes or columns that you're reading in a with a simple SQL statement. The SQLExecutor and SQLCreator are much more robust and can handle more complex SQL statements (like the one you're using that has comments).


Hi @peterz, the SELECT statement on the Reader Feature Type was designed to help limit the number of attributes or columns that you're reading in a with a simple SQL statement. The SQLExecutor and SQLCreator are much more robust and can handle more complex SQL statements (like the one you're using that has comments).

Thanks for your response Tia. I guess it's a grey area what constitutes 'complex SQL', but in this case it was just a matter of eliminating the comments to make it work.

 

 

I'm in the habit of always putting some comments in my SQL so that when I come back to it in 12 months time, I know what it's doing and why. This also benefits other people in my team who may not be as familiar with my workspace as I am. I might start using the SQLExecutor/SQLCreator more from now on.

 

 


Reply