Question

How I can download featureclass/tables from Enterprise gdb to text file?

M

Dear everyone, I have a workbench which we is using for data download purpose. This workbench is connected with an enterprise gdb (sql server) and reading feature classes and tables and we are downloading data in different format in need basis. So far, through "Published Parameter" we have been successfully downloaded data in csv2, XLSXW, GEODATABASE_FILE formats without doing any programming but when I added "TEXTLINE" writer format in Published Parameter then the download result is producing an empty text file.

Please someone help me out from this problem.

 

Thanking you

With kinds Regards

Muqitbild1

11 replies

H
Badge +2

Hi @mailmuqit​ ,

 

To write to TEXTFILE all the feature information must be stored in a single attribute called text_line_data, it cannot write out any other attributes. Therefore you'll probably need to add a step into your workspace so that if the chosen DEST_FORMAT is TEXTFILE, all the source data is aggregated into this single text_line_data attribute.

M

Hi @mailmuqit​ ,

 

To write to TEXTFILE all the feature information must be stored in a single attribute called text_line_data, it cannot write out any other attributes. Therefore you'll probably need to add a step into your workspace so that if the chosen DEST_FORMAT is TEXTFILE, all the source data is aggregated into this single text_line_data attribute.

Hi, thanks for your reply. Actually my workbench is pretty straightforward and the writer type is Dynamic, so I am not sure why I can change something to get the output in text format.bild1 

H
Badge +2

Hi, thanks for your reply. Actually my workbench is pretty straightforward and the writer type is Dynamic, so I am not sure why I can change something to get the output in text format.bild1 

Hi @mailmuqit​ ,

As noted in the format documentation although you can use this writer in a generic translation it requires a customised workspace to produce a useful result.

 

The textfile Writer is not like the other writers you are using in that it will only write out data that is stored in an attribute called text_line_data, if this attribute is empty the resulting text file will be empty. In your current set up all the attributes are being parsed to the writer with the names they have at the source hence no textfile is written.

 

You should add a Tester after the Reader to check if the destination format parameter was set to Text File. If it wasn't, the features will go out of the Failed port which can be connected to the CsmapReprojector. If Text File was selected, the features will come out of the Passed port for further processing, where you can then concatenate them into the text_line_data attribute and redefine the schema before the writer.

 

I have attached a workspace to demonstrate one way you could set this up. After the Passed port of the Tester you can see:

  1. SchemaScanner - creates a schema feature that contains a list feature attribute{}.name (Note: this transformer is new and only available in 2021.1)
  2. Aggregator - Merge all incoming attributes (merge schema feature with data)
  3. AttributeConcatenator - Create a single feature containing all attribute values separated by a comma
  4. SchemaSetter - Reset the Schema to prepare for writer

Lastly in the Writer make sure the Schema Sources is set to accept "Schema from a Schema Feature" as well as the source Reader.

 

The workspace contains annotations that expand on how to set up these transformers in more detail. These resources are also a great tool to help with getting started with more complex dynamic workflows:

M

Hi, thanks for your reply. Actually my workbench is pretty straightforward and the writer type is Dynamic, so I am not sure why I can change something to get the output in text format.bild1 

Hi @hollyatsafe​ 

I'll try working on this and see where I get. Very much appreciated.

M

Hi, thanks for your reply. Actually my workbench is pretty straightforward and the writer type is Dynamic, so I am not sure why I can change something to get the output in text format.bild1 

Hi, I have followed your footstep and still no clue why my output file is empty. I am seding you my workflow screenshort. Any further help will be highly appreciated.fme

M

Hi, my fme workbench is 2020, so I dont have any Schema Scanner, is there any alternate way to solve this problem ?

 

Regards

Muqit

H
Badge +2

Hi, my fme workbench is 2020, so I dont have any Schema Scanner, is there any alternate way to solve this problem ?

 

Regards

Muqit

Hi @mailmuqit​ ,

 

Without access to the SchemaScanner, the workspace I attached will not work as intended. Since this is a dynamic workflow so you don't know what the attributes to combine are beforehand, it is a rather complex problem. The only other method I can think of would be to instead use a ListBuilder after the Tester and then concatenate every item in the list into the single text_line_data attribute before the SchemaSetter.

 

To concatenate the list you'd need to use Python, I can't provide much guidance with that as I don't write python myself, but I think the answer by jdh on this Community Post would be a good starting point. If you're not familiar with Python, perhaps consider installing FME 2021.1.

M

Hi @mailmuqit​ ,

 

Without access to the SchemaScanner, the workspace I attached will not work as intended. Since this is a dynamic workflow so you don't know what the attributes to combine are beforehand, it is a rather complex problem. The only other method I can think of would be to instead use a ListBuilder after the Tester and then concatenate every item in the list into the single text_line_data attribute before the SchemaSetter.

 

To concatenate the list you'd need to use Python, I can't provide much guidance with that as I don't write python myself, but I think the answer by jdh on this Community Post would be a good starting point. If you're not familiar with Python, perhaps consider installing FME 2021.1.

Dear @hollyatsafe​ ,

I really appreciate your help. I have installed fme 2021.1 version and now schemaScanner is there, but I am getting only the first row of the table. Following is the example table:

f1And I getting data as output like:

f2but I also want the column names and after second line values will come seperated by (,) and need to get all values in row wise like a csv file. but the file extension will be .txt .

 

Thanks

M

Hi @mailmuqit​ ,

 

Without access to the SchemaScanner, the workspace I attached will not work as intended. Since this is a dynamic workflow so you don't know what the attributes to combine are beforehand, it is a rather complex problem. The only other method I can think of would be to instead use a ListBuilder after the Tester and then concatenate every item in the list into the single text_line_data attribute before the SchemaSetter.

 

To concatenate the list you'd need to use Python, I can't provide much guidance with that as I don't write python myself, but I think the answer by jdh on this Community Post would be a good starting point. If you're not familiar with Python, perhaps consider installing FME 2021.1.

Status:

  • We have installed FME 2021.1 version in test server.
  • Tested the new updated workbench
  • result is some missing values , cloumn names, and first row data (but in one single line one after another seperated by (,))

Result should look like:

  • output text file should look like a normal text file , first the column names seperated by comma then all the values in different rows
  • if we have more than one table or feature to download then output will be seperate text files

@Takashi Iijima​  some years back you also helped me a lot to solve some problem, so do you have suggestions regarding this problem of mine? please follow this post from the beginning. Already i have build the workflow based on @hollyatsafe​  her suggestions and its working good but not completely ..

 

H
Badge +2

Hi @mailmuqit​ ,

 

Without access to the SchemaScanner, the workspace I attached will not work as intended. Since this is a dynamic workflow so you don't know what the attributes to combine are beforehand, it is a rather complex problem. The only other method I can think of would be to instead use a ListBuilder after the Tester and then concatenate every item in the list into the single text_line_data attribute before the SchemaSetter.

 

To concatenate the list you'd need to use Python, I can't provide much guidance with that as I don't write python myself, but I think the answer by jdh on this Community Post would be a good starting point. If you're not familiar with Python, perhaps consider installing FME 2021.1.

Hi @mailmuqit​ ,

 

This certainly is a head-scratcher, I've been working with a colleague and we think we've come up with a much simpler approach, although there are still some kinks you'll need to work out.

 

I'm proposing that you disregard my previous suggestion entirely, and instead rather than use the Text File Writer, use a CSV Writer but change the file extension to txt. This way you don't have to deal with changing the schema, and should still get the desired format with attribute names and values separated by commas with a feature per row.

 

To do this you should:

1. Update the DEST_FORMAT parameter so that the value for TEXT FILE is CSV2

2. Add a CSV2 Writer to the canvas, set the CSV File Definition to None (Advanced) then open the Parameters and change the extension to txt.

Screen Shot 2021-08-17 at 10.54.53 AMNow, whenever the Generic Writer is set to the CSV2 format, it will use these settings.

 

The problem with this method is that you also have CSV as a format option, so you may need to link the extension to a published parameter. Alternatively, instead of adding a separate CSV writer, you could change the Generic Writer to a Feature Writer, and when a user selects Textfile have the workspace write to CSV but then use a SystemCaller to change the file extension to text afterwards. Hopefully one of these ideas gives you something to work with.

 

 

 

M

Hi @mailmuqit​ ,

 

Without access to the SchemaScanner, the workspace I attached will not work as intended. Since this is a dynamic workflow so you don't know what the attributes to combine are beforehand, it is a rather complex problem. The only other method I can think of would be to instead use a ListBuilder after the Tester and then concatenate every item in the list into the single text_line_data attribute before the SchemaSetter.

 

To concatenate the list you'd need to use Python, I can't provide much guidance with that as I don't write python myself, but I think the answer by jdh on this Community Post would be a good starting point. If you're not familiar with Python, perhaps consider installing FME 2021.1.

Hi @hollyatsafe​ 

I really appreciate your efforts. Your outline worked for me and it was a simple solution. I did not even add SystemCalled at all. Please see the screenshot of my workbench:

f1By using tester I just bypass the texfile to a FeatureWritter (which is a csv writer) and in parameters I have used txt as extension.

I kept this featureWriter for text output and for other formats I am using Dynamic writer.

Now all set, Thanks a lot once again.

Reply


Community Stats

28,723
Posts
109,511
Replies
36,559
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.