Question

Writing to AzureBlob with FME Server (Source


Dear community,

I have a small issue writing to my Azure BlobStorage with FME Server.

When I run my script locally with ABSOLUT PATH it works fine, but when I try to change to a relativ path, it will not upload anything.

 

Any advises. I guess it has something to to with the InputFile for Uplaoding to the Blob, which ist not specfied in the right way.

(dem10 INVALID_FILE Source file does not exist)

 

--> Inputpath:

$(FME_DATA_REPOSITORY)+@Value(fme_basename)

 

Regards,

Alex

 

 

 

 

 


9 replies

Badge +5

You don't need a + to concatenate strings try removing that.

well thanks...this helped, but it is still just working on FME Desktop. When I run the script in my server, it wont! But it says:

 

109 2021-7-23 10:01:36 | dem10 (SurfaceModelFactory): Processed 445536 features and 0 breaklines. 445536 were added to the model successfully. 0 were not added. The number of DEM points output was 0. The number of triangles output was 0. The number of Voronoi Polygons was 0. The number of contours output was 0. The number of vertex points output was 0. The number of features draped was 0. The number of raster features output was 1. The number of TIN Surface features output was 0. The total number of features output was 1

 

110 2021-7-23 10:01:36 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

111 2021-7-23 10:01:36 | Feature output statistics for `GEOTIFF' writer using keyword `FeatureWriter_0':

112 2021-7-23 10:01:36 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

113 2021-7-23 10:01:36 | Features Written

114 2021-7-23 10:01:36 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

115 2021-7-23 10:01:36 | dem10.tif 1

116 2021-7-23 10:01:36 | ==============================================================================

117 2021-7-23 10:01:36 | Total Features Written 1

118 2021-7-23 10:01:36 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

119 2021-7-23 10:01:36 | AzureBlobStorageConnector_CATCHER (TeeFactory): Cloned 1 input feature(s) into 2 output feature(s)

120 2021-7-23 10:01:36 | AzureBlobStorageConnector_ROUTER (TestFactory): Tested 1 input feature(s) -- 0 feature(s) passed and 1 feature(s) failed

121 2021-7-23 10:01:36 | AzureBlobStorageConnector OUTPUT Transformer Output Nuker (TeeFactory): Cloned 0 input feature(s) into 0 output feature(s)

122 2021-7-23 10:01:36 | AzureBlobStorageConnector INITIATOR Transformer Output Nuker (TeeFactory): Cloned 1 input feature(s) into 0 output feature(s)

123 2021-7-23 10:01:36 | AzureBlobStorageConnector <REJECTED> Transformer Output Nuker (TeeFactory): Cloned 1 input feature(s) into 0 output feature(s)

124 2021-7-23 10:01:36 | Destination Feature Type Routing Correlator (RoutingFactory): Tested 0 input feature(s), wrote 0 output feature(s): 0 matched merge filters, 0 were routed to output, 0 could not be routed.

125 2021-7-23 10:01:36 | Final Output Nuker (TeeFactory): Cloned 0 input feature(s) into 0 output feature(s)

 

 

126 2021-7-23 10:01:36 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

127 2021-7-23 10:01:36 | Features Read Summary

128 2021-7-23 10:01:36 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

129 2021-7-23 10:01:36 | ==============================================================================

130 2021-7-23 10:01:36 | Total Features Read 0

131 2021-7-23 10:01:36 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

132 2021-7-23 10:01:36 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

133 2021-7-23 10:01:36 | Features Written Summary

134 2021-7-23 10:01:36 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

135 2021-7-23 10:01:36 | ==============================================================================

136 2021-7-23 10:01:36 | Total Features Written 0

137 2021-7-23 10:01:36 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

138 2021-7-23 10:01:36 | Translation was SUCCESSFUL with 0 warning(s) (0 feature(s) output)

139 2021-7-23 10:01:36 | FME Session Duration: 7.5 seconds. (CPU: 7.0s user, 0.0s system)

140 2021-7-23 10:01:36 | END - ProcessID: 1147, peak process heap usage: 75684 kB, current process heap usage: 28624 kB

Badge +5

Can you put a Logger on the Summary port of the FeatureWriter and show the results and also a screenshot of the AzureBlobStorageConnector setup. My guess is that the AzureBlobStorageConnector is not finding the file that has been written.

You are abolsutly rigth....

 

"SourceFile not found...."

But where does the FeatureWrite saves the data, so the next Task can UPLOAD the specific FILE!

 

(FeatureWriter)

FeatureWriter0 

Any Ideas? I tried 50 times with different "FME Parameters", but it wont work for me. Everytime ist give backs "Soruce file not found" in the Log (AzureBlob...)(

AzureBlob2

Badge +5

You are probably better off using FME_SHAREDRESCOURCE_TEMP. Stand by and I'll test it myself.

Badge +5

Try in the writer having Dataset just be $(FME_SHAREDRESOURCE_TEMP) and have Raster File Name as just dem10 (the extension is added automatically).

 

In the Azure uploader the file to upload is

$(FME_SHAREDRESOURCE_TEMP)\\dem10.tif

 

 

 

but even if i solve the problem, by uploading a folder (which actually works in FME Desktop)...i will get the message vom FME Server:

 

 

 

982021-7-29 11:38:20 | dem10 (SurfaceModelFactory): Processed 445536 features and 0 breaklines. 445536 were added to the model successfully. 0 were not added. The number of DEM points output was 0. The number of triangles output was 0. The number of Voronoi Polygons was 0. The number of contours output was 0. The number of vertex points output was 0. The number of features draped was 0. The number of raster features output was 1. The number of TIN Surface features output was 0. The total number of features output was 

 

1992021-7-29 11:38:20 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

002021-7-29 11:38:20 | Feature output statistics for `GEOTIFF' writer using keyword `FeatureWriter_0':

 

1012021-7-29 11:38:20 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

1022021-7-29 11:38:20 | Features Written1032021-7-29 11:38:20 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

1042021-7-29 11:38:20 | dem10 11052021-7-29 11:38:20 | ==============================================================================

 

1062021-7-29 11:38:20 | Total Features Written 11072021-7-29 11:38:20 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

1082021-7-29 11:38:20 | AzureBlobStorageConnector_CATCHER (TeeFactory): Cloned 1 input feature(s) into 2 output feature(s)

 

1092021-7-29 11:38:20 | AzureBlobStorageConnector_ROUTER (TestFactory): Tested 1 input feature(s) -- 0 feature(s) passed and 1 feature(s) failed

 

1102021-7-29 11:38:20 | AzureBlobStorageConnector INITIATOR Transformer Output Nuker (TeeFactory): Cloned 1 input feature(s) into 0 output feature(s)

 

1112021-7-29 11:38:20 | AzureBlobStorageConnector <REJECTED> Transformer Output Nuker (TeeFactory): Cloned 1 input feature(s) into 0 output feature(s)

 

1122021-7-29 11:38:20 | AzureBlobStorageConnector_Output (Disabled) Nuker (TeeFactory): Cloned 0 input feature(s) into 0 output feature(s)

 

1132021-7-29 11:38:20 | Destination Feature Type Routing Correlator (RoutingFactory): Tested 0 input feature(s), wrote 0 output feature(s): 0 matched merge filters, 0 were routed to output, 0 could not be routed.

 

1142021-7-29 11:38:20 | Final Output Nuker (TeeFactory): Cloned 0 input feature(s) into 0 output feature(s)

 

 

 

FME Server reads the FILE, but is not able to write it

Badge +5

 

 

but even if i solve the problem, by uploading a folder (which actually works in FME Desktop)...i will get the message vom FME Server:

 

 

 

982021-7-29 11:38:20 | dem10 (SurfaceModelFactory): Processed 445536 features and 0 breaklines. 445536 were added to the model successfully. 0 were not added. The number of DEM points output was 0. The number of triangles output was 0. The number of Voronoi Polygons was 0. The number of contours output was 0. The number of vertex points output was 0. The number of features draped was 0. The number of raster features output was 1. The number of TIN Surface features output was 0. The total number of features output was 

 

1992021-7-29 11:38:20 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

002021-7-29 11:38:20 | Feature output statistics for `GEOTIFF' writer using keyword `FeatureWriter_0':

 

1012021-7-29 11:38:20 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

1022021-7-29 11:38:20 | Features Written1032021-7-29 11:38:20 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

1042021-7-29 11:38:20 | dem10 11052021-7-29 11:38:20 | ==============================================================================

 

1062021-7-29 11:38:20 | Total Features Written 11072021-7-29 11:38:20 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

1082021-7-29 11:38:20 | AzureBlobStorageConnector_CATCHER (TeeFactory): Cloned 1 input feature(s) into 2 output feature(s)

 

1092021-7-29 11:38:20 | AzureBlobStorageConnector_ROUTER (TestFactory): Tested 1 input feature(s) -- 0 feature(s) passed and 1 feature(s) failed

 

1102021-7-29 11:38:20 | AzureBlobStorageConnector INITIATOR Transformer Output Nuker (TeeFactory): Cloned 1 input feature(s) into 0 output feature(s)

 

1112021-7-29 11:38:20 | AzureBlobStorageConnector <REJECTED> Transformer Output Nuker (TeeFactory): Cloned 1 input feature(s) into 0 output feature(s)

 

1122021-7-29 11:38:20 | AzureBlobStorageConnector_Output (Disabled) Nuker (TeeFactory): Cloned 0 input feature(s) into 0 output feature(s)

 

1132021-7-29 11:38:20 | Destination Feature Type Routing Correlator (RoutingFactory): Tested 0 input feature(s), wrote 0 output feature(s): 0 matched merge filters, 0 were routed to output, 0 could not be routed.

 

1142021-7-29 11:38:20 | Final Output Nuker (TeeFactory): Cloned 0 input feature(s) into 0 output feature(s)

 

 

 

FME Server reads the FILE, but is not able to write it

Could you post screenshots of your latest settings as you did above please?

Create a Folder in Blob and uploads the File "dem10.tif.tif" (wierd - but working in Desktop)

AzureBlobStoareg

Reply