I have an FMW that writes a CSV file and then uploads that CSV to Azure BLOB.
The CSV that's created on the server's disk is okay. It has 201 records as expected and opens up just fine.
That CSV--after FME uploads it to Azure BLOB via AzureBlobStorageConnector has about 25 less records--seemingly randomly removed. To rule out Azure BLOB itself as the problem, I used Microsoft Azure Storage Explorer to upload the CSV--when I do that, all records are in the CSV file when downloaded back down from Azure.
I have experimented w/ a folder upload (upload the file w/ a containing folder) vs. a file upload--no luck w/ that, same result.
What really bothers me about this is how it makes no sense. The CSV that gets uploaded should be the CSV that's on disk. It's a file and file-upload process. What could possibly be removing certain lines/records from the file?
Page 1 / 1
I figured it out. It was a timing issue.
I was trying to use a Connection Run Time Order to run a CSV writer and then run AzureBlobStorageConnector (branched). Apparently, the CSV writer was still writing when the AzureBlobStorageConnector started.
I switched from using a CSV writer to a FeatureWriter. The FeatureWriter can write to CSV. And, because FeatureWriter is a transformer, the ETL can continue on from its output--no need to branch out and be concerned about order.