s3 Watch - CREATE and MODIFY - how does FME Server know when something is new or modified?

Hi @virtualcitymatt​ ,

When an S3 Watch publication is started, we first fetch all of the objects in the S3 bucket to keep track of the initial state of the bucket. The AmazonS3.listObjects() API returns a maximum of 1000 objects so if a bucket has more than 1000 objects, then several requests will need to be made to get all the objects.

Events trigger based on changes to the entity Tag (ETag), which is a representation of file contents. No message is triggered if a change to an existing file does not update the ETag. If a file is overwritten and the ETag value changes, a MODIFY event is triggered (from Amazon S3 Bucket Triggers documentation).

Hope this helps!

-Kezia

OK great, thanks for this explanation - I thiiink what may have happened is that a few files with 0KB size were present in the folder when the service started. These files were deleted and later reuploaded (still with 0KB). I assume that the ETag was the same and so the polling did not pick up a change? could that be? The other possibility was that an ETag was never created.

Seems to be working now as expected