Question

Automation trigger stops working after a 503 error

jorma
Rank
Badge

We have a simple automation that checks an S3 bucket every 30 seconds and triggers a workbench when a file is added to the bucket. It all seems to work fine, except I noticed today that although the automation was still running it was not triggering the workbench when new files were added to the bucket. Looking at the automation log I see...

 

 Service Unavailable (Service: Amazon S3; Status Code: 503; Error Code: 503 Service Unavailable; Request ID: 4F1445BF8D4DF69F; S3 Extended Request ID: bNSlvkHBzAjPZYX7hSRdG86fwpdaYqp2WrrGNa0IpPm4Qk4XeLb9qRXDdUk5EkWY/NdK1PcNYahUZduRItZf4t9NeiHv1+A8juV+dCYgLfc=)

 

It appears that there was an intermittent issue that caused the 503 error. When this happened the automation continued to run, but no longer worked correctly (new files were added to the S3 bucket, but the automation did not trigger the workbench).

 

I added a notifier to the S3 Trigger in the automation so I can be notified of future failures, but wondering if there is another solution to resolving the issue so the automation will continue to work correctly even if it gets an occasional 503 error. I see that you can configure automated retries (https://community.safe.com/s/article/Configuring-Guaranteed-Delivery-in-FME-Server-Automations-with-Automated-Retries), but I presume this is only for when the workbench fails and not the automation trigger. Thanks! - Jorma

 

4 replies

markw
Badge

Hi @jorma​! My apologies for the late response. The 503 error from Amazon S3 may be caused by the 30-second polling rate. I'd suggest increasing the polling rate to ~1-5 minutes to see if that avoids the error. These resources from AWS may be helpful: https://aws.amazon.com/premiumsupport/knowledge-center/s3-troubleshoot-503-cross-region/

jorma
Rank
Badge

Hi @Mark Warren​ - Thanks for the info! I will take a look at the link you provided and try increasing the polling rate. Seems to have just happened that one time, so not an ongoing issue fortunately. Cheers

R

A 503 Service Unavailable Error is an HTTP response status code that indicates your web server operates properly, but it can't handle a request at the moment. Since it's just a generic error message, it's difficult to pinpoint the issue's exact cause.

jorma
Rank
Badge

A 503 Service Unavailable Error is an HTTP response status code that indicates your web server operates properly, but it can't handle a request at the moment. Since it's just a generic error message, it's difficult to pinpoint the issue's exact cause.

Hi @rosstaylor11223​ - Thanks for your comment. The issue is not with the S3 bucket being intermittently unavailable, but with the way the automation handles the error. When this happened the automation continued to run, but no longer worked correctly (new files were added to the S3 bucket, but the automation did not trigger the workbench).

 

My solution was to create a workbench that is triggered in the automation when there is a failure with the S3 listener. When the listener fails I have an 'automation_restart' workbench that stops the automation, waits 30 seconds and then starts the automation. This looks to have resolved it.

Reply


Community Stats

28,707
Posts
109,479
Replies
36,553
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.