Skip to main content
Solved

Is there a recent issue with the Sentinel-2 on AWS reader?

abal
Contributor
Forum|alt.badge.img+6
  • abalContributor
  • Contributor

Hey there,

I recently tried to run a workspace I created a month ago where I download the latest Sentinel-2 image via the FeatureReader (Sentinel-2 on AWS format) based on an intersection with a shape file. The workspace used to work out fine.

Now, the workspace fails early at the FeatureReader with the following error message:

Imagery on AWS Reader: Error listing 'http://sentinel-s2-l1c.s3.amazonaws.com/?delimiter=/&prefix;=tiles/32/U/MC/'. Check the reader filters or try again later. The error was 'FMEException: 931363: Imagery on AWS Reader: Error downloading 'http://sentinel-s2-l1c.s3.amazonaws.com/?delimiter=/&prefix;=tiles/32/U/MC/'. Check the reader filters or try again later. The error was '403 Client Error: Forbidden for url: http://sentinel-s2-l1c.s3.amazonaws.com/?delimiter=/&prefix;=tiles/32/U/MC/''

Does anyone encountered the same error message? Is it possible that the imagery downloading service is temporarily unavailable or did it changed the permissions (regarding the the 403 Client Error)?  

Cheers,

André

EDIT: In addition, the Landsat-8 and Sentinel-2 selector at the Server Playground demos (http://demos.fmeserver.com/landsat-sentinel-selector/) only provides Landsat-8 images for a given location (tested in Vancouver and Northern Germany). 

Best answer by jdh

Sentinel-2 data on amazon is now on a pay per use system.

 

 

See https://forum.sentinel-hub.com/t/changes-of-the-access-rights-to-l1c-bucket-at-aws-public-datasets-requester-pays/172 for details.
View original
Did this help you find an answer to your question?

12 replies

jdh
Contributor
Forum|alt.badge.img+28
  • jdhContributor
  • Contributor
  • Best Answer
  • August 13, 2018

Sentinel-2 data on amazon is now on a pay per use system.

 

 

See https://forum.sentinel-hub.com/t/changes-of-the-access-rights-to-l1c-bucket-at-aws-public-datasets-requester-pays/172 for details.
fmelizard
Contributor
Forum|alt.badge.img+17
  • fmelizardContributor
  • Contributor
  • August 14, 2018
jdh wrote:

Sentinel-2 data on amazon is now on a pay per use system.

 

 

See https://forum.sentinel-hub.com/t/changes-of-the-access-rights-to-l1c-bucket-at-aws-public-datasets-requester-pays/172 for details.
Thanks for letting us know @jdh -- I've asked the team to see how we can change our reader to include AWS credentials.

 

abal
Contributor
Forum|alt.badge.img+6
  • abalContributor
  • Author
  • Contributor
  • August 14, 2018
jdh wrote:

Sentinel-2 data on amazon is now on a pay per use system.

 

 

See https://forum.sentinel-hub.com/t/changes-of-the-access-rights-to-l1c-bucket-at-aws-public-datasets-requester-pays/172 for details.
Thank you for the information, @jdh!

 

Could this have a massive impact to the free, full and open data policy of Copernicus/Sentinel, @daleatsafe? Could you or the team suggest another workaround to get Sentinel data?
jdh
Contributor
Forum|alt.badge.img+28
  • jdhContributor
  • Contributor
  • August 14, 2018
abal wrote:
Thank you for the information, @jdh!

 

Could this have a massive impact to the free, full and open data policy of Copernicus/Sentinel, @daleatsafe? Could you or the team suggest another workaround to get Sentinel data?
As far as I am aware, the copernicus scihub has no intention of becoming pay to play.

 

You can access sentinel data for free from https://scihub.copernicus.eu/.

 

 

Use the open hub for an interactive session, and the api hub if you want to script your data access.

 

 

Both require a free registration.

 

 

A
  • andeh
  • August 24, 2018
jdh wrote:
As far as I am aware, the copernicus scihub has no intention of becoming pay to play.

 

You can access sentinel data for free from https://scihub.copernicus.eu/.

 

 

Use the open hub for an interactive session, and the api hub if you want to script your data access.

 

 

Both require a free registration.

 

 

Has anyone any experience with using for instance the httpcaller transformer for downloading from scihub via FME? And wants to share it? :-)
abal
Contributor
Forum|alt.badge.img+6
  • abalContributor
  • Author
  • Contributor
  • August 24, 2018
abal wrote:
Thank you for the information, @jdh!

 

Could this have a massive impact to the free, full and open data policy of Copernicus/Sentinel, @daleatsafe? Could you or the team suggest another workaround to get Sentinel data?
Hey @andeh, I'm currently working on this thing. I will share my solution within next week!

 

A
  • andeh
  • August 24, 2018
abal wrote:
Hey @andeh, I'm currently working on this thing. I will share my solution within next week!

 

Many thanks @abal! Excited to see what you come up with:-)
fmelizard
Contributor
Forum|alt.badge.img+17
  • fmelizardContributor
  • Contributor
  • August 26, 2018
fmelizard wrote:
Thanks for letting us know @jdh -- I've asked the team to see how we can change our reader to include AWS credentials.

 

Just saw that a fix will be put into FME 2018.1.x shortly -- will be available by early Sept. latest.

 

 

H
abal wrote:
Hey @andeh, I'm currently working on this thing. I will share my solution within next week!

 

Same issue, would love to see a scihub x FME workflow solution. :)

 

 

jdh
Contributor
Forum|alt.badge.img+28
  • jdhContributor
  • Contributor
  • August 29, 2018

So I had a bit of time today and decided to put together a scihub custom transformer for downloading Sentinel-2 imagery. sentinel2querier.fmx

 

This is a beta version, and I don't guarantee it will work in all circumstances, feedback is welcome.

 

 

This transformer does not immediately download the data, it creates footprints for all images that intersect the area of interest (trigger polygon) that meet the criteria (date range and cloud cover).

 

 

These footprints contain all the metadata available for the image as well as an explicit url to download the full version of the data (_data_URL).

@andeh , @harijsijabs , @abal

xiaomengatsafe
Safer
Forum|alt.badge.img+3

Update: The Sentinal-2 AWS reader now allows user to supply s3 credentials for access. The updated feature can be found in our latest 2018.1.0.2 available here.

abal
Contributor
Forum|alt.badge.img+6
  • abalContributor
  • Author
  • Contributor
  • November 22, 2018

Just arrived in the FME Hub: The SentinelDownloader Transformer:

https://hub.safe.com/transformers/sentineldownloader

This looks like a pretty good alternative to the SentinelQuerier since it outputs the data bandwise and let you automate upcoming processes.

Reply


Helpful Members This Week

Community Stats

30,944
Posts
117,265
Replies
38,742
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.