Solved

Is there a recent issue with the Sentinel-2 on AWS reader?


Badge

Hey there,

I recently tried to run a workspace I created a month ago where I download the latest Sentinel-2 image via the FeatureReader (Sentinel-2 on AWS format) based on an intersection with a shape file. The workspace used to work out fine.

Now, the workspace fails early at the FeatureReader with the following error message:

Imagery on AWS Reader: Error listing 'http://sentinel-s2-l1c.s3.amazonaws.com/?delimiter=/&prefix;=tiles/32/U/MC/'. Check the reader filters or try again later. The error was 'FMEException: 931363: Imagery on AWS Reader: Error downloading 'http://sentinel-s2-l1c.s3.amazonaws.com/?delimiter=/&prefix;=tiles/32/U/MC/'. Check the reader filters or try again later. The error was '403 Client Error: Forbidden for url: http://sentinel-s2-l1c.s3.amazonaws.com/?delimiter=/&prefix;=tiles/32/U/MC/''

Does anyone encountered the same error message? Is it possible that the imagery downloading service is temporarily unavailable or did it changed the permissions (regarding the the 403 Client Error)?  

Cheers,

André

EDIT: In addition, the Landsat-8 and Sentinel-2 selector at the Server Playground demos (http://demos.fmeserver.com/landsat-sentinel-selector/) only provides Landsat-8 images for a given location (tested in Vancouver and Northern Germany). 

icon

Best answer by jdh 13 August 2018, 17:12

View original

12 replies

Badge +22

Sentinel-2 data on amazon is now on a pay per use system.

 

 

See https://forum.sentinel-hub.com/t/changes-of-the-access-rights-to-l1c-bucket-at-aws-public-datasets-requester-pays/172 for details.
Userlevel 3
Badge +13

Sentinel-2 data on amazon is now on a pay per use system.

 

 

See https://forum.sentinel-hub.com/t/changes-of-the-access-rights-to-l1c-bucket-at-aws-public-datasets-requester-pays/172 for details.
Thanks for letting us know @jdh -- I've asked the team to see how we can change our reader to include AWS credentials.

 

Badge

Sentinel-2 data on amazon is now on a pay per use system.

 

 

See https://forum.sentinel-hub.com/t/changes-of-the-access-rights-to-l1c-bucket-at-aws-public-datasets-requester-pays/172 for details.
Thank you for the information, @jdh!

 

Could this have a massive impact to the free, full and open data policy of Copernicus/Sentinel, @daleatsafe? Could you or the team suggest another workaround to get Sentinel data?
Badge +22
Thank you for the information, @jdh!

 

Could this have a massive impact to the free, full and open data policy of Copernicus/Sentinel, @daleatsafe? Could you or the team suggest another workaround to get Sentinel data?
As far as I am aware, the copernicus scihub has no intention of becoming pay to play.

 

You can access sentinel data for free from https://scihub.copernicus.eu/.

 

 

Use the open hub for an interactive session, and the api hub if you want to script your data access.

 

 

Both require a free registration.

 

 

As far as I am aware, the copernicus scihub has no intention of becoming pay to play.

 

You can access sentinel data for free from https://scihub.copernicus.eu/.

 

 

Use the open hub for an interactive session, and the api hub if you want to script your data access.

 

 

Both require a free registration.

 

 

Has anyone any experience with using for instance the httpcaller transformer for downloading from scihub via FME? And wants to share it? :-)
Badge
Thank you for the information, @jdh!

 

Could this have a massive impact to the free, full and open data policy of Copernicus/Sentinel, @daleatsafe? Could you or the team suggest another workaround to get Sentinel data?
Hey @andeh, I'm currently working on this thing. I will share my solution within next week!

 

Hey @andeh, I'm currently working on this thing. I will share my solution within next week!

 

Many thanks @abal! Excited to see what you come up with:-)
Userlevel 3
Badge +13
Thanks for letting us know @jdh -- I've asked the team to see how we can change our reader to include AWS credentials.

 

Just saw that a fix will be put into FME 2018.1.x shortly -- will be available by early Sept. latest.

 

 

Hey @andeh, I'm currently working on this thing. I will share my solution within next week!

 

Same issue, would love to see a scihub x FME workflow solution. :)

 

 

Badge +22

So I had a bit of time today and decided to put together a scihub custom transformer for downloading Sentinel-2 imagery. sentinel2querier.fmx

 

This is a beta version, and I don't guarantee it will work in all circumstances, feedback is welcome.

 

 

This transformer does not immediately download the data, it creates footprints for all images that intersect the area of interest (trigger polygon) that meet the criteria (date range and cloud cover).

 

 

These footprints contain all the metadata available for the image as well as an explicit url to download the full version of the data (_data_URL).

@andeh , @harijsijabs , @abal

Badge +2

Update: The Sentinal-2 AWS reader now allows user to supply s3 credentials for access. The updated feature can be found in our latest 2018.1.0.2 available here.

Badge

Just arrived in the FME Hub: The SentinelDownloader Transformer:

https://hub.safe.com/transformers/sentineldownloader

This looks like a pretty good alternative to the SentinelQuerier since it outputs the data bandwise and let you automate upcoming processes.

Reply