You will have to look at the API documentation on how it implements paging (sometimes also called pagination), meaning request data in increment blocks.

Here's one way of implementing it in FME: https://knowledge.safe.com/questions/51270/loop-on-httpcaller.html

Hoi @perry,

The issue with pagination is that you need to know how many requests you should make and that it can overload the service.

Quering the service for the total number of features, using $count=true doenst work, unfortunately.

Luckily you can get the data using the ODatafeed and not the OdataApi, this way you can download the file and read it into the workspace or save it into an attribute.

Hope this helps,

Itay

Yes,

this works ok, thanks all.

@ all Is it possible to limit the URL with a parameter such as Municipaly so I don't select the full data?

@Itay: I suppose I have to use XML fragmenter to read the data.

The I filled Elements to match (select all), Options: enable flattening and attributes to Expose.

The output I get however I do not understand because it seems to have no common item between the separete records.

Can you give me a clue how to combine these records grouped by municipaly?

Do I have to make a list or do I have to use a different transformer?

Do you have an example preferable with this ODataFeed?

Manu thanks,

Perry

Hi @perry,

You can use a FeatureReader (XML format) with the following elements to match: feed/entry/content/properties.

This will provide you with features that you can group by Gemeentenaam_1 attribute.

Unfortunately filtering of the request is allowed on a limited number of attributes, see for more information (https://www.cbs.nl/nl-nl/onze-diensten/open-data/website-open-data ) and the feed is limited for 10000 features per request So that means you probably need more that one request to get all the data.cbs-open-data-services.pdf

Succes!

Itay