I would love the ability to read OSM data in pbf rather than xml format http://wiki.openstreetmap.org/wiki/PBF_Format
Page 1 / 1
This is something we could use big time as OSM is the most important source for us in any time of crisis.
@mb_fdfa I do have an inelegant solution using scripted paramters and OSMConvert where there is a published parameter (Filename (Existing) with a configuration of *.pbf;*.osm;*.o5m;*.osm.gz and the OSM reader points to a scripted parameter. Mine is more complicated with filtering and clipping options, but it boils down to
import os.path
import subprocess
#get parameters
inOSM = FME_MacroValuesl'OSM']
toolsDir = FME_MacroValuesa'OSMTools']
#get osm basename
(filepath, ext) = os.path.splitext(inOSM)
#create temp output file
outOSM = filepath + '-temp.osm'
#Create and run OSMconvert command line
runP = toolsDir+'/osmconvert ' + inOSM + ' -o=' + outOSM
subprocess.call(runP)
#return temp filename
return outOSM
File size differences are huge (11GB in pbf, 171GB in XML) this would really save some space/processing if we could get the pbf format.
@jdh @d2 @mb_fdfa -- I can advise that an Open Street Map PBF Reader has been in FME 2017 betas since build 17037 -- we've had no feedback on it yet and so would definitely welcome some. Thanks for your interest!
Are there plans to allow for common osm feature types like those available in the osm reader (highway, waterway, natural, etc), or will it only support data primitives (nodes, ways, relations).
Thanks for the feedback. To be clear, you'd prefer the features be emitted with feature types more like the OSM reader? We'll look and see what we can do -- it does seem very reasonable.
The osm reader has a parameter as to whether to use basic (node/way/relation), broad (highway), specific (highway_primary) or user defined (fmi pipeline file).
The pbf reader uses the gdal osm config .ini, which produces nodes/way/relations. Even something like offering alternate config files with the distribution would be beneficial*, though my preference would be a parameter.
*Most of my users would not be able to create a custom config file.
Will there be a pbf writer as well?
Is there a way to try this reader in FME 2016 already?
You need to grab
FME 2017 betas to try it, but these can co-exist with FME 2016.
FME 2017 betas to try it, but these can co-exist with FME 2016.
No plans at the current time for a PBF writer. Feel free to suggest as an Idea so we can track interest.
Finally I have FME 2017 on my main Workstation and can try the new reader. Thanks, great it is there!
Although I haven't been able to try any real scenarios yet, I think that the suggestion from @jdh would be a useful enhancement.
This has been released as part of FME 2017.0 . Visit http://www.safe.com/downloads to download it and give it a spin.
Hi
@jdh,
@jdh,
I'm pleased to let you know that the 2018.0 version of the now reader allows you to choose from the 26 broad categories and its sub categories (via OSM Map Features), which are represented as feature types similar to the OSM XML reader.
Let us know how it looks!