Skip to main content
Question

reading jpegs - speed


owen
Forum|alt.badge.img+1
Hello, 

 

 

I have a workspace that extracts gps data from geotagged photos. In order to do this fme has to read in each file. I just want to create points with a link to the photo, so I remove the geometry straight away.

 

 

The problem is that the workspace takes a long time to run, reading hunrdreds of  jpegs.

 

 

Is there a way just to read the exif metadata somehow to speed things up?

 

 

Thanks, 

 

 

Owen

6 replies

david_r
Celebrity
  • June 11, 2013
Hi Owen,

 

 

how about using a "Directory and File Pathnames" reader to pick up all the .jpg files, then passing them through ExifTool?

 

 

David

david_r
Celebrity
  • June 11, 2013
There is even a Python wrapper for ExifTool, so you could also call it from a PythonCaller. Perhaps put it into a Custom Transforer for easy reuse?

 

 

David

owen
Forum|alt.badge.img+1
  • Author
  • June 11, 2013

Hi David,

 

 

Thanks for that. I was hoping to avoid python if possible! I'm also unable to install open source apps - our IT is quite locked down.

 

 

Owen

geospatiallover
Participant
Forum|alt.badge.img+6

I also need a solution for this. The deprecated transformer JPEGPointExtractor replaced by the PhotoCoordinateExtractor will transform after the reader has done its job. So I hope there's a better way to read the JPEGs. I like @david_r suggestion in using the python wrapper with the exiftool. I submitted a New Idea to have an option for the JPEG reader to turn of GDAL intermediate processing if its possible.


geospatiallover
Participant
Forum|alt.badge.img+6

I forgot to mention that the JPEG reader during its intermediate processing could easily fill up the hard drive space because of the intermediate processing. During this process, every pixel of the picture is being read for its pixel information. That's why the reader takes so much time to read.


geospatiallover
Participant
Forum|alt.badge.img+6
geospatiallover wrote:

I also need a solution for this. The deprecated transformer JPEGPointExtractor replaced by the PhotoCoordinateExtractor will transform after the reader has done its job. So I hope there's a better way to read the JPEGs. I like @david_r suggestion in using the python wrapper with the exiftool. I submitted a New Idea to have an option for the JPEG reader to turn of GDAL intermediate processing if its possible.

I turned off the only user attribute on the jpeg reader called exif_brightnessvalue but it did not help much. I chatted with Safe support and I got help from Dave Campanas and he suggested adding a geometry remover after the reader. On my test data of 48 raster files, the processing was reduced from 266 seconds to 74.6 or almost a 4x faster. This solved my slowness issue.

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings