Question

reading jpegs - speed

  • 11 June 2013
  • 6 replies
  • 14 views

Badge +1
Hello, 

 

 

I have a workspace that extracts gps data from geotagged photos. In order to do this fme has to read in each file. I just want to create points with a link to the photo, so I remove the geometry straight away.

 

 

The problem is that the workspace takes a long time to run, reading hunrdreds of  jpegs.

 

 

Is there a way just to read the exif metadata somehow to speed things up?

 

 

Thanks, 

 

 

Owen

6 replies

Userlevel 4
Hi Owen,

 

 

how about using a "Directory and File Pathnames" reader to pick up all the .jpg files, then passing them through ExifTool?

 

 

David
Userlevel 4
There is even a Python wrapper for ExifTool, so you could also call it from a PythonCaller. Perhaps put it into a Custom Transforer for easy reuse?

 

 

David
Badge +1

Hi David,

 

 

Thanks for that. I was hoping to avoid python if possible! I'm also unable to install open source apps - our IT is quite locked down.

 

 

Owen
Badge

I also need a solution for this. The deprecated transformer JPEGPointExtractor replaced by the PhotoCoordinateExtractor will transform after the reader has done its job. So I hope there's a better way to read the JPEGs. I like @david_r suggestion in using the python wrapper with the exiftool. I submitted a New Idea to have an option for the JPEG reader to turn of GDAL intermediate processing if its possible.

Badge

I forgot to mention that the JPEG reader during its intermediate processing could easily fill up the hard drive space because of the intermediate processing. During this process, every pixel of the picture is being read for its pixel information. That's why the reader takes so much time to read.

Badge

I also need a solution for this. The deprecated transformer JPEGPointExtractor replaced by the PhotoCoordinateExtractor will transform after the reader has done its job. So I hope there's a better way to read the JPEGs. I like @david_r suggestion in using the python wrapper with the exiftool. I submitted a New Idea to have an option for the JPEG reader to turn of GDAL intermediate processing if its possible.

I turned off the only user attribute on the jpeg reader called exif_brightnessvalue but it did not help much. I chatted with Safe support and I got help from Dave Campanas and he suggested adding a geometry remover after the reader. On my test data of 48 raster files, the processing was reduced from 266 seconds to 74.6 or almost a 4x faster. This solved my slowness issue.

Reply