Skip to main content

Hello,

 

I am trying to retrieve lat/long/rotation attributes from jpegs sitting on Azure but can't get it working.

 

My workflow is as follows:

I read features from AGOL with attachments, attachments get stored on Azure Blob Storage, then I try to retrieve lat/long/direction of jpegs and write these data somewhere else.

Attachments coming from AGOL or Azure come in binary format; which makes exposing jpeg's metadata tricky. I have no problem getting lat/long/direction if jpegs sit on my harddrive.

 

  1. I've tried to use BinaryDecoder, got (TextEncoderFactory): Invalid Base64 character 'ÿ' error.
  2. I've tried RasterReplacer. It shows the actual photo but AttributeExposer doesn't expose metadata this way.

 

Below are screenshots of my workspace as well as attachments' attributes:

 

Annotation 2020-08-27 152129workflow Annotation 2020-08-27 154741

 

Please help.

I assume the metadata is the EXIF tags?

If you are reading the JPEGs with a reader, then there should be various EXIF format attributes that just need to be exposed.

If you are downloading the JPEG as binary, then I think you'll need to save the file and then read it back to get that EXIF info. So the sequence of transformers would be something like:

  • TempPathnameCreator
  • AttributeFileWriter (target encoding = Binary)
  • FeatureReader

I hope this helps. I'll try and come up with an example workspace if I have a moment, but this should get you started.


Thank you for the response, Mark.

I assume the metadata is the EXIF tags? - correct

If you are downloading the JPEG as binary, then I think you'll need to save the file and then read it back to get that EXIF info - thanks for the tip, this will work; however, I intend to publish the workspace to FME server later and would like to keep the whole translation in cache, without saving files in the middle. I believe temporary saving might drastically slow down my workflow as there will be hundreds of photos processed daily.

 

Is it possible to retrieve lat/long/direction right from binary data?

 

Cheers.


Thank you for the response, Mark.

I assume the metadata is the EXIF tags? - correct

If you are downloading the JPEG as binary, then I think you'll need to save the file and then read it back to get that EXIF info - thanks for the tip, this will work; however, I intend to publish the workspace to FME server later and would like to keep the whole translation in cache, without saving files in the middle. I believe temporary saving might drastically slow down my workflow as there will be hundreds of photos processed daily.

 

Is it possible to retrieve lat/long/direction right from binary data?

 

Cheers.

"Is it possible to retrieve lat/long/direction right from binary data?"

 

In theory the RasterReplacer transformer should do this for you. I found that it didn't work for me when I read the data with a HTTPCaller, but you could try it. So place a RasterReplacer and it will replace the current geometry with raster. For me it gave me the image, but it didn't return the exif tags; but you could try it.

 

So, to interpret the raster data directly, as strange as it sounds you would use the BinaryEncoder (not BinaryDecoder) to convert the data from binary to hex.

 

Now you have a hex string that you can interpret according to the structure of a jpeg. For example, I have:

FFD8FFE000104A46494600010100000100010000..........

The part "FFD8" is a Start Of Image marker that denotes that this is a jpeg file, in other words the file signature. Then "FFE0" means a header for JFIF information (followed by a pointer to that info), and then... well FFE1 should denote a pointer to the exif data, but I can't find that. This analysis tool does, which proves I must be looking at it wrong somehow.

 

But decoding data like that is all about bytes and pointers and big-endianness and little-endianness (like perhaps I should be looking for E1FF not FFE1). If you aren't familiar with that, then I really wouldn't suggest you try to interpret the data this way. Either do a temporary file write (which shouldn't really be much overhead) or try and find a Python package that reads exif from jpeg data (I did a quick search and there seem to be a few around).


Reply