Hello FME'ers,

Here is the chosen Question-of-the-Week. As always, to view these regularly, hit the Follow button in the top-right of the forum.

This week, I'm looking at a question about JPEG files and how to get their info from an online blob storage...

Question of the Week

In this question, user @alex.west​ asks...

Q) I am trying to retrieve lat/long/rotation attributes from jpegs sitting on Azure but can't get it working. Attachments coming from AGOL or Azure come in binary format; which makes exposing jpeg's metadata tricky. I have no problem getting lat/long/direction if jpegs sit on my harddrive.

A) I'm assuming that the metadata you need are what we call exif tags. It's true that if you read a JPEG file directly, then you get these tags as attributes. But if you read the file as a binary attribute, then it's not as simple. So let's see how we can do that, plus how my new favourite transformer - the TempPathnameCreator - will help us.

My Scenario

I don't have an Azure setup I can quickly access, so I'm just going to use a file that I found online, specifically this one:

The exact URL for this is: http://images.safe.com/photos/safe-event/fmeuc-2017-don-dale.jpg

I can read that with the JPEG reader easily enough, and that returns the following tags:

Hmmm. Although it's supposed to be from the FME UC, 2017, the exif datetime is 2019. Looks like the marketing team are engaging in some "fake moon landing" type of conspiracy here. Either that or Windows Photo Editor updates the exif datetimes.

Anyway... let's say that for some reason I couldn't read the file directly. I could only get it returned as a binary "string" from a web service.

How can I get the exif tags from that?!

The RasterReplacer to the Rescue?

Alex tried using the BinaryDecoder transformer to pull information out of the binary string. I think the transformer you need is actually the BinaryEncoder, setting Hex as the destination.

That certainly could work - the metadata is encoded in there somewhere - but you would need to be really familiar with the JPEG format to know where.

Ordinarily, if I wanted to convert a binary string to a proper raster, I'd use the RasterReplacer. That does work in this scenario, except for one little thing: it doesn't recreate the exif tags!

I've filed an enhancement request, but that doesn't help us today. So what to try next?

AttributeFileWriter and FeatureReader

Because the JPEG reader will return this information, the obvious thing is to use the reader. How do we do that mid-translation? With a FeatureReader transformer of course. But the binary attribute needs to be written to a file first, and we do that with the AttributeFileWriter:

Now I've got the JPEG back with the exif tags as format attributes, so we're all good.

Where Does the TempPathnameCreator Come In?

I need to write the raster file and read it back. But there are various shortcomings with doing that. Firstly, if I repeat the task multiple times I could end up with multiple files cluttering my disk. In other words, I'd need to do a manual cleanup.

Secondly, if I transfer the workspace to Server - or to another Desktop machine - I've no guarantee that the path I choose will still exist (or that I'll have permission to write to it).

So this is where the TempPathnameCreator comes in!

The TempPathnameCreator transformer creates me a temporary path and filename on my computer. It uses the FME_TEMP location so I know that it will always exist, and that I will always have permission to write there; even on Server.

Additionally, because that is a temporary location, FME will automatically clean any files that I create once the session is complete. In other words, when the workspace finishes, the last thing FME does is remove any temporary files.

So I simply add a TempPathnameCreator before the AttributeFileWriter:

...and set the AttributeFileWriter and FeatureReader to use the _pathname that it creates. On my Mac computer it created: /private/tmp/159863673299_90078_d/tor930

That file no longer exists once the workspace is run (and I'm really not clear about how a partial run with Feature Caching will affect that) but I've already read the data, and presumably then processed it, so I don't mind.

I know this works on Server because I had cause to use it this week for a new hub template I created: the SundialCalculator. I had to write the image to a temporary file because the HTMLReportGenerator only adds an image from a file, not a feature or attribute. So my workspace looked like this:

You can find it on the hub, and if you want to create your own working sundial (I'm a bit obsessed with them) then give it a try. It's also a great example of using the Geometry parameter to select a location from a map, and of generating a HTML report in response to an action.

Other Notable Questions

Here are a couple of other questions I found of interest this week:

• Can you set an admin password on Server using the API, asked @skime​ 
  • Yes, says @richardatsafe​. At least, you can avoid having to do that by using the keyword FIRSTLOGINCHANGEPASSWORD=false during a silent install, which deals with the user's question nicely.
• @jseigneuret​ asked why their PostGIS connection failed.
  • They then diagnosed their own problem: an accented character in the connection name. I mention this in case anyone else comes across this issue, but I'll also file it with our developers to see if that's fixable.
• User @checcosisani​ wondered how you can update an NLP model.
  • The short answer is... you can't. NLP is Natural Language Processing. You create a model by training FME to recognize various words in a "corpus" of source data. That model is then used to assess further data that hasn't yet been categorized. However, as far as I know, you can't update that original model with more information. You have to create it from scratch again, incorporating the new information into the original corpus.
• Can you access detailed billing information for FME Cloud through the API, asks @antoine​ 
  • Frankly, I don't know. Do you? If so, check out the question and provide your answer. Fame and fortune - well, reputation and badges - await the correct answer.