Local coordinates of the raster to the geometry.

You can try the RasterGCPSetter to georeference the raster. I think that might resample it though so that would mean your area of interest is off. Probably best to extract that first from the original raster (set other pixels to nodata) but I'm not quite sure how.

The AffineWarper will distort and warp a raster, but you may find you don't actually need to do that. Possibly the RubberSheeter may work better.

You have already figured out the real-world ground extents of the raster, so if you set up four control vectors to warp the four corners of the raster, from photo-space to real-space (first vertex of control vector is corner of photo, second vertex is the coordinates of that location) then that transformation can be used to warp everything: your raster, its polygon extent, and the area of interest polygon to the correct real-world location.

If you have many of these ground geometries for the images and need the FME process to derive those affine warp vectors, I have some ideas.

Have a look at what I've attached. It doesn't work exactly as I had hoped but might get you closer.

My idea was to generate the warp by centering the image over the target location first, then fitting the image to the nearest corner. It relies on the fact that images are rectangles.

Basically, AffineWarper applies a single warp which has the effect of preserving the original shape to some degree, whereas RubberSheeter warps each vertex independently which will distort the original shape to fit the warp exactly. But the RubberSheeter doesn't produce a usable result for the area of interest, because all the control vectors are for the base image polygon.

Could you explain a little further as to what the above workspace was not able to achieve? Also can you share your data and workspace/workspace template if possible?