At the risk of answering my own question, for unexplained reasons PointOnRasterExtractor massively sped up if I instead fed it data from an intermediate transformer rather than straight from the Reader?

For instance, if I use a Blocker like FeatureHolder in between the DEM Raster Reader and PointOnRasterExtractor, then the same number of Points (77,000) will calculate their Raster Values in 50 seconds!!

That's an interesting observation. If you're able to reproduce it, I'm sure Safe would love to take a look to try and understand why it happens.

Ran the same process on 160000 points and the process took 50.2 seconds.

I also tried using the 'RasterCellCoercer' on the dem to convert to polygon features. I then connected the transformer to an 'AttributeManager' transformer adding new attribute 'Elevation' and 'Attribute Value': '_band{0}.value'. Next I connected this transformer to the 'Spatial Relator' transformer 'Supplier' import port. and connected the points to the 'Requester' input port. Within this transformer I updated the 'Spatial Predicates to Test': 'Requestor is Within Supplier' and expanded 'Attribute Accumulation' and ticked 'Merge Attributes'. This process took 42.6 seconds.

Similarly further testing suggests this may be largely a function of FME WorkBench in 32 bit, in which I was forced to use as the DEM was an ESRI FGDB. There was some weirdness in this though. As above, certain Intermediate Transformers when used allowed the workflow to execute much faster, whereas a direct feed from the FGDB Reader to the PointOnRasterExtractor significantly slowed it down to ~1 hour + other trials/symptoms suggest to me it is memory management related.

With the DEM converted to to FFS and FME WorkBench 64 bit, it is all good. No Intermediate Transformers required, could connect the FFS Reader direct to PointOnRasterExtractor.