FME is a set of tools and functions to process and manipulate data. To take it to an extreme, FME is like a specialized carpenters tool belt, whereas AGOL is more like a home handyman's shed/garage. The carpenter has a very specific set of specialised tools and skills to produce furniture. In a home handy mans shed, you could still build a piece of furniture, fix up a car, play darts etc - the later you can't do with a carpenters tool belt
Taking this back to FME vs AGOL, FME's focus is data processing and manipulation (more than just spatial data). AGOLs focus is GIS (Spatial), it has some of the same tools that FME does, but a much broader focus
FME Server and AGOL are two very different beasts, with little or almost no overlap in functionality. I would be very surprised if Safe Software were to go into direct competition with AGOL in the web mapping space.