You have to use a template. Just create a blank word document, set you page format and orientation and set the path to it in your Microsoft Word Writer Parameter: Base Microsoft Word File.

Thanks for your help @caracadrian​. This does seem to work for orienting the page, although it's created another problem.

By including a base file, FME requires me to include a placeholder for every type of element that I'm planning on writing to my MS Word file. The documentation for the MS Word writer is terrible, needs work or some examples or something.

You do have to configure the Headings and the Table Styles in you template but you don't have to include placeholders for them.

The first Paragraph is probably a bug but you can set it's font size to 1 in the template to use as less space as possible.