If you search the Knowledge Base for articles about CityGML you can find some tutorials including sample workspaces. Something like:

BIM to GIS (Basic) | IFC LOD 100 to LOD 2 CityGML
Creating a Textured CityGML Model
 

I have zero experience with CityGML but this is a good place to start.

There is also this template workspace here: https://hub.safe.com/publishers/con-terra-lab/templates/citygml-create-building-lod1

You can open it up in FME and see how it works - the input data here is shapefile and there are some nice comments in the workspace so you can see what happens at each step.

This workspace, I think, just puts the bottom of the Buildings at 0 elevation - depending on your application it might be important to consider the terrain height in order to put the buildings correctly in 3D Space. But that can be step two. 

This template is a pretty good starting point to get you going

Thank you! I’ll check it up.

I am quite new to FME so I literally don’t have any idea how to operate the software. This is a big help. again, thanks!

Heh, Well while these are super old video it should give you a pretty quick overview of what is possible with FME and how it should look - videos less than 5 mins each: 

 And another old video which is a little more hands on: 
 

To get more into FME you can check out the FME Acadamy which is full of free on demand courses https://academy.safe.com/