In FME features can have exposed (visible in Table View in the Visual Preview) and unexposed (visible in the Feature Information window in the Visual Preview) attributes.
The attribute you are looking for is fme_dataset, which is unexposed by default. You can expose this attribute by manually add the attribute name in the AttributeExposer or by checking the attribute in the "Additional Attributes to Expose" in the (Feature)Reader.
As an alternative you can use a "Directory and File Pathnames" Reader to scan for folders / files and use the path_windows attribute as input for the next reader. FeatureReaders can be chained together. This is the way I prefer working.
Also, personally, I prefer to have the results in a different window, on a second screen. This can be switched on with the checkbox "Inspect with Data Inspector when Visual Prevuew window closed" which you can find via Tools, Options, tab Workbench.