You should be able to read the csv by just appending the path to the csv after the zip path. Something like this:
path/to/zipfile.zip/path/to/csvfile.csv
Have you tried that?
Heres the doc: https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Desktop/Workbench/archive_files_reading.htm
Thanks for your reply @virtualcitymatt. That was what I thought but I am guessing that this might be a bug on the feature reader. Let me clarify, I could read the file but I was in some sort of binary format and didnt recognize the file schema or data and caused my workbench to run for more than 30 mins when it should have taken 3-4 tops.
I was able to modify the python caller on the zip extractor to only extract the csv file. It took some digging around on the python libraries to figure this out particularly since I dont know python.