Hmmm. It's quite rare to see this encoding so I don't have much experience with it in FME. What you could try is using the EBCDIC encoding but also setting the reader parameter (in the Navigator window, usually on the left-hand side) called Features to Read > Max Features to Read to 1. Then it will only try reading the first record. That might help at least show if FME is reading the data correctly. I'm hoping it shouldn't fail with memory issues on the very first record (also make sure that the parameter Read Whole File at Once = No).
You're probably using the Text File format reader. You could try a couple of other things: there's a Data File format, which might make a difference. Also you could try using an AttributeFileReader transformer, to read the file into an attribute. From there an AttributeSplitter/ListExploder combo would split the data into separate records.
Finally, I'd be interested in what error messages are occuring. Does FME crash? Or does it log an error in the log window? Also, is the log showing any performance issues (like a message stating it is reorganising memory use)?
That's about all I have, I'm afraid. If you're still having problems, it might be worth logging a support case. I don't see why FME couldn't read an EBCDIC file (in fact I see specific examples that tells me that it should) so it would be useful to let our support team examine the file and see if something is wrong with it (or maybe FME is having problems with the packing).