When I switch computers and open up the same workspace I get errors on the web connections. So I have to reconnect to the same AGOL feature service but on a different computer.
You don't.
When you create a web connection, you have the option to make it private or public.
The information is stored in a file (SQLLite database).
The location of that file is per default on your own computer, but that can be changed to a network location, using these settings for connection storage:
If you use a network location, you can reuse the same web connections on all machines.
Hope this helps.
That's so if you pass the workspace to a colleague, they can't use your connection details to log in; they have to provide their own. It just happens here that you are passing them to yourself on another machine, but FME can't know that.
But if you have lots of these you should be able to export them from one machine and import them to another (go to Tools > FME Options > Connections, right-click in there, and choose Export).