Note: I don't use FF.

If you look up "portable application" on Wikipedia, I'm sure you can find some starting points to look into how to install FF on a USB stick -- I believe this will also put the cache there.

I've used Opera in that way (but on a network share instead of on USB memory), and it works pretty well.

Of course, your cache size is going to be limited by the size of the memory you choose to install it on, so you might have to tweak the settings a bit. (It shouldn't be that hard.) And if you're doing that, you might as well disable the disk cache altogether, and skip the part where you carry around the browser in your pocket. (There's likely still a memory cache, so it will still be usable.)