I don't think that Gantz cares for what people do, except when it interferes with his little killing aliens' game.
Also, I don't think that people can hurt him with his own weapons. Maybe with a knife, I dunno... But then, he can most likely regenerate himself.
But again, is Gantz really the man in the Ball? Why is he there unprotected if he doesn't want to be hurt?
Nah, the guy inside is probably an organic component to Gantz himself, used to tune up the machine and to give some organic feedback, like heat and sound. I think Gantz is the Ball itself.