You’re allowed, but surely you’d be aware enough not to walk in your costume outside the set. The same goes for theatre or anything. An actor who thinks it’s a good idea to walk outside in a nazi costume has it coming, and that’s a good thing.