The disc floats like that, as to allow the caliper grab at whatever angle the disc maybe at. It's not so much a problem that it might allow the key walk out, then you will have problems.
I don't think you need to take off the axle to remove the caliper. If the disc floats already, just remove the axle covers, and whatever stops for the disc. Slide the disc and caliper over to remove, install in reverse.