That is a tough place to grind, and getting a hole saw lined up would be difficult. I never thought about welding though.
It's tempting to weld the old bearing in, problem is, may never get it back out!
I'm liking the shim idea though, interesting ideas, could use a flapper wheel on the inside...