A cam won't necessarily ruin your engine by itself, but if your engine can't breathe, you won't benefit from it very much at all, and if you have removed the governor and are running stock carb and exhaust, you'll likely end up running lean at high throttle. These engines do that with the STOCK cam.
A cam increases the amount of air and fuel sucked in on your intake stroke. If you have a carb able to provide that air and fuel, and an exhaust capable of getting rid of it, you will see some gains for sure.
The valves will still float at 5400 rpm so if you've removed your governor, that will still act as a crude governing mechanism.
However, that extra air and fuel is going to increase your cylinder head pressure, which will put extra strain on the bottom end components.
So it might work, or it might go bang. Like you said, I don't know if anyone has actually done just a cam before. It's down near the middle of the "what mods to do in what order" list.
From a stock engine, you want to do exhaust, carb, and a governor delete before anything else, and see where that puts you. You'll probably be happy with that, at least for a while. Then start looking at internal components.
