The governor cannot be used with the VM34 carb. I would remove it if using the VM style carb. And remember that the stock carb linkage holds the throttle open a bit when starting. Aftermarket carbs need to be given a bit of throttle during starting.
Sounds like an issue with the low / pilot / idle jet. Once the throttle opens a bit and you are into the needle it runs. But if the throttle is closed (like at idle) then you have issues.
First thing I would do is check for air leaks. Get it running and spray some of your spray around the carb adapter gasket surfaces. Listen for a change in RPMs. If there is a leak, your spray should make its way in and change the air fuel ratio. The RPMs might increase. It's possible the RPMs decrease. If the RPMs change in either direction when you spray the gasket surfaces, you probably have a leak.
Next I would try to set the float bowl to specs. Not sure what the specs are, but you should be able to find them for a true Mikuni VM34. There are videos and how to's about setting float bowl heights.
After that, work on the jetting. Starting with the pilot jet.
What jet are you using?
How many turns out is the air mixture screw?
If you turn the screw in, does it improve or get worse?
If you turn the screw out, does it improve or get worse?
Are you using choke (or enrichment) when starting? Does it run better or worse with the choke (enrichment) partially on?
In this case, I suppose improvement means you can keep it running at a lower rpm or you can start it without carb cleaner.
Getting worse means the engine will only run at even higher rpms.
Turning an air screw in will limit the air. This enriches the mixture. If the engine likes it when you turn the air screw in, it may want a larger pilot jet. Turning an air screw out will allow more air. This leans out the mixture. If the engine likes it when you turn the air screw out, it may want a smaller pilot jet. The screw should not be all the way in, nor should it be more than about 3 or 3.5 turns out. My initial thought was that the pilot was too small and not giving enough fuel. You are supplementing the inadequate amount of fuel with started fluid (or carb cleaner). But that is just a first guess.
Engaging the choke / enrichment will richen the mixture up. If the engine likes it better when the choke is partially engaged, then it may want a larger pilot jet.
There are a number of videos available on youtube about tuning the idle circuit on dirtbike / motorcylce carbs that will do a much better job than me.
