I had the same thing with my thunderkart...it was good at slow speeds, but twitchy and hard to keep straight when going fast. Had to keep a death-grip on the wheel, lol.
I'll bet you've got either zero, or negative caster. Take a look at your kingbolts (the ones going through the spindles, indicated by the yellow lines in the pic). It may be the camera angle, but it kinda looks like the left one is leaning forward (negative caster), which is bad.
You really really want them to lean backwards relative to the ground (positive caster) so that the top of the bolt is behind the bottom of the bolt. Anywhere from 10 to 15 degrees works. This makes it more self-centering and stable at speed.
All I did to fix mine was heat the arm where I circled in the pic, then twist it back with some large channel-locks.
Made a WORLD of difference!