Your picture shows a very cockeyed position of the band brake itself, halfway off center at the bottom, with the pivot bolt bent downward at the top, and looks like band itself might be twisted. Never work right like that. Level out/reinforce the pivot bolt, twist the band in a vise or something as needed to get it as squared as a wheel. If twisting off as brake band is pulled, then the pivot bolt isn't filling up the space in the pivot hole end of the band, allowing for it to flop around on the bolt. Use a fatter bolt or whatever is needed to make the band swing as squarely as a door hinge on your house. Just make sure everything is squared up first.