Will it "wiggle" any? If this "adapter pipe" is long enough and fits close enough, it should be good. if not, try making it longer and then put two bolts through it (going through both pipes), one at the top and one at the bottom of this "adapter pipe" that are 90 degrees to each other.
Viola! A fix from outside the box.