I think if it's just 4 motors you can get away with just using a 4X configuration, as it places the motors in the same location (just the structure getting there is different). If it's 6 or 8 motors I'm not sure other than to try and see what happens. Trying to work that part out myself, got a 8 motor tilt rotor, H config (4 front wing, 4 back wing) and currently treating it as a 4X.
Edit: Go to Firmware/src/lib/mixer/geometries/
I haven't tried myself, but it looks like you can manually set the positions for each motor. Hope that helps