Simulink SITL erratic behavior

I am using the following interface to connect my Simulink plant model to PX4 v1.12.3 in lockstep:

However, regardless of my PID gains, my quadcopter makes erratic movements and crashes in a few seconds. I tried running my model in 250 Hz as suggested in the interface setup, tried several combinations with my model running at a lower/higher frequency. Nevertheless, I couldn’t make any progress.

My setup only uses 1 Windows machine with everything running on it. I established connection on UDP.

Below is a flight log in which the behavior can be seen after 20 secs (Note that I added a switch in my Simulink model that feeds the same PWM to all motors in the first 20 secs to take off and eliminate my landing gear forces):