PixRacer Offboard Control ESP8266 MAVSDK Python

Very new user here.
I want to control a drone using a pixracer by sending it velocity setpoint commands over MAVSDK.
http://mavsdk-python-docs.s3-website.eu-central-1.amazonaws.com/plugins/offboard.html

My overall setup is a pc receiving OptiTrack motion capture information via the NatNetSDK python version. The pc connects to the drone via a wifi adapter to an ESP8266 flashed with the latest firmware. The drone is using a pixracer v1 that also has the latest firmware.

Instead of the pixracer receiving the mocap data and position setpoints, I want to use an offboard LQR controller that just sends in velocity commands.

To do this I think I need to communicate over mavlink with the MAVSDK. I can get MAVSDK running, just not connected to the drone. I am using a windows system so I have to use

to get it running I believe. The command I am using to connect is
drone = System(mavsdk_server_address=“localhost”, port=50040)
I am setting up the server using
.\mavsdk_server_win32.exe -p 50040 udp://:14540

I am also connecting to QGroundControl through the same ESP8266 module. I am using the same computer for QGC as well as the MAVSDK connection.

I am not too familiar with wifi connections but I think this should be possible since it should use different ports but the same connection?

With QGC using port 14550 and the offboard control using port 14540.

I am not building my own code so I am trying to make do with out using any dev tools.

I think my issue is with the way I am setting up my parameters, but I am unsure.
Another difficulty I will face is simply getting into offboard control mode as that requires a GPS/mocap information so I may need to figure out how to send that with MAVSDK as well.

Let me know if there is any more information I need to provide.