Elevation offset seems too much?

Hello all,
I’m making a multi-copter simulation for my Master’s degree. I’m currently moving all multicopters to specific locations inside a warehouse using the offboard mode (kinda like a rendezvous to a known position), but I see that the height is not accurate: when giving 3m as desired height, each drone stops at a different level and is clearly visible in the simulation…
The offset (taken from topics and logs) seems to be in the ±0.5m, and it changes from run to run (well, I guess that’s to be expected)…

I’m just using local NED positions as references, manually transforming the global position I want (which should be fine, I’ve tested it multiple times).
Is there anything I should check? Maybe the estimator is off?
I’m prone to think about the estimator because of the result of some tests I did, like this one where I did not move the copters but the local z position drifts
(note: this is not offboard, it was just a test to inspect the altitude)

As an additional info, I’ve seen that the GPS reference in the altitude estimate is 500m, but I cannot figure out why! Since it’s a simulation, I would expect it to be a somewhat more reasonable value in the spawned environment…