Estimator's z position does not represent actual height during flight at all times

Hi all,
We are having a problem during altitude flight mode. In the log file the z-estimated seems to follow the z-setpoint within a range of about 8cm. However, during the flight we noticed that this offset seemed actually way higher. When we checked the raw data of the range sensor we use (LidarLiteV3) during a flight in QGroundControl, this showed an offset of 40cm at it’s worst. When we check the local_position.z during a flight, this shows the 8cm offset we see back in the log files. Which is clearly not correct when we observe the drone in the air.
We suspect that there is something going wrong in the estimator. However, we don’t have a lot of knowledge about this. Does somebody have an idea what could cause this problem?
The log file of a flight in altitude mode can be found here:
Thanks in advance!