RC commands are not immediately reflected on flight behavious

Sometimes, it takes a couple of seconds to respond.

While taking off, I increased throttle above 50% and after 3 seconds it took off.

And while in mid-air, it started drifting even though it is in position mode. Tried to perform correction but too late as it rammed into a tree.

Did anyone face a similar issue? I noticed this won’t happen if I arm and disarm a couple of times before take-off.

Using the latest firmware version.

Thanks!