I just made a minor Modification to the Code in this Part that detects the Serial Port an will stay in Bootloader Mode (not yet in production Code).
As I had the opposite Problem, my Board was staying in Bootloader mode as the RX Line was pulled low by unpowered Equipment, which caused this. The Code will watch for at least 3 Cycles of LOW RX (Break Signal) which triggers the Bootloader. Problem is, that this is only checked in a small Window about 200ms long.
You should check which Version of the Bootloader is running ("Version X") during FW Installation in QGC with USB Connection. What Board are you trying to flash and why is it not possible to do via USB?