In multirotor mode (when CA_AIRFRAME is set to 0), the control allocator module takes the geometry from CA_MC_xxx parameters, builds the effectiveness matrix and invert it to produce the control allocation matrix (see PX4-Autopilot/ControlAllocationPseudoInverse.cpp at d0f89f7fff0328ff92cd62af0506b42b2d31106d · PX4/PX4-Autopilot · GitHub). Then, whenever the geometry is changed (e.g.: motor failure), the control allocation matrix can be recomputed using the same process (failure detection still needs to be implementer, but you can already try to change it via the CA_MC_xxx parameters).