There are few phrases as infuriating, as deeply unfair, as 'but why didn’t you leave?'