More brokers throwing in the towel