While some are working out how to get rid of the prime minister, others are already plotting against his rival