In the race to implement and deploy AI in our organizations, we risk making the same mistakes so many gamers do in their battles for the top spot. I say that with some authority: I’ve always been a gamer. Real-time strategy games like Age of Empires, StarCraft, and Command & Conquer; online battle arenas like DotA and League of Legends. I had a competitive streak, too, pouring countless hours into mastering their intricacies and sharpening my strategies. After all those late nights, the analogies between gaming and the emerging world of AI are hard to ignore.
In years past, a common focal point for gamers trying to elevate their skills was actions per minute or APM: the pace of actions executed by the player. This was often seen as a proxy for player skill, as more advanced players could better handle the cognitive rush of commanding dozens of units across multiple fronts, presumably leading to more effective play, eventually overwhelming the opponent.
This metric became so popular that it became a target in and of itself. Novice players would look to the recorded APMs of expert-level players and set that as a goal to be achieved. And that’s exactly when it lost its value, a perfect example of Goodhart’s Law: when a measure becomes a target, it ceases to be a good measure. Merely pumping up one’s APM by frantically clicking around the game did nothing to help you win, but could certainly generate the illusion you were improving.
