Anu Choudhury is the Chief Technology Officer of Reputation, a global leader in reputation intelligence.getty​All software companies are thinking about AI right now. If not, they probably won’t be around much longer. But how many are thinking about it the right way?There’s the obvious stuff: How do we use AI to generate and test code more efficiently? How do we release products to our customers faster? Those are real benefits that matter tremendously, and they’re fun to think about. But they’re the easy part. The hard part is the data, and most companies aren’t anywhere near where they need to be in that department.Here’s the thing about legacy software companies: Over the decades, they’ve accumulated enormous amounts of data, and they tend to view that data as a competitive advantage—their pot of gold in the AI era. If only it were that easy. The reality is far more complicated.The problem is that most software companies host data in outdated, siloed architectures. Typically, this data is inconsistently labeled and poorly cross-referenced, which means it’s not an asset. In fact, it’s often a liability. If you try to build AI on top of a broken foundation, just see what happens. You’ll likely end up with products that look great in a demo but fall apart in production.There is also a regulatory dimension that doesn’t get enough attention. Siloed legacy data is actually a major compliance risk, and that risk gets exacerbated with AI. Models trained on incomplete or noncompliant data create downstream exposure. It may take months or years to appear, but when it does inevitably rise to the surface, it’s very expensive to fix. That's why it’s so important to get AI right at the architecture level.The Winning StrategyMany software companies still think the best plan of action is to move at a breakneck speed and bolt AI onto whatever they already have. But that’s not the case at all. The ones that want to get ahead have to fix their data foundation first.This is increasingly true as AI models become commoditized. Anyone can now access state-of-the-art capabilities through an API. But what that API will never provide is clean, rich, domain-specific data. That’s the real moat. Bridging over it takes time, discipline and a willingness to do the unglamorous work first before embarking on the more exciting stuff that will naturally follow.The good news is you don’t have to entirely finish modernizing your platform before you start leaning into AI. In fact, the modernization process itself should rely heavily on AI. The people who built your legacy systems in the first place may have retired or moved on to other organizations. Much of their institutional knowledge has simply vanished.AI can help you fill in those gaps. You can use it to write code, modernize your services and get your platform into a state where you can build on top of it. Once you’ve done that, you can go further. For instance, you can use AI to build proofs of concept and develop full features and products from the ground up.The Barriers To SuccessOf course, this is all easier said than done, and there will be real challenges along your journey.From where I sit, one of the biggest challenges is culture. Engineering teams have built their careers around writing code. That’s their bread and butter, and it’s what they were trained to do best. And now they’re being asked to work in a totally different way, where they’re no longer the ones actually writing the code. Let’s face it: That’s a very big ask.Leaders need to help people understand that AI isn’t replacing them. They’re still the ones who need to do the bulk of the thinking and planning. In my opinion, there’s an 80/20 rule of AI: It can handle the initial 80% of the work, like the scaffolding and more routine tasks, but that last 20% is where humans must be involved—that's where the real magic happens.The next challenge, as we discussed earlier, is data. If your data is scattered across silos and disconnected systems, you have a problem. The quality of what AI produces is only as good as the data you feed it. You know the old saying: Garbage in, garbage out. Getting your data cleaned up and ready for prime time is a major hurdle, and it only gets harder the longer you wait.Then there’s the fact that AI tools are changing and improving so fast. Often, by the time you get comfortable with a tool, it’s been replaced by a better one. You can’t assume what you learned six months ago is relevant today. That’s why you need to build a culture that’s willing to embrace change and isn’t afraid of continuous learning.The Benefits Of An AI-Ready PlatformSo, what’s the ultimate payoff in all this? One huge benefit is feature velocity. New product features that used to take months or quarters to develop can now be completed in weeks.Of course, it’s still on you and your team to understand and plan around specific customer problems. The last thing you want is to spend time and tokens churning out features that aren’t meaningful to customers. You can quickly spit out a cool new feature, yes, but if it’s not useful to your customer, it will have little value or utility.Then there’s quality control. When you use AI to catch glitches, there are fewer problems with your data and fewer bugs in your code. As a result, customer confidence can grow because customers themselves encounter fewer problems with your software.ConclusionSo, will software companies that don’t modernize their legacy platforms survive? Probably not. They’ll never be able to deliver at the same velocity or match the operational efficiency of their competitors. The market and their customers won’t wait. They’ll move on. Get AI right, and maybe they’ll move on to you.​​Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?