AI: Crunching vast amounts of data in quick time
| Photo Credit:
Foryou13
The period between 1950 and 1980 saw a burgeoning of three intellectual disciplines. One was sociology, the second was political science and the third was economics.But by the end of the 20th century sociology and political science had lost their attractions and turned into a broad river called anthropology. That river is a nebulous set of pursuits. It doesn’t have many takers because of the absence of intellectual rigour underpinned by logic but there are opinions propped up by in attempts at weak empiricism.Economics, however, managed to survive largely because of the massive increases in data collection and computing power.These two enabled opinion and theory to be tested. People, who were actually no more than people who knew how to ‘mine the data’ started to rule the roost of economics.Nothing wrong with that, you might say and quite rightly, too. But here’s the problem: if all that economists do is to look for patterns in the data, why not use AI instead? It will be more accurate, fast and bias free.I fear, therefore, that economics is about to go the way of sociology and political science. It’s very likely to suffer an identity crisis from which it never recovers.The transition started happening in the late 1970s when the politicians and aid agencies of the West started demanding more data before they’d unbelt more money. Put your data where my money is, they said.So economics went from comprising ‘intellectual puzzles’ as a Nobel winning economist once described it, to a pretence that it really was a science that uses ‘scientific’ methods to validate theories.Data, data, dataThis forced governments to collect more data and statistical methods to grow evermore sophisticated. The new term of certification was ‘robustness’ to say that the data had been put through all the washing machines and driers and was now final proof. Except of course that it wasn’t because accursed humans don’t behave consistently.But, if I may invent a term, this is how ‘post-modern macroeconomics’ was born. Then in the last decade, along came big tech where vast amounts of micro data started gathering in mainframes, awaiting analysis.Both processes are now reaching maturity and, with the coming of LLMs, the job will get done. Economists, as they are now, I would say would have become redundant unless they are trained to ask the right questions. This can be very hard if your knowledge of basic theory doesn’t measure up.You only have to read the economics research of the last three decades to see how the obsession with data has left economics with thousands of unprovable conditional statements drawing trivial conclusions.Please don’t get me wrong. I am not denying the importance of data. It is important for testing a theory and for detecting patterns. But the fact that it is necessary does not mean that it is sufficient. Far from it.The big challengeThis is where modern economists face the biggest challenge. What they do now is mostly the necessary data crunching which, it turns out, can be done more efficiently by machines. But unless economists ask the right questions, the exercises will not be sufficient.And all these questions have to do, in the end, with causality: the why — and not just the what, which is what data collection and analysis mostly does.For example, it has been obvious for many years now that group behaviour and individual preferences play a large role in determining economic outcomes and that they mostly diverge. The former forms the basis of macroeconomics and the latter of microeconomics.So let’s ask: what’s the overarching question today in macroeconomics? Let’s take the Keynesian identity that justifies massive income support. But how do you apply it when capital and labour are both in excess supply and government debt globally, too, is excessive?If we leave out the well known cause of excess labour, the excess supply of capital and excess government debt need to be analysed by asking which has caused which.Then when some theory is postulated it can be tested. Excess, by the way, can be defined as a factor earning extremely low or zero returns.And what of microeconomics? This is relevant largely for market structure analysis which is currently a completely neglected area of research.No one seems to be using the huge mass of data that is now available to come up with new, or any kind of, theories to ask what the optimum size of an oligopoly should be.There are lots of such questions that ought to be bothering economics and economists. Sadly, both the discipline and its practitioners are unaware that by their complacence they are floating even further away into intellectual irrelevance.Published on July 6, 2026
