Unraveling Market Mysteries One Equation at a Time

Using Greek letters, numbers, and equations to describe the world is strange. Before I embarked on a near decade-long academic pursuit in the field of theoretical physics, I distinctly remember thinking about this peculiarity while sitting in my high school physics class staring at Newton’s laws of kinematics. It really is strange that we have specific rules for combining funny-looking letters, and near miraculous that those specific combinations tell us – with an impressive degree of accuracy – how the world around us behaves. It has always fascinated me what we – humans, that is – have been able to do by building on those relatively simple primitives. One way or another, it all boils down to the “unreasonable effectiveness of mathematic(al)” abstractions of an underlying physical reality. (Wigner, 1960)

In the world of economics, however, things are different. While quantitative finance and economics have built an impressive repertoire of mathematical jiu-jitsu to describe financial markets, the prevailing theories and models are – at their core – abstractions of an already abstract underlying substrate: markets, money, and assets. This is why quantitative economic models often need to be taken with a sizeable grain of salt. It’s easy to lose oneself in the technical details of an increasingly complex field of literature where cutting edge results can tend to look like an article on pure math, exhibiting a “degree of rigor that is inversely proportional to their minimal usefulness” – to quote the esteemed ex-Goldman quant Emanuel Derman.

However, the field of quantitative finance has certainly had its wins. Chief amongst those would likely be the impressive strides made in understanding derivatives pricing and its use in effectively diffusing risk throughout the financial network. That being said, you can’t simply throw complex math at any problem and expect it to work.  In the emerging field of microstructure theory, however, it seems that the quants may once again have found their day in the sun.  Over the past many years, the study of financial order books has led to some surprisingly impactful and enlightening realizations about the underlying dynamics of markets that bear an interpretable – albeit complex – mathematical underpinning. For instance, first-principles derivations of the seemingly ubiquitous square-root impact law from the dynamics of an order book (Benzaquen & Bouchaud, 2018) or finding that the so-called “zero-intelligence” model of investor behaviour can lead to a market that appears “efficient” (in the statistical sense) (Smith et. al., 2006) both serve to provide an improved understanding of how high-level market behaviours emerge from a more granular scale.

One reason why this specific field has been fertile ground for quantitative analysis is that transaction-level data serves to be about as granular as you can get in finance. This data is still, however, referred to as macroscopic, as it emerges due to the behaviour of microscopic actions (the trading behaviour of market participants). That being said, if there are any data that will give us a microscopic understanding of why prices change, it is hidden in the order books. As stated by Fabrizio Lillo in a recent APS article, “the law of supply and demand posits that an increase in supply lowers prices, while an increase in demand raises them, yet the inner workings of this mechanism are partially obscured.” He goes on to state that while there are many theoretical frameworks to explain exactly how this works, to gain a truly scientific understanding of the process, the results need to be backed by empirical validation.

For instance, one of the puzzling observations that has come out of the study of transaction-level data is the observed long-term correlation in order flows. Ultimately reflecting changes in supply or demand, this observed phenomenon implies that when a market participant submits an order to sell an asset, it is likely that more sell orders will follow, and similarly for buy orders.[1] This long-range correlation has proved to be somewhat of a mystery in the microstructure literature. (Lillo & Farmer, 2008) While there have been several theoretical models to explain its origin, one of particular note is due to Lillo, Mike, and Farmer (2005), which claims that the long-lived correlation in order flow is due to the impact of large participants in the market executing big orders through a series of chunks, known as a metaorder. (Lillo et. al., 2005) Furthermore, an important aspect of the Lillo-Mike-Farmer (LMF) model is how it quantitatively links the timescale of order flow imbalances to the size of market participants, making a strong claim about exactly how a macroscopic – readily observed – quantity (order flow correlation) is related to the microscopic – unobserved – quantity (distribution of market participant sizes).

While some of Lillo and Farmer’s subsequent work shows agreement with market data on the London Stock Exchange (Lillo & Farmer, 2008), it has always been difficult to tease out whether or not their theory is correct, given that publicly available data only yields the transactions themselves, not the identity of the participant who transmits each order. However, the LMF theory has recently been given new legs, as a research group from Kyoto University in Japan has interrogated the quantitative predictions of the LMF model in a novel dataset from the Tokyo Stock Exchange, giving the authors the ability to identify orders that originate from the same market participant. (Sato & Kanazawa, 2023) Ultimately, the authors find excellent quantitative agreement between the theoretical microscopic model and the order book.

While this result is satisfying to a theorist, there are also very interesting practical implications. First, the link between so-called microscopic and macroscopic quantities allows us to infer the microscopic terms in other markets where we do not have access to the detailed dataset used by Sato & Kanazawa (2023). Furthermore, the microscopic parameters provide additional clarity on the true depth of the order book. For instance, from these parameters, you might be able to detect the presence of large participants that could cause market fragility. Put another way, they allow you to estimate the true liquidity of a market in a much more precise and nuanced way than conventional metrics, such as bid/ask spread, market depth, or observed market impact. This work takes an exciting step forward in the field of market microstructure, giving validation to a theory that makes strong quantitative predictions about the nature of markets. Ultimately, this work provides a relatively unique example of how the true force of scientific rigor can be effectively applied in the financial world and serves as a promising field of research to follow.

References

• Wigner, E. (1960). “The Unreasonable Effectiveness of Mathematics in the Natural Sciences”, Communications in Pure and Applied Mathematics (13).
• Benzaquen, M. & Bouchaud, J.P. (2018). “Market Impact with Multi-Timescale Liquidity”, Quantitative Finance (18).
• Smith, E., Farmer, J. D., Gillemot, L., & Krishnamurthy, S. (2006). “Statistical Theory of Continuous Double Auction”, Quantitative Finance, (3).
• Lillo, F. & Farmer, J. D. (2008). “The long memory effect of the efficient market”, Studies in Nonlinear Dynamics & Econometrics, (8).
• Lillo, F., Mike, S., & Farmer, J. D. (2005). “Theory for long memory in supply and demand”, Physical Review E. (71).
• Sato, Y. & Kanazawa, K. (2023). “Inferring Microscopic Financial Information from the Long Memory in Market-Order Flow: A Quantitative Test of the Lillo-Mike-Farmer Model”, Physical Review Letters, (131).

Footnotes

[1] To the astute reader, this may seem to fly in the face of market efficiency, which is a widely discussed topic. However, the reader can be put at ease, because – as pointed out by Lillo & Farmer (2008) – the predictability of prices due to the long-range correlation is offset by other factors, such as the relative size of the buy and sell orders.