Reinforcement Learning for Trade Execution with Market Impact

2025-07-08

AI Summary

This paper addresses the optimal trade execution problem in a limit order book (LOB) under market impact, aiming to maximize expected revenue. We propose a reinforcement learning–based dynamic order allocation framework that jointly determines the type (market vs. limit), quantity, and price level of orders. A key methodological innovation is the use of a multivariate logistic-normal distribution to model the stochastic order allocation policy, enhancing both exploration efficiency and policy trainability and generalizability. The approach is evaluated in a high-fidelity LOB simulator integrating noise traders, tactical traders, and strategic traders. Empirical results demonstrate significant outperformance over conventional benchmark strategies across all key metrics: execution cost, fill rate, and net revenue—achieving substantial improvements in each.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Reinforcement Learning
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

In this paper, we introduce a novel reinforcement learning framework for optimal trade execution in a limit order book. We formulate the trade execution problem as a dynamic allocation task whose objective is the optimal placement of market and limit orders to maximize expected revenue. By employing multivariate logistic-normal distributions to model random allocations, the framework enables efficient training of the reinforcement learning algorithm. Numerical experiments show that the proposed method outperforms traditional benchmark strategies in simulated limit order book environments featuring noise traders submitting random orders, tactical traders responding to order book imbalances, and a strategic trader seeking to acquire or liquidate an asset position.

Problem

• Optimal trade execution in limit order books
• Dynamic allocation of market and limit orders
• Maximizing expected revenue under market impact

Innovation

• Reinforcement learning optimizes trade execution
• Multivariate logistic-normal models random allocations
• Outperforms benchmarks in simulated order books