DeepScalper: A Risk-Aware Deep Reinforcement Learning Framework for Intraday Trading with Micro-level Market Embedding
Auteurs : Shuo Sun, Rundong Wang, Xu He, Junlei Zhu, Jian Li, Bo An
Résumé scientifique
Reinforcement learning (RL) techniques have shown great success in quantitative investment tasks, such as portfolio management and algorithmic trading. Especially, intraday trading is one of the most profitable and risky tasks because of the intraday behaviors of the financial market that reflect billions of rapidly fluctuating values. However, it is hard to apply existing RL methods to intraday trading due to the following three limitations: 1) overlooking micro-level market information (e.g., limit order book); 2) only focusing on local price fluctuation and failing to capture the overall trend of the whole trading day; 3) neglecting the impact of market risk. To tackle these limitations, we propose DeepScalper, a deep reinforcement learning framework for intraday trading. Specifically, we adopt an encoder-decoder architecture to learn robust market embedding incorporating both macro-level and micro-level market information. Moreover, a novel hindsight reward function is designed to provide the agent a long-term horizon for capturing the overall price trend. In addition, we propose a risk-aware auxiliary task by predicting future volatility, which helps the agent take market risk into consideration while maximizing profit. Finally, extensive experiments on two stock index futures and four treasury bond futures demonstrate that DeepScalper achieves significant improvement against many state-of-the-art approaches.
Les articles autour de lui, par date et par proximité
Plus haut, c’est plus proche de cet article ; à gauche, c’est plus ancien. Les traits relient les articles proches entre eux : les groupes de traits sont des sous-thèmes. Sélectionnez un point pour voir pourquoi il est là. Retour au résumé de cet article
Les articles autour de celui-ci
Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.
Les articles les plus proches
-
FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative FinanceXiao-Yang Liu, Hongyang Yang, Jiechao Gao et al. · 2021 · ACM International Conference on AI in Finance, 2021 · cité 189 fois · Résumé IA disponible
Résumé scientifique
Deep reinforcement learning (DRL) has been envisioned to have a competitive edge in quantitative finance. However, there is a steep development curve for quantitative traders to obtain an agent that automatically positions to win in the market, namely \textit{to decide where to trade, at what price} and \textit{what quantity}, due to the error-prone programming and arduous debugging. In this paper, we present the first open-source framework \textit{FinRL} as a full pipeline to help quantitative traders overcome the steep learning curve. FinRL is featured with simplicity, applicability and extensibility under the key principles, \textit{full-stack framework, customization, reproducibility} an…
-
Deep reinforcement learning with positional context for intraday tradingSven Goluža, Tomislav Kovačević, Tessa Bauman et al. · 2024 · Evolutionary Systematics · cité 9 fois · Pas encore résumé
Résumé scientifique
Deep reinforcement learning (DRL) is a well-suited approach to financial decision-making, where an agent makes decisions based on its trading strategy developed from market observations. Existing DRL intraday trading strategies mainly use price-based features to construct the state space. They neglect the contextual information related to the position of the strategy, which is an important aspect given the sequential nature of intraday trading. In this study, we propose a novel DRL model for intraday trading that introduces positional features encapsulating the contextual information into its sparse state space. The model is evaluated over an extended period of almost a decade and across var…
-
DeepTrader: A Deep Reinforcement Learning Approach for Risk-Return Balanced Portfolio Management with Market Conditions EmbeddingZhicheng Wang, Bi-Wei Huang, Shikui Tu et al. · 2021 · AAAI Conference on Artificial Intelligence · cité 187 fois · Pas encore résumé
Résumé scientifique
Most existing reinforcement learning (RL)-based portfolio management models do not take into account the market conditions, which limits their performance in risk-return balancing. In this paper, we propose DeepTrader, a deep RL method to optimize the investment policy. In particular, to tackle the risk-return balancing problem, our model embeds macro market conditions as an indicator to dynamically adjust the proportion between long and short funds, to lower the risk of market fluctuations, with the negative maximum drawdown as the reward function. Additionally, the model involves a unit to evaluate individual assets, which learns dynamic patterns from historical data with the price rising …
-
Deep reinforcement learning on a multi-asset environment for tradingAli Hirsa, Joerg Osterrieder, Branka Hadji-Misheva et al. · 2021 · Trading and Market Microstructure · cité 9 fois · Résumé IA disponible
Résumé scientifique
Financial trading has been widely analyzed for decades with market participants and academics always looking for advanced methods to improve trading performance. Deep reinforcement learning (DRL), a recently reinvigorated method with significant success in multiple domains, still has to show its benefit in the financial markets. We use a deep Q-network (DQN) to design long-short trading strategies for futures contracts. The state space consists of volatility-normalized daily returns, with buying or selling being the reinforcement learning action and the total reward defined as the cumulative profits from our actions. Our trading strategy is trained and tested both on real and simulated price…
-
FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative FinanceXiao-Yang Liu, Hongyang Yang, Qian Chen et al. · 2020 · Trading and Market Microstructure · cité 213 fois · Résumé IA disponible
Résumé scientifique
As deep reinforcement learning (DRL) has been recognized as an effective approach in quantitative finance, getting hands-on experiences is attractive to beginners. However, to train a practical DRL trading agent that decides where to trade, at what price, and what quantity involves error-prone and arduous development and debugging. In this paper, we introduce a DRL library FinRL that facilitates beginners to expose themselves to quantitative finance and to develop their own stock trading strategies. Along with easily-reproducible tutorials, FinRL library allows users to streamline their own developments and to compare with existing schemes easily. Within FinRL, virtual environments are confi…
-
Deep Reinforcement Learning Approach for Trading Automation in The Stock MarketTaylan Kabbani, Ekrem Duman · 2022 · Trading and Market Microstructure · cité 89 fois · Résumé IA disponible
Résumé scientifique
Deep Reinforcement Learning (DRL) algorithms can scale to previously intractable problems. The automation of profit generation in the stock market is possible using DRL, by combining the financial assets price "prediction" step and the "allocation" step of the portfolio in one unified process to produce fully autonomous systems capable of interacting with their environment to make optimal decisions through trial and error. This work represents a DRL model to generate profitable trades in the stock market, effectively overcoming the limitations of supervised learning approaches. We formulate the trading problem as a Partially Observed Markov Decision Process (POMDP) model, considering the con…
-
Deep Reinforcement Learning in Quantitative Algorithmic Trading: A ReviewTidor-Vlad Pricope · 2021 · Machine Learning · preprint, non relu par les pairs · cité 59 fois · Résumé IA disponible
Résumé scientifique
Algorithmic stock trading has become a staple in today's financial market, the majority of trades being now fully automated. Deep Reinforcement Learning (DRL) agents proved to be to a force to be reckon with in many complex games like Chess and Go. We can look at the stock market historical price series and movements as a complex imperfect information environment in which we try to maximize return - profit and minimize risk. This paper reviews the progress made so far with deep reinforcement learning in the subdomain of AI in finance, more precisely, automated low-frequency quantitative stock trading. Many of the reviewed studies had only proof-of-concept ideals with experiments conducted in…
-
Safe-FinRL: A Low Bias and Variance Deep Reinforcement Learning Implementation for High-Freq Stock TradingZitao Song, Xuyang Jin, Chenliang Li · 2022 · Portfolio Management · preprint, non relu par les pairs · cité 3 fois · Résumé IA disponible
Résumé scientifique
In recent years, many practitioners in quantitative finance have attempted to use Deep Reinforcement Learning (DRL) to build better quantitative trading (QT) strategies. Nevertheless, many existing studies fail to address several serious challenges, such as the non-stationary financial environment and the bias and variance trade-off when applying DRL in the real financial market. In this work, we proposed Safe-FinRL, a novel DRL-based high-freq stock trading strategy enhanced by the near-stationary financial environment and low bias and variance estimation. Our main contributions are twofold: firstly, we separate the long financial time series into the near-stationary short environment; seco…
Les travaux antérieurs sur lesquels il s’appuie
-
Deep Direct Reinforcement Learning for Financial Signal Representation and TradingYue Deng, Feng Bao, Youyong Kong et al. · 2017 · IEEE Transactions on Neural Networks and Learning Systems · cité 826 fois · Pas encore résumé
-
Adaptive Quantitative Trading: An Imitative Deep Reinforcement Learning ApproachYang Liu, Qi Liu, Hong-Ke Zhao et al. · 2020 · AAAI Conference on Artificial Intelligence · cité 164 fois · Pas encore résumé
Résumé scientifique
In recent years, considerable efforts have been devoted to developing AI techniques for finance research and applications. For instance, AI techniques (e.g., machine learning) can help traders in quantitative trading (QT) by automating two tasks: market condition recognition and trading strategies execution. However, existing methods in QT face challenges such as representing noisy high-frequent financial data and finding the balance between exploration and exploitation of the trading agent with AI techniques. To address the challenges, we propose an adaptive trading model, namely iRDPG, to automatically develop QT strategies by an intelligent trading agent. Our model is enhanced by deep rei…
-
A Deep Reinforcement Learning Framework for the Financial Portfolio Management ProblemZhengyao Jiang, Dixing Xu, Jinjun Liang · 2017 · Computational Finance · preprint, non relu par les pairs · cité 522 fois · Résumé IA disponible
Résumé scientifique
Financial portfolio management is the process of constant redistribution of a fund into different financial products. This paper presents a financial-model-free Reinforcement Learning framework to provide a deep machine learning solution to the portfolio management problem. The framework consists of the Ensemble of Identical Independent Evaluators (EIIE) topology, a Portfolio-Vector Memory (PVM), an Online Stochastic Batch Learning (OSBL) scheme, and a fully exploiting and explicit reward function. This framework is realized in three instants in this work with a Convolutional Neural Network (CNN), a basic Recurrent Neural Network (RNN), and a Long Short-Term Memory (LSTM). They are, along wi…
-
Model-based Deep Reinforcement Learning for Dynamic Portfolio OptimizationPengqian Yu, J. Lee, Ilya Kulyatin et al. · 2019 · arXiv.org · preprint, non relu par les pairs · cité 79 fois · Pas encore résumé
Résumé scientifique
Dynamic portfolio optimization is the process of sequentially allocating wealth to a collection of assets in some consecutive trading periods, based on investors' return-risk profile. Automating this process with machine learning remains a challenging problem. Here, we design a deep reinforcement learning (RL) architecture with an autonomous trading agent such that, investment decisions and actions are made periodically, based on a global objective, with autonomy. In particular, without relying on a purely model-free RL agent, we train our trading agent using a novel RL architecture consisting of an infused prediction module (IPM), a generative adversarial data augmentation module (DAM) and …
-
Quantitative Stock Investment by Routing Uncertainty-Aware Trading Experts: A Multi-Task Learning ApproachShuo Sun, Rundong Wang, Bo An · 2022 · Trading and Market Microstructure · preprint, non relu par les pairs · cité 5 fois · Résumé IA disponible
Résumé scientifique
Quantitative investment is a fundamental financial task that highly relies on accurate stock prediction and profitable investment decision making. Despite recent advances in deep learning (DL) have shown stellar performance on capturing trading opportunities in the stochastic stock market, we observe that the performance of existing DL methods is sensitive to random seeds and network initialization. To design more profitable DL methods, we analyze this phenomenon and find two major limitations of existing works. First, there is a noticeable gap between accurate financial predictions and profitable investment strategies. Second, investment decisions are made based on only one individual predi…
Les articles qui le citent
-
A Methodology for Developing Deep Reinforcement Learning Trading Strategies: A Case Study in the Futures MarketLeonardo Conegundes, Adriano C. M. Pereira · 2024 · IEEE Conference on Computational Intelligence for Financial Engineering & Economics · cité 1 fois · Pas encore résumé
Résumé scientifique
Quantitative algorithmic trading and machine learning have transformed financial markets by introducing advanced computational techniques for market analysis, decision-making, risk management, and order execution. This paper explores Deep Reinforcement Learning (DRL) for developing day trading strategies within the Ibovespa futures market, the world's most traded stock index futures contract. We chose the Ibovespa market for its high liquidity and volatility, which present unique challenges and opportunities for trading. We propose a novel methodology for the development, evaluation, and deployment of DRL-powered strategies. This methodology involves model generation, approval, selection, an…
-
TradeMaster: A Holistic Quantitative Trading Platform Empowered by Reinforcement LearningShuo Sun, Molei Qin, Wen-Tao Zhang et al. · 2023 · Neural Information Processing Systems · cité 40 fois · Pas encore résumé
Résumé scientifique
The financial markets, which involve over $90 trillion market capitals, attract the attention of innumerable profit-seeking investors globally. Recent explosion of reinforcement learning in financial trading (RLFT) research has shown stellar performance on many quantitative trading tasks. However, it is still challenging to deploy reinforcement learning (RL) methods into real-world financial markets due to the highly composite nature of this domain, which entails design choices and interactions between components that collect financial data, conduct feature engineering, build market environments, make investment decisions, evaluate model behaviors and provides user interfaces. Despite the av…
-
Evolution and Challenges of Reinforcement Learning in Intraday Algorithmic Trading: A Systematic Literature ReviewT. Amorgianiotis, G. Beligiannis, E. Georgopoulos · 2026 · Applied Sciences · Pas encore résumé
Résumé scientifique
Reinforcement learning has been increasingly applied to intraday financial decision-making, but the available evidence remains fragmented across heterogeneous tasks, datasets, and evaluation protocols. This systematic literature review examined peer-reviewed studies indexed in ScienceDirect and Scopus from 2018 to 2025. Across the initial search and the subsequent update, 315 bibliographic occurrences were imported; 51 redundant occurrences were removed, leaving 264 unique records for title and abstract screening. Twenty-one reports were retrieved and assessed in full text. Five were excluded during the historical StArt workflow, and 16 were historically accepted. During manuscript revision,…
-
Deep Reinforcement Learning for Financial Trading: Enhanced by Cluster Embedding and Zero-Shot PredictionHaoran Zhang, Xiaofei Li, Tian-Jiao Wan et al. · 2026 · Symmetry · cité 1 fois · Pas encore résumé
Résumé scientifique
Deep reinforcement learning (DRL) plays a pivotal role in decision-making within financial markets. However, DRL models are highly reliant on raw market data and often overlook the impact of future trends on model performance. To address these challenges, we propose a novel framework named Cluster Embedding-Proximal Policy Optimization (CE-PPO) for trading decision-making in financial markets. Specifically, the framework groups feature channels with intrinsic similarities and enhances the original model by leveraging clustering information instead of features from individual channels. Meanwhile, zero-shot prediction for unseen samples is achieved by assigning them to appropriate clusters. Fu…
-
EarnHFT: Efficient Hierarchical Reinforcement Learning for High Frequency TradingMolei Qin, Shuo Sun, Wen-Tao Zhang et al. · 2023 · AAAI Conference on Artificial Intelligence · cité 40 fois · Pas encore résumé
Résumé scientifique
High-frequency trading (HFT) is using computer algorithms to make trading decisions in short time scales (e.g., second-level), which is widely used in the Cryptocurrency (Crypto) market, (e.g., Bitcoin). Reinforcement learning (RL) in financial research has shown stellar performance on many quantitative trading tasks. However, most methods focus on low-frequency trading, e.g., day-level, which cannot be directly applied to HFT because of two challenges. First, RL for HFT involves dealing with extremely long trajectories (e.g., 2.4 million steps per month), which is hard to optimize and evaluate. Second, the dramatic price fluctuations and market trend changes of Crypto make existing algorith…
Aller plus loin
Comment ces articles sont-ils trouvés ?
Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.
Mis à jour le 5 novembre 2026.