Task 2: LLM-Engineered Signals with RLMF

This task aims to develop LLMs that can generate and engineer effective signals from news by using Reinforcement Learning from Market Feedback (RLMF). By incorporating market feedback in the fine-tuning process, LLMs can learn from and adapt to financial market behaviors.

In this task, the LLM will be used to generate one type of signal (e.g., a sentiment score) based on the content of news. Contestants will develop models that leverage RLMF to adjust the signals based on the market feedback. Contestants are expected to explore useful market feedback from market data and innovative reward models to fine-tune their LLMs.

Starter Kit

Task 2 Starter Kit

Dataset

An OHLCV dataset and a corresponding news dataset for a list of stocks are provided. Contestants are free to use external datasets to develop RLMF methods and fine-tune the LLMs.

The task2_dsets.zip file contains:

  • A 70-30 data split into a train and test directory.

Each split includes:

  • task2_stocks_{split}.csv: an OHLCV dataset of a list of stocks.

  • task2_news_{split}.csv: a corresponding news dataset for the list of stocks.

Contestants are free to use external datasets to develop RLMF methods and fine-tune the LLMs. The evaluation phase will use the testing datasets, which have the same fields as the datasets provided.

Requirements

  • Contestants should use an LLM that can be loaded with Huggingface and can perform inference on a 20GB GPU.

  • Contestants can determine what signal to generate and engineer.

  • Contestants should provide the prompt used for their LLM. For example: “What is the sentiment score of {stock ticker} after the release of this news: {news}. Give and only return the score in the range of [-1, 1]. Answer:”

Model Evaluation

To assess the effectiveness of the signal engineered by the LLMs, we will apply the signal to a simple and practical trading strategy:

  • Buy the top 3 stocks with the highest signals on the day of the news release and sell them 3 days later.

  • Short-sell the 3 stocks with the lowest signals on the day of the news release and cover the short position 3 days later.

  • The initial cash is $1 million, which will be allocated equally to each trading day and each stock.

The performance will be assessed by the following metrics:

  • Cumulative return: Total return generated by the trading strategy over a trading period.

  • Win/loss ratio: Calculated by dividing the number of winning trades by the number of losing trades over a trading period.