24 lines
17 KiB
Markdown
24 lines
17 KiB
Markdown
# 19 · 50-Line Trading Bot（50 行代码 · 标普成分股调整）
|
||
|
||
- **系列**：Full Algorithmic Trading Using Python
|
||
- **频道**：TradeOptionsWithMe ｜ **本集**：50-Line Trading Bot（50 行代码 · 标普成分股调整）
|
||
- **时长**：17 分 29 秒 ｜ **原视频**：https://youtu.be/EbUEMW5135M
|
||
- **本地视频**：[videos/19-trading-bot-50-lines.mp4](videos/19-trading-bot-50-lines.mp4)
|
||
|
||
## 🎯 本集要点（中文导读）
|
||
|
||
**50 行代码的选股 bot**：围绕 **标普 500 指数成分股调整** 套利（新手友好）。
|
||
|
||
- **策略逻辑**：标普 500 定期**增删成分股**；新纳入的股票会迎来关注度与被动买盘，被剔除的会被抛售。于是——**买入新纳入的股票、卖空（或卖出）被剔除的股票**（通常同时发生，故常同时开一多一空）。入场后用**跟踪止损（trailing stop）** 做风控。
|
||
- **数据准备**：从维基百科《List of S&P 500 companies》的增删表格抓数据，用 **Jupyter + pandas**（`read_html`）清洗成可用格式（作者提供了数据链接）。
|
||
- **实现**：用 QuantConnect，**不到 50 行 Python** 完成。
|
||
- ⚠️ 提醒：
|
||
- 维基数据的**质量/完整性无保证**，用自定义数据要警惕数据质量问题（含此前提到的 **ticker/symbol 陷阱**，可用 QuantConnect 的 SPY 成分股数据集改进，其中还含各成分权重）。
|
||
- 实盘需**实时数据源**；可扩展方向：把"剔除原因"标准化后纳入决策、或扩展到其它大指数。
|
||
|
||
## 📝 完整文稿（英文 · 自动转写）
|
||
|
||
> 由本地 ASR（faster-whisper）从视频音轨转录，未人工校对，供检索/精读使用。原始讲解以上方视频为准。
|
||
|
||
Welcome back to another video. In this video we will create yet another trading algorithm using the QuantConnect Algorithmic Trading Platform. The bot that we will implement will look at changes in the constituents of the biggest and most followed market index, namely the S&P 500 and trade based on these changes. As you probably are aware of, the S&P 500 index tracks the 500 biggest companies trading on US stock exchanges. Since markets are in constant motion, the list of the biggest companies occasionally changes. Sometimes certain stocks are removed from the index while others are included. Unsurprisingly, the addition or removal of a stock into one of the most followed market indexes will attract some attention. Furthermore, a wide plethora of funds and investors have holdings in the S&P and will thus allocate capital to the stock newly included in the index. The idea of our strategy is to capitalize on this spike of attention and buy pressure by buying stocks that are being newly added to the index and sell stocks that have been removed. Since most of the time a stock will be removed at the same time as another stock will be added to the index, we will usually open a long and a short position at the same time. Once a position has been opened, we want to use a trailing stop loss a few percentage points away from the entry price for risk management. In this video, we will code this entire strategy in Python in under 50 lines of code using QuantConnect. I highly recommend following along, so if you haven't already, make sure to use the link in the description box below to create your free account and code along. Note that this will be a relatively easy beginner-friendly bot to code along with. However, in case you want a complete walkthrough of how to develop your own trading bots using the QuantConnect API, check out my free algorithmic trading course in the description box below. That said, before we can actually start implementing the code for this trading bot, we first need to find the data of the constituent changes and preprocess it so that we can actually use it as an input for our trading algorithm. Luckily, I found some data on changes in the S&P 500 members on Wikipedia in the article list of S&P 500 companies. Here we can see a table with additions and removals of stocks and a reason for the change going back at least 20 years. Note that I don't have any information on the quality or completeness of this data, but nonetheless I will use this data for the testing of our strategy. However, before we want to import this data and actually use it in our trading algorithm, we need to preprocess this data and transform it into the right format. For this, I will open a new Jupyter notebook to scrape the data and perform these changes. Note that I will make this data available through a link in the description box below so that you can copy and download it for your own trading algorithm. The first thing we want to do in our Jupyter notebook is import pandas so that we can work on the data in a pandas data frame. Next, we use the pandas readHTML method to read the data from the Wikipedia page. What this method does is it takes all HTML and tables on the given page and turns them into data frames and then adds these to a list. Since the table we are interested in is the second one, we index this list to get the right data frame. As you can see, we now have the data here, however, instead of having the newer states at the top, we want the reverse order. Next up, we want to flatten the column names of the data frame since right now it consists of multiple layers. Furthermore, we are actually only interested in the columns, date, ticker, added and ticker removed. Before we move on to refactoring this data a bit more, let's create a list of all the tickers mentioned in the remove and add columns. We will use this list inside of QuantConnect to later add data for these stocks to our algorithm. To create this list, we turn the columns into sets and union them after dropping non-values. After this, we want to refactor the data a bit. More specifically, right now the problem with the data is that we have multiple rows with the same date. Instead of having many rows with the same date, we want to have a list of all added and removed stocks for each date. For this, we can simply loop through the rows of the data frame and create a dictionary with the lists for each date in the data frame. Then all we have to do is turn that dictionary into a new Pandas data frame, reset the indexing, sort it correctly and we basically have the format we want. One last thing we want to do however is turn the lists of stocks into strings where the tickers are separated by spaces instead of commas. We do this since we will later import the data into QuantConnect as a comma-separated file and thus not using commas here as separators will make this step easier later. With that, we now have the data in the format that we desire. So now we can export it to a CSV file. This CSV file I will now upload to Dropbox. We can then use the Dropbox URL to this data in our algorithm to access this data and use it in our algorithm inside of QuantConnect. I will leave the Dropbox URL in the description box below so that you can download this data too. With that being said, we can now head over to QuantConnect and start writing the code for the actual trading bot. To create the trading bot, we will go to the lab tab and click on Create New Algorithm. Once a blank template algorithm was being created, we want to start by implementing the initialize method in which we can set some algorithm variables and backtest settings. First off, we set the backtest time frame to 2015 until 2021. Note that to backtest a strategy properly, you need to specify a backtest time frame for which we actually have the custom data. If you backtest on a time frame without the data, you won't get any meaningful results. After that, I will leave the starting cache balance for the backtest at $100,000. In the next step, we will take advantage of QuantConnect's algorithm framework which allows us to specify a risk model that is pre-made for us. Here I will go with the Trading Stop Risk Model and pass a percentage of how far I want the trading stop loss to follow our positions. For this parameter, I will go with 5%. This will automatically create a trading stop loss for us so we won't have to worry about any risk management in the remainder of the implementation. Next up, I will paste the ticker list that we created from the custom data before and save it to a variable. Note that there is a potential problem with this. To understand this, we have to know that QuantConnect differentiates between symbols and tickers. Symbols are permanent identifiers that stay consistent even after big changes. While tickers can change and lead to some ambiguity problems. In other words, since some of these companies and tickers are old, there might be a chance that the algorithm will think this is a different security. One approach to fixing this problem would be to use QuantConnect's curated SPY Constituents dataset that I linked to in the description box below. However, here we have to know that the SPY Constituents might slightly vary from those of the S&P. Furthermore, I want to demonstrate how we can work with custom data since this opens up so many doors for developing interesting strategies. That's why I will still stick to this approach for now, but for more details on the differences between tickers and symbols, check out the third video in my algorithmic trading course. We will now subscribe to the data of all these tickers at an hourly resolution. Note that this could be optimized since we might not need all the data depending on the back test time frame, but since there aren't that many symbols, this is just the easiest approach. Using Minitly or an even higher resolution would allow us to access quote information. This would then lead to more accurate back tests since the trades are based on the bits and asks, but at the same time the back test would take way longer due to the abundance of data points. That's why we stick to hourly data for now. After adding the equities data, it is time to add the custom data file that we uploaded to Dropbox. To import custom data to QuantConnect, you need to create and implement a custom class. We will name this class SPX change and add the data at an hourly resolution and save it symbol to a class variable name change. Before we can actually use this data, however, we must implement this SPX change class. This class has to inherit from the Python data class and it needs to implement two methods, namely the get source and reader methods. Like the name implies, the get source method is there to get the source of the custom data. In this method, we will specify as the source of the data the dropbox you are ill from before. However, note that here we need to set the download flag to 1 instead of the default 0. Then next up, we specify that this is a remote file that we want to get the data from and return a subscription data source object. After that, we can move on to implementing the reader method, which is there to read from the custom data source and return a custom data object. This method basically receives the data from our data file line by line. So first, we need to check that the line that the method received is in fact one with the right data. If for instance, we already went through all the data and receive an empty line, we don't want to return anything. Otherwise, we want to split the line by commas since it's a comma separated file and create a custom data object. An important thing we need to do here is set the symbol and datetime of the just created custom data object. Since the time is specified in the first column of our data file, we can index it with zero. However, then it is important to tell the method what format the data has. Since the Wikipedia data did not have any time of day information, we'll just make it available at 10am. Next up, we need to set some other attributes of the custom data object so that we can access the added and removed tickers. By default, there is the value attribute for custom data that we can use for this. But this is only for decimal data, which we don't have here. So instead, we just set the attribute to zero, since it's not recommended to not set it to anything. Then we create two more attributes in which we save lists of the added and remove tickers respectively. Lastly, if while doing any of this, we run into an error, we just want to return nothing. In case everything works out fine, we return this custom data object. Now that we have implemented the custom data class, we can move on to implementing the actual trade logic, which will only be a few lines of code. For this, we will use the on data method that is called every time the algorithm receives new data. This new data is submitted in the data parameter. Since we only want to perform any opening trades in case the stock has been added or removed from the index, we first need to check if we have any custom data object in our data parameter. In case we do, we want to loop through the added tickers and set 10% of our portfolio to invest into this stock. However, before we do so, we first check whether we have data for the given stock since a few of the tickers might not be supported or have data issues. Furthermore, we basically want to do the same for the stocks that have been removed from the index. However, here we want to open a short position with only 5% of our portfolios capital. We open a smaller position here since short positions are considered more risky. To achieve the right allocation, we use portfolio targets and set holdings. With that, we are now actually already done with implementing our trading bot. As you can see, this didn't even take 50 lines of code. So let us now build and backtest this algorithm to see the results over the backtest time frame. After a relatively short time, the backtest should have finished and you will see a performance report like this one. As you can see, the performance of this bot seems to be relatively good. However, I will not go over this report in any detail. But note that, if you haven't coded along, you can clone the entire algorithm, test it and play around with it yourself. I left a link in the description box below for that. Note that this algorithm in general plays it quite safe considering that the S&P 500 constituents don't change very often and it only invests about 15% at once into a long and short position. Instead of analyzing these results in detail, let me provide some suggestions on how this strategy could be improved. Firstly, usually the news that a stock will be added or removed from the S&P 500 is known before it actually happens. So a possible improvement of the strategy would be to try to find out about an inclusion or removal earlier so that the bots can act earlier. Possible approaches to this would be to track the inclusion metrics of the S&P 500 and check which stocks fulfill the criteria. You can, for example, find the requirements on Wikipedia. Another approach for this would be to analyze news and social media from mentions of stocks being included or removed. You might be able to get a rough idea of what impact getting the information earlier might have by making the custom data available a bit earlier. However, note that getting in earlier will mean that you will be exposed to directional movements for some time before the actual inclusion or removal. So here it might potentially also make sense to adjust the trailing stop. Besides that, another variation would be to track the reason for the addition or removal to the index as well. The idea behind this is that some reasons might impact the price more than others, so accounting for this in the strategy might be interesting to try out. This would not be very hard to implement since the Wikipedia data actually includes the reason already. However, note that this would require some additional pre-processing since the reasons aren't standardized. So you would first have to standardize the reason column to the most common reasons such as market cap requirements or acquisitions to name two common examples. Besides that, it might be an interesting idea to expand the strategy to more big name indexes than just the S&P 500. With that being said, note that for live trading you would have to find a real-time data source, but since the S&P 500 is one of the most well-known indexes, this shouldn't be too hard to find. Furthermore, please keep in mind that I can't guarantee the accuracy of the Wikipedia data in this video. In general, especially when working with custom data, always make sure to consider any potential data quality issues. For example, here you should remember the ticker and symbol problem that I referenced earlier. Like I said then, this could be improved by using QuantConnect's SPY constituents dataset. The cool thing about this dataset is that it also includes the allocation weight of each member of the SPY which could also be incorporated into the decision making of your strategy. With that being said, I really hope you enjoyed this video. If you want to learn more about developing your own trading algorithms using Python and QuantConnect, make sure to check out my free algorithmic trading course. Otherwise, make sure to smash the like button, subscribe and turn on the notification bell. Thanks for watching.
|