quanxiel/course/Full Algorithmic Trading Using Python/15-algorithm-framework.md

15 · The Algorithm Framework（算法框架）

• 系列：Full Algorithmic Trading Using Python
• 频道：TradeOptionsWithMe ｜ 本集：The Algorithm Framework（算法框架）
• 时长：34 分 14 秒 ｜ 原视频：https://youtu.be/YQ3xwZuly8I
• 本地视频：videos/15-algorithm-framework.mp4

🎯 本集要点（中文导读）

本集讲 Algorithm Framework（算法框架）——用"职责分离"的类结构组织策略。

• 传统写法把逻辑都塞在 OnData/Initialize 里，职责不清；框架把算法拆成 5~6 个模块（可插拔、可复用）：
  1. Universe Selection Model：决定"考虑哪些标的"（选资产类别、剔除流动性差/无数据者、按主题等过滤）。
  2. Alpha Model：你的 edge 所在——对标的给出 alpha 分数（方向 + 信心），即交易信号（还只是"建议"，不是订单）。
  3. Portfolio Construction Model：把信号变成目标组合（每个信号分配多少资金；最简单的如等权）。
  4. Risk Management Model：风控（限制敞口、止损等）。
  5. Execution Model：把目标组合转成实际订单并执行。
  6. Transaction Cost Model：计入交易成本（有时并入执行模型）。
• 流程：Universe → Alpha（信号）→ Portfolio Construction（定仓位）→ Risk（风控）→ Execution（下单）。
• 示例 bot：用框架实现一个"按板块/基本面选股、生成 Insights（价格预判）、做多"的策略；示例用 Insight.Price(symbol, expiry, direction) 生成 insight。
• 优点：清晰、可替换各模块、易扩展；初期看着"绕"，但用熟后收益很大。

📝 完整文稿（英文 · 自动转写）

由本地 ASR（faster-whisper）从视频音轨转录，未人工校对，供检索/精读使用。原始讲解以上方视频为准。

Welcome back to the next video of my Python algorithmic trading course. So far we have only looked at one way of developing algorithms inside of QuantConnect, namely by creating different methods that handle different functions of the algorithm. Even though this is a simple and straightforward way to accomplish this, it does not really logically separate the different responsibilities of your strategy. Following a class-based approach with a clear separation of concerns instead can have many advantages. That's why QuantConnect offers the so-called algorithm framework which does exactly that. So in this video I will introduce the algorithm framework and its advantages. Furthermore, we will code a trading bot that uses this algorithm framework so that you can see it put into practice. With that being said, let's start breaking down what goes into a typical trading algorithm. Most trading algorithms are divided into six distinct components. The first component is the universe selection model. This is where your algorithm decides which assets it considers when looking for new trades. First, you will have to choose which asset class your strategy focuses on. Next up, you might want to implement further restrictions. At the very least you will want to filter out illiquid securities. Furthermore, you obviously only want to consider assets that you have data for. Besides that, your strategy might focus on a specific subset of the remaining assets. If for instance your strategy focuses on tech stocks, you will want to filter out all non-tech stocks. Alternatively, you might want to filter out too cheap or expensive securities or add other filters. After the universe selection model has filtered out those assets that don't meet all the required criteria, it can pass on the list of remaining securities to the alpha model. The alpha model is where your edge comes into play. This is where your algorithm assigns alpha scores to the securities that were passed on from the universe selection model. These alpha scores represent a directional assumption as well as a level of confidence in this prediction. In other words, this is where a trade signal is generated. But note that this trade signal is not directly a trade order. A trade signal can be thought of as a trade recommendation. Whether or not this recommendation is turned into a trade order is decided by the next model, namely the portfolio construction model. This is where the trade signals from the alpha model are turned into a potential portfolio. In other words, this is where it is decided how much capital is allocated to each trade signal. One of the easiest portfolio construction models is an equal weighted model that allocates the same amount of capital to each position. But sometimes you might want to allocate more to specific trades due to a higher level of confidence or some other reason. How this is decided depends on your implementation of this model. Next up is the transaction cost model. This is where you aggregate the accounts for transaction costs. Sometimes this model is integrated into the execution or portfolio construction model. The reason why this model is important is that sometimes it might not be worth it to open a new position due to high transaction costs. If you don't expect to make enough money to cover the transaction costs, you should not open the position. Besides the obvious transaction costs such as trade commissions, the transaction cost model also estimates other costs such as slippage and market impact costs. Depending on your order size and trade volume, these can have a significant impact to your bottom line which is why it is very important to consider them before sending out any trade orders. The model that is responsible for sending out the actual trade orders is the execution model. This model decides how a new position will be opened. It decides on how many trade orders your position is split up, what kind of trade orders are used, when these orders are sent out and more. Especially for bigger orders, it can be very important to split up the trade entry into multiple smaller steps. Furthermore, depending on the importance of the trade, you might want to use limit orders, market orders or some other type. Last but not least, let's talk about the risk model. This is the part of your trading algorithm that monitors your portfolio's risk and manages positions when necessary. A simple example of a risk model would be one that closes a position as soon as it is down more than a certain percentage, but obviously more sophisticated risk models can be much more complex. Note that, even though I presented all these models individually, they are all interdependent and very much rely on each other. All these models are very important. Even if an algorithm has a great alpha model, it can still perform terribly. Ten algorithms with the exact same alpha model can perform vastly different depending on the implementation of the other models. Now let me quickly go through each of these components for a simple example algorithm. This example algorithm bases its trades on the 50-day moving average and I would not recommend using it for any trading. Let's start with the Universe selection model. This algorithm only wants to trade liquid US financial stocks. That's why it orders all US financial stocks by the $80 volume and picks the top 10 as its trading universe. The alpha model then compares the current trading price of these 10 stocks to their 50-day moving average. If the current price is above the 50-day moving average, it sends a sales signal, otherwise it sends a buy signal. These signals are then picked up by the portfolio construction model which gives each of these 10 potential positions equal weights. Furthermore, it makes sure that not more than 75% of the available capital is allocated. The transaction cost model is very simple and assumes $5 per trade as commissions and simply disregards the cost of slippage and market impact. This means that the transaction cost model does not reject any trades due to two high transaction costs. The risk model only monitors the current percentage P&L of each position. If the profit of a position reaches 5% or a position loses more than 2%, the risk model sends a closed signal. First but not least, the execution model is called once a week on Monday at 10am. It then opens or closes positions with market orders depending on the information it received from the portfolio construction and risk model. I hope this brief example gives you an idea of what these models can look like. Of course, this is a very simple example. In reality, these models can get incredibly complex depending on the strategy and its implementation. But in the beginning, it is best to start out simple and move forward from there. You should now have a much better understanding of the design framework that more advanced trading algorithms usually follow. Before we move on to how you can use this algorithm framework inside of QuantConnect, let me first go over the advantages of using such a framework. First and foremost, this framework allows the coding principle of separation of concerns. This means that components of your algorithm that have their own functionalities will be separated into different classes. One of the main advantages of this is that the code is much easier modifiable and readable. If for example, you want to change how your algorithm handles the execution of your trades, you would only have to go and modify the code inside of the execution model while leaving the rest of your code untouched. Another advantage of dividing everything into its own module is that you can easily replace entire aspects of your strategy by using a different model. Besides developing your own custom models, you can also use community-created or other ready-to-use modules. If for example, you create a really promising risk model, you could just reuse the exact same model for different algorithms. Furthermore, QuantConnect offers a bunch of ready-made versions for pretty much all of the models that you can easily import into your code and use. I will demonstrate this later when we get to the coding part of this video. In general, using the algorithm framework allows you to focus on what you are good at. If for example, you don't like implementing the details of how trades are executed but instead prefer focusing on the alpha generation, you can just dedicate your time to the alpha model and use a pre-built execution model. That said, designing algorithms according to the algorithm framework does usually require a little more effort on the implementation side. Furthermore, it requires an understanding of class and object-oriented programming. That's why I recommend starting with basic classical algorithms, which is also exactly what we did in this series so far. However, once you get the grip of them, it can definitely be beneficial to try out the algorithm framework. Before we move on to actually coding a bot, let me quickly break down the details of how QuantConnect handles the algorithm framework. Firstly, let me quickly outline the workflow order in which QuantConnect executes these models. The first model that is executed is the Universe Selection model which will then filter out securities for the active universe. These securities will then be sent to the alpha model which generates trade signals based on the received data on these securities. These trade signals are then sent to the portfolio construction model which constructs the portfolio of investments out of them. However, before this portfolio is turned into actual trade orders, it is first sent to the risk model which makes sure that the theoretical new portfolio doesn't violate any risk requirements. Only after all that, the execution model will be called to actually turn the theoretical trades into actual real trade orders. Note that the alpha model does not generate any trade orders. It simply generates trade suggestions. Inside of QuantConnect, these trade suggestions are so-called inside objects. They are passed on to other models and the portfolio construction and execution model can then decide how to turn these insights into actual trade orders. An inside object has eight properties of which four are optional. The first argument of its constructor specifies the symbol for which the inside should be generated. The second one determines the time period that the inside should be valid. Depending on the strategy, this can be everything from a few seconds until multiple years. For example, if you want to base a trade on a foundational macroeconomic decision, you likely would go for a longer period than for a momentum intraday trading decision. The third argument is the type of insight. Here, you can either go with an insight for the price of the specified symbol or its volatility. In the vast majority of cases, you would want to emit a price insight. That's also why there is a direct helper method that allows you to skip this argument. The next argument is the direction of the insight. This can either be up, down or flat. The remaining arguments are optional, but depending on the implementation of the other models, you might want to add values for them as well. The first of these optional arguments is the magnitude of the insight. This is where you can specify how big you expect the upcoming move to be. Another parameter you can specify is the confidence in this insight which you can use to define how strong the signal is. Two more arguments are the source model and the weight of your insights. The source model is used to link an insight to a given alpha model. Since an algorithm can get insight from multiple alpha models, it would use this attribute to find out from which alpha model a given insight was emitted. The last weight argument can be used to state how much emphasis should be put on this insight compared to any other emitted insights. Here is an example of what an insight for AAPL could look like. This insight states that it expects Apple's price to rise 5% within the next 30 days with the confidence of 90%. Note that this is merely a trade suggestion and not an actual trade order. After the alpha model emits such an insight, the other models can decide whether to turn this into a trade order and if so, how to do so. Since you now hopefully understand how insights work, let me outline the structure of an alpha model inside of QuantConnect. To create an alpha model class in QuantConnect, you need to implement the i alpha model interface which consists of two mandatory class methods, namely an update method and an onsecurities changed method. The update method of an alpha model works very similar to the on data method since it updates the alpha model with new data and emits insights. To actually emit insights, you simply return your list of insights in this update method. To emit insights elsewhere, in your algorithm, you can use the emit insights helper method. As the name implies, the onsecurities changed method is where the alpha model handles changes in the active universe. Typically, you would handle positions that have left your active universe and apply the decision making logic to newly added securities. For helper class variables and initialization, you would use the standard Python init constructor. After this brief theoretical breakdown, I hope you now have a basic understanding of the algorithm framework inside of QuantConnect. To help facilitate this knowledge and to see it in action, let's create an actual trading algorithm that uses this algorithm framework. In the coding part of this video, we will implement an alpha model and a universe selection model. For the other model, such as the execution model and portfolio construction model, we will use some of QuantConnect's pre-built classes. However, before we actually start coding, let me quickly outline what strategy we will be implementing. The goal of the algorithm that we will implement will be to find high quality companies to invest in for the longer term. To identify quality stocks, we will use different quality criteria. Firstly, we will filter out stocks that have IPOed more than five years ago since we want the companies to have proven themselves by existing for some time. Next up, we want to diversify our holdings by investing in different sectors. For this algorithm, we will focus on the sector's financial services, real estate, healthcare, utilities and technology. But feel free to try out different variations. We then consider the factors price to earnings ratio, profit margin and return on equity. For each sector, we rank all companies in that given sector by these factors and establish a long position in the top 20% of the companies in each sector. Since the investment philosophy of this algorithm focuses on the long run, we only rebalance our portfolio on a quarterly basis. So summed up, this algorithm will rank US stocks in the previously mentioned sectors by some quality factor, once a quarter and invest in the top 20% of each sector. That said, you now hopefully understand what the bot is supposed to do, so let's now head over to QuantConnect and actually start writing some code. Inside of QuantConnect, we will go to the lab tab and create a new blank algorithm. As always, we will start by implementing the initialize method. But unlike in the previous videos, we will not use the on-data method since we will be using the algorithm framework instead. As usual, we will begin by setting the time frame and the starting cash balance for the back test. For this, I will just go with the year 2020 and $100,000. Note that this algorithm might have trouble with a too low balance since it wants to invest in a bunch of large cap US equities. Thereafter, we will create two helper variables, namely self.month, which we will use for keeping track of rebalancing times and self.nom course, which will be the number of stocks that will be considered in our course universe selection filter. I will initialize this variable to be 500. Then we set the resolution of the universe to daily, which we can do by accessing the universe settings property. Now, it's already time to add our universe by specifying the course and fine filters that we will use for this universe selection process. However, before we actually implement these filters, let me quickly implement a rebalancing function that will keep track of when to rebalance the universe and the portfolio for this algorithm. We will call this function is rebalance due and it will return the time to rebalance. As a parameter, it takes a time variable. Since we want to rebalance on a quarterly basis, we want to rebalance in the first, fourth, seventh and tenth months of every year. That's why we test whether the current month has passed or the current month is not one of these. If that's the case, we just return an empty object since it's not time to rebalance. If this is not the case, we set the self.month help variable to the current month and return the submitted time which will tell the other function that it is time to rebalance. With that, let's now move on to implementing the course universe selection filter. All we want to do here is check whether it's time to rebalance and if it is, we want to return a list of the 500 most liquid US stocks with a price above $5. To check whether it's time to rebalance, we use the just coded rebalancing check function. If it's not time, we just return the unchanged universe. Otherwise, we sort the list of available securities by dollar volume. Here, we also check whether the security has fundamental data and that its price is above $5. Thereafter, we return the symbols of the first 500 securities in this sorted list. This list of symbols will now be passed on to the fine universe selection filter which we will implement next. Here we want to find those securities that are in one of the desired sectors. For that, let's first quickly create a list containing the sector codes for financial securities, real estate, healthcare, utilities and technology. We can use Morningstar sector codes for this. Now, we once again create a list containing the symbols of the desired securities. However, we only add those symbols of the securities that I peeled more than 5 years ago and that are in one of the desired sectors. Furthermore, we test that there is a non-zero value for the fundamental factors, return on equity, net profit margin and price to earnings ratio. Last but not least, we return this filtered fine list. Next up, we will implement an alpha model for this algorithm which will take this filtered fine universe of stocks and rank them according to the just mentioned factors and generate buy insights for the top 20% of each sector. To add an alpha model, we will go back to the initialized function and use the helper method add alpha. Here we will add an instance of the alpha model fundamental factor alpha model. Note that this model does not exist yet, so what we will do is create a new file, name it alpha model dot pi and create the fundamental factor alpha model class inheriting from the alpha model class here. However, before we actually start implementing this class, let's first go back to the initialized method of the main file and import the alpha model file so that we can use this class. Furthermore, let's quickly finish the initialized method by adding the remaining models. First, we will add QuantConnect ready to use equal weighting portfolio construction model which will allocate an equal amount of capital to all of the insights it receives from an alpha model. As an argument, we passed the rebalancing function which it will use to find out when it is time to rebalance the portfolio. Next up, we set the risk management model to the null risk management model which basically does nothing. Alternatively, you could use one of QuantConnect's other pre-made risk models such as the trailing stop-wiz model or even implement your own as an exercise. Last but not least, we set the execution model to the immediate execution model which will always send out trade orders as soon as the portfolio construction model sends a trade signal. With that, we can now head back to the alpha model file and start implementing the alpha model of this algorithm which is where the actual trade decision making will take place. Just like for the main class, we will also use the initialised method for the alpha model. However, here we will just use Python's standard init constructor. In this constructor, we will create two class variables. One will be the rebalance time which we will use for rebalancing purposes and the other is a dictionary that will keep track of the securities in each sector. Here it is important to understand what the self dot sector's dictionary looks like since we will be using it a lot in the alpha model. The keys for this dictionary will be the sectors that we specified earlier. The value saved under such a key will be a Python set containing the stocks that are in the given sector. Besides an initialised method, an alpha model always has an update and onsecurities changed method. The update method is called every time there is new information for the alpha model and it should return a list of insights that will be passed on through the portfolio construction model. The unsecurities changed method is called ifsecurities are added or removed from the universe which is in our case should happen in about one secorture. We will start by implementing the unsecurities changed method. In this method, we will first of all iterate through the securities that were just removed from our universe. For each of these securities, we now want to remove those securities from the self dot sector's dictionary sets. To do so, we iterate over all sectors and check whether the given security is saved under that sector and if it is, we remove it. Next up, we basically do the exact opposite for those securities that were just added to the universe. We iterate over those and look at the sector that the given security trades in. If that sector is not yet in the self dot sector's dictionary, we add it to the dictionary with an empty set as the value. Then we add the security to the set that is saved under that sector. As you can see, the unsecurities changed method now simply takes care of keeping the self dot sector's dictionary up to date with the actual universe of stocks that are in our universe. Next, we will implement the update method to actually rank these stocks and send out buy insights for the top 20% of each sector. The first thing that we do in the update method is check whether it's time to rebalance. If it's not, we simply return an empty list since we do not want to send out any new insights. Otherwise, we set the rebalancing time to the end of the current quarter. Note that we have to use algorithm dot time to access the time since only the main QC algorithm class has this property. After that, we create an empty list and save it into the local insights variable. Now it's time to iterate over all the sectors in our self dot sectors dictionary and rank all the stocks in them. First, we sort the securities in that sector by return on equity, by profit margin and by price to earnings ratio. Since the higher return on equity and high profit margins are good, we set the reverse flag to be true for these two. On the other hand, we want to prioritize stocks with a low price to earnings ratio since we consider these to be relatively cheap, which is why we set the reverse to be false for the PE ratio. To calculate the scores, we create a dictionary named scores. We then iterate over the securities in the current e-considered sector and sum up the positions of that security in each of the sorted factor lists. So for example, if a stock would have the highest margins, highest return on equity and you would have the lowest PE ratio, it would be first place in all of them and receive the lowest and best score. Now that we have a score for each security in one sector, we want to find the best 20% of them. For this, we check how many elements there are in the scores dictionary so that we find out how many stocks are 20% in that sector. Note however, due to diversification purposes, we want this to be at least one. To say for example, there only would be two stocks in the current sector, the top 20% would be rounded down to be no stocks. But in that case, we would still want to invest in the first stock. With that, we return the scores dictionary into a key value pairs list and iterate over the 20% best ones. For each of these, we append an inside object to our insides list. Since we're just interested in price insights, we use insides.price. As arguments, we first specify the symbol of the security, then the expiration of the inside, namely the end of the current culture, and finally the direction in our case is up. We did all this in a for loop of all the sectors, which means that after this outer for loop is done, the insides list should have all the desired insides. This means all that's left to do is return this insides list. And with that, we're now done implementing this algorithm. So let's now build and backtest it to view its performance over the backtest time frame. As you can see, this bot managed to generate a positive return in the year 2020, however, not without quite the volatility during the early 2020 market decline. If we look at the benchmark chart of the S&P 500, we can see that this strategy seems to be highly correlated to the S&P 500. This should not be very surprising since they both invest in large-cap US equities. Since this bot trades a long-only strategy, it will be heavily affected by market downturns. If you want to analyze this performance report in more detail or add your own variations to this strategy, you can clone this code using the link in the description box below. Otherwise, I hope that this video gave you a good overview of the algorithm framework. I know it can be a little overwhelming at first, but once you get used to it, it has a multitude of benefits. That said, hopefully you are enjoying this video series. If you are, make sure to smash the like button, subscribe and turn on the notification bell. Thanks for watching.