Creating an AI strategy: 4. Model inputs (features)

Defining your model inputs is a make-it-or-break-it kind of a thing. With awesome inputs (that is, "inputs legitimately having predictive power for the signal"), even the simplest model will learn well and work well. With poor inputs (that is, "inputs having nothing to do with the actual signal"), even the best model in the world won’t make you happy.

In order to understand how to approach inputs, you should understand something about how machine learning works.

How ML models learn at all

Think of training an ML model as taking a fresh model which does not know anything, showing it a bunch of examples of valid signals from the past and then shaking it aggressively, screaming “watch and learn”.

“Learning” means model discovering relations between “signal should be here” and “here’s what’s happening on this market at all”. ML models can not perceive markets in their entire complexity automatically. Sadly, they can not consider every factor possible and then just spit out predictions. The reasons for that have something to do with the amount of actual information vs noise, model overfitting and computational resources.

When training a model, you should define what it should pay attention to. You don’t have to explain how to use what you give. You should give the model things you believe might have something to do with the signals you want to predict. Some of the things you name will be useless, and that’s fine for some models (they will simply ignore these things). Defining what the model should use to identify relations with the actual signals is what they call “feature engineering” in the world of machine learning.

If your model inputs have predictive power for a given type of signal, your will have a useful model. If they don't, then your model wont be useful. For the purpose of trading, it takes a trader to define inputs, not a machine learning engineer. Luckily, that’s exactly you!

Feature engineering for traders

On TrendSpider, we offer you two kinds of inputs: indicators and formulas.

Using indicators seems intuitive. You just take an indicator and shove it into the ML model. Indeed, it’s that easy. There is some fine print below, but in general, you can use any of the indicators for inputs, either directly or via a formula.

Using formulas is not as intuitive, but that’s where the real power is. That’s how you apply feature engineering tricks without coding anything. You can have RSI as input A, and in your input B you can have a formula like (A - A[1]) / A[1] which will be “1 candle change% of the RSI value”. The formula language is very powerful; see Appendix B for a full explanation.

ML models seek relations between inputs and signals, and the very fact of using conventional indicators like RSI or BOP does not automatically mean that there’s no alpha to seek here. Individual inputs matter, but the actual combination of inputs and model type matter as well, and maybe even more. Moreover, deriving features from indicators offers endless amount of opportunities.

What not to do with features

You want your inputs to have some predictive power (that is, “have something to do with the signal you want to capture”), and it takes a trader to come up with the ideas for that. That’s hard to formalize. However, it’s rather easy to formalize what is definitely not going to work. Here are the things to remember when designing inputs.

A. Pay attention to the input variance range

Never use values (indicators or formulas) which have unbound variance range as direct input features. That’s going to be a classic grave ML mistake and the TrendSpider interface attempts to prevent you from that. You can’t push SMA into a model. Don't use formulas which return pure price values, for the same reason.

The reason for that is fairly simple: values of unbound indicators are plenty, and they are fairly unlikely to occur again in the future. This means that your model has a lot of chances for over-fitting itself to the training data set while having virtually zero chances for working with the previously unseen data. To make it worse, a lot of models demand their input values to be normalized prior to the actual training process (i..e, into a 0..1 range). Normalizing unbound variables on a completely different value range will produce results which are essentially random, when compared to your training data set.

However, if you believe that an unbound value carries a signal, you can apply some feature engineering techniques to transform it into something ML will gladly consume. In example, if you want SMA(20) badly, then you can do any of the below, depending on what exactly do you believe might carry a signal:

1. Distance from price: use SMA(20) as input A (not used for training) and a formula like (A - C) / C for input B. That will give you “distance from SMA to price as percentage of the price”. Logarithm of this distance might be even better. That way, Input B will have a variance range which is way closer to being predictable and finite, if compared to the variance range of the SMA per se.
2. Distance between 2 SMAs: use SMA(20) as input A (not used for training), use SMA(50) as input B (not used for training) and a formula like (A - B) / C for input D. Or, alternatively, use ma(C, 20) - ma(C, 50) as your only input.
3. Take change% of the SMA. Use SMA(20) as input A (not used for training) and a formula like (A - A[1]) / A[1] for input B.
4. Extract the mean from SMA. Use SMA(20) as input A (not used for training) and a formula like (A - ma(A, 50)) / A for input B, which will give you “distance between SMA20 and SMA50 built off this SMA20, as percentage vs SMA20 value”.

B. Pay attention to distribution of values of your features

1. Prefer features which have their values changing smoothly (like RSI, for example) rather than in a jumpy fashion.
2. Prefer features which have their values distributed (roughly) evenly over the entire variance range of this feature. In general, inputs which tend to stick at certain levels won’t be as good as inputs which don’t. For example, in that sense RSI will be better than an indicator which tends to max out and stick at the maximum value level.
3. Prefer features which have their mean value and variance range not changing much over time.

Indicators which do not meet these requirements "off the shelf" are not neceesarily useless. You still can apply feature engineering approaches (that is, mathematical transformations) to use them. For eaxmple,

1. If you believe that mean value is changing too much, then apply mean extraction (A.4 above). In example, during a long bullish market, mean value of your RSI might drift away from 50( which you might expect to be the mean value).
2. If you believe that variance range is changing too much over time, then try replacing the feature with its derivative and see if that’s better (A.3 above).

C. Avoid inputs which have outliers (spikes)

Make sure your inputs don’t have outliers. ML has a hard time dealing with outliers. When model inputs are normalized before ML training, outliers can skew the entire data set and make it really hard for the training process to come up with anything meaningful. If your features have outliers then the best thing you can do is to get rid of them completely. There are a few approaches, like clamping into a fixed range (which violates p.3 above) or replacing values with their logarithm (which still skews the data set, but not as much). There are other techniques as well, but if you want to keep it simple, then stay away from features which have outliers.

In general, built-in oscillating indicators never have outliers. Your best chances for introducing outliers are when you apply custom formulas which divide by values which can potentially become very small. Or by letting you formulas to produce values which are price values. There are other ways, too. When using formulas (and you should do that a lot!), make sure that your expressions are not prone to returning incredibly small or incredibly large values in some circumstances.

Use LLM to give you inputs

While challenges of feature engineering are plenty, the very first one you’ll face would be the problem of a blank slate. “What do I start from?” is how it feels. While it might sound funny, that’s a real problem and that’s the first and the last problem quite a few people face when approaching classic algo trading. Yes, a lot of people give up because they don’t know where to start.

We solve this problem by letting you ask a large language model (GPT) to give you a start. We have designed a good prompt for it, and oftentimes what you’ll get from LLM will actually be interesting. We have seen many not-bad models produced by simply using input features proposed by the language model.

Think of LLM as a fellow trader who knows a lot about indicators and typical strategies, and who’s very much eager to help you, but has no idea about what you’re after. Explain what your idea is. Don’t be afraid to be vague. Feel free to ask LLM to be creative. Or frame it into only using one indicator type and see what happens. Use examples of LLM requests we have provided.

Have fun with it. Surprisingly, that might be fruitful.