Workflow for experimenting with AI models

There are many ways to go with the AI strategies. If you have particular ideas, then you definitely should start from them. If you’ve got hands-on experience with training AI models, then you might want to jump straight to the point, too.

However, we have found that framing yourself into following a basic process makes the learning curve easier on you. You will eventually graduate from it, but starting here is a good idea.

1. Focus on one market. Pick a symbol, time frame and a date range.
2. If you have ideas on which markets are “less efficient” than others, then pick one of those.
3. Select the date range which is not expanding “up to now”. Leave some room (like ~2,000 candles) for the final backtesting. So if it’s October 2024 and you are training a model on 60 min stock charts, then don’t use anything after May 2023 in your training data set selectors.
4. Focus on one kind of signal. Pick your R/R and horizon and stick to them.
5. Start from KNN and RF as model types. Create 5 of each type. Wait for the learning process to converge.
6. Use LLM to generate inputs. You can be purposeful with your LLM queries, or you can use our examples.
7. For every model type (KNN, RF etc), pick 3 best models. Crossbreed them. Wait for the learning process to converge.
8. From there, you can either dive deeper or keep on going with crossbreeding.
9. If you go deeper, then analyze which inputs made sense. Work around them. Add more indicators like that, just with different length values.
10. If you keep on going with crossbreeding, then consider adding 1-2 new models to every generation before running the next cycle of crossbreeding. Introduce some fresh blood into the process.
11. Pick best models and backtest them on unseen data.

Here are a few ideas which you might find useful, too

1. If you expect a seasonal pattern occurring, then use the Date&Time indicator. Pick the series which you believe might have something to do with the pattern. Do not pick large chunks like “year” though, as these are unbound.
2. If you’re just exploring and willing to try a new indicator, then try a few different lengths of it. Also try the first and second derivatives of it. In example, if yo uwant to play with RSI then try A: RSI(7), B: (A - A[1]) / A[1], D: (B - B[1]) / B[1], E: RSI(14), F: (E - E[1]) / E[1], G: (F - F[1]) / F[1].
3. Any time one input has its importance obviously dominating, always try 2 options
   1. Add more inputs like that
   2. Remove this input
4. If you’re seeing a damn awesome Confidence vs Win% chart, double check whether any of your inputs happen to be unbound. If they are then backtest on unseen data is almost guaranteed to look bad.