Two Ways to Write the Same Signal
Back in the anatomy lesson we drew a deliberately simplifying line: in the basic architecture this course uses, machine learning most often lives in one box — the signal engine — where it produces a score, never a trade. (More advanced systems also put machine learning to work elsewhere — in execution and order routing, transaction-cost and market-impact estimation, and risk monitoring — but the signal engine is where it most visibly earns its reputation, so it's the box we open here.) There are two ways to fill that box, and the difference is entirely about who writes the pattern.
In a rules-based algorithm, a person writes the formula by hand: "score a stock +0.8 if it's cheap relative to its average." In a machine-learning approach, the person writes no formula at all. Instead they hand the software thousands of past examples, each labeled with what happened next, and let it learn the pattern on its own. The classic everyday version is a spam filter: nobody writes a rule listing every spammy word — you just show it a pile of emails marked "spam" or "not-spam," and it works out the giveaways itself. A trading model does the same trick on market data. Crucially, the output is identical to the hand-written version — a score. The model is a pattern-finder, not a decision-maker.
So the two approaches fill the same slot in different ways. Here's how they line up on the things that actually matter:
| Aspect | Rules-based signal | Machine-learning signal |
|---|---|---|
| Who writes the logic | A person writes the formula by hand | A person feeds examples; the software learns the formula |
| What it needs | A clear idea and a few parameters | Lots of labeled historical examples and computing power |
| Main failure mode | The idea itself is simply wrong | Overfitting — memorizing noise that won't repeat |
| How you'd check it | Backtest, then test on data it never saw | The same — plus a strict split it was never trained on |
Same signal-engine slot, two ways to fill it. Notice the bottom two rows barely change — both approaches live or die on whether they hold up on data they've never seen.
That last row is the quiet punchline: swapping a human formula for a learned one doesn't change the test that matters. Both have to prove themselves on fresh data. To see why the learned version is especially easy to fool, we first need the handful of words machine-learning people actually use.
The Words Behind Machine Learning
Four terms cover almost everything, and each has a plain-English meaning:
- Features — the inputs, the clues. For a stock these might be recent returns, volume, how far the price sits from its average, or a sentiment score pulled from news. Choosing good features is most of the work.
- Training — the learning step. The model looks at many historical examples, each paired with what happened next, and adjusts itself to fit that history as well as it can.
- Prediction (or inference) — using the trained model on new data it hasn't seen, to produce a fresh score.
- Train/test split — deliberately hiding some of the history. You train on one slice and check the model on a different slice it never touched. Judging a model on the same data it learned from is like grading a student on the exact questions they were given the answers to.
That last idea is the whole ballgame, and it connects straight to last lesson's villain. A model with enough freedom can memorize the training history perfectly — every wiggle, including the pure luck — and look brilliant. Then it meets fresh data and falls apart. That's overfitting, and the train/test split is the main tool for catching it before real money is involved.
Why Markets Fight Machine Learning
Machine learning does genuinely well on problems like reading handwriting or filtering spam. Markets are a much harder arena — arguably one of the hardest there is — for four honest reasons:
- Terrible signal-to-noise. A photo of a cat is almost entirely signal. A price chart is mostly randomness with a faint pattern buried inside — if one is there at all. The model spends most of its effort trying not to mistake noise for meaning.
- Non-stationarity. In most machine-learning problems the rules stay put: a cat looks like a cat every year. Markets don't sit still — conditions, participants, and behavior all shift, so a pattern that held for years can quietly decay and stop working.
- Reflexivity and competition. If a real edge exists and enough people trade it, their own buying and selling can move prices until the edge disappears. The act of using a pattern can erase it — a problem a handwriting reader never faces.
- Tiny data. Twenty years of daily prices is only about 5,000 data points. To a field that trains on millions of images, that's a scrap. Small data plus a flexible model is the perfect recipe for overfitting.
Why this combination is dangerous
Each of those four on its own makes finding a durable edge hard. Together they make it very easy to believe you've found one that isn't real. A flexible model, pointed at noisy, shifting, scarce data, will almost always discover a beautiful pattern in the training set — and that pattern is often just memorized luck. This is why serious quant teams treat an impressive backtest with suspicion, not excitement.
What AI Genuinely Helps With
None of that means machine learning is useless here — it means the honest use is narrower than the headlines suggest. Where it earns its keep is on jobs that play to its real strength, which is chewing through more information than a person can:
- Sifting huge, messy inputs — turning piles of news text, filings, or other unstructured data into a tidy number a strategy can use.
- Spotting non-obvious combinations — weighing dozens of features at once to notice interactions a human might not think to write into a formula by hand.
- Automating research — testing many candidate ideas quickly, so people spend their time on the promising few instead of the drudgery.
In every one of these, the model is a tool that proposes a signal. A human still has to validate it on unseen data, decide whether it makes sense, and wrap it in the risk checks from earlier lessons. AI widens the funnel of ideas; it doesn't remove the need for judgment at the end of it. And the signal engine isn't the only place it turns up: in the simplified architecture used throughout this course, AI most commonly appears there, but more advanced systems also apply machine learning to execution, order routing, transaction-cost and market-impact estimation, risk monitoring, and anomaly detection. The same honest caveats — noise, shifting patterns, and the need for out-of-sample checks — follow it into each of those jobs.
Same Slot, a Different Score
Make it concrete. A hand-written signal might be one line: "score +1 to buy if the price is above its 200-day moving average, otherwise 0." One clue, one rule, written by a person in an afternoon. A machine-learning version of the same slot says instead: "here are 20 years of examples, each tagged with whether the next month was up or down, and 50 possible clues — you figure out how to weigh them into a score."
Both drop a number into the exact same signal-engine box, and both feed the identical rules, risk checks, and execution downstream. The difference is only in how the score is produced — one from a formula a human chose, one from a pattern the software learned. And both face the same final exam: does the score still work on data the system has never seen? If the answer isn't a confident yes, the extra machinery of the learned version bought nothing but a more convincing way to fool yourself.
The Honest Bottom Line
Machine learning is a tool for finding patterns, not a machine that prints money. When a model you don't fully understand is generating the signal, the discipline from earlier lessons — out-of-sample testing, position limits, drawdown caps, a kill switch — matters more, not less, because you can no longer eyeball the logic and sanity-check it in your head. A learned signal deserves more suspicion than a hand-written one, not a pass because it sounds sophisticated.
That's also why this lesson names no specific tools or models. The particular systems change constantly; the principles here — noise, shifting patterns, competition, tiny data, and the discipline they demand — don't. Understand those, and you can read any timely piece about the newest AI trading tool without being dazzled by it.
Sources & Further Reading
- Staff Report on Algorithmic Trading in U.S. Capital Markets (2020) — U.S. Securities and Exchange Commission (describes AI/ML in smart order routing, best execution, and allocation)
- Concept Release on Equity Market Structure (2010) — U.S. Securities and Exchange Commission
Educational use only
Educational content only. StockCram isn't a broker or adviser, and we have no affiliation with any institution or tool we mention. Nothing here is a recommendation to trade in any particular way.