“Learn from the market. Let the market be your teacher.”
This sounds like the path to true trading wisdom (and it is), but there’s an obvious question: how do we learn from the market?
Everything in trading reduces down to patterns and statistics. If you find that statement challenging, perhaps it’s because you are locked into one way of understanding “patterns.” We don’t just mean patterns on charts.
It’s all patterns
Fundamental analysis is a type of pattern recognition—when companies have certain characteristics in their financial structure, something tends to happen to the stock price. When a company has a certain position relative to its competitors, something tends to happen to the stock price.
Even indexing and dollar cost averaging rely on a type of pattern. Options premium sellers rely on patterns (though I would argue they usually have an incomplete understanding of the full pattern!).
If you doubt the validity of my claim (that it’s all patterns) then consider this—at the end of your trading or investing career, your accounts will stand as a statistical test of the patterns of your own trading. You likely made money at some points. You lost at others, and you paid a constant vig in the form of commissions, fees, and spreads that were probably offset by some form of yield. The overall gains and losses in your account will be the record of how these patterns played out in the market.
The real question is something like “how can we separate out valid, repeatable patterns from a noisy market, and do so within our limited lifetimes?” There are, broadly speaking, three approaches to solving this problem.
- Trade for a very long time. Over time, a trader will gain some intuitive understanding of how the market works and can speak with some authority on the odds in any situation.
- Do heavy duty statistical analysis on the market, perhaps aided by some machine learning or AI.
- Do semi-discretionary pattern analysis, including a lot of work by hand.
Each of these approaches gives a slightly different perspective, and each has some specific issues or problems.
Old trader smart trader?
When we trade for a very long time, we see the same patterns over and over. Once we’ve seen enough, we start to get a sense of what usually happens. An experienced trader can say “this usually happens when this appears,” but there are some very serious issues to consider.
First, a trader’s intuitive sense of probabilities—even an experienced, profitable trader—may be nonsense. It’s not uncommon to hear a trader say, “This is an 80% sure thing.” Well, no, it’s not. That type of probability (assuming a 1:1 movement) simply does not exist in financial markets.
I once had a trader who was a principal for a trading firm tell me, “I always trade with a 5 to 1 risk ratio and only take trades that have an 80% chance of working out.” Significantly, this trader was not profitable, at least during the period in which I knew his trading. Stats like that simply do not exist, and if they did, this trader would have been able to buy a small nation of his choice with a year’s trading profits.
Another issue is that cognitive biases are strong. Traders can easily anchor and attach too much significance to winning and losing trades, especially if they were dramatic. It’s very difficult to achieve the emotional equipoise needed to pull out valid stats from actual trading.
Still, don’t shut yourself off from this kind of insight. Many of my good trading ideas have come from observations while in trades—seeing things work or not work in a certain way, and then digging deeper with other tools. Let’s talk about those tools.
Maybe a little science will help?
There’s much glitz and glamour around data science. The success of quant firms such as Renaissance, the easy availability of data, and access to powerful analytical tools (R, Python, Matlab, etc.) seem to put success within the reach of anyone with a computer.
And indeed, we can glean meaningful insights into the way markets move from the right kind of analysis. It is not, however, as easy as you might think. First, we have to specify very precisely what we are going to test. Any statistical test is a joint result of both the precise specification of the test and of whatever underlying tendency might or might not be in the market.
Consider this: let’s say you want to test something during a trending phase in the market. Great—how do you define trending? Is it the close relative to a moving average? What moving average? How many closes to reverse a trend? Is there a band of uncertainty around the average? Should we be using two averages? Which averages? What about periods where they chop? Must this be accepted or should there be some rules? Should we use a pure price action specification? How about swing point analysis?
You can see that simply saying “in an uptrend” raises many questions, if we are going to do real statistical work. We might, for instance, find something to be worthless in a test, but perhaps a different way of defining the condition might have given wildly different results.
There are also questions about which techniques are appropriate. To give a single example: it’s very easy to (literally) curve fit with a polynomial regression but assuming linearity is not necessarily the best course. Much of the ignorance abounding in technical analysis over correlation and causation could be overcome with a little Granger Causality testing (but for people stuck on Fibonacci ratios, we’re not likely to get here!). Significant testing and understanding economic significance (i.e., is the edge big enough to overcome the vig?) are further questions, and all of these tools must be applied rigorously and with understanding.
These are valid—and almost essential—tools, but they are not everything. Taleb said, “Believe half of what you read, none of what you hear. Never study a theory before doing your own observation and thinking. Read every piece of theoretical research you can—but stay a trader. An unguarded study of lower quantitative methods will rob you of your insight.”
How do we gain real insight?
The third path
The last possibility is to combine your discretion with some hard statistics. Simply looking at a chart is a passive exercise, and I think traders gain relatively little from it. (Though this is one of the most common ways new traders start to learn, it’s largely a waste of time.)
Rather than looking at a chart and “looking for patterns,” it’s far better to precisely define a condition, and then to comb through charts, bar by bar, looking for occurrences. For a new trader, it can be something simple like an outside bar, a bar whose high is the highest high of the last 3 bars, etc. At first, you’re not necessarily even looking for a valid trading approach; you’re just looking to engage the pattern recognition part of your brain.
How many charts is enough? Well, I have some bad news for you. Contrary to the glitzy scammers who tell you that all you have to do is to pay them $3000 for a course, trading is hard work. If you want to succeed, it’s going to take time and energy, and you’re going to have to physically change your brain through exposure to data.
I would say a useful “run” of looking at a single pattern probably covers about 4,000 – 5,000 bars. This is roughly a quarter of 5 minute intraday bars, or 20 markets of daily bars over a year. It will take several hours to do a single pattern run on this dataset, if you do it carefully.
Why would you do this, and spend 50-100 hours on a research project? Because there is insight and wisdom available here that you simply cannot find anywhere else. It’s hard work. It’s boring. It’s a sacrifice because you could be doing something more fulfilling with the time you spend on this, but it’s a sacrifice that will be repaid in the end.
What does this mean for you?
Most of you reading this have done one or two of these approaches, but I would challenge you to rethink your own development process and see if there’s something you’re missing, something you could do better. In my next post, I’ll share some of my personal experiences from these past few months, and tell you what I’ve learned new from markets, even after decades of trading full time.
