Blog

[dc]M[/dc]y blogging this week turned into a bit more of a stream of consciousness exercise than I’d expected; I needed more words to make fewer points than I’d hoped, and I thank you, faithful reader, for sticking with me for the journey. If you kept thinking “get to the point already!”, know you were not alone. I felt the same way, and, today, I do.

[This is a long post. Expected reading time is about 9 minutes, or you can read the bold highlighted sections in less than 2 minutes.]

I left you yesterday with an interesting situation: I was making money as a trader, but when I really investigated the tools of technical analysis I found deep and serious problems. When I went to experts in the field, I was rebuffed and usually told that it was wrong to focus too much on statistics, rather “you just have to make it work for you.“, backed up with vague appeals to the authority, and that authority came from pattern research done in an era without any computer support, when the field of data analysis was still in development, and when access to market data was spotty and limited–certainly not ideal conditions. Furthermore, it was proving difficult to substantiate their findings because the things that I tested generally tested poorly.

What is technical analysis?

The problem might begin with how we define technical analysis. Here is a brief excerpt from Edwards and Magee:

…two quite distinctive schools of thought have arisen, two radically different methods of arriving at the answers to the trader’s problem of what and when. …one of these is commonly referred to as the fundamental or statistical, and the other as the technical. (In recent years a third approach, the cyclical, has made rapid progress and, although still beset by a “lunatic fringe”, it promises to contribute a great deal to our understanding of economic trends.) The stock market fundamentalist depends on statistics…

Right away, the modern quantitative analyst is likely to be confused by the word “statistics”, but, in this case, the term refers to financial statements, aspects of the company’s competitive position in the industry. So what we have here is just a semantic difference, and fundamental analysis today still broadly encompasses the same areas. They go on to say:

The term “technical”, in its application to the stock market, has come to have a very special meaning, quite different from its ordinary dictionary definition. It refers to the study of the action of the market itself as opposed to the study of the good in which the market deals. Technical Analysis is the science of recording, usually in graphic form, the actual history of trading (price changes, volume of transactions, etc.)… and then deducing from that pictured history the probable future trend.

And, here, I believe we arrive at the central problem of traditional technical analysis: it is presented as a visual discipline. We look at pictures on charts and try to predict the future direction of the market based on those pictures. If we accept that definition at face value (and most of the certification programs in the discipline and most practitioners have wholeheartedly embraced that definition), then technical analysis is mostly untestable and is, at least, highly subjective. People may look at the same picture and draw different conclusions, and who can say who is right and who is wrong? While it might be possible to generate a track record of a specific analyst’s calls, it is not possible to test the actual method. This was not acceptable to me because I didn’t have a lot of money and couldn’t afford to lose. I wanted the security of knowing I was doing something that was likely to work in the future; so, you see, my obsession was driven by necessity.

In the opening pages of my book, I defined technical analysis more broadly by saying, “technical traders and analysts make decisions based on information contained in past price changes themselves.” That, to me, is the essence of the discipline, and the key distinction between technical and fundamental. Technical decisions are made based on information contained in the movements of the market (perhaps with additional information beyond price such as volume, whether a trade was on the bid or ask, coincident movements in related markets, etc.), while fundamental analysis incorporates outside factors. With those definitions, we are ready to make some progress.

Charts are useful

Now, don’t get me wrong. I’m not saying we should throw charts out. Some investment professionals are very prejudiced against charts, but it is the application of charts that is the problem. A chart is a visual display of data. The visual display may be useful because it speaks to “more” of our cognitive capacity than a simple list of numbers can. In other words, by seeing pictures and shapes on charts, it is quite possible that we are engaging some inductive reasoning, and some other ways of knowing that may transcend logical analysis. So, don’t throw your charts out, yet, and if you laugh at charts because you’re tired of hearing people talk about the “double dipping two dog reversal pattern aligning with the harmonic convergence of the third wave’s structural retracement ratios”, know that your problem is with the way charts are used, not with charts themselves.

There is no free lunch

This is one of the core principles of finance. Though the details may be complex, I believe core truths must always be simple and readily explainable–I use the “smart eight year old” test. ((The other principles of modern finance, in my opinion, are also simple: 1) A dollar today is worth more than a dollar tomorrow. 2) People should demand more reward for more risk, and 3) people sometimes make mistakes.)) Markets are highly competitive, with a lot of smart people with a lot of money trying to make profits. Arbitrage opportunities (basically, free money) do exist, but they are rare, usually disappear quickly, and can often only be profitable for people with special access or conditions. There’s that old joke about an economist who comes upon a $1,000 bill lying on the sidewalk and won’t pick it up because it “can’t be there”, and there’s a grain of truth in that joke. We may sometimes find “gifts” in the market, but it is exceedingly rare, and we can’t count on it. If you think you’ve found a magical system that beats the market consistently and soundly, you’ve made a mistake in your analysis. If someone is selling you such a system, they are lying.

There are patterns in prices

Here, I depart from some of the academic research, that suggests that prices are wholly described by random walks. More research, however, supports the idea that there are some patterns and some predictable elements in prices. This should tell you just how thin the statistical edges (if they exist) actually are. Many traders seem to discount academic research as irrelevant; I think that’s a mistake, but if you choose to do so you must at least admit that researchers are highly educated, intelligent, and spend a lot of time carefully looking at data. Some researchers find contradictory evidence. In some cases, this points to problems with methodology, but, more often, it is that different standards of proof are used and the statistical edges only exist in some markets for some time periods. My work suggests that there are some rather simple patterns that work with a degree of reliability, but, again, the edges are small.

We can test patterns objectively, but we must be careful

One of the perennial defenses of technical analysis is “you can’t test it. You just have to figure out what works for you“, with the emphasis clearly on the “for you”. If you can’t make money with the method, then the problem is yours because you just didn’t figure out how to use it, or maybe you just need to work on your trading psychology. However, back up a minute. Consider the well-known nugget of statistical wisdom: correlation does not imply causation. Yes, I think most of us know this, but what might not be so clear is that causation does imply correlation. In other words, you can’t have a relationship in the data that is both real and important, and also invisible. Now, step back and apply that to technical analysis. If something works, then it should work objectively. Fibonacci ratios define and limit retracements in financial markets? Ok, then retracements should terminate more often near those ratios. Moving averages provide support and resistance? Ok, then when price touches a moving average, we should see non-random price movement follow. (Note that I don’t know what will happen after price touches, but I do know it must somehow be non-random. If what happens to a market after an event is no different from any other time, it was, by definition, a “non-event”.) Bollinger bands set +/- 2.0 standard deviations around a moving average contain 95% of price action? Ok, well that’s not too hard to test, either.

We have to be humble and objective in our testing; any test is a joint test of the test itself and whatever pattern might or might not be in the data. In other words, we have to think of a way to test an idea, and maybe we thought of the wrong way or missed something. That’s certainly possible, but I don’t see much of this work being done in technical analysis. The examples I gave in the previous paragraph range from fairly hard to test to very simple. The Fibonacci tests require many assumptions and we can always say maybe we missed something (though, again, once we test it enough ways and don’t find support, we have to seriously ask how useful it can be if it is that difficult to tease it out of the data). The Bollinger test is extremely simple and anyone can do it with free data in a few hours, yet nearly every book on technical analysis repeats that completely bogus statistic on Bollinger Bands. (Hint: the rule of thumb would only be valid if price deviations from a moving average were normally distributed, and they clearly are not.)

It is easy to be misled by patterns

I’ve written on this fairly extensively in the past, as have many other writers, so we don’t need to belabor the point here. Just know that we are basically wired to find patterns, and we will find them whether or not they actually exist. (Think of faces in clouds or voices in the seashell.) Furthermore, we tend to attach too much importance to memorable events. It is entirely possible to have a series of trades that are, overall, losing trades, but to remember them as an overall win because of one big or exciting trade. (Be careful how you do your trade review.) You can’t “fix” these problems because they are fundamental part of the way we think and process information. What you can do, what you must do, is to counterbalance these foibles with objective, statistical analysis.

You have to be literate in basic math and finance

There’s no way around this. Though we all like the idea that you just have to learn some pattern and trade it to make money, the world doesn’t work like that. (There is no free lunch, remember?) You should understand everything you can about the market, finance in general, the world, human behavior… the list goes on and on. (In response to a question, I created a reading list here. This might seem daunting, but what it really is is a curriculum of study.) You don’t need to do this work to draw trendlines, but you do need a pretty deep understanding of statistics and probability to understand what might happen after a trendline is touched. While you’re at it, it would be helpful to have some ability to analyze data. Microsoft Excel is not ideal, but better than nothing (and it has a short learning curve.) Consider Stata, R, Python or learning another “real” programming language. You don’t want to be a programmer, but you do need to have the skills to rip through a dataset.

Think objectively

Other writers have made this call, and I will reiterate it. The discipline of technical analysis must move toward understanding the objective tendencies of price movement. Doing so requires correct methodology, careful thought and review, and a willingness to discard what is not useful. Actually, I don’t know if it “must”. This work is already being done by swarms of quantitative analysts, but traditional technical analysis is stuck. Perhaps it is because of the roots of the discipline as a visual exercise, but I also wonder if it is, to some degree, simply laziness and a resistance to change. A friend of mine pointed out that many of the certification programs for technical analysis include a provision that you cannot criticize any work done by any other technical analyst. While that might work toward building a supportive community, that requirement is anathema to scientific progress. Science depends on people asking questions and working to find answers, and part of that process is reviewing the work done by other people, expanding on it, finding problems, and fixing what is wrong. Why is technical analysis any different? Why is peer review so thoroughly shunned by the community?

Intuition is real

Now, I realize I have perhaps just offended some people, so let me offend a different group here. (I’m joking, or at least I hope I am.) Objective, statistical analysis is critical, but there is also a place for intuition. There are other ways of knowing that go beyond rational analysis; intuition is real, valuable, and, I think, it can be trained. I have done extensive work on this subject, and even slipped a very powerful intuition building exercise into my trading course. (If you took the course, did you find it?) I have done some research and statistical work on what might be charitably called the “fringe” of perception, and there appears to be statistical support for intuition, even bordering on some aspects that people might call “ESP”. There are probably aspects of market behavior, and certainly aspects of our individual decisions as traders or managers that we might not be able to quantify, and that is ok. Leave some room for intuition.

However, intuition must be guided by an understanding of how markets really move. For instance, I once sat in a training class for new intraday traders who were told to hold their stocks overnight if they made very large moves the day before. When I asked why, I was told that stocks usually continue the next day after a big move. Now, one of the strongest statistical tendencies in the market is for stocks to mean revert after large one day moves. This is only one example, but does it make any sense to train intuition against a strong statistical bias? There may be situations in which you wish to take a position against the statistical bias, but you still must understand which direction the wind usually blows.

Why does this matter?

This all matters because, very simply, we have a responsibility. If we write books, blogs, or even just share ideas online, don’t we have a responsibility to make sure the tools we use are the best they can be? If you are trading your own money, then you have an obligation to yourself and to your family. If you mange client money, then you have a supreme responsibility to the client. The point of all our work and analysis must be to manage risk and find opportunity in the markets, and how can we do that unless we are willing to ask the hard questions of ourselves and of our methodology? Ask those questions, and don’t be afraid of the answers.

Tomorrow, I’ll wrap this up with a look at some simple, quantitative patterns that do work, and how charts can sometimes be used to find them.