On my way to writing about Information Theory, I realized that there is something missing from about 99% of all writing done regarding stock trading theories and methodologies. And it bugs the crap out of me, it really does.

How many times have you heard someone say “I have this trading method that can’t go wrong. I can time both the markets and individual stocks coming and going. I got it on lock and won’t let go..blah blah boast, self aggrandizement, ego stroke, blah blah” ?

And when questioned closely about their strategy, if said braggart is willing to come clean with their performance, you may get a more reasonable response of “I am right on my calls by 60-70%”. And one thinks, wow perhaps they do know whereof they speak of. It’s common knowledge that most fund managers and professionals are at best right about 60% of the time.

If, at this point they want your money to explain in nauseating detail exactly how you too, could become wealthy beyond your wildest dreams, then my advice is to run, not walk away. It’s a con pure and simple.  As Ice-T once said “real killers move in silence”. [and if I'm wrong on the quote, post it in the comments and I'll correct myself. I am known to be wrong on many occasions. Just ask my wife.]

However, if you have some time you want to kill and like shooting down egos that have grown large enough to shove the air out of an economy sized sedan, then pay attention because the surest way to do that is to use the idea of significance to judge the performance.

The easiest way to judge data coming at you, whether in the form of some self aggrandizing bozo that has too much time on their hands to post on a blog or forum, or in the form of self serving statistics coming from your boss, your local politician, or spouse (90% of the time you do such and such when you take the trash out) is to engage in Significance Testing.

I could bore you stupid with the example that is listed in the above link but, I won’t. I think there is a much simpler way to put it and more relevant for iBC and so here goes.

Bozo: If you use dizzlebop indicator on the third minute of the fourth hour and it matches the spinning doji then there is an 80% chance that the stock will retrace enough for you to buy.

Intrepid iBC reader: 80%? Hmmm. One question – when you were doing your testing how large was your sample size?

Bozo: Large enough. If you want full details, please send monthly payments of $600 to this postal box in Maryland.

So, let’s regroup here. If the dizzlebop indicator and the doji are in alignment then there is (probably) an 80% chance of retracement. Well, how significant is 80%?

At first glance, 80% seems pretty good. After all, Van Tharp states that with proper money management even 60% is enough to make serious bank if one has enough plays. He even has this nifty app written in Visual Basic that one can download the first three levels of at his website. The rest of his training is only $195 in one easy payment.

Put another way, you’re at a dinner party and the spouse of your best friend from high school has announced the sudden outbreak of stupendous psychic powers. You grab hattery’s deck of cards and decide to test this (potentially) amazing Greshkin.

The question is: How many cards will it take for them to predict to support their claim?

To make things simple for us all, let’s say that the Amazing Greshkin must have the same performance level as the dizzlebop/doji indicator – 80%. To do that we will need to check against whether their prediction is correct or if they are experiencing a run of good luck.

OK – so how many cards do they have to pick right?

The fast way to calculate this is to use the Chi Square test. I suggest using the one at Graphpad because it’s fast and cool and relates to biostatistics (which I find kind of interesting anyways).

So, we decide that someone with a random streak would be right 50% of the time if we’re choosing Red or Black. (Pretend you’re at hattery’s casino).

The fast way would be to shuffle the cards and then do a draw of ten.  80% of 10 would be 8 so you decide that if they pick correctly 8  times then they support their claim.

and you’d be wrong.

To follow along with this post, go to the link above and put in the following

Category /Observed#/Expected

Right/8/5

Wrong/2/5

The results are:

Chi squared equals 3.600 with 1 degrees of freedom.
The two-tailed P value equals 0.0578
By conventional criteria, this difference is considered to be not quite statistically significant.

The P value answers this question: If the theory that generated the expected values were correct, what is the probability of observing such a large discrepancy (or larger) between observed and expected values? A small P value is evidence that the data are not sampled from the distribution you expected.

 So, with a sample size of 10 – 80% is not nearly enough.

So how big a sample size? If you double all the figures above making a sample size of twenty – then the results say:

Chi squared equals 7.200 with 1 degrees of freedom.
The two-tailed P value equals 0.0073
By conventional criteria, this difference is considered to be very statistically significant.

In Conclusion:

I left a whole lot out about designing proper experiments, how to create a hypothesis etc. because I feel it would probably go over about as well as a lead balloon. Suffice it enough to say that when Curtis Faith talks about using his Trading Blox software to backtest various methodologies – I pay attention because it doesn’t matter how right one is – it only matters how consistently right one is. Even the best backtested trading strategy in his book was right about 57% of the time.

57% properly dealt with can mean lots of money – but, there needs to be levels of transparency with regards to presenting a trading technique. If there isn’t – meaning if they haven’t checked against the possibility that the process could be just a streak of good luck moving in their direction – then don’t spend any money.

This also is a good way to test one’s own history. If a trader keeps a journal and can put in the number of times their strategy was right and the number of times it was wrong and has a history of at least twenty or more incidents then perhaps some objective praise is warranted. If not, well you’re better off at your local bingo parlour.