EVT is a branch of statistics that deals with everything at the fringe. You see, the majority of statisticians are like scientists of all stripes and they would rather focus on the 60-70% of the data that conforms to their thesis.

This allows for a faster means of getting things like a Phd, a job and the like. Generally this means that the statisticians are the emperors of the land that Taleb calls ‘mediocristan’. Unfortunately, for you and me - that is not a real place. After all - statistics is an ‘applied’ science. No unified theory here please sir, thank you very much.

And following maps in reality from maps made in another dimension (putting it politely) can get you hurt in a hurry.

Insurance companies on the other hand, are very much interested in reality and they make it their business to understand how it really is - after all, when things go wrong they go wrong in very big ways so, since it’s their business to make money on knowing when and how things could go wrong they use a different kind of map.

Remember the infamous bell curve? Well, if you’ve read ‘The Black Swan’ you’ll know that it doesn’t work like that all the time - in a big enough sample there are extreme ends. Focusing on the middle 1-1.5 deviations will not aptly protect you from the big changes that seem to come over night.

To boil down a lot of academia, that I know most of you are not interested in (except Woody) I’ll put it this way: Extreme Value Theory deals with understanding the data at the far ends of the spectrum, the 2.5-3+ areas of a standard deviation. For an insurance company this is making a data model of a ‘Katrina’ or a 6+ point earthquake in California.

How?

There are two methods, one is to look at ALL the data range and then come up with a formula that describes the far edge cases. This takes a lot of work and most of it is mathematical navel gazing as far as I can tell.

The second method is called ‘Peak over Threshold’ and it categorizes the data into two groups: Group 1 where all the data fits nicely within the bell curve and Group 2 where all the data does not.

Peak over Threshold methodology builds it’s analysis on the idea that in every distribution, one has the ‘tame’ group and the ‘wild’ group. Each has a different behaviour and a different shape.

Therefore, if one wants to examine the ‘wild’ group then one takes the data that is past the threshold and separate it from the rest creating a whole new data set.

From here, the rest of statistics takes over and any tool normally used there can be at will.

Case Study

Yesterday, the The Dow Chemical Company [[DOW]] moved over 300 points. This is a big deal and lots of pundits are calling it the ‘end of the recession’ or whatever their script says for them to mouth.

But in the 2050 days of the market (from finance.yahoo.com) starting from October 1, 1928 - there have been only 25 days where the market has moved up by that much. In fact, up and down the market has only moved 52 times more than an absolute value of 300+ points.

By using EVT, one can separate the two data sets and see the following:

Minimum move: 300.01 points

Maximum move: 684.81 points

Mean (EVT): 385.52

Standard Dev (EVT): 79.96

This means that the average BIG move (which accounts for 3% of the data set) ranges between 305-465 points (adding and subtracting the standard deviation from the mean).

One interesting point:

The first time this happened was 1987. It didn’t happen again until ten years later (there abouts).

Data: