EVT is a branch of statistics that deals with everything at the fringe. You see, the majority of statisticians are like scientists of all stripes and they would rather focus on the 60-70% of the data that conforms to their thesis.

This allows for a faster means of getting things like a Phd, a job and the like. Generally this means that the statisticians are the emperors of the land that Taleb calls ‘mediocristan’. Unfortunately, for you and me – that is not a real place. After all – statistics is an ‘applied’ science. No unified theory here please sir, thank you very much.

And following maps in reality from maps made in another dimension (putting it politely) can get you hurt in a hurry.

Insurance companies on the other hand, are very much interested in reality and they make it their business to understand how it really is – after all, when things go wrong they go wrong in very big ways so, since it’s their business to make money on knowing when and how things could go wrong they use a different kind of map.

Remember the infamous bell curve? Well, if you’ve read ‘The Black Swan’ you’ll know that it doesn’t work like that all the time – in a big enough sample there are extreme ends. Focusing on the middle 1-1.5 deviations will not aptly protect you from the big changes that seem to come over night.

To boil down a lot of academia, that I know most of you are not interested in (except Woody) I’ll put it this way: Extreme Value Theory deals with understanding the data at the far ends of the spectrum, the 2.5-3+ areas of a standard deviation. For an insurance company this is making a data model of a ‘Katrina’ or a 6+ point earthquake in California.

How?

There are two methods, one is to look at ALL the data range and then come up with a formula that describes the far edge cases. This takes a lot of work and most of it is mathematical navel gazing as far as I can tell.

The second method is called ‘Peak over Threshold’ and it categorizes the data into two groups: Group 1 where all the data fits nicely within the bell curve and Group 2 where all the data does not.

Peak over Threshold methodology builds it’s analysis on the idea that in every distribution, one has the ‘tame’ group and the ‘wild’ group. Each has a different behaviour and a different shape.

Therefore, if one wants to examine the ‘wild’ group then one takes the data that is past the threshold and separate it from the rest creating a whole new data set.

From here, the rest of statistics takes over and any tool normally used there can be at will.

Case Study

Yesterday, the The Dow Chemical Company [[DOW]] moved over 300 points. This is a big deal and lots of pundits are calling it the ‘end of the recession’ or whatever their script says for them to mouth.

But in the 2050 days of the market (from finance.yahoo.com) starting from October 1, 1928 – there have been only 25 days where the market has moved up by that much. In fact, up and down the market has only moved 52 times more than an absolute value of 300+ points.

By using EVT, one can separate the two data sets and see the following:

Minimum move: 300.01 points

Maximum move: 684.81 points

Mean (EVT): 385.52

Standard Dev (EVT): 79.96

This means that the average BIG move (which accounts for 3% of the data set) ranges between 305-465 points (adding and subtracting the standard deviation from the mean).

One interesting point:

The first time this happened was 1987. It didn’t happen again until ten years later (there abouts).

Data:

Date	Open	High	Low	Close	Volume	Adj Close	Move	Absolute	up over 300?	down > 300	move more 300
10/19/87	2164.16	2164.16	1677.55	1738.74	604300000	1738.74	-507.99	507.99	0	1	1
10/27/97	7715.41	7717.37	7150.1	7161.15	693730000	7161.15	-554.26	554.26	0	1	1
10/28/97	7161.15	7553.57	6936.45	7498.32	1202550000	7498.32	337.17	337.17	1	0	1
08/27/98	8377.89	8448.69	8062.24	8165.99	938600000	8165.99	-357.36	357.36	0	1	1
08/31/98	8078.97	8149	7517.7	7539.07	917500000	7539.07	-512.61	512.61	0	1	1
09/08/98	7964.91	8103.69	7779.03	8020.78	814800000	8020.78	380.53	380.53	1	0	1
10/15/98	7953.07	8375.57	7885.62	8299.36	937600000	8299.36	330.58	330.58	1	0	1
01/04/00	11349.75	11358.44	10907.03	10997.93	1009000000	10997.93	-359.58	359.58	0	1	1
03/07/00	10197.61	10208.66	9651.77	9796.03	1314100000	9796.03	-374.47	374.47	0	1	1
03/15/00	9808.15	10294.6	9676.9	10131.41	1302800000	10131.41	320.17	320.17	1	0	1
03/16/00	10139.58	10716.23	10139.58	10630.6	1482300000	10630.6	499.19	499.19	1	0	1
04/03/00	10863.28	11344.17	10821.71	11221.93	1021700000	11221.93	300.01	300.01	1	0	1
04/14/00	10922.85	10922.85	10173.92	10305.77	1279700000	10305.77	-617.78	617.78	0	1	1
10/12/00	10424.14	10462.25	9873.66	10034.58	1388600000	10034.58	-379.21	379.21	0	1	1
12/05/00	10576.78	11044.7	10504.36	10898.72	900300000	10898.72	337.77	337.77	1	0	1
03/12/01	10638.52	10638.63	10138.9	10208.25	1229000000	10208.25	-436.37	436.37	0	1	1
03/14/01	10279.42	10279.42	9817.74	9973.46	1397400000	9973.46	-317.34	317.34	0	1	1
04/05/01	9527.21	9969.92	9527.21	9918.05	1368000000	9918.05	402.63	402.63	1	0	1
04/18/01	10226.88	10806.41	10215.69	10615.83	1918900000	10615.83	399.1	399.1	1	0	1
05/16/01	10864.74	11258.21	10779.66	11215.92	1405300000	11215.92	342.95	342.95	1	0	1
09/17/01	9294.55	9294.55	8755.46	8920.7	2330830000	8920.7	-684.81	684.81	0	1	1
09/20/01	8375.72	8711.38	8304.45	8376.21	2004800000	8376.21	-382.92	382.92	0	1	1
09/24/01	8242.32	8733.39	8242.32	8603.86	1746600000	8603.86	368.05	368.05	1	0	1
05/08/02	9847.96	10203.76	9847.96	10141.83	1502000000	10141.83	305.28	305.28	1	0	1
07/05/02	9061.54	9399.65	9054.97	9379.5	699400000	9379.5	324.53	324.53	1	0	1
07/19/02	8356.74	8356.74	7940.83	8019.26	2654100000	8019.26	-390.23	390.23	0	1	1
07/24/02	7698.46	8243.07	7489.53	8191.29	2775560000	8191.29	488.95	488.95	1	0	1
07/29/02	8267.99	8749.12	8267.99	8711.88	1778650000	8711.88	447.49	447.49	1	0	1
09/03/02	8659.27	8659.27	8282.87	8308.05	1323400000	8308.05	-355.45	355.45	0	1	1
10/01/02	7593.04	7964.24	7558.36	7938.79	1780900000	7938.79	346.86	346.86	1	0	1
10/11/02	7540.74	7919.57	7540.74	7850.29	1854130000	7850.29	316.34	316.34	1	0	1
10/15/02	7883.23	8304.58	7883.23	8255.68	1956000000	8255.68	378.28	378.28	1	0	1
03/24/03	8514.82	8514.82	8166.78	8214.68	1293000000	8214.68	-307.29	307.29	0	1	1
02/27/07	12628.9	12628.9	12078.85	12216.24	4065230000	12216.24	-416.02	416.02	0	1	1
07/26/07	13783.12	13793.61	13307.74	13473.57	4472550000	13473.57	-312.22	312.22	0	1	1
08/09/07	13652.33	13675.66	13196.05	13270.68	5889600000	13270.68	-387.18	387.18	0	1	1
09/18/07	13403.18	13772.15	13379.68	13739.39	3708940000	13739.39	335.97	335.97	1	0	1
10/19/07	13888.47	13888.47	13478.94	13522.02	4160970000	13522.02	-366.94	366.94	0	1	1
11/01/07	13924.16	13924.16	13522.75	13567.87	4241470000	13567.87	-362.14	362.14	0	1	1
11/07/07	13646.72	13646.72	13269.46	13300.02	4353160000	13300.02	-360.92	360.92	0	1	1
11/13/07	12975.11	13357.57	12975.11	13307.09	4141310000	13307.09	319.54	319.54	1	0	1
11/28/07	12958.04	13353.51	12958.04	13289.45	4508020000	13289.45	331.01	331.01	1	0	1
01/17/08	12467.05	12597.85	12089.38	12159.21	5303130000	12159.21	-306.95	306.95	0	1	1
02/05/08	12631.85	12631.85	12234.97	12265.13	4315740000	12265.13	-370.03	370.03	0	1	1
02/29/08	12579.58	12579.58	12210.3	12266.39	4426730000	12266.39	-315.79	315.79	0	1	1
03/11/08	11741.33	12205.98	11741.33	12156.81	5109080000	12156.81	416.66	416.66	1	0	1
03/18/08	11975.92	12411.63	11975.92	12392.66	5335630000	12392.66	420.41	420.41	1	0	1
04/01/08	12266.64	12693.93	12266.64	12654.36	4745120000	12654.36	391.47	391.47	1	0	1
06/06/08	12602.74	12602.74	12180.5	12209.81	4771660000	12209.81	-394.64	394.64	0	1	1
06/26/08	11808.57	11808.57	11431.92	11453.42	5231280000	11453.42	-358.41	358.41	0	1	1
08/05/08	11286.02	11652.24	11286.02	11615.77	1219310000	11615.77	331.62	331.62	1	0	1