Frank,
This is where Bayesian theory (and more accurate "a priori" beliefs)
allow more accurate probability calculations.
If you use past data, and assume P(all events) = equal, then you often
run into pre-selection bias; e.g. I picked a weird set of data.
So Bayesian analysis will use P(my data set is unusual) = whatever you
set.
Another example, say I'm considering video poker games in Las Vegas at
1) major casino in Las Vegas - P(prior belief in gaffed machine) < 0.01
2) non-name casino at Indian reservation where other people are
reporting suspicious result - P(prior belief in gaffed machines) = 0.25
Then P(belief machine is gaffed after test | prior belief) = function
of test result and P(prior belief in gaffed machines).
If you use a non-random set of data (e.g. data you have gathered
before), then
P(belief machine is gaffed) = function of test result and P(prior
belief in gaffed machines) and selection-bias-in-original test)
Mitchell
A similar example of selection-bias is one about weather.
My friend tells me that last week it rained 6 out of 7 days, and they
ask how unusual that is...
Most people will just calculate how unlikely it is to have rain 6 of 7
days.
A better calculation will take into account that my friend is only
telling me this because it is "somehow weird" (e.g. no royals in
120,000 hands)
and factor in the "selection-bias".
On Apr 16, 2012, at 3:09 PM, Frank wrote:
> OK. You completely misunderstood what I was saying. It will
> completely invalidate the testing utility I'm making if people use
> their currently existing data. Why? Imagine this.
>
> You post in the newspaper that you'd like to do a study into how
> likely it is to be hit by lighting. Not surprisingly, the people
> that answer your add are those most concerned about this issue (AKA
> people that have been hit). After looking at all your volunteer test
> subjects you conclude that the chances of being hit by lighting are
> 1 in 1.
>
