Weblog:Bayesian data analysis in English - Revision history

Seth at 21:49, 9 March 2011

2011-03-09T21:49:24Z

Seth at 21:46, 9 March 2011

2011-03-09T21:46:05Z

Seth: Created page with 'There is a paper currently in press at an important social psychology journal. Using a really simple procedure, this fella [http://dbem.ws/ Bem] has provided [http://dbem.ws/ so…'

2011-03-09T20:58:56Z

Created page with 'There is a paper currently in press at an important social psychology journal. Using a really simple procedure, this fella [http://dbem.ws/ Bem] has provided [http://dbem.ws/ so…'

New page

There is a paper currently in press at an important social psychology journal. Using a really simple procedure, this fella [http://dbem.ws/ Bem] has provided [http://dbem.ws/ some evidence] for the existence of precognition (having carefully excluded both psychokinesis and clairvoyance as alternate possibilities). He used the same basic statistics that are the basis of all empirical science for the past century-or-so.

So, do scientists accept the existence of precognition or uproot one of the foundations of science? They are doing the second one. What's even better is that fact that that's OK. It turns out that the statistical foundation of science, null hypothesis significance testing, has a lot of problems. But it is so entrenched that it takes a really compelling false positive to motivate people to do anything about it.

So the alternative statistical approaches--contenders for the title, are getting more attention. One of the more notable approaches is called "Bayesian data analysis." It is nice because it is a mathy way of framing what scientists do in the day-to-day:

You take what you came in believing, then you see something new, and you come out with beliefs that have changed given what you just saw. They might be the same, they might be different. It might be that you didn't know what you believed at first---that a lot of perspectives on the world made sense before---but after seeing what you saw only one or a few of those beliefs make sense anymore. Or it could go the either way, that you had strong beliefs but seeing the world made you realize that other perspectives are possible. Either way, your beliefs change in response to what you are experiencing. Of course, this doesn't represent all of science, but pinning down this small part of it is really useful.

And you actually can describe this same process in math. You had some ''prior'' plausible beliefs about what kinds of things are likely to happen in the world (''A'' and ''B''). Then you actually saw things happening in the world (''X'', the data). What is the liklihood of what just happened from the perspectives of the different things you believe? (What are the probabilities of ''X'' given ''A'' and of ''X'' given ''B''? In symbols, what is ''p(X|A)''). Once you calculate that, you can figure out the reverse: How likely are your beliefs given what just happened---what are the probabilities of ''A'' and ''B'' given ''X'' (''p(A|X)''). This figuring out the reverse is where the name comes from---Bayes rules is an expression relating p(A|X) to p(X|A). With these values, you can do other cool things, like find the number for how much more likely A is than B (''p(A|X)/p(B|X)'').

So a good experiment will speak to many possible incompatible beliefs, and make most of them really unlikely no matter how it turns out. Bayesian data analysis is a way of calculating which of your priors you believe in the most after seeing what just happened. Designing a good "what just happened" is up to the scientist, and that is where a lot of the creativity comes in.

Bayesian methods have actually been around for a while, but you need computers to calculate all of the different values involved. So the technique has only started to become popular with the personal computer and modern computing. Another post describes how to do it on a mac.

@@ Line 7: / Line 7: @@
 You go into a room with prior beliefs about the world  In the room you see something new, and you come out with beliefs that have changed given what you just saw.  They might be mostly the same or completely different.  It might be that you didn't know what you believed at first---that a lot of perspectives on the world made sense before---but after seeing what you saw only one or a few of those beliefs make sense anymore.  Or it could go the either way, that you had strong beliefs but seeing the world made you realize that other perspectives are possible.  Either way, your beliefs change in response to what you experienced.  Of course, this doesn't represent all of science, but pinning down this small part of it is a really useful way of doing statistics.
-You actually can describe this same process in math.  You had some plausible ''prior'' beliefs about what kinds of things are likely to happen in the world (''A'' and ''B'').  Then you actually saw things happening in the world (''X'', the data).  What is the likelihood of what just happened from the perspectives of the different things you believe? (What is the probability of ''X'' given ''A'', what is ''p(X|A)''?  And what about ''B'').  Once you calculate that, you can figure out the reverse: How likely are your beliefs given what just happened---what are the probabilities of ''A'' and ''B'' given ''X'' (''p(A|X)'').  Figuring out the reverse is where the name comes from---Bayes rules is an expression relating p(A|X) to p(X|A).  With these values, you can do other cool things, like find the number for how much more likely A is than B (''p(A|X)/p(B|X)'').
+You actually can describe this same process in math.  You had some plausible ''prior'' beliefs about what kinds of things are likely to happen in the world (''A'' and ''B'').  Then you actually saw things happening in the world (''X'', the data).  What is the likelihood of what just happened from the perspectives of the different things you believe? (What is the probability of ''X'' given ''A'', what is ''p(X|A)''?  And what about ''B''?).  Once you calculate that, you can figure out the reverse: How likely are your beliefs given what just happened---what are the probabilities of ''A'' and ''B'' given ''X'' (''p(A|X)'')"  Figuring out the reverse is where the name comes from---[http://en.wikipedia.org/wiki/Bayes%27_theorem Bayes' rule] is an expression relating p(A|X) to p(X|A).  With these values, you can do other important things, like find the number for how much more likely belief ''A'' is than ''B'' (''p(A|X)/p(B|X)'').
 So a good experiment will speak to many possible incompatible beliefs, and make most of them really unlikely no matter how it turns out.  Bayesian data analysis is a way of calculating which of your priors you believe in the most after seeing what just happened.  Designing a good "what just happened" is up to the scientist, and that is where a lot of the creativity comes in.
 Bayesian methods have actually been around for a while, but you need computers to calculate all of the  different values involved.  So the technique has only started to become popular with the personal computer and modern computing.  Another post describes how to do it [[Blog:Bayesian_data_analysis_on_a_Mac|on a Mac]].

@@ Line 1: / Line 1: @@
 There is a paper currently in press at an important social psychology journal.  Using a really simple procedure, this fella [http://dbem.ws/ Bem] has provided [http://dbem.ws/ some evidence] for the existence of precognition (having carefully excluded both psychokinesis and clairvoyance as alternate possibilities).  He used the same basic statistics that are the basis of all empirical science for the past century-or-so.
-So, do scientists accept the existence of precognition or uproot one of the foundations of science?  They are doing the second one.  What's even better is that fact that that's OK.  It turns out that the statistical foundation of science, null hypothesis significance testing, has a lot of problems.  But it is so entrenched that it takes a really compelling false positive to motivate people to do anything about it.
+So, do scientists accept the existence of precognition or uproot one of the foundations of science?  They do the second one.  What's even better is that that's OK.  It turns out that the statistical foundation of 20th century science, null hypothesis significance testing, has a lot of problems.  But it is so entrenched that it takes a really compelling false positive to motivate people to do anything about it.
-So the alternative statistical approaches--contenders for the title, are getting more attention.  One of the more notable approaches is called "Bayesian data analysis."  It is nice because it is a mathy way of framing what scientists do in the day-to-day:
+So the alternative statistical approaches---contenders for hegemony over 21st century statistics---are getting more attention.  One of the more notable approaches is called "Bayesian data analysis."  It is nice because it is a mathy way of framing what scientists do in the day-to-day:
-You take what you came in believing, then you see something new, and you come out with beliefs that have changed given what you just saw.  They might be the same, they might be different.  It might be that you didn't know what you believed at first---that a lot of perspectives on the world made sense before---but after seeing what you saw only one or a few of those beliefs make sense anymore.  Or it could go the either way, that you had strong beliefs but seeing the world made you realize that other perspectives are possible.  Either way, your beliefs change in response to what you are experiencing.  Of course, this doesn't represent all of science, but pinning down this small part of it is really useful.
+You go into a room with prior beliefs about the world  In the room you see something new, and you come out with beliefs that have changed given what you just saw.  They might be mostly the same or completely different.  It might be that you didn't know what you believed at first---that a lot of perspectives on the world made sense before---but after seeing what you saw only one or a few of those beliefs make sense anymore.  Or it could go the either way, that you had strong beliefs but seeing the world made you realize that other perspectives are possible.  Either way, your beliefs change in response to what you experienced.  Of course, this doesn't represent all of science, but pinning down this small part of it is a really useful way of doing statistics.
-And you actually can describe this same process in math.  You had some ''prior'' plausible beliefs about what kinds of things are likely to happen in the world (''A'' and ''B'').  Then you actually saw things happening in the world (''X'', the data).  What is the liklihood of what just happened from the perspectives of the different things you believe? (What are the probabilities of ''X'' given ''A''  and of ''X'' given ''B''?  In symbols, what is ''p(X|A)'').  Once you calculate that, you can figure out the reverse: How likely are your beliefs given what just happened---what are the probabilities of ''A'' and ''B'' given ''X'' (''p(A|X)'').  This figuring out the reverse is where the name comes from---Bayes rules is an expression relating p(A|X) to p(X|A).  With these values, you can do other cool things, like find the number for how much more likely A is than B (''p(A|X)/p(B|X)'').
+You actually can describe this same process in math.  You had some plausible ''prior'' beliefs about what kinds of things are likely to happen in the world (''A'' and ''B'').  Then you actually saw things happening in the world (''X'', the data).  What is the likelihood of what just happened from the perspectives of the different things you believe? (What is the probability of ''X'' given ''A'', what is ''p(X|A)''?  And what about ''B'').  Once you calculate that, you can figure out the reverse: How likely are your beliefs given what just happened---what are the probabilities of ''A'' and ''B'' given ''X'' (''p(A|X)'').  Figuring out the reverse is where the name comes from---Bayes rules is an expression relating p(A|X) to p(X|A).  With these values, you can do other cool things, like find the number for how much more likely A is than B (''p(A|X)/p(B|X)'').
 So a good experiment will speak to many possible incompatible beliefs, and make most of them really unlikely no matter how it turns out.  Bayesian data analysis is a way of calculating which of your priors you believe in the most after seeing what just happened.  Designing a good "what just happened" is up to the scientist, and that is where a lot of the creativity comes in.
-Bayesian methods have actually been around for a while, but you need computers to calculate all of the  different values involved.  So the technique has only started to become popular with the personal computer and modern computing.  Another post describes how to do it on a mac.
+Bayesian methods have actually been around for a while, but you need computers to calculate all of the  different values involved.  So the technique has only started to become popular with the personal computer and modern computing.  Another post describes how to do it [[Blog:Bayesian_data_analysis_on_a_Mac|on a Mac]].