Academically
speaking, being Bayesian is more difficult.
It requires a solid mathematical understanding of Bayesian inference. After all, an academic Bayesian must perform
such inference, or derive proofs concerning it.
Fortunately, Bayesians have produced a large and distinguished
literature. Unfortunately, statisticians,
mathematicians and physicists have written much of it, intending it (reasonably)
for consumption by other statisticians, mathematicians and physicists. For a cognitive scientist like me this literature
is intimidatingly difficult. How does a cognitive
scientist become Bayesian?
Recent
research developments in my lab require that I become Bayesian, and must
quickly train my students to become Bayesian as well. After extensively searching the literature,
and buying a number of classic texts, I believe that the fastest way for a cognitive
scientist to become Bayesian is to read three key books. More reading of the technical literature is
required for a cognitive scientist to be Bayesian, but this additional
reading will be much more rewarding if one begins as follows:
First,
get excited about becoming Bayesian. To
do so, read The Theory That Would Not Die
by Sharon Bertsch McGrayne. (Full
references for books are provided at the end of this blog entry.)
McGrayne’s
book gives the history of Bayesian inference, beginning with Presbyterian
minister Thomas Bayes’ 18th century rule for calculating, and
updating, conditional probabilities.
McGrayne traces the evolution of this rule to its modern usage,
providing a dizzying array of case studies.
She describes the role of Bayes’ rule for such real world problems as
cracking the German WWII enigma code, searching for missing submarines and
atomic bombs, and evaluating medical diagnoses.
Cognitive scientists may be particularly interested in her chapter on
Alan Turing. She also describes the continuing
controversy surrounding Bayesian inference – Bayesianism is repeatedly
abandoned because of concerns about some of its core assumptions (such as its
subjective definition of probability) or because of the difficult calculations
it entails when applied to real world problems.
Experimentally minded cognitive scientists will be interested in the
biographical portraits that McGrayne provides of famous statisticians like
Fischer, Pearson, and Savage. McGrayne
also chronicles the continual re-adoption and re-discovery of Bayes’ rule,
because of its practical significance and because of the advent of new
computational approaches. It is a theory that refuses to die – and that makes
it very enticing!
Second,
develop a working understanding of Bayesian inference. To do so, read Doing Bayesian Data Analysis by John Kruschke.
McGrayne’s
book gets one excited about Bayes, but provides very few technical details
about what Bayesian inference involves.
Kruschke’s book solves this problem, by beginning with some basic
elements of probability theory, then developing Bayes’ rule, introducing
Bayesian inference for a coin-flipping scenario, and moving on to Bayesian alternatives
for a number of core non-Bayesian statistical tasks. Kruschke modestly notes that his book “is
definitely not a mathematical statistics textbook in that it does not emphasize
theorem proving, and any mathematical statistician would be totally bummed at
the informality, dude.” However, this lack of formality is the book’s strength
– it is informal, but informative.
Kruschke does not assume any statistical background at all, and builds
up an understanding of Bayesian inference from first principles. Furthermore, he replaces theorem proving with
working computer programs, for which he provides generous comments. The result: hands-on experience with core
Bayesian concepts, a deep practical understanding of what Bayesian inference is
all about, and an ability to actually apply Bayesian techniques to real
data. Kruschke’s book lays the
foundation – and provides the courage – for moving on to classic, formal
treatments of Bayesian inference.
Third,
reflect on why being Bayesian might be an important foundation for cognitive science. To do so, read Bayesian Rationality by Mike Oaksford and Nick Chater.
Classical
cognitive science has evolved from logicism, which views thinking as carrying
out operations in some kind of mental logic, a logic that ultimately defines
judgments as being true or false.
Oaksford and Chater argue that such logicism is a mistake. They instead argue that a better formalism
for classical cognitive science is probability theory: “Logic-based approaches
to cognition appeared to be viable for mathematical theorem proving and simple
formal game playing, but seemed fundamentally ill-suited to representation and
reasoning with real-world, common sense knowledge.” Their book begins by considering logicism in
order to contrast it with the Bayesian approach that they favor. They develop their Bayesian move – which they
name ‘the probabilistic turn’ – over several early chapters. They then proceed to consider a number of
core paradigms in the study of deductive (i.e. logicist) reasoning from the
Bayesian perspective, making the case that the probabilistic approach is more
appropriate. This book is stimulating,
because it links Bayesian inference to foundational assumptions in cognitive
science – a link that is now leading to a lively and growing debate in the literature. It also provides a satisfying tie-in to the
Bayesian view of probability in statistics – which subjectively and
controversially defines probability in terms of degree of confidence in beliefs.
To
recap: if you are a cognitive scientist who is considering becoming Bayesian,
then I recommend that you read these three books. To be Bayesian you will need to read more –
key authors are Cox, Jaynes, Jeffreys, and Savage – but rest assured that you
will be well prepared to do so.
The Three Key Books To
Read:
Below
are the bibliographic details for the three books that I recommend, as well as
links to information about them on amazon.ca:
No comments:
Post a Comment