-
Notifications
You must be signed in to change notification settings - Fork 1
/
mere_reality.tex
22741 lines (16802 loc) · 866 KB
/
mere_reality.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
\part{Mere Reality}
\mysectiontwo{The World: An Introduction}{The World: An Introduction\newline
by Rob Bensinger}
Previous essays have discussed human reasoning, language, goals,
and social dynamics. Mathematics, physics, and biology were cited to
explain patterns in human behavior, but little has been said about
humanity's place in nature, or about the natural world
in its own right.
Just as it was useful to contrast humans \textit{as goal-oriented
systems} with inhuman processes in evolutionary biology and artificial
intelligence, it will be useful in the coming sequences of essays to
contrast humans \textit{as physical systems} with inhuman processes
that \textit{aren't} mind-like.
{
We humans are, after all, built out of inhuman parts. The world of
atoms looks nothing like the world as we ordinarily think of it, and
certainly looks nothing like the world's conscious
denizens as we ordinarily think of them. As Giulio Giorello put the
point in an interview with Daniel Dennett: ``Yes, we
have a soul. But it's made of lots of tiny
robots.''\footnote{Daniel C. Dennett, \textit{Freedom Evolves} (Viking Books,
2003).\comment{1}}}
\textit{Mere Reality} collects seven sequences of essays on this
topic. The first three introduce the question of how the human world
relates to the world revealed by physics: ``Lawful
Truth'' (on the basic links between physics and human
cognition), ``Reductionism 101'' (on
the project of scientifically explaining phenomena), and
``Joy in the Merely Real'' (on the
emotional, personal significance of the scientific world-view). This is
followed by two sequences that go into more depth on specific academic
debates: ``Physicalism 201'' (on the
hard problem of consciousness) and ``Quantum Physics
and Many Worlds'' (on the measurement problem in
physics). Finally, the sequence ``Science and
Rationality'' and the essay A Technical Explanation
of Technical Explanation tie these ideas together and relate them to
scientific practice.
The discussions of consciousness and quantum physics illustrate
the relevance of reductionism to present-day controversies in science
and philosophy. For those interested in a bit of extra context,
I'll say a few more words about those two topics here.
For those eager to skip ahead: skip ahead!
\subsection{Minds in the World}
Can we ever know what it's like to be a bat?
{
We can certainly develop better cognitive models for predicting
bat behavior, or more fine-grained models of bat neurology---but it
isn't obvious that this would tell us what echolocation
subjectively feels like, or what flying feels like, \textit{from the
bat's point of view}.}
Indeed, it seems as though we could never even be certain that
there \textit{is} anything it's like to be a bat. Why
couldn't an unconscious automaton replicate all the
overt behaviors of a conscious agent to arbitrary precision?
(Philosophers call such automata
``zombies,'' though they have little
in common with the zombies of folklore---who are \textit{quite visibly}
different from conscious agents!)
A race of alien psychologists would run into the same problem in
trying to model \textit{human} consciousness. They might arrive at a
perfect predictive model of what we say and do when we see a red rose,
but that wouldn't mean that the aliens fully understand
what redness feels like ``from the
inside.''
Running with examples like these, philosophers like Thomas Nagel
and David Chalmers have argued that third-person cognitive and neural
models can never fully capture first-person
consciousness.\footnote{David J. Chalmers, \textit{The Conscious Mind: In Search of a
Fundamental Theory} (New York: Oxford University Press, 1996).\comment{2}}\supercomma\footnote{Thomas Nagel, ``What Is It Like to Be a
Bat?,'' \textit{Philosophical Review} 83, no. 4
(1974): 435--450, \url{http://www.jstor.org/stable/2183914}.\comment{3}} No matter how much we know about a
physical system, it is always logically possible, on this view, that
the system has no first-person experiences. Traditional dualism, with
its immaterial souls freely floating around violating physical laws,
may be false; but Chalmers insists on a weaker thesis, that
consciousness is a ``further fact''
not fully explainable by the physical facts.
A number of philosophers and scientists have found this line of
reasoning persuasive.\footnote{In a survey of Anglophone professional philosophers, 56.5\%
endorsed physicalism, 27.1\% endorsed anti-physicalism, and 16.4\%
endorsed other views (e.g., ``I don't
know'').\footnotemark Most philosophers reject
the metaphysical possibility of Chalmers's
``zombies,'' but there is no
consensus about \textit{why}, exactly, Chalmers's
zombie argument fails. Kirk summarizes contemporary positions on
phenomenal consciousness, giving arguments that resemble
Yudkowsky's against the possibility of knowing or
referring to irreducible qualia.\footnotemark\comment{4}}
\footback{2}\footnext\footnotetext{David Bourget and David J. Chalmers, ``What
Do Philosophers Believe?,'' \textit{Philosophical
Studies} (2013): 1--36.\comment{11}}\footnext\footnotetext{Robert Kirk, \textit{Mind and Body}
(McGill-Queen's University Press, 2003).\comment{12}}If we feel this
argument's intuitive force, should we grant its
conclusion and ditch physicalism?
We certainly shouldn't reject it just because it
\textit{sounds strange} or feels vaguely unscientific. But how does the
argument stand up to a \textit{technical} understanding of how
explanation and belief work? Are there any hints we can take from the
history of science, or from our understanding of the physical
mechanisms underlying evidence? ``Physicalism
201'' will return to this question.
\subsection{Worlds in the World}
Quantum mechanics is our best mathematical model of the universe
to date, powerfully confirmed by a century of tests. The theory posits
a complex-numbered ``probability
amplitude,'' so called because a specific operation
(squaring the number's absolute value---the Born rule)
lets us probabilistically predict phenomena at small scales and extreme
energy levels. This amplitude changes deterministically in accord with
the Schrödinger equation. In the process, it often enters odd states
called ``superpositions.''
Yet when we perform experiments, the superpositions seem to vanish
without a trace. When we aren't looking, the
Schrödinger equation appears to capture everything there is to know
about the dynamics of physical systems. When we \textit{are} looking,
though, this clean determinism is replaced by Born's
probabilistic rule. It's as though the ordinary laws of
physics are suddenly suspended whenever we make
``observations.'' As John Stewart
Bell put the point:
\begin{quote}
{
It would seem that the theory is exclusively concerned about
``results of measurements'' and has
nothing to say about anything else. What exactly qualifies some
physical systems to play the role of the
``measurer''? Was the wavefunction
of the world waiting to jump for thousands of millions of years until a
single-celled living creature appeared? Or did it have to wait a little
longer, for some better qualified system\,\ldots with a PhD?}
\end{quote}
Everyone agrees that this strange mix of Schrödinger and
Born's rules has proved empirically adequate. However,
the question of exactly \textit{when} Born's rule
enters the mix, and what it all \textit{means}, has produced a chaos of
different views on the nature of quantum mechanics.
Early on, the Copenhagen school---Niels Bohr and other originators
of quantum theory---splintered into several standard ways of talking
about the experimental results and the odd formalism used to predict
them. Some, taking the theory's focus on
``measurements'' and
``observations'' quite literally,
proposed that consciousness plays a fundamental role in physical law,
intervening to cause complex amplitudes to
``collapse'' into observables.
Others, led by Werner Heisenberg, advocated a non-realistic view
according to which physics is about our states of knowledge rather than
about any objective reality. Yet another Copenhagen tradition, summed
up in the slogan ``shut up and
calculate,'' warned against metaphysical speculation
of all kinds.
Yudkowsky uses this scientific controversy as a proving ground for
some central ideas from previous sequences: map-territory distinctions,
mysterious answers, Bayesianism, and Occam's Razor.
Since he is not a physicist---and neither am I---I'll
provide some outside sources here for readers who want to vet his
arguments or learn more about his physics examples.
Tegmark's \textit{Our Mathematical Universe}
discusses a number of relevant ideas in philosophy and
physics.\footnote{Max Tegmark, \textit{Our Mathematical Universe: My Quest for
the Ultimate Nature }\textit{of Reality} (Random House LLC, 2014).\comment{5}} Among Tegmark's more novel
ideas is his argument that all consistent mathematical structures
exist, including worlds with physical laws and boundary conditions
entirely unlike our own. He distinguishes these Tegmark worlds from
multiverses in more scientifically mainstream hypotheses---e.g., worlds
in stochastic eternal inflationary models of the Big Bang and in Hugh
Everett's many-worlds interpretation of quantum
physics.
{
Yudkowsky discusses many-worlds interpretations at greater length,
as a response to the Copenhagen interpretations of quantum mechanics.
Many-worlds has become very popular in recent decades among physicists,
especially cosmologists. However, a number of physicists continue to
reject it or maintain agnosticism. For a (mostly) philosophically
mainstream introduction to this debate, see Albert's
\textit{Quantum Mechanics and Experience}.\footnote{David Z. Albert, \textit{Quantum Mechanics and Experience}
(Harvard University Press, 1994).\comment{6}} See also
the \textit{Stanford Encyclopedia of Philosophy}'s
introduction to ``Measurement in Quantum
Theory,''\footnote{Henry Krips, ``Measurement in Quantum
Theory,'' in \textit{The Stanford Encyclopedia of
Philosophy}, Fall 2013, ed. Edward N. Zalta.\comment{7}} and their introduction
to several of the views associated with ``many
worlds'' in
``Everett's Relative-State
Formulation''\footnote{Jeffrey Barrett, \textit{Everett's
Relative-State Formulation of Quantum Mechanics}, ed. Edward N. Zalta,
\url{http://plato.stanford.edu/archives/fall2008/entries/qm-everett/}.\comment{8}} and
``Many-Worlds
Interpretation.''\footnote{Lev Vaidman, ``Many-Worlds Interpretation of
Quantum Mechanics,'' in \textit{The Stanford
Encyclopedia of Philosophy}, Fall 2008, ed. Edward N. Zalta.\comment{9}}}
On the less theoretical side, Epstein's
\textit{Thinking Physics} is a great text for training physical
intuitions.\footnote{Lewis Carroll Epstein, \textit{Thinking Physics:
Understandable Practical Reality, 3rd Edition} (Insight Press, 2009).\comment{10}} It's worth keeping in
mind that just as one can understand most of cognitive science without
understanding the nature of subjective awareness, one can understand
most of physics without having a settled view of the ultimate nature
(and size!) of the physical world.
\chapter{Lawful Truth}
\mysection{Universal Fire}
In L. Sprague de Camp's fantasy story \textit{The
Incomplete Enchanter} (which set the mold for the many imitations that
followed), the hero, Harold Shea, is transported from our own universe
into the universe of Norse mythology.\footnote{Lyon Sprague de Camp and Fletcher Pratt, \textit{The Incomplete
Enchanter} (New York: Henry Holt \& Company, 1941).\comment{1}} This world is
based on magic rather than technology; so naturally, when Our Hero
tries to light a fire with a match brought along from Earth, the match
fails to strike.
I realize it was only a fantasy story, but\,\ldots how do I put this\,\ldots
\textit{No.}
In the late eighteenth century, Antoine-Laurent de Lavoisier
discovered fire. ``What?'' you say.
``Hasn't the use of fire been dated
back for hundreds of thousands of years?'' Well, yes,
people \textit{used} fire; it was hot, bright, sort of orangey-colored,
and you could use it to cook things. But nobody knew how it worked.
Greek and medieval alchemists thought that Fire was a basic thing, one
of the Four Elements. In Lavoisier's time the
alchemical paradigm had been gradually amended and greatly complicated,
but fire was still held to be basic---in the form of
``phlogiston,'' a rather mysterious
substance which was said to explain fire, and also every other
phenomenon in alchemy.
Lavoisier's great innovation was to weigh
\textit{all} the pieces of the chemical puzzle, both before and after
the chemical reaction. It had previously been thought that some
chemical transmutations changed the weight of the total material: If
you subjected finely ground antimony to the focused sunlight of a
burning glass, the antimony would be reduced to ashes after one hour,
and the ashes would weigh one-tenth more than the original
antimony---even though the burning had been accompanied by the loss of
a thick white smoke. Lavoisier weighed \textit{all} the components of
such reactions, including the air in which the reaction took place, and
discovered that matter was neither created nor destroyed. If the burnt
ashes increased in weight, there was a corresponding decrease in the
weight of the air.
{
Lavoisier also knew how to separate gases, and discovered that a
burning candle diminished the amount of one kind of gas, \textit{vital
air}, and produced another gas, \textit{fixed air}. Today we would call
them \textit{oxygen} and \textit{carbon dioxide}. When the
\textit{vital air} was exhausted, the fire went out. One might guess,
perhaps, that combustion transformed \textit{vital air} into
\textit{fixed air} and fuel to ash, and that the ability of this
transformation to continue was limited by the amount of \textit{vital
air} available.}
Lavoisier's proposal directly contradicted the
then-current phlogiston theory. That alone would have been shocking
enough, but it also turned out\,\ldots
To appreciate what comes next, you must put yourself into an
eighteenth-century frame of mind. Forget the discovery of DNA, which
occurred only in 1953. Unlearn the cell theory of biology, which was
formulated in 1839. Imagine looking at your hand, flexing your fingers\,\ldots and having absolutely no idea how it worked. The anatomy of
muscle and bone was known, but no one had any notion of
``what makes it go''---why a muscle
moves and flexes, while clay molded into a similar shape just sits
there. Imagine \textit{your own body} being composed of mysterious,
incomprehensible gloop. And then, imagine discovering\,\ldots
\ldots that humans, in the course of breathing, consumed
\textit{vital air} and breathed out \textit{fixed air}. People also ran
on combustion! Lavoisier measured the amount of heat that animals (and
Lavoisier's assistant, Seguin) produced when
exercising, the amount of \textit{vital air} consumed, and the
\textit{fixed air} breathed out. When animals produced more heat, they
consumed more \textit{vital air} and exhaled more \textit{fixed air}.
People, like fire, consumed fuel and oxygen; people, like fire,
produced heat and carbon dioxide. Deprive people of oxygen, or fuel,
and the light goes out.
Matches catch fire because of
phosphorus---``safety matches'' have
phosphorus on the ignition strip; strike-anywhere matches have
phosphorus in the match heads. Phosphorus is highly reactive; pure
phosphorus glows in the dark and may spontaneously combust. (Henning
Brand, who purified phosphorus in 1669, announced that he had
discovered Elemental Fire.) Phosphorus is thus also well-suited to its
role in \textit{adenosine triphosphate}, ATP, your
body's chief method of storing chemical energy. ATP is
sometimes called the ``molecular
currency.'' It invigorates your muscles and charges
up your neurons. Almost every metabolic reaction in biology relies on
ATP, and therefore on the chemical properties of phosphorus.
If a match stops working, so do you. You can't
change just one thing.
The surface-level rules, ``Matches catch fire
when struck,'' and ``Humans need air
to breathe,'' are not obviously connected. It took
centuries to discover the connection, and even then, it still seems
like some distant fact learned in school, relevant only to a few
specialists. It is all too easy to imagine a world where one surface
rule holds, and the other doesn't; to suppress our
credence in one belief, but not the other. But that is
\textit{imagination}, not reality. If your map breaks into four pieces
for easy storage, it doesn't mean the territory is also
broken into disconnected parts. Our minds store different surface-level
rules in different compartments, but this does not reflect any division
in the laws that govern Nature.
We can take the lesson further. Phosphorus derives its behavior
from even deeper laws, electrodynamics and chromodynamics.
``Phosphorus'' is merely our
\textit{word} for electrons and quarks arranged a certain way. You
cannot change the chemical properties of phosphorus without changing
the laws governing electrons and quarks.
If you stepped into a world where matches failed to strike, you
would cease to exist as organized matter.
Reality is laced together a lot more tightly than humans might
like to believe.
\myendsectiontext
\bigskip
\mysection{Universal Law}
Antoine-Laurent de Lavoisier discovered that breathing
(respiration) and fire (combustion) operated on the same principle. It
was one of the most startling unifications in the history of science,
for it brought together the mundane realm of matter and the sacred
realm of life, which humans had divided into separate magisteria.
The first great simplification was that of Isaac Newton, who
unified the course of the planets with the trajectory of a falling
apple. The shock of this discovery was greater by far than
Lavoisier's. It wasn't just that Newton
had dared to unify the Earthly realm of base matter with the obviously
different and sacred celestial realm, once thought to be the abode of
the gods. Newton's discovery gave rise to the notion of
a \textit{universal law}, one that is the same everywhere and
everywhen, with literally \textit{zero} exceptions.
Human beings live in a world of surface phenomena, and surface
phenomena are divided into leaky categories with plenty of exceptions.
A tiger does not behave like a buffalo. Most buffalo have four legs,
but perhaps this one has three. Why would anyone think there would be
laws that hold everywhere? It's just so obviously
untrue.
The only time when it seems like we would \textit{want} a law to
hold everywhere is when we are talking about moral laws---tribal rules
of behavior. Some tribe members may try to take more than their fair
share of the buffalo meat---perhaps coming up with some clever
excuse---so in the case of moral laws we do seem to have an instinct to
universality. Yes, the rule about dividing the meat evenly applies to
\textit{you}, right now, whether you like it or not. But even here
there are exceptions. If---for some bizarre reason---a more powerful
tribe threatened to spear all of you unless Bob received twice as much
meat on just this one occasion, you'd give Bob twice as
much meat. The idea of a rule with literally \textit{no} exceptions
seems insanely rigid, the product of closed-minded thinking by fanatics
so in the grip of their one big idea that they can't
see the richness and complexity of the real universe.
This is the customary accusation made against scientists---the
professional students of the richness and complexity of the real
universe. Because \textit{when you actually look at the universe}, it
turns out to be, by human standards, insanely rigid in applying its
rules. As far as we know, there has been \textit{not one single}
violation of Conservation of Momentum from the uttermost dawn of time
up until now.
Sometimes---very rarely---we observe an apparent violation of our
\textit{models} of the fundamental laws. Though our scientific models
may last for a generation or two, they are not stable over the course
of centuries\,\ldots but do not fancy that this makes the universe itself
whimsical. That is mixing up the map with the territory. For when the
dust subsides and the old theory is overthrown, it turns out that the
universe \textit{always was} acting according to the new generalization
we have discovered, which once again is absolutely universal as far as
humanity's knowledge extends. When it was discovered
that Newtonian gravitation was a special case of General Relativity, it
was seen that General Relativity had been governing the orbit of
Mercury for decades before any human being knew about it; and it would
later become apparent that General Relativity had been governing the
collapse of stars for billions of years before humanity. It is only our
model that was mistaken---the Law itself was always absolutely
constant---or so our new model tells us.
{
I may repose only 80\% confidence that the lightspeed limit will
last out the next hundred thousand years, but this does not mean that I
think the lightspeed limit holds only 80\% of the time, with occasional
exceptions. The proposition to which I assign 80\% probability is that
the lightspeed law is \textit{absolutely inviolable throughout the
entirety of space and time}.}
One of the reasons the ancient Greeks didn't
discover science is that they didn't realize you could
generalize from experiments. The Greek philosophers were interested in
``normal'' phenomena. If you set up
a contrived experiment, you would probably get a
``monstrous'' result, one that had
no implications for how things really worked.
So that is how humans tend to dream, before they learn better; but
what of the universe's own quiet dreams that it dreamed
to itself before ever it dreamed of humans? If you would learn to think
like reality, then here is the Tao:
\begin{quote}
{
\textit{Since the beginning}\newline
\textit{ not one unusual thing}\newline
\textit{ has ever happened.}}
\end{quote}
\myendsectiontext
\mysection{Is Reality Ugly?}
Consider the cubes, \{1, 8, 27,
64, 125,\,\ldots~\}. Their first
differences \{7, 19, 37, 61,\,\ldots~\} might at first seem to lack
an obvious pattern, but taking the second differences
\{12, 18, 24,\,\ldots~\} takes you down to the simply
related level. Taking the third differences
\{6, 6,\,\ldots~\} brings us to the perfectly
stable level, where chaos dissolves into order.
But this is a handpicked example. Perhaps the
``messy real world'' lacks the
beauty of these abstract mathematical objects? Perhaps it would be more
appropriate to talk about neuroscience or gene expression networks?
Abstract math, being constructed solely in imagination, arises
from simple foundations---a small set of initial axioms---and is a
closed system; conditions that might seem \textit{unnaturally}
conducive to neatness.
Which is to say: In pure math, you don't have to
worry about a tiger leaping out of the bushes and eating
Pascal's Triangle.
So is the real world uglier than mathematics?
Strange that people ask this. I mean, the question might have been
sensible two and a half millennia ago\,\ldots Back when the Greek
philosophers were debating what this ``real
world'' thingy might be made of, there were many
positions. Heraclitus said, ``All is
fire.'' Thales said, ``All is
water.'' Pythagoras said, ``All is
number.''
Score:
\begin{center}
\begin{tabular}{|l|l|}
\hline
Heraclitus: & 0\\
\hline
Thales: & 0\\
\hline
Pythagoras: & 1\\
\hline
\end{tabular}
\end{center}
Beneath the complex forms and shapes of the surface world, there
is a simple level, an exact and stable level, whose laws we name
``physics.'' This discovery, the
Great Surprise, has already taken place at our point in human
history---but it does not do to forget that it was surprising. Once
upon a time, people went in search of underlying beauty, with no
guarantee of finding it; and once upon a time, they found it; and now
it is a known thing, and taken for granted.
Then why can't we predict the location of every
tiger in the bushes as easily as we predict the sixth cube?
I count three sources of uncertainty even within worlds of pure
math---two obvious sources, and one not so obvious.
The first source of uncertainty is that even a creature of pure
math, living embedded in a world of pure math, may not know the math.
Humans walked the Earth long before Galileo/Newton/Einstein discovered
the law of gravity that prevents us from being flung off into space.
You can be governed by stable fundamental rules without knowing them.
There is no law of physics which says that laws of physics must be
explicitly represented, as knowledge, in brains that run under them.
We do not yet have the Theory of Everything. Our best current
theories are things of math, but they are not perfectly integrated with
each other. The most probable explanation is that---as has previously
proved to be the case---we are seeing surface manifestations of deeper
math. So by far the best guess is that reality is made of math; but we
do not fully know which math, yet.
But physicists have to construct huge particle accelerators to
distinguish between theories---to manifest their remaining uncertainty
in any visible fashion. That physicists must go to such lengths to be
unsure, suggests that this is not the source of our uncertainty about
stock prices.
The second obvious source of uncertainty is that even when you
know all the relevant laws of physics, you may not have enough
computing power to extrapolate them. We know every fundamental physical
law that is relevant to a chain of amino acids folding itself into a
protein. But we still can't predict the shape of the
protein from the amino acids. Some tiny little 5-nanometer molecule
that folds in a microsecond is \textit{too much information} for
current computers to handle (never mind tigers and stock prices). Our
frontier efforts in protein folding use clever approximations, rather
than the underlying Schrödinger equation. When it comes to describing a
5-nanometer object using \textit{really} basic physics, over
quarks---well, you don't even bother trying.
We have to use instruments like X-ray crystallography and NMR to
discover the shapes of proteins that are fully determined by physics we
know and a DNA sequence we know. We are not logically omniscient; we
cannot see all the implications of our thoughts; we do not know what we
believe.
The third source of uncertainty is the most difficult to
understand, and Nick Bostrom has written a book about it. Suppose that
the sequence \{1, 8, 27, 64, 125,\,\ldots~\} exists; suppose that this is
a fact. And suppose that atop each cube is a little person---one person
per cube---and suppose that this is also a fact.
If you stand on the outside and take a global
perspective---looking down from above at the sequence of cubes and the
little people perched on top---then these two facts say everything
there is to know about the sequence and the people.
But if you are one of the little people perched atop a cube, and
you know these two facts, there is still a third piece of information
you need to make predictions: ``Which cube am
\textit{I} standing on?''
You expect to find yourself standing on a cube; you do not expect
to find yourself standing on the number 7. Your anticipations are
definitely constrained by your knowledge of the basic physics; your
beliefs are falsifiable. But you still have to look down to find out
whether you're standing on 1,728 or 5,177,717. If you
can do fast mental arithmetic, then seeing that the first two digits of
a four-digit cube are 17\_\_ will be sufficient to guess that the last
digits are 2 and 8. Otherwise you may have to look to discover the 2
and 8 as well.
To figure out what the night sky should look like,
it's not enough to know the laws of physics.
It's not even enough to have logical omniscience over
their consequences. You have to know \textit{where} you are in the
universe. You have to know that you're looking up at
the night sky \textit{from Earth.} The information required is not just
the information to locate Earth in the \textit{visible} universe, but
in the entire universe, including all the parts that our telescopes
can't see because they are too distant, and different
inflationary universes, and alternate Everett branches.
It's a good bet that
``uncertainty about initial conditions at the
boundary'' is really indexical uncertainty. But if
not, it's empirical uncertainty, uncertainty about how
the universe \textit{is} from a global perspective, which puts it in
the same class as uncertainty about fundamental laws.
Wherever our best guess is that the ``real
world'' has an \textit{irretrievably} messy
component, it is because of the second and third sources of
uncertainty---logical uncertainty and indexical uncertainty.
Ignorance of fundamental laws does not tell you that a
messy-looking pattern really is messy. It might just be that you
haven't figured out the order yet.
But when it comes to messy gene expression networks,
we've \textit{already found} the hidden beauty---the
stable level of underlying physics. \textit{Because}
we've already found the master order, we can guess that
we won't find any \textit{additional} secret patterns
that will make biology as easy as a sequence of cubes. Knowing the
rules of the game, we know that the game is hard. We
don't have enough computing power to do protein
chemistry from physics (the second source of uncertainty) and
evolutionary pathways may have gone different ways on different planets
(the third source of uncertainty). New discoveries in basic physics
won't help us here.
If you were an ancient Greek staring at the raw data from a
biology experiment, you would be much wiser to look for some hidden
structure of Pythagorean elegance, all the proteins lining up in a
perfect icosahedron. But in biology we already know where the
Pythagorean elegance is, and we know it's too far down
to help us overcome our indexical and logical uncertainty.
Similarly, we can be confident that no one will ever be able to
predict the results of certain quantum experiments, only because our
fundamental theory tells us quite definitely that different versions of
us will see different results. If your knowledge of fundamental laws
tells you that there's a sequence of cubes, and that
there's one little person standing on top of each cube,
and that the little people are all alike except for being on different
cubes, and that you are one of these little people, then you
\textit{know} that you have no way of deducing which cube
you're on except by looking.
The best current knowledge says that the ``real
world'' is a perfectly regular, deterministic, and
\textit{very large} mathematical object which is highly expensive to
simulate. So ``real life'' is less
like predicting the next cube in a sequence of cubes, and more like
knowing that lots of little people are standing on top of cubes, but
not knowing who \textit{you personally} are, and also not being very
good at mental arithmetic. Our knowledge of the rules does constrain
our anticipations, quite a bit, but not perfectly.
There, now doesn't \textit{that} sound like real
life?
But uncertainty exists in the map, not in the territory. If we are
ignorant of a phenomenon, that is a fact about our state of mind, not a
fact about the phenomenon itself. Empirical uncertainty, logical
uncertainty, and indexical uncertainty are just names for our own
bewilderment. The best current guess is that the world is math and the
math is perfectly regular. The messiness is only in the eye of the
beholder.
Even the huge morass of the blogosphere is embedded in this
perfect physics, which is ultimately as orderly as
\{1, 8, 27, 64, 125,\,\ldots~\}.
So the Internet is not a big muck\,\ldots it's a
series of cubes.
\myendsectiontext
\mysection{Beautiful Probability}
Should we expect rationality to be, \textit{on some level},
simple? Should we search and hope for \textit{underlying} beauty in the
arts of belief and choice?
{
Let me introduce this issue by borrowing a complaint of the late
great Bayesian Master, E. T. Jaynes:\footnote{Edwin T. Jaynes, ``Probability Theory as
Logic,'' in \textit{Maximum Entropy and Bayesian
Methods}, ed. Paul F. Fougère (Springer Netherlands, 1990).\comment{1}}}
\begin{quote}
{
Two medical researchers use the same treatment independently, in
different hospitals. Neither would stoop to falsifying the data, but
one had decided beforehand that because of finite resources he would
stop after treating n = 100 patients, however many cures were observed
by then. The other had staked his reputation on the efficacy of the
treatment, and decided he would not stop until he had data indicating a
rate of cures definitely greater than 60\%, however many patients that
might require. But in fact, both stopped with exactly the same data: n
= 100 [patients], r = 70 [cures]. Should we then draw different
conclusions from their experiments?'' [Presumably the
two control groups also had equal results.]}
\end{quote}
{
Cyan\footnote{\url{http://lesswrong.com/lw/mt/beautiful_probability/hnz}} directs us to chapter 37\footnote{\url{http://www.inference.phy.cam.ac.uk/mackay/itprnn/ps/457.466.pdf}} of MacKay's
excellent statistics book, free online, for a more thorough explanation
of this problem.\footnote{David J. C. MacKay, \textit{Information Theory, Inference, and
Learning Algorithms} (New York: Cambridge University Press, 2003). \url{http://www.inference.phy.cam.ac.uk/mackay/itila/book.html}\comment{2}}}
According to old-fashioned statistical procedure---which I believe
is still being taught today---the two researchers have performed
different experiments with different stopping conditions. The two
experiments \textit{could} have terminated with different data, and
therefore represent different tests of the hypothesis, requiring
different statistical analyses. It's quite possible
that the first experiment will be ``statistically
significant,'' the second not.
Whether or not you are disturbed by this says a good deal about
your attitude toward probability theory, and indeed, rationality
itself.
Non-Bayesian statisticians might shrug, saying,
``Well, not all statistical tools have the same
strengths and weaknesses, y'know---a hammer
isn't like a screwdriver---and if you apply different
statistical tools you may get different results, just like using the
same data to compute a linear regression or train a regularized neural
network. You've got to use the right tool for the
occasion. Life is messy---''
And then there's the Bayesian reply:
``Excuse \textit{you}? The evidential impact of a
fixed experimental method, producing the same data, depends on the
researcher's private thoughts? And you have the nerve
to accuse \textit{us} of being `too
subjective'?''
If Nature is one way, the likelihood of the data coming out the
way we have seen will be one thing. If Nature is another way, the
likelihood of the data coming out that way will be something else. But
the likelihood of a given state of Nature producing the data we have
seen, has nothing to do with the researcher's private
intentions. So whatever our hypotheses about Nature, the likelihood
ratio is the same, and the evidential impact is the same, and the
posterior belief should be the same, between the two experiments. At
least one of the two Old Style methods must discard relevant
information---or simply do the wrong calculation---for the two methods
to arrive at different answers.
The ancient war between the Bayesians and the accursèd
frequentists stretches back through decades, and I'm
not going to try to recount that elder history in this essay.
But one of the central conflicts is that Bayesians expect
probability theory to be\,\ldots what's the word
I'm looking for?
``Neat?''
``Clean?''
``Self-consistent?''
As Jaynes says, the theorems of Bayesian probability are just
that, \textit{theorems} in a coherent proof system. No matter what
derivations you use, in what order, the results of Bayesian probability
theory should always be consistent---every theorem compatible with
every other theorem.
If you want to know the sum 10 + 10, you can redefine it as (2
{\texttimes} 5) + (7 + 3) or as (2 {\texttimes} (4 + 6)) or use
whatever other \textit{legal} tricks you like, but the result always
has to come out to be the same, in this case, 20. If it comes out as 20
one way and 19 the other way, then you may conclude you did something
illegal on at least one of the two occasions. (In arithmetic, the
illegal operation is usually division by zero; in probability theory,
it is usually an infinity that was not taken as a the limit of a finite
process.)
If you get the result 19 = 20, look hard for that error you just
made, because it's unlikely that you've
sent arithmetic itself up in smoke. If anyone should ever succeed in
deriving a \textit{real} contradiction from Bayesian probability
theory---like, say, two different evidential impacts from the same
experimental method yielding the same results---then the whole edifice
goes up in smoke. Along with set theory, 'cause
I'm pretty sure \textsf{ZF} provides a model for probability
theory.
Math! That's the word I was looking for. Bayes\-ians
expect probability theory to be \textit{math.} That's
why we're interested in Cox's Theorem
and its many extensions, showing that any representation of uncertainty
which obeys certain constraints has to map onto probability theory.
Coherent math is great, but unique math is even better.
And yet\,\ldots \textit{should} rationality be math? It is by no
means a foregone conclusion that probability should be pretty. The real
world is messy---so shouldn't you need messy reasoning
to handle it? Maybe the non-Bayesian statisticians, with their vast
collection of ad-hoc methods and ad-hoc justifications, are strictly
more competent because they have a strictly larger toolbox.
It's nice when problems are clean, but they usually
aren't, and you have to live with that.
After all, it's a well-known fact that you
can't use Bayesian methods on many problems because the
Bayesian calculation is computationally intractable. So why not let
many flowers bloom? Why not have more than one tool in your toolbox?
\textit{That's} the fundamental difference in
mindset. Old School statisticians thought in terms of \textit{tools},
tricks to throw at particular problems. Bayesians---at least this
Bayesian, though I don't think I'm
speaking only for myself---we think in terms of \textit{laws.}
Looking for laws isn't the same as looking for
especially neat and pretty tools. The Second Law of Thermodynamics
isn't an especially neat and pretty refrigerator.
The Carnot cycle is an ideal engine---in fact, \textit{the} ideal
engine. No engine powered by two heat reservoirs can be more efficient
than a Carnot engine. As a corollary, all thermodynamically reversible
engines operating between the same heat reservoirs are equally
efficient.
But, of course, you can't use a Carnot engine to
power a real car. A real car's engine bears the same
resemblance to a Carnot engine that the car's tires
bear to perfect rolling cylinders.
Clearly, then, a Carnot engine is a useless \textit{tool} for
building a real-world car. The Second Law of Thermodynamics, obviously,
is not applicable here. It's too hard to make an engine
that obeys it, in the real world. Just ignore thermodynamics---use
whatever works.
This is the sort of confusion that I think reigns over they who
still cling to the Old Ways.
No, you can't always do the exact Bayesian
calculation for a problem. Sometimes you must seek an approximation;
often, indeed. This doesn't mean that probability
theory has ceased to apply, any more than your inability to calculate
the aerodynamics of a 747 on an atom-by-atom basis implies that the 747
is not made out of atoms. Whatever approximation you use, it works to
the extent that it approximates the ideal Bayesian calculation---and
fails to the extent that it departs.
Bayesianism's coherence and uniqueness proofs cut
both ways. Just as any calculation that obeys Cox's
coherency axioms (or any of the many reformulations and
generalizations) must map onto probabilities, so too, anything that is
not Bayesian must fail one of the coherency tests. This, in turn, opens
you to punishments like Dutch-booking (accepting combinations of bets
that are sure losses, or rejecting combinations of bets that are sure
gains).
You may not be able to compute the optimal answer. But whatever
approximation you use, both its failures and successes will be
\textit{explainable} in terms of Bayesian probability theory. You may
not know the explanation; that does not mean no explanation exists.
So you want to use a linear regression, instead of doing Bayesian
updates? But look to the underlying structure of the linear regression,
and you see that it corresponds to picking the best point estimate
given a Gaussian likelihood function and a uniform prior over the
parameters.
You want to use a regularized linear regression, because that
works better in practice? Well, that corresponds (says the Bayesian) to
having a Gaussian prior over the weights.
Sometimes you can't use Bayesian methods
\textit{literally}; often, indeed. But when you \textit{can} use the
exact Bayesian calculation that uses every scrap of available
knowledge, you are done. You will never find a statistical method that
yields a \textit{better} answer. You may find a cheap approximation
that works excellently nearly all the time, and it will be cheaper, but
it will not be more accurate. Not unless the other method uses
knowledge, perhaps in the form of disguised prior information, that you
are not allowing into the Bayesian calculation; and then when you feed
the prior information into the Bayesian calculation, the Bayesian
calculation will again be equal or superior.
When you use an Old Style ad-hoc statistical tool with an ad-hoc
(but often quite interesting) justification, you never know if someone
else will come up with an even more clever tool tomorrow. But when you
\textit{can} directly use a calculation that mirrors the Bayesian law,
you're \textit{done}{}---like managing to put a Carnot
heat engine into your car. It is, as the saying goes,
``Bayes-optimal.''
It seems to me that the toolboxers are looking at the sequence of
cubes \{1, 8, 27, 64, 125,\,\ldots~\} and pointing to the first
differences \{7, 19, 37, 61,\,\ldots~\} and saying
``Look, life isn't always so
neat---you've got to adapt to
circumstances.'' And the Bayesians are pointing to
the third differences, the underlying stable level
\{6, 6, 6, 6, 6,\,\ldots~\}. And the critics are saying,
``What the heck are you talking about?
It's 7, 19, 37 not 6, 6, 6. You are oversimplifying
this messy problem; you are too attached to
simplicity.''