Matt McAlister's List: Less Wrong 0 - Map and Territory

• Epistemic rationality: believing, and updating on evidence, so as to systematically improve the correspondence between your map and the territory.  The art of obtaining beliefs that correspond to reality as closely as possible.  This correspondence is commonly termed "truth" or "accuracy", and we're happy to call it that.
• Instrumental rationality: achieving your values.  Not necessarily "your values" in the sense of being selfish values or unshared values: "your values" means anything you care about.  The art of choosing actions that steer the future toward outcomes ranked higher in your preferences.  On LW we sometimes refer to this as "winning".
• Beliefs that conform to a coherent probability distribution, and decisions that maximize the probabilistic expectation of a coherent utility function, are called "Bayesian".
• There's a whole further art to finding the truth and accomplishing value from inside a human mind: we have to learn our own flaws, overcome our biases, prevent ourselves from self-deceiving, get ourselves into good emotional shape to confront the truth and do what needs doing, etcetera etcetera and so on.
• The point of attaching sequences of letters to particular concepts is to let two people communicate - to help transport thoughts from one mind to another.  You cannot change reality, or prove the thought, by manipulating which meanings go with which words.
• If you speak overmuch of the Way you will not attain it.

4 more annotations...

• Curiosity is one reason to seek truth, and it may not be the only one, but it has a special and admirable purity. 
• I label an emotion as "not rational" if it rests on mistaken beliefs, or rather, on irrational epistemic conduct
• an emotion which is evoked by correct beliefs or epistemically rational thinking is a "rational emotion"
• When people think of "emotion" and "rationality" as opposed, I suspect that they are really thinking of System 1 and System 2 - fast perceptual judgments versus slow deliberative judgments. Deliberative judgments aren't always true, and perceptual judgments aren't always false; so it is very important to distinguish that dichotomy from "rationality". Both systems can serve the goal of truth, or defeat it, according to how they are used.
• Pure curiosity is a wonderful thing, but it may not linger too long on verifying its answers, once the attractive mystery is gone.  Curiosity, as a human emotion, has been around since long before the ancient Greeks.  But what set humanity firmly on the path of Science was noticing that certain modes of thinking uncovered beliefs that let us manipulate the world.  As far as sheer curiosity goes, spinning campfire tales of gods and heroes satisfied that desire just as well, and no one realized that anything was wrong with that.
• Are there motives for seeking truth besides curiosity and pragmatism?  The third reason that I can think of is morality
• Being curious about what's behind the curtain doesn't feel the same as believing that you have a moral duty to look there.  In the latter state of mind, you are a lot more likely to believe that someone else should look behind the curtain, too, or castigate them if they deliberately close their eyes.  For this reason, I would also label as "morality" the belief that truthseeking is pragmatically important to society, and therefore is incumbent as a duty upon all.  Your priorities, under this motivation, will be determined by your ideals about which truths are most important (not most useful or most intriguing); or your moral ideals about when, under what circumstances, the duty to seek truth is at its strongest.
• To make rationality into a moral duty is to give it all the dreadful degrees of freedom of an arbitrary tribal custom.  People arrive at the wrong answer, and then indignantly protest that they acted with propriety, rather than learning from their mistake.

6 more annotations...

• Not every influence creates the kind of "entanglement" required for evidence.  It's no help to have a machine that beeps when you enter winning lottery numbers, if the machine also beeps when you enter losing lottery numbers.  The light reflected from your shoes would not be useful evidence about your shoelaces, if the photons ended up in the same physical state whether your shoelaces were tied or untied.
• This is why rationalists put such a heavy premium on the paradoxical-seeming claim that a belief is only really worthwhile if you could, in principle, be persuaded to believe otherwise.  If your retina ended up in the same state regardless of what light entered it, you would be blind.  Some belief systems, in a rather obvious trick to reinforce themselves, say that certain beliefs are only really worthwhile if you believe them unconditionally - no matter what you see, no matter what you think.  Your brain is supposed to end up in the same state regardless.  Hence the phrase, "blind faith".
• If your beliefs are entangled with reality, they should be contagious
• Explain why you think you're rational.  Why you think that, using thought processes like the ones you use, minds will end up believing "snow is white" if and only if snow is white.
• "For me to consider an event to be evidence about a target of inquiry, I must first possess or construct a model of that event and that target and also a model of the world that contains and relates the event and target with each other. Then, for the event to be evidence CORROBORATING a particular theory about the target, I must imagine plausible alternative events that would that would CONTRADICT that theory."

3 more annotations...

• To win the lottery, you would need evidence selective enough to visibly favor one combination over 131,115,984 alternatives.
• Suppose there are some tests you can perform which discriminate, probabilistically, between winning and losing lottery numbers.  For example, you can punch a combination into a little black box that always beeps if the combination is the winner, and has only a 1/4 (25%) chance of beeping if the combination is wrong.  In Bayesian terms, we would say the likelihood ratio is 4 to 1.  This means that the box is 4 times as likely to beep when we punch in a correct combination, compared to how likely it is to beep for an incorrect combination.
• There are still a whole lot of possible combinations.  If you punch in 20 incorrect combinations, the box will beep on 5 of them by sheer chance (on average).  If you punch in all 131,115,985 possible combinations, then while the box is certain to beep for the one winning combination, it will also beep for 32,778,996 losing combinations (on average).
• In general, the rules for weighing "how much evidence it takes" follow a similar pattern:  The larger the space of possibilities in which the hypothesis lies, or the more unlikely the hypothesis seems a priori compared to its neighbors, or the more confident you wish to be, the more evidence you need.
• You cannot defy the rules; you cannot form accurate beliefs based on inadequate evidence.  Let's say you've got 10 boxes lined up in a row, and you start punching combinations into the boxes.  You cannot stop on the first combination that gets beeps from all 10 boxes, saying, "But the odds of that happening for a losing combination are a million to one!  I'll just ignore those ivory-tower Bayesian rules and stop here."  On average, 131 losing tickets will pass such a test for every winner.  Considering the space of possibilities and the prior improbability, you jumped to a too-strong conclusion based on insufficient evidence.  That's not a pointless bureaucratic regulation, it's math.
• Of course, you can still believe based on inadequate evidence, if that is your whim; but you will not be able to believe accurately.  It is like trying to drive your car without any fuel, because you don't believe in the silly-dilly fuddy-duddy concept that it ought to take fuel to go places.  It would be so much more fun, and so much less expensive, if we just decided to repeal the law that cars need fuel.  Isn't it just obviously better for everyone?  Well, you can try, if that is your whim.  You can even shut your eyes and pretend the car is moving.  But to really arrive at accurate beliefs requires evidence-fuel, and the further you want to go, the more fuel you need.

4 more annotations...

• What would convince me that 2 + 2 = 3, in other words, is exactly the same kind of evidence that currently convinces me that 2 + 2 = 4:  The evidential crossfire of physical observation, mental visualization, and social agreement.
• There's really only two possibilities, for a belief of fact - either the belief got there via a mind-reality entangling process, or not.  If not, the belief can't be correct except by coincidence.  For beliefs with the slightest shred of internal complexity (requiring a computer program of more than 10 bits to simulate), the space of possibilities is large enough that coincidence vanishes.
• Unconditional facts are not the same as unconditional beliefs.  If entangled evidence convinces me that a fact is unconditional, this doesn't mean I always believed in the fact without need of entangled evidence.

• If mountains moved when Christians told them to, every time, and no one else could effectively command mountains to move, I think most of us non-believers would start going to church.

  Alternatively, if the world looked like it was designed and regulated by a loving being, it would help. That might not promote Christianity specifically, but it would be a much better start than what we actually see.
• ...Consider a basket with 2 apples in it. Now toss in 2 more apples. Examine the basket, and you will find (surprise!) 4 apples. However, you cannot prove a priori that there will be 4 apples in the basket. It is an empirical question, albeit a trivial one, whether baskets of apples (which are physical things) behave in the same manner as the non-negative integers under addition (which is an abstract logical construct).
• Eliezer is right; numbers are first an abstraction of the world around us. There are a vast number of possible abstractions; the reason we have been so very interested in numbers, compared to all the other possible abstractions, is that numbers happen to describe the world around us. It need not have been so.
• This is right and proper: why should I allow myself to be convinced by something that would not be valid evidence?
• A priori reasoning does take place inside the brain, and neuroscientists do use a posteriori reasoning to associate physical events in the brain with a priori reasoning. Despite this, a priori claims exist and have their own rules for establishing truth.
• As Damien Broderick observed: "If you build a snazzy alife sim ... you'd be a kind of bridging `first cause', and might even have the power to intervene in their lives - even obliterate their entire experienced cosmos - but that wouldn't make you a god in any interesting sense. Gods are ontologically distinct from creatures, or they're not worth the paper they're written on."

7 more annotations...

• The more complex an explanation is, the more evidence you need just to find it in belief-space.
• One observes that the length of an English sentence is not a good way to measure "complexity".
• Why, exactly, is the length of an English sentence a poor measure of complexity?  Because when you speak a sentence aloud, you are using labels for concepts that the listener shares - the receiver has already stored the complexity in them.
• "Witch", itself, is a label for some extraordinary assertions - just because we all know what it means doesn't mean the concept is simple.
• To a human, Maxwell's Equations take much longer to explain than Thor.  Humans don't have a built-in vocabulary for calculus the way we have a built-in vocabulary for anger.  You've got to explain your language, and the language behind the language, and the very concept of mathematics, before you can start on electricity.
• there should be some sense in which Maxwell's Equations are simpler than a human brain, or Thor the thunder-agent.
• It's enormously easier (as it turns out) to write a computer program that simulates Maxwell's Equations, compared to a computer program that simulates an intelligent emotional mind like Thor.
• The formalism of Solomonoff Induction measures the "complexity of a description" by the length of the shortest computer program which produces that description as an output.
• Witchcraft may fit our observations in the sense of qualitatively permitting them; but this is because witchcraft permits everything, like saying "Phlogiston!"  So, even after you say "witch", you still have to describe all the observed data in full detail.  You have not compressed the total length of the message describing your observations by transmitting the message about witchcraft; you have simply added a useless prologue, increasing the total length.
• The real sneakiness was concealed in the word "it" of "A witch did it".  A witch did what?

8 more annotations...

• Here is the secret of deliberate rationality - this whole entanglement process is not magic, and you can understand it.  You can understand how you see your shoelaces.  You can think about which sort of thinking processes will create beliefs which mirror reality, and which thinking processes will not.
• A mouse lives in a mental world that includes cats, holes, cheese and mousetraps - but not mouse brains.  Their camera does not take pictures of its own lens.  But we, as humans, can look at a seemingly bizarre image, and realize that part of what we're seeing is the lens itself.  You don't always have to believe your own eyes, but you have to realize that you have eyes - you must have distinct mental buckets for the map and the territory, for the senses and reality. 
• The whole idea of Science is, simply, reflective reasoning about a more reliable process for making the contents of your mind mirror the contents of the world.
• Some clever fellow is bound to say, "Ah, but since I have hope, I'll work a little harder at my job, pump up the global economy, and thus help to prevent countries from sliding into the angry and hopeless state where nuclear war is a possibility.  So the two events are related after all."  At this point, we have to drag in Bayes's Theorem and measure the charge of entanglement quantitatively.  Your optimistic nature cannot have that large an effect on the world; it cannot, of itself, decrease the probability of nuclear war by 20%, or however much your optimistic nature shifted your beliefs.  Shifting your beliefs by a large amount, due to an event that only carries a very tiny charge of entanglement, will still mess up your mapping.
• To ask which beliefs make you happy, is to turn inward, not outward - it tells you something about yourself, but it is not evidence entangled with the environment.  I have nothing anything against happiness, but it should follow from your picture of the world, rather than tampering with the mental paintbrushes.
• If you can see this - if you can see that hope is shifting your first-order thoughts by too large a degree - if you can understand your mind as a mapping-engine with flaws in it - then you can apply a reflective correction.  The brain is a flawed lens through which to see reality.  This is true of both mouse brains and human brains.  But a human brain is a flawed lens that can understand its own flaws - its systematic errors, its biases - and apply second-order corrections to them.  This, in practice, makes the flawed lens far more powerful.  Not perfect, but far more powerful.
• I do not understand your bizarre concept, illusion. Whatever is, is real. Sometimes the underlying levels of organization are different from what you expected.

5 more annotations...