Matt McAlister's List: Less Wrong 3 - How to Actually Change Your Mind

• A sequence on the various ways that politics damages our sanity

• Procopius said of the warring factions:  "So there grows up in them against their fellow men a hostility which has no cause, and at no time does it cease or disappear, for it gives place neither to the ties of marriage nor of relationship nor of friendship, and the case is the same even though those who differ with respect to these colors be brothers or any other kin." 
• Yet from a truce originally born of exhaustion, there is a quietly growing spirit of tolerance, even friendship.
• The stone corridor wends upward... and upward... and finally terminates in a hole carved out of the world, a place where all stone ends.  Distance, endless distance, stretches away into forever; a gathering space to hold a thousand cities.  Unimaginably far above, too bright to look at directly, a searing spark casts light over all visible space, the naked filament of some huge light bulb.  In the air, hanging unsupported, are great incomprehensible tufts of white cotton.
• Aditya the Blue stood under the blue forever, and slowly smiled.  It was not a pleasant smile.  There was hatred, and wounded pride; it recalled every argument she'd ever had with a Green, every rivalry, every contested promotion.  "You were right all along," the sky whispered down at her, "and now you can prove it."  For a moment Aditya stood there, absorbing the message, glorying in it, and then she turned back to the stone corridor to tell the world.  As Aditya walked, she curled her hand into a clenched fist.  "The truce," she said, "is over."
• "How can you be on their side?" Barron screamed at the sky, and then he began to weep; because he knew, standing under the malevolent blue glare, that the universe had always been a place of evil.
• The sky was a metaphysical construct, and cerulean a color that could be seen in more than one way.
• Charles sighed, and turned to go back into the corridor.  Tomorrow he would come back alone and block off the passageway.
• Daria, once Green, tried to breathe amid the ashes of her world.  I will not flinch, Daria told herself, I will not look away.  She had been Green all her life, and now she must be Blue.  Her friends, her family, would turn from her.  Speak the truth, even if your voice trembles, her father had told her; but her father was dead now, and her mother would never understand.  Daria stared down the calm blue gaze of the sky, trying to accept it, and finally her breathing quietened.  I was wrong, she said to herself mournfully; it's not so complicated, after all.  She would find new friends, and perhaps her family would forgive her... or, she wondered with a tinge of hope, rise to this same test, standing underneath this same sky?  "The sky is blue," Daria said experimentally, and nothing dire happened to her; but she couldn't bring herself to smile.  Daria the Blue exhaled sadly, and went back into the world, wondering what she would say.
• Eddin, a Green, looked up at the blue sky and began to laugh cynically.  The course of his world's history came clear at last; even he couldn't believe they'd been such fools.  "Stupid," Eddin said, "stupid, stupid, and all the time it was right here."  Hatred, murders, wars, and all along it was just a thing somewhere, that someone had written about like they'd write about any other thing.  No poetry, no beauty, nothing that any sane person would ever care about, just one pointless thing that had been blown out of all proportion.  Eddin leaned against the cave mouth wearily, trying to think of a way to prevent this information from blowing up the world, and wondering if they didn't all deserve it.
• Ferris gasped involuntarily, frozen by sheer wonder and delight.  Ferris's eyes darted hungrily about, fastening on each sight in turn before moving reluctantly to the next; the blue sky, the white clouds, the vast unknown outside, full of places and things (and people?) that no Undergrounder had ever seen.  "Oh, so that's what color it is," Ferris said, and went exploring.

8 more annotations...

• People go funny in the head when talking about politics.
• When, today, you get into an argument about whether "we" ought to raise the minimum wage, you're executing adaptations for an ancestral environment where being on the wrong side of the argument could get you killed.
• Politics is an important domain to which we should individually apply our rationality - but it's a terrible domain in which to learn rationality, or discuss rationality, unless all the discussants are already rational.
• Politics is an extension of war by other means.  Arguments are soldiers.  Once you know which side you're on, you must support all arguments of that side, and attack all arguments that appear to favor the enemy side; otherwise it's like stabbing your soldiers in the back - providing aid and comfort to the enemy.  People who would be level-headed about evenhandedly weighing all sides of an issue in their professional life as scientists, can suddenly turn into slogan-chanting zombies when there's a Blue or Green position on an issue.
• try to resist getting in those good, solid digs if you can possibly avoid it.
• I see politics as unimportant. For most of us, our political opinions have essentially no impact on the world. Their main effect is in our personal lives, our interactions with friends and family. On that basis, one should choose a political position that facilitates such "local" goals. There is little point in trying to be correct and accurate on large-scale political matters, other than as a bias-stretching mental exercise on a par with doing Sudoku.
• To deny politics is a form of political ideology itself.
• I don't think that talking about politics is either a mind-killer or time-wasting. For me personally it's a motivator both to understand more about the facts, so that I can present arguments; to understand more about other people, so I know why they disagree; and to understand more about myself, so that I can make sure that my convictions are solid. I actually believe that trying to find a way to influence politics to become more sensible is the most I can do to make a positive difference in the lives of other people.
• The pressure to commit "mind killing politics" type biases is proportional to how much status one or one's side has to lose for being wrong in any given disagreement.
• If you are talking to your friends or people close to you that you trust then the pressure to be right will be reduced because they are less likely to subtract status from you for being wrong. Talking to strangers will increase it.
• Having an audience will increase the pressure to be right. That's because the loss of status for being wrong is multiplied by the number of people that see you lose
• If someone is considered an 'expert', the pressure to be right will be enormous. Thats because experts have special status for being knowledgeable about a topic and getting answers about it right.
• Humans need rivals as much as they need comradery.

11 more annotations...

• monotonicity indicates that learning a new piece of knowledge cannot reduce the set of what is known

• some poor, honest, not overwhelmingly educated mother of 5 children is going to go into these stores and buy a "Dr. Snakeoil's Sulfuric Acid Drink" for her arthritis and die, leaving her orphans to weep on national television.
• under the Bayesian definition of evidence, "strong evidence" is just that sort of evidence which we only expect to find on one side of an argument.
• But there is no reason for complex actions with many consequences to exhibit this onesidedness property. 
• If you abide within that pattern, policy debates will also appear one-sided to you - the costs and drawbacks of your favored policy are enemy soldiers, to be attacked by any means necessary.
• A policy may legitimately have lopsided costs or benefits.
• But there is also a human tendency to deny all costs of a favored policy, or deny all benefits of a disfavored policy
• If you allow shops that sell otherwise banned products, some poor, honest, poorly educated mother of 5 kids is going to buy something that kills her.  This is a prediction about a factual consequence, and as a factual question it appears rather straightforward - a sane person should readily confess this to be true regardless of which stance they take on the policy issue.  You may also think that making things illegal just makes them more expensive, that regulators will abuse their power, or that her individual freedom trumps your desire to meddle with her life.  But, as a matter of simple fact, she's still going to die.
• We live in an unfair universe.  Like all primates, humans have strong negative reactions to perceived unfairness; thus we find this fact stressful.  There are two popular methods of dealing with the resulting cognitive dissonance.  First, one may change one's view of the facts - deny that the unfair events took place, or edit the history to make it appear fair.  Second, one may change one's morality - deny that the events are unfair.
• Some libertarians might say that if you go into a "banned products shop", passing clear warning labels that say "THINGS IN THIS STORE MAY KILL YOU", and buy something that kills you, then it's your own fault and you deserve it.  If that were a moral truth, there would be no downside to having shops that sell banned products.  It wouldn't just be a net benefit, it would be a one-sided tradeoff with no drawbacks.
• Others argue that regulators can be trained to choose rationally and in harmony with consumer interests; if those were the facts of the matter then (in their moral view) there would be no downside to regulation.
• Like it or not, there's a birth lottery for intelligence - though this is one of the cases where the universe's unfairness is so extreme that many people choose to deny the facts.  The experimental evidence for a purely genetic component of 0.6-0.8 is overwhelming, but even if this were to be denied, you don't choose your parental upbringing or your early schools either.
• Do you really think you're so smart that you would have been a proper scientific skeptic even if you'd been born in 500 C.E.?  Yes, there is a birth lottery, no matter what you believe about genes.
• Saying "People who buy dangerous products deserve to get hurt!" is not tough-minded.  It is a way of refusing to live in an unfair universe.  Real tough-mindedness is saying, "Yes, sulfuric acid is a horrible painful death, and no, that mother of 5 children didn't deserve it, but we're going to keep the shops open anyway because we did this cost-benefit calculation."  Can you imagine a politician saying that? 
• I don't think that when someone makes a stupid choice and dies, this is a cause for celebration.  I count it as a tragedy.  It is not always helping people, to save them from the consequences of their own actions; but I draw a moral line at capital punishment.  If you're dead, you can't learn from your mistakes.

12 more annotations...

• In human discourse there is a natural tendency to treat discussion as a form of combat, an extension of war, a sport; and in sports you only need to keep track of how many points have been scored by each team.
• "The Affect Heuristic in Judgments of Risks and Benefits" studied whether subjects mixed up their judgments of the possible benefits of a technology (e.g. nuclear power), and the possible risks of that technology, into a single overall good or bad feeling about the technology.
• If the reactor is more likely to melt down, this seems like a 'point against' the reactor, or a 'point against' someone who argues for building the reactor.  And if the reactor produces less waste, this is a 'point for' the reactor, or a 'point for' building it.  So are these two facts opposed to each other?  No.  In the real world, no.  These two facts may be cited by different sides of the same debate, but they are logically distinct; the facts don't know whose side they're on.  The amount of waste produced by the reactor arises from physical properties of that reactor design.  Other physical properties of the reactor make the nuclear reaction more unstable.  Even if some of the same design properties are involved, you have to separately consider the probability of meltdown, and the expected annual waste generated.  These are two different physical questions with two different factual answers.
• you have mixed up logically distinct questions - treated facts like human soldiers on different sides of a war, thinking that any soldier on one side can be used to fight any soldier on the other side.
• all Bayesian evidence consists of probability flows between hypotheses; there is no such thing as evidence that "supports" or "contradicts" a single hypothesis, except insofar as other hypotheses do worse or better.
• Lady Rationality carries a notebook, wherein she writes down all the facts that aren't on anyone's side.

4 more annotations...

• The correspondence bias is the tendency to draw inferences about a person's unique and enduring dispositions from behaviors that can be entirely explained by the situations in which they occur.
• We attribute our own actions to our situations, seeing our behaviors as perfectly normal responses to experience. But when someone else kicks a vending machine, we don't see their past history trailing behind them in the air.  We just see the kick, for no reason we know about, and we think this must be a naturally angry person - since they lashed out without any provocation.
• Yet consider the prior probabilities.  There are more late buses in the world, than mutants born with unnaturally high anger levels that cause them to sometimes spontaneously kick vending machines.
• any given aspect of someone's disposition is probably not very far from average.  To suggest otherwise is to shoulder a burden of improbability.
• Even when people are informed explicitly of situational causes, they don't seem to properly discount the observed behavior.  When subjects are told that a pro-abortion or anti-abortion speaker was randomly assigned to give a speech on that position, subjects still think the speakers harbor leanings in the direction randomly assigned. 
• mechanisms sound more complicated than essences; they are harder to think of, less available.  So when someone kicks a vending machine, we think they have an innate vending-machine-kicking-tendency.
• To understand why people act the way they do, we must first realize that everyone sees themselves as behaving normally.  Don't ask what strange, mutant disposition they were born with, which directly corresponds to their surface behavior.  Rather, ask what situations people see themselves as being in.  Yes, people do have dispositions - but there are not enough heritable quirks of disposition to directly account for all the surface behaviors you see.
• Suppose I gave you a control with two buttons, a red button and a green button.  The red button destroys the world, and the green button stops the red button from being pressed.  Which button would you press?  The green one.  Anyone who gives a different answer is probably overcomplicating the question.
• Most people see themselves as perfectly normal, from the inside.  Even people you hate, people who do terrible things, are not exceptional mutants.  No mutations are required, alas.  When you understand this, you are ready to stop being surprised by human events.

7 more annotations...

• Whenever I hear someone describe quantum physics as "weird"
• then I think to myself:  This person will never understand physics no matter how many books they read.
• Reality has been around since long before you showed up.  Don't go calling it nasty names like "bizarre" or "incredible".  The universe was propagating complex amplitudes through configuration space for ten billion years before life ever emerged on Earth.  Quantum physics is not "weird".  You are weird.  You have the absolutely bizarre idea that reality ought to consist of little billiard balls bopping around, when in fact reality is a perfectly normal cloud of complex amplitude in configuration space.  This is your problem, not reality's, and you are the one who needs to change.
• When you go down to the fundamental level, the level on which the laws are stable, global, and exception-free, there aren't any tigers.  In fact there aren't any persistent objects bouncing around in 3 spatial dimensions.  Deal with it.
• Calling reality "weird" keeps you inside a viewpoint already proven erroneous.  Probability theory tells us that surprise is the measure of a poor hypothesis; if a model is consistently stupid  - consistently hits on events the model assigns tiny probabilities - then it's time to discard that model.  A good model makes reality look normal, not weird; a good model assigns high probability to that which is actually the case.  Intuition is only a model by another name: poor intuitions are shocked by reality, good intuitions make reality feel natural.  You want to reshape your intuitions so that the universe looks normal.  You want to think like reality.
• Spending emotional energy on incredulity wastes time you could be using to update.  It repeatedly throws you back into the frame of the old, wrong viewpoint.  It feeds your sense of righteous indignation at reality daring to contradict you.
• if you literally don't understand, then your indignation is stopping you from coming to terms with reality.  It shouldn't be hard to imagine how a PhD physicist ends up believing in astrology.  People compartmentalize, enough said.
• Surprise exists in the map, not in the territory.  There are no surprising facts, only models that are surprised by facts. 

6 more annotations...

• We see unusual dispositions that exactly match the unusual behavior, rather than asking after real situations or imagined situations that could explain the behavior.  We hypothesize mutants.
• Not as a moral point, but as a strict question of prior probability, we should ask what the Enemy might believe about their situation which would reduce the seeming bizarrity of their behavior.
• Realistically, most people don't construct their life stories with themselves as the villains.  Everyone is the hero of their own story.  The Enemy's story, as seen by the Enemy, is not going to make the Enemy look bad.  If you try to construe motivations that would make the Enemy look bad, you'll end up flat wrong about what actually goes on in the Enemy's mind.
• If the Enemy did have an evil disposition, that would be an argument in favor of your side.  And any argument that favors your side must be supported, no matter how silly - otherwise you're letting up the pressure somewhere on the battlefront.  Everyone strives to outshine their neighbor in patriotic denunciation, and no one dares to contradict.  Soon the Enemy has horns, bat wings, flaming breath, and fangs that drip corrosive venom.  If you deny any aspect of this on merely factual grounds, you are arguing the Enemy's side; you are a traitor.  Very few people will understand that you aren't defending the Enemy, just defending the truth.
• Or maybe the fear is that understanding will lead to forgiveness.  It's easier to shoot down evil mutants.  It is a more inspiring battle cry to scream, "Die, vicious scum!" instead of "Die, people who could have been just like me but grew up in a different environment!"  You might feel guilty killing people who weren't pure darkness.

  • Important.
• If an army is crossing the border or a lunatic is coming at you with a knife, the policy alternatives are (a) defend yourself (b) lie down and die.  If you defend yourself, you may have to kill.  If you kill someone who could, in another world, have been your friend, that is a tragedy.  And it is a tragedy.  The other option, lying down and dying, is also a tragedy.  Why must there be a non-tragic option?  Who says that the best policy available must have no downside?  If someone has to die, it may as well be the initiator of force, to discourage future violence and thereby minimize the total sum of death.
• When you accurately estimate the Enemy's psychology - when you know what is really in the Enemy's mind - that knowledge won't feel like landing a delicious punch on the opposing side.  It won't give you a warm feeling of righteous indignation.  It won't make you feel good about yourself.  If your estimate makes you feel unbearably sad, you may be seeing the world as it really is.  More rarely, an accurate estimate may send shivers of serious horror down your spine, as when dealing with true psychopaths, or neurologically intact people with beliefs that have utterly destroyed their sanity (Scientologists or Jesus Camp).
• It does mean that in another world, if they had been raised in a different environment, those hijackers might have been police officers.  And that is indeed a tragedy.  Welcome to Earth.

6 more annotations...

• How would they spark an intergroup conflict to investigate?  Well, the 22 boys were divided into two groups of 11 campers, and -

   

  - and that turned out to be quite sufficient.
• There was hostility almost from the moment each group became aware of the other group's existence:  They were using our campground, our baseball diamond.  On their first meeting, the two groups began hurling insults.
• Each group developed a negative stereotype of Them and a contrasting positive stereotype of Us.  The Rattlers swore heavily.
• In Stage 3, the researchers tried to reduce friction between the two groups.
• Mere contact (being present without contesting) did not reduce friction between the two groups.  Attending pleasant events together - for example, shooting off Fourth of July fireworks - did not reduce friction; instead it developed into a food fight.
• The boys were informed that there might be a water shortage in the whole camp, due to mysterious trouble with the water system - possibly due to vandals.  (The Outside Enemy, one of the oldest tricks in the book.)
• It wasn't the end of the rivalry.  There was another food fight, with insults, the next morning.  But a few more common tasks, requiring cooperation from both groups - e.g. restarting a stalled truck - did the job.  At the end of the trip, the Rattlers used $5 won in a bean-toss contest to buy malts for all the boys in both groups.
• (Sometimes I think humanity's second-greatest need is a supervillain.  Maybe I'll go into that line of work after I finish my current job.)
• Sherif, M., Harvey, O. J., White, B. J., Hood, W. R., & Sherif, C. W. 1954/1961. Study of positive and negative intergroup attitudes between experimentally produced groups: Robbers Cave study. University of Oklahoma.

7 more annotations...

• "The standard smother-out technique," Verkan Vall grinned.
• You and I believe that flying saucer cults arose in the total absence of any flying saucers.  Cults can arise around almost any idea, thanks to human silliness.  This silliness operates orthogonally to alien intervention:  We would expect to see flying saucer cults whether or not there were flying saucers. 
• By the Bayesian definition of evidence, the observation "flying saucer cults exist" is not evidence against the existence of flying saucers.  It's not much evidence one way or the other.
• as Robert Pirsig puts it, "The world's greatest fool may say the Sun is shining, but that doesn't make it dark out."
• If you knew someone who was wrong 99.99% of the time on yes-or-no questions, you could obtain 99.99% accuracy just by reversing their answers.  They would need to do all the work of obtaining good evidence entangled with reality, and processing that evidence coherently, just to anticorrelate that reliably.  They would have to be superintelligent to be that stupid.
• A car with a broken engine cannot drive backward at 200 mph, even if the engine is really really broken.
• If stupidity does not reliably anticorrelate with truth, how much less should human evil anticorrelate with truth? 
• To argue against an idea honestly, you should argue against the best arguments of the strongest advocates.  Arguing against weaker advocates proves nothing, because even the strongest idea will attract weak advocates.
• The least convenient path is the only valid one.
• Exhibiting sad, pathetic lunatics, driven to madness by their apprehension of an Idea, is no evidence against that Idea.
• Someone once said, "Not all conservatives are stupid, but most stupid people are conservatives."  If you cannot place yourself in a state of mind where this statement, true or false, seems completely irrelevant as a critique of conservatism, you are not ready to think rationally about politics.

  • Interesting.
• Your instinctive willingness to believe something will change along with your willingness to affiliate with people who are known for believing it - quite apart from whether the belief is actually true. 
• If your current computer stops working, you can't conclude that everything about the current system is wrong and that you need a new system without an AMD processor, an ATI video card, a Maxtor hard drive, or case fans - even though your current system has all these things and it doesn't work.  Maybe you just need a new power cord.

11 more annotations...

• Indeed, if the technical arguments are good enough, Barry's advantage over Charles may not be worth tracking.  A good technical argument is one that eliminates reliance on the personal authority of the speaker.
• if we really believe Ernie that the argument he gave is the best argument he could give, which includes all of the inferential steps that Ernie executed, and all of the support that Ernie took into account - citing any authorities that Ernie may have listened to himself - then we can pretty much ignore any information about Ernie's credentials.  Ernie can be a physicist or a clown, it shouldn't matter. 
• there's an asymmetry between argument and authority.  If we know authority we are still interested in hearing the arguments; but if we know the arguments fully, we have very little left to learn from authority.
• Suppose that H is "My sidewalk is slippery", E1 is "My sprinkler is running", and E2 is "It's night."  The sidewalk is slippery starting from 1 minute after the sprinkler starts, until just after the sprinkler finishes, and the sprinkler runs for 10 minutes.  So if we know the sprinkler is on, the probability is 90% that the sidewalk is slippery.  The sprinkler is on during 10% of the nighttime, so if we know that it's night, the probability of the sidewalk being slippery is 9%.  If we know that it's night and the sprinkler is on - that is, if we know both facts - the probability of the sidewalk being slippery is 90%.

  • Nice.
• p(slippery|night, sprinkler) = p(slippery|sprinkler)
• Knowledge of the Sprinkler has made knowledge of the Night irrelevant to inferences about Slipperiness.
• This is known as screening off, and the criterion that lets us read such conditional independences off causal graphs is known as D-separation.
• if we know the value of the Argument node, this D-separates the node "Truth" from the node "Expert Belief" by blocking all paths between them
• p(truth|argument,expert) = p(truth|argument)
• Authority and argument don't need two different kinds of probability, any more than sprinklers are made out of ontologically different stuff than sunlight.
• In practice you can never completely eliminate reliance on authority.  Good authorities are more likely to know about any counterevidence that exists and should be taken into account; a lesser authority is less likely to know this, which makes their arguments less reliable.  This is not a factor you can eliminate merely by hearing the evidence they did take into account.
• It's also very hard to reduce arguments to pure math; and otherwise, judging the strength of an inferential step may rely on intuitions you can't duplicate without the same thirty years of experience.
• There is an ineradicable legitimacy to assigning slightly higher probability to what E. T. Jaynes tells you about Bayesian probability, than you assign to Eliezer Yudkowsky making the exact same statement.  Fifty additional years of experience should not count for literally zero influence.
• the arguments presented are often only a small fraction of what convinced someone to support a position.

12 more annotations...

• observe evidence that is as near to the original question as possible, so that it screens off as many other arguments as possible.
• If you actually watch the plane fly, the calculations themselves become moot for many purposes, and Kelvin's authority not even worth considering.
• you can never get more information from distant nodes, than from strictly closer nodes that screen off the distant ones.
• it is also superior to argue physics than rationality
• It's all too easy to argue that someone is exhibiting Bias #182 in your repertoire of fully generic accusations, but you can't settle a factual issue without closer evidence.  If there are biased reasons to say the sun is shining, that doesn't make it dark out.
• Whenever you can, dance as near to the original question as possible - press yourself up against it - get close enough to hug the query!

4 more annotations...

• Prior attitude effect. Subjects who feel strongly about an issue - even when encouraged to be objective - will evaluate supportive arguments more favorably than contrary arguments.
• Disconfirmation bias. Subjects will spend more time and cognitive resources denigrating contrary arguments than supportive arguments.
• Confirmation bias. Subjects free to choose their information sources will seek out supportive rather than contrary sources.
• Attitude polarization. Exposing subjects to an apparently balanced set of pro and con arguments will exaggerate their initial polarization.
• Attitude strength effect. Subjects voicing stronger attitudes will be more prone to the above biases.
• Sophistication effect. Politically knowledgeable subjects, because they possess greater ammunition with which to counter-argue incongruent facts and arguments, will be more prone to the above biases.
• If you're irrational to start with, having more knowledge can hurt you.
• too much ready ammunition - is one of the primary ways that people with high mental agility end up stupid, in Stanovich's "dysrationalia" sense of stupidity.
• hey have more ammunition with which to argue against anything they don't like.
• Somehow, when it came to trusting his own preferred conclusions, all those biases and fallacies seemed much less salient - leapt much less readily to mind - than when he needed to counter-argue someone else.
• I told the one about the problem of disconfirmation bias and sophisticated argument, and lo and behold, the next time I said something he didn't like, he accused me of being a sophisticated arguer.  He didn't try to point out any particular sophisticated argument, any particular flaw - just shook his head and sighed sadly over how I was apparently using my own intelligence to defeat itself.  He had acquired yet another Fully General Counterargument.
• the failure to generalize from knowledge of others' failures to appreciation of your own.
• the bias is generalizing from knowledge of others' failures to skepticism about disliked conclusions, but failing to generalize to skepticism about preferred conclusions or one's own conclusions. That is, the error is not absence of generalization, but imbalance of generalization, which is far deadlier.

11 more annotations...

• Envy me!
  Aim at me!
  Rival me!
  Transcend me!
• I was never your city,
  Just a stretch of your road.

• Every now and then, yet another atheist is struck by the amazing idea that atheists should have hymns, just like religious people have hymns, and they take some existing religious song and turn out an atheistic version.  And then this "atheistic hymn" is, almost without exception, absolutely awful.  But the author can't see how dreadful the verse is as verse.  They're too busy congratulating themselves on having said "Religion sure sucks, amen."  Landing a punch on the Hated Enemy feels so good that they overlook the hymn's lack of any other merit.  Verse of the same quality about something unpolitical, like mountain streams, would be seen as something a kindergartener's mother would post on her refrigerator. 
• The prosody - the pattern of stressed syllables - was all wrong.
• "I was never your city" came to me, not by thinking about rationality, but by thinking about prosody.  The constraints of art force us to toss out the first, old, tired phrasing that comes to mind; and in searching for a less obvious phrasing, often lead us to less obvious thoughts.
• If I'd said, "Well, this is such a wonderful thought about rationality, that I don't have to worry about the prosodic problem", then I would have not received the benefit of being constrained.
• If you allow affirmation of My-Favorite-Idea to compensate for lack of rhythm in a song, lack of beauty in a painting, lack of poignancy in fiction, then your art will, inevitably, suck.
• There is powerful politicized art, just as there are great religious paintings.  But merit in politicized art is more the exception than the rule.  Most of it ends up as New Soviet Man Heroically Crushing Capitalist Snakes.  It's an easy living.  If anyone criticizes your art on grounds of general suckiness, they'll be executed for siding with the capitalist snakes.
• Tolerance of awful art, just because it lands a delicious punch on the Enemy, or just because it affirms the Great Truth, is a dangerous sign:  It indicates an affective death spiral entering the supercritical phase where you can no longer criticize any argument whose conclusion is the "right" one.

5 more annotations...

• It suffices that the Hated Enemy gets hurt.  It's like humor, only without the humor.
• atire is a much more demanding art than just punching the Enemy in the nose
• In fact, it was the very first example that popped into my mind when I thought "political cartoon about Bush and science".  This degree of obviousness is a bad sign.
• humor relies on surprise - the ridiculous, the unexpected, the absurd.
• (Satire achieves surprise by saying, out loud, the thoughts you didn't dare think.  Fake satires repeat thoughts you were already thinking.)
• The general test against awful political art is to ask whether the art would seem worthwhile if it were not political. 
• one test for derisive laughter is to ask if the joke would still be funny, if it weren't the Hated Enemy getting the kick in the pants.
• If you find yourself in a group of people who tell consistently unfunny jokes about the Hated Enemy, it may be a good idea to head for the hills, before you start to laugh as well...
• The most scathing accusation I ever heard against Objectivism is that hardcore Objectivists have no sense of humor

  • Robots!

  • Interesting.
• Laughter, of all things, has to come naturally.
• Obstacles to humor are a sign of dreadful things.

9 more annotations...

• If you really want an artist's perspective on rationality, then read Orwell; he is mandatory reading for rationalists as well as authors.  Orwell was not a scientist, but a writer; his tools were not numbers, but words; his adversary was not Nature, but human evil.  If you wish to imprison people for years without trial, you must think of some other way to say it than "I'm going to imprison Mr. Jennings for years without trial."  You must muddy the listener's thinking, prevent clear images from outraging conscience.  You say, "Unreliable elements were subjected to an alternative justice process."
• With enough static noun phrases, you can keep anything unpleasant from actually happening.
• Passive voice obscures reality.
• It doesn't seem like a logical flaw.  And this is why rationalists need to read Orwell, not just Feynman or even Jaynes.
• Nonfiction conveys knowledge, fiction conveys experience.  Medical science can extrapolate what would happen to a human unprotected in a vacuum.  Fiction can make you live through it.
• A writer knows that readers will not stop for a minute to think.  A fictional experience is a continuous stream of first impressions.  A writer-rationalist pays attention to the experience words create.  If you are evaluating the public rationality of a statement, and you analyze the words deliberatively, rephrasing propositions, trying out different meanings, searching for nuggets of truthiness, then you're losing track of the first impression - what the audience sees, or rather feels.
• A novelist would notice the screaming wrongness of "The subjects were administered Progenitorivox."  What life is here for a reader to live?  This sentence creates a distant feeling of authoritativeness, and that's all - the only experience is the feeling of being told something reliable.  A novelist would see nouns too abstract to show what actually happened - the postdoc with the bottle in his hand, trying to look stern; the student listening with a nervous grin.
• My point is not to say that journal articles should be written like novels, but that a rationalist should become consciously aware of the experiences which words create.  A rationalist must understand the mind and how to operate it.  That includes the stream of consciousness, the part of yourself that unfolds in language.  A rationalist must become consciously aware of the actual, experiential impact of phrases, beyond their mere propositional semantics.
• Meaning does not excuse impact!
• I don't care what rational interpretation you can construct for an applause light like "AI should be developed through democratic processes".  That cannot excuse its irrational impact of signaling the audience to applaud, not to mention its cloudy question-begging vagueness.
• More than one path can lead to the Way.

9 more annotations...

• If we really cared what artists thought, we would find some artists and ask them questions, not call for artists to participate.  We don't actually want to hear from artists.  We think your opinions are stupid.
• You are not an artist, you are a human being; art is only one facet in which you express your humanity.  Your reactions to the Singularity should arise from your entire self.  It's perfectly all right to have a boringly normal and nonunique reaction like "I'm afraid," or "I don't think we should do this," or "I want to help, where do I send the check?"  The right answer is not always unusual.  Your natural reaction does not need to be unique, and that's why you don't need to try to come up with an "artist's viewpoint" on the Singularity.  I would call on you to participate as a human being, not an artist.  If your artistry has something to say, it will express itself naturally in your responses, without you needing to make a conscious effort to say something artist-like.