One of my favorite essays, by Sarah Constantin (one of my favorite contemporary thinkers), is Fact Posts: How and Why. Fact posts are “an exercise in original seeing and showing your reasoning, not finding the official last word on a topic or doing the best analysis in the world”, so it’s okay if you get things wrong. Starting with an empirical question or general topic (“how common are hate crimes?”):
You look for quantitative data from conventionally reliable sources. CDC data for incidences of diseases and other health risks in the US; WHO data for global health issues; Bureau of Labor Statistics data for US employment; and so on. Published scientific journal articles, especially from reputable journals and large randomized studies.
You explicitly do not look for opinion, even expert opinion. You avoid news, and you’re wary of think-tank white papers. You’re looking for raw information. You are taking a sola scriptura approach, for better and for worse.
And then you start letting the data show you things.
You see things that are surprising or odd, and you note that.
You see facts that seem to be inconsistent with each other, and you look into the data sources and methodology until you clear up the mystery.
You orient towards the random, the unfamiliar, the things that are totally unfamiliar to your experience. One of the major exports of Germany is valves? When was the last time I even thought about valves? Why valves, what do you use valves in? OK, show me a list of all the different kinds of machine parts, by percent of total exports.
And so, you dig in a little bit, to this part of the world that you hadn’t looked at before. You cultivate the ability to spin up a lightweight sort of fannish obsessive curiosity when something seems like it might be a big deal.
And you take casual notes and impressions (though keeping track of all the numbers and their sources in your notes).
You do a little bit of arithmetic to compare things to familiar reference points. How does this source of risk compare to the risk of smoking or going horseback riding? How does the effect size of this drug compare to the effect size of psychotherapy?
You don’t really want to do statistics. You might take percents, means, standard deviations, maybe a Cohen’s d here and there, but nothing fancy. You’re just trying to figure out what’s going on.
It’s often a good idea to rank things by raw scale. What is responsible for the bulk of deaths, the bulk of money moved, etc? What is big? Then pay attention more to things, and ask more questions about things, that are big. (Or disproportionately high-impact.)
You may find that this process gives you contrarian beliefs, but often you won’t, you’ll just have a strongly fact-based assessment of why you believe the usual thing.
(I love that long quote. I keep coming back to it again and again.)
The thing that’s relevant here for what I’ll quote next is what Sarah calls “a sense of the world that stays in place, even as you discover new facts, instead of wildly swinging around at every new stimulus”:
There’s a quality of ordinariness about fact-based beliefs. It’s not that they’re never surprising — they often are. But if you do fact-checking frequently enough, you begin to have a sense of the world overall that stays in place, even as you discover new facts, instead of swinging wildly around at every new stimulus. For example, after doing lots and lots of reading of the biomedical literature, I have sort of a “sense of the world” of biomedical science — what sorts of things I expect to see, and what sorts of things I don’t. My “sense of the world” isn’t that the world itself is boring — I actually believe in a world rich in discoveries and low-hanging fruit — but the sense itself has stabilized, feels like “yeah, that’s how things are” rather than “omg what is even going on.”
In areas where I’m less familiar, I feel more like “omg what is even going on”, which sometimes motivates me to go accumulate facts.
Scott Alexander adds:
Don’t underestimate Wikipedia as a really good place to get a (usually) unbiased overview of things and links to more in-depth sources.
The warning against biased sources is well-taken, but if you’re looking into something controversial, you might have to just read the biased sources on both sides, then try to reconcile them. I’ve found it helpful to find a seemingly compelling argument, google something like “why X is wrong” or “X debunked” into Google, and see what the other side has to say about it. Then repeat until you feel like both sides are talking past each other or disagreeing on minutiae. This is important to do even with published papers!
Success often feels like realizing that a topic you thought would have one clear answer actually has a million different answers depending on how you ask the question. You start with something like “did the economy do better or worse this year?”, you find that it’s actually a thousand different questions like “did unemployment get better or worse this year?” vs. “did the stock market get better or worse this year?” and end up with things even more complicated like “did employment as measured in percentage of job-seekers finding a job within six months get better” vs. “did employment as measured in total percent of workforce working get better?”. Then finally once you’ve disentangled all that and realized that the people saying “employment is getting better” or “employment is getting worse” are using statistics about subtly different things and talking past each other, you use all of the specific things you’ve discovered to reconstruct a picture of whether, in the ways important to you, the economy really is getting better or worse.
(That last part, that people operationalize vague questions plausibly-but-differently, is well worth internalizing.)
This segues into a recent essay on The Scholar’s Stage (so recent it just appeared last month!). Let’s say you’re interested in the following kinds of questions:
What makes human society work?
Why do people do what they do?
How does culture/wealth/geography/[enter your favorite variable here] change human behavior?
What is the relationship between human behavior seen at the micro-scale and at the macro-scale?
Do ideas matter?
How much does individual choice matter?
Is it possible to live morally in human society?
Is it possible for societies as a whole to become more or less moral over time?
Let’s say you have no special background in any particular fields, and you aren’t especially mathematically inclined (if you are, I’d recommend whatever Cosma Shalizi recommends, or just his Bactra Review.) Then you, like lots of (quote) “bookish overly-intellectual American teenagers”, will be especially drawn to a certain genre of ‘soft’ SF exemplified by Banks’ Culture series, Card’s Ender’s Game, Herbert’s Dune, Heinlein’s Starship Troopers, Le Guin’s The Dispossessed, Ayn Rand’s books, Nietzsche’s works, etc. The Scholar (who has no ‘About’ page, so I don’t know a single thing about her/him) writes:
None of these books (well, maybe a few of Hermann Hesse’s…) were designed for the ‘young adult’ audience. Almost all were written before publishers considered ‘YA’ a distinct consumer demographic. Much of their attraction to the teenage mind comes from this fact. These books are adult works written for adult audiences. They are meant to be taken seriously. And these young readers do take them seriously.
These are all books with big ideas. The ideas rest at the intersection of action and thought. Foundational to all of these works is a critique of the conventional. This is quite explicit in the work of the philosophers and existentialists, who write directly of what bothers them in human life, and how humans might do better. The critique is more subtle in the science fiction novels. Here readers are presented with societies vastly different than their own, fictional utopias and dystopias that discard all of the assumptions of American middle class life. They operate on a different set of values than that taught in classrooms and living rooms of suburbia. They force readers to reassess their own values and assumptions about what makes society work. No matter what else might be packed into it, this is an underlying message behind any thoughtful work of ‘soft’ science fiction: things could be different.
You could learn this other ways, of course. A look at the political philosophy of the Aztecs or the feuding laws of Medieval Iceland will force you to rethink your assumptions about what makes humans tick. But that is hard. In contrast, science fiction writers wrote with modern audiences in mind and package their material into engaging narratives. You can read them without bothering with supporting class lectures or extended footnotes. That appeals to an intellectual 16 year old. Well written science fiction is history and political philosophy on the cheap.
Now the issue is that if you don’t know history, all these depictions of alternate societies will sound extremely plausible, because they’re so detailed. To someone like the Scholar, who does know history, and is familiar with civilizational failure modes both legible and less-legible, they’re obviously doomed to failure.
What if you really are interested in the questions above, want your “sense of the world to stay in place” as you come across new convincingly-detailed depictions of alternate societies that contradict ones you already know, but don’t have tens of thousands of hours to spare? (This is the crux of this whole post.)
The Scholar suggests a five-step reading strategy: history+archaeology/ethnography, literature, behavioral science, political/moral philosophy, and social science. I’m attracted to their explanation of the first step:
History is the most important thing you can read. Why? Only a strong background in history can you tell you when writers in other fields are full of crap. I cannot tell you the number of times I have a found a political argument (or even fairly well regarded work of social science) that reads compelling at the 10,000 foot view but falls apart when you stack it up against concrete facts of history seen from the ground view. Humans are motivated reasoners. We bend the data to fit our theories. If you are not familiar with the data, you will not realize when it is being distorted or misused.
The data of the social sciences is history.
The problem with history is that it is too big. It is impossible to get a fine grained picture of every people and era on the planet. There is just too much of it.
My recommendation is to pick three very different historical periods that you find fascinating. They can be any three, really, but ideally they will be a bit separated from each other in space, time, and culture. For example, you might choose pre-Columbian Mesoamerica, the Abbasid empire, and revolutionary Russia. Or maybe your interests lie with Republican Rome, the Protestant reformation, and 20th century India. That all works. It does not really matter what you choose, as long as you have decent spread (at least one is ‘modern,’ at least one is ‘ancient,’ and at least one is from a non-Western civilization). The important thing is that you have a genuine interest in these societies strong enough that you could gladly read 4-6 books about each of them without getting bored.
Because that is what you should do. Read 4-6 books about each of the eras in question.
Your goal here is to build up a fairly granular knowledge of a particular time or event than can be called on to test and assess theories and narratives that will be thrown at you. “Famous scholar X proposes that y leads to z, but did y lead to z in each of the eras I am most familiar with?” You will know you have the background knowledge to do this right when you can answer questions like the following for a given era of expertise: “What are some of the biggest disagreements historians have about this era/event? What are the main sources historians or archaeologists use to try and understand the era, and how might they bias this understanding? If you had to pick one small incident or detail about the era that seems insignificant at first, but is actually very revealing example of the way this society/event worked, what would it be?”
You don’t need PhD levels answers to these questions. Just something more insightful that you would get from the Wikipedia page.
From that point, you can broaden out to more general histories. If you read fast enough to keep reading 4-5 books on different eras, keep on doing that. More normal people will probably want to transition to broader surveys that fill in the blank spaces they have with the rest of the world. There are plenty of fine histories that cover entire countries or regions from antiquity to the present (e.g., India: A History Japan and the Shackles of the Past). Others might follow the history of a specific topic (say, war, the environment, or the financial system) over multiple centuries (e.g. the Pursuit of Power, Ecological Imperialism, the Cash Nexus). Others might do the same thing, but restrict themselves to a slightly smaller geographical scale (e.g. Asian Military Revolution, China: An Environmental History, An Economic History of China). Global histories of entire centuries are also somewhat in vogue (I blame Hobsawm’s series for this development). Others will be comparative histories—works of history or ethnography that line up dozens of societies (e.g. The Lifeways of Hunter-Gatherers, Understanding Early Civilizations, War in Human Civilization, Dynamics of Ancient Empires), or just a few (Islamic Gunpowder Empires, Empires of the Atlantic World, The Industrial Revolution in World History). All of these will do.
If that seems overwhelming, one way to make it easier would be to focus one particular macro-topic that can be explored in almost every single society. I personally have a special interest in warfare and military affairs. Reading about the wars and military institutions of different societies across history of human civilization has proved useful for learning much about the broader history of the societies involved. Something similar can be said for economic, religious, institutional, and environmental history.…
The last group of history books are the ones you are likely the most eager to read. These are books like Jared Diamond’s Guns, Germs, and Steel, Francis Fukuyama’s Political Order, Ian Morris’s Why the West Rules. While methodologically these books are properly considered histories, for the purpose of this series I group them with the social sciences. They are concerned with the same questions that animate works of social science like Acemoglu and Robinson’s Why Nations Fail: The Origins of Power, Prosperity, and Poverty or the entire oeuvre of Peter Turchin. Why do some countries become wealthy while others do not? What explains the rise and fall of civilizations? Why did Western countries conquer the world instead of the other way around?
These books are fine to read and fun to contemplate, but if you start here you are doing it wrong. I have collected fifteen separate 400+ page books that try to answer the question “why did the West get rich first.” And that was seven years ago! The number of books tackling this question has only grown larger. But if that is all you read, you are in trouble. How will you know who is right and who is wrong? If you have not read widely in history and anthropology, who are you to judge? There is absolutely no point, for example, in reading one of Peter Turchin’s books if you don’t have the background knowledge needed to assess whether his models match the historical record. There is no point reading Diamond’s explanation for why China stagnated and why Europe did not if you do not know anything about Chinese or European history yourself (I am not convinced Diamond does). Grand theories of civilization should be at the bottom of your list. They are worth reading, but not before you have the foundation in facts that you need to distinguish between the good work and the ill.
So how do you find the history books worth reading? Occasionally people you can trust will put up reading lists. I have a reading list here on books to read in Chinese history. Here is Razib Khan’s recommendations on Roman history. Will Buckner has a list of valuable ethnographies over at Traditions of Conflict. Bryn Hammond has an absolutely fabulous set of reviews on just about every book ever written on the Mongols and Inner Asian nomads. Website like Five Books are another good place to start.
But if no reading lists come to mind, there are two methods in particular I have often have useful. The first is to Google syllabi. If you are interested in the history of the Roman Republic, Google “Roman Republic syllabus” and see what pops up. Read a few courses and see what books are included. Alternatively, if you just read a book you thought was particularly good, put its title into Google and then the word “syllabus” afterwards and see what other readings college professors have paired with that book in their courses.
The other route is the more old-fashioned: read the footnotes. A significant percentage of what I read comes not from book reviews or book lists, but by looking up and purchasing the books mentioned in the footnotes of other books I found interesting. Often times the best book on a topic is not the newest one. This is how experienced academics and researchers fill up their own reading lists. What works for us will work for you.
(Don’t just read my quotes – if you’re interested in the Scholar’s post, go to their blog!)