In the last newsletter I put forward a list of the many common logical fallacies that are used to present bullshit as truth. In this newsletter I intend to give you a set of tools that will allow you to better discover the truth. Be warned that searching for and discovering the truth is not always a socially acceptable activity. Our world is built out of bullshit and debunking things that are held as truth can be inherently anti-social. But knowing the truth does have its advantages and thus it is always useful to know how to go about finding out how things really work.

The best tool we have today for discovering the truth is called the scientific method. It is not perfect, but given the technological progress of the last two centuries, and knowing that it was made possible by people using the scientific method to discover how the physical world behaves, it far surpasses any other available approach in effectiveness.

It is disappointing that the scientific method is all too often NOT taught in science classes. Generally students are taught facts and theories which were discovered by those using the scientific method but are never taught the procedure for establishing fact and supporting theories with evidence. As a result, science is sometimes perceived as being a body of knowledge that is supported by an argument from authority, in essence "we are scientists and thus we know the truth". The statement is bullshit, and truth should always be open to scrutiny as it stands exclusively on the facts which support it. Everything should be so open, but most things are not.

Being taught "the truth" rather than how to get to the truth is a serious and unfortunately common occurrence. It leaves us without the ability to think critically, and failing to scrutinize that which is presented as the truth when it is important to you, whether you accept it or dismiss it, is a mistake. The truth can help you and it can hurt you too. So whether a claim is made by a scientist, a religious leader, a politician, or anyone, you should approach it the same way, using an analytic approach, to determine its validity.

The following process is the scientific method. It is used by people who use test tubes and microscopes in labs, but you don't need any special equipment to make it work. And you don't need to restrict yourself as to the subject that you can study with it, it can be used for anything. The important part is to know the process, to understand it, and to use it effectively. Here are the basic steps:

1. A number of related facts are observed
2. A theory is conjectured to explain the facts
3. More facts are gathered which could support or refute the theory
4. The theory is tested via experiments
5. The theory and method are reviewed by others
6. The experiments are replicated by others

Once a theory survives all of these steps, it becomes well supported, provided that other well supported theories do not exist which contradict it, but the theory remains open to criticism and refutation until such a time when it is disproved. Note that there is no litmus test for truth, as at any time in the future a fact may be presented which refutes any theory, but a theory with plenty of support which has stood the test of time is generally considered to be the truth.

The process looks pretty dry and technical, but here is how it can be applied to daily life.

1. A number of related facts are observed

This could be simplified as saying "you notice a few things that seem to be connected". This is how everything starts. Something catches your attention and you recognize it as being similar to something that has happened before. Perhaps it is something good that you want to have happen again. Perhaps it is something bad that you want to avoid in the future. In any event, you want to gain an understanding of the phenomenon in question to help you cope with your life.

Here is an everyday example that I will use to demonstrate the scientific method. Let's say that you wake up sick to your stomach for the third Saturday in the last two months. Those are related facts that I think we can all relate to. You decide that you want to stop it from happening again and that means that you need to find out what is causing it.

2. A theory is conjectured to explain the facts

Since related facts have similarities, it's not a wild guess to assume that they have a similar cause. At this point you can come up with one theory, but it's always good to come up with more than one theory so that your mind is open when you are searching for more facts to support or refute your theories. By focusing on only one you may pass over evidence for other theories and hamper your search for the truth.

So in our example let's say that you come up with a few theories to explain the sickness: it was something you ate; you drank too much on Friday night; or you slept in and threw off your sleep/wake cycle. It may be that none of these theories are correct, but at least now you have a place to start. Always be open to other theories which could explain the facts as you may have to expand your search.

3. More facts are gathered which could support or refute the theory

Now that you have a few possible theories it is time to gather together more facts to determine which theories are worth following up on with experiments. Fact gathering is a fairly straight forward process. The theories themselves dictate where to look for supporting evidence. Each theory involves a number of things which need to happen to cause the observed effect. Theories, like criminals, leave a trail of evidence behind. As an investigator you need to follow the trail and see if there is any evidence which would support each theory.

It is important at this stage to avoid getting fixated on any one theory and to avoid searching only for things which support that theory. Favoritism can result in skewed data gathering where you seek only for facts which support your theory and ignore looking for things which refute your theory or support others. Setting aside favoritism and treating each theory equally is often referred to as being objective and dispassionate. These are qualities that will help you find the truth and avoid making mistakes.

So in our example let's say that you look at each theory and you examine what you did each Friday for the last two months. For the food poisoning theory you recall where you ate and what you ate on each Friday, looking for a pattern. Perhaps on two of those Fridays you ate dinner at the same restaurant. Those facts support that theory. But on the third Friday you ate dinner at home, which refutes the theory. However, it is possible that both the restaurant and your home cooked meal made you sick. Then you remember having leftovers from that home cooked meal and that you ate the leftovers without getting sick. This fact takes away support for the restaurant being the source of possible food poisoning. So you keep looking for a single food which you ate on each of the three occasions and at no other time and find none. Thus you assume that either you were hit with food poisoning on three different occasions from different sources, which is unlikely but possible, or it is something else. That trail has gone rather cold.

You do the same analysis with the theory of drinking too much and you find that you drank heavily on all those nights, plus on almost every other Friday night and a few Saturdays as well. You look closer to see if there is a pattern and you realize that on those Fridays in question you went to more than one place to drink and had different drinks at each place. Thus, there is a correlation between mixing drinks on a given night and being sick the following morning. Though you are fairly sure that the drinking theory is correct, you still think about the sleep/wake cycle theory. So you remember having a day off where you didn't drink the night before and slept in late and suffered no ill effects. So that theory too, seems doubtful.

Now you have a theory which is supported by the known facts much more so than your other theories. But it is still too early to give up on the other theories. Don't rule out anything yet.

4. The theory is tested via experiments

Putting your theories to the test is critically important. A theory is effectively a predictive tool. It is a piece of knowledge that allows you to know what will happen in the future given what you know in the present or the past. If you've ever heard the phrase, "you've got it down to a science", it means that you understand something so well that you know exactly what is going to happen before it happens. It also means that you can effectively control a thing, making it happen or not happen at will. Testing a theory is a first step toward having such knowledge. It is a demonstration of the validity of your theory.

Testing involves isolating the variables. If you have three variables, which have yielded three possible theories, you want to be able to rule out the theories that are not correct. So the way to do that is make experiments where only one of the variables is present. You also want to run a "control" experiments where none of the variables are present. Ideally, the control experiments and the experiments which test the "wrong" theories will yield similar nominal results, while the experiment which tests the "right" theory will show a strong correlation with the expected result. Of course, the actual case may be that only a combination of variables produces the result and thus all the experiments will come up with no correlation. In that case you will need to do follow-up experiments with multiple variables. Using the scientific method to discover complicated relationships is not easy, but it allows you to simplify things and methodically rule out theories until you finally discover the truth.

In our example let's say that you pose a few experiments, isolating one variable at a time: eating dinner at your usual Friday night restaurant, ordering the same dish as you had the other two nights when you got sick; drinking heavily and mixing drinks; and sleeping in late. On the first Friday you do none of the three things and that is your control experiment. Then on each Friday after that you do only one of the three things listed above. Ideally you would want to do each experiment multiple times to eliminate random elements, such as having some unrelated thing make you sick which would throw off your study. Also, it would be ideal if you were to do "double blind" experiments, where neither the experimenter nor the subject, which in this case is you both times, knows which variable is being tested. In our example doing a double blind study is virtually impossible, since you obviously know whether you are drinking, eating, or sleeping in, but doing double blind experiments can be critically important when subjective criteria is involved, such as if you were only feeling "down" instead of sick. If you are partial to a particular theory and you are using a subjective standard to measure the results, you may rate how you feel differently knowing which experiment it is that you are evaluating. This can happen unconsciously, so it is best to try to rule it out altogether. Double blind studies are a good way to be truly objective.

But for the purposes of our example you have to deal with the pitfalls of having a small sample size (one experiment for each variable) and no blind evaluation. So you keep that in mind and run your experiments. As anticipated, the night you mixed drinks is the night where you woke up the next morning feeling sick. You have know confirmed that theory with data. Hooray! You are closer to the truth.

5. The theory and method are reviewed by others

In practice, scientists do this by submitting their studies to "peer-reviewed" journals. These studies are some of the driest reading that you'll ever come across, but the point is to present the experiments that were performed and the reasons for performing them in sufficient detail that there is a permanent record of them somewhere and that others can replicate the experiments exactly to see if they get the same results. Peer review is important since it allows others with a different perspective to examine the works of others and see if a mistake was made or if something was overlooked. People aren't perfect, but a lot of different people checking each other's work gets closer to perfection than one person working alone. Also, in the future a new theory may arise and it is very useful to be able to go back and examine old experiments, data, and results to see if the new theory fits the old data.

In our example, peer-review would involve talking to friends and others about your theories and findings. So you talk to your friends about their experiences with mixing drinks, eating at a particular restaurant, sleeping in, and how you came to your conclusion that mixing drinks makes you sick. You find out that a few of your friends have had similar experiences with getting sick after a night out and find that while all three things have made them sick, mixing drinks seems to do it far more frequently. But while talking with your friends they come up with a few other theories that could explain why you got sick and you also find that some of them can mix drinks with no problems. This suggests the need to go back and run a few more experiments to see if you can further isolate your variables. But hey, unless you are really curious, you probably won't bother doing more experiments unless you start getting sick again. So lets say that you enjoy mixing your drinks and that you don't want to stop doing it. You want to gain a deep understanding of the problem.

6. The experiments are replicated by others

This is like peer review only one step farther. Others will try to test the theory by replicating the experiment. In this way they can often find problems in the experimental method if the data does not agree with the initial study. This happens often enough that a single study is generally not considered at all conclusive. Furthermore, even if the method is proper, there will usually be a difference in the subjects, who may be very different from those tested in the original experiment and thus give different results. Finding differences in experiments helps to refine a theory and expand our understanding of things. If the theory is useful, many people will end up testing it either in experiments or in practice and further refinement will occur. Gradually, through usage, theories get refined into scientific knowledge.

In our example, let's say that you convince a few of your friends to replicate your experiments by doing the same thing that you did, isolating the variables. At the end of the month you find that no one got sick except when they mixed drinks and not everyone got sick when they did. You get your heads together and try to figure out if it matters which type of drinks, when mixed, make you get sick. So you run a few more experiments and soon you discover that Zima and Tequila are best kept separate. Eureka!

And that is the scientific method.

It may occur to you that you have never gone about trying to figure anything out so methodically. Indeed, most of us simply latch onto a theory that we like and use it until we decide that it doesn't work and then move on to another. That works to some degree, provided our problems aren't complicated, but it is often the case that our most serious problems are not simple at all, which is why they are serious problems. Without the ability to solve complex problems we tend to either live with them or take simplistic, heavy handed measures to make them go away. In our example a typical person may have chosen to either just live with being sick now and then or have sworn off drinking, partying, eating at that suspicious restaurant, and sleeping late completely. The latter reaction is overkill, but it would work since the person would no longer be drinking both Zima and Tequila. Of course, the benefits of understanding the problem would allow someone to keep their wild lifestyle without getting sick. Isn't science wonderful?

It is difficult to use the scientific method in practice when dealing with things that do not involve you personally since it is hard for you to gather the facts to fit your personal theories. The most obvious example is when you are reading or watching the news to try to become an informed voter. Facts about issues are scarce, painfully so. Most of what we read and hear are opinions which may be substantiated by as little as one fact and which have no experimental support whatsoever. Too often, opinions are based entirely on ideology rather than reality. It's frustrating. But if you know how to look for the truth it is easier to separate good advice from ideological ranting. Here are some good ideas:

1. Look for facts, proof, and evidence. Talk is cheap.

2. Look for examples of the current situation, as many as possible, and draw connections between them. Debunk false analogies and treat history as a case study - provided you get enough historical facts to understand what really happened rather than what a few people would like you to believe.

3. Look for "experiments" or places where an approach has been tried and try to find as many cases as possible. The more samples you have, the better your understanding will be.

4. Look for possible pitfalls with every opinion of what should work. One good exercise is to build up an example of how to make something work and then try to figure out ways that it could be subverted. Then figure out ways that such subversion could be prevented.

5. Argue both sides of an issue seriously and passionately. This helps you to see flaws in both sides.

Other things that can help you to find the truth are trying to be as impartial as you can, being methodical, not making up your mind quickly, and admitting it when you are wrong. Reality will usually show you when you are wrong, but sometimes you don't have the luxury of waiting until that happens.