‘Science Says’ or ‘Study Reveals’ Does Not Automatically Mean Something Is True

In the news recently, there was an article floating around about a study conducted at UNC Chapel Hill. In this study, people were asked to define what they believed God looked like by comparing about 300 pairs of images the researchers presented them with. At the end of this study, a sketch was drawn up of “what Americans believe God looks like”. The final sketch was this:

 

J. Jackson et al., PLOS ONE 10.1371 (2018)

 

While I have to admit that this does strike some interest with me, it unsettles me even more. First of all, the bias of making people choose between pairs of images rather than having them provide specific details makes me think these researchers already had a sketch in mind and simply modified it or chose something based on other faces which appeared to look closest to their final options.

Secondly, and most importantly, these researchers only survey 511 Americans.

I’ll let you sit on that before I tell you why it matters.

Did you figure it out?

Well, there are about 326 million people in the United States. If you divide 511 into 326 million, you get the percentage, which is about .00015% of people. I spent about 6 years in college and I remember, even in high school, learning that sample size is everything. If you want to conduct a study or run a test, you have to consult the laws of statistics. Sample sizes determine your level of confidence–the accuracy of whatever you’re about to test.

I used a calculator below to determine what sample size would have given a confidence level of about 95%, using a confidence interval of 0.1, and entering the population of the United States, and it said I needed to survey at least 95,579 to get an accurate representation of whatever I was testing.

https://www.surveysystem.com/sscalc.htm

Therefore, I’m skeptical and horrified when ‘studies’ like this are posted online. They mean nothing to me until I see sample size. Sometimes you can do the research and find sample size used. I do this for things that matter, such as the flu shot statistics or with pap smear recommendations. When I refused the pap smear screening just recently after my son was born, the nurse tried to use fancy words like ‘the ACOG’ (which simply refers to the American College of Gynecologists) and ‘cancer screening’, but I was not scared or fooled into making a decision that went against what I wanted.

The American College of Gynecologists has changed its opinion multiple times over the years about how often a woman should get screened. They then relay their knowledge to the United States Preventative Task Force which tells doctors what they should be recommending. This uncomfortable test which most women either avoid or dread is not required. How did I determine that? Well, I don’t have a background which would indicate that cervical cancer is in my family and that I’m more prone to it, and I don’t have symptoms.

When I went to the United States Preventative Task Force website to check numbers, I found that the studies they’re using to force women into stirrups are mostly from other countries and the sample sizes vary greatly. Some studies used 4,000 women while larger studies used about 150,000 women. So how do they see what’s really happening? One example says that 4/1,000 women showed an abnormality or detected HPV. That’s only 0.4%. Some women say, ‘well if there’s even the smallest risk of having cancer, I would rather check’, and that’s fine, but if that were my belief, I would also be checking for mouth cancer and skin cancer and pancreatic cancer—all of which have higher percentages of probability than cervical cancer.

math

https://www.cancer.org/cancer/cervical-cancer/about/key-statistics.html

The American Cancer Society even says that this is most likely to affect women between 35-44, so why are there guidelines harassing girls as young as 18 and every year into their 20s about this dumb test?

I won’t delve more into this topic since I personally believe they just push these exams to collect insurance money when most of us are not affected, and the results are usually inaccurate (i.e. they could tell you you have an abnormality after a screen but a second screen would reveal that you are fine). Also, most women have HPV already so a simple urine test would be sufficient screening. There is no need to collect cells the way doctors want to do these days. But my point remains… SAMPLE SIZE IS KEY.

Do you know why the flu season was so disastrous this year? Because the flu shot only attacks a minimal amount of the many flu strains. These flu strains change every year and are constantly mutating. There are eve multiple categories. Our typically recommended vaccines don’t even cover all of the types. This year, and probably many other years, it was not effective for the strain which hit Americans which is why so many people fell ill and even died, despite having been vaccinated.

http://theconversation.com/flu-why-this-years-outbreak-is-one-of-the-worst-89802

So you see, you don’t have to be a scientist or engineer or even a mathematician to realize that numbers are important. When someone starts out by saying, ‘science says’ or a ‘study reveals’, know that anyone can do a study. I can survey 6 people in my office and ask them if they like donuts. If 3 of them say yes, I wouldn’t print an article that says studies show that 50% of lawyers like donuts, but someone might. They assume a reader will only read the results of a study without doing the math to determine how accurate it is, which is true of most Americans, but now you don’t have to be one of them.

This easy math will let you know what percentage of people actually believed something or participated in something. Confidence levels is another equation but you can always Google it if you’re curious. Otherwise, it’s best to just be careful next time the news or some other company tries to fool you or make you believe something that was based on a weak study.

Next time, we can talk about how things are correlated versus coincidental or ‘scientifically proven’, because I have a hard time believing that teaching your kids to swear is beneficial and am sure this study happened to find some successful kids and then asked them if their parents used swear words…

While Jay acknowledges that strong language and verbal abuse can lead to violence, he’s almost never seen it happen. “I’ve recorded over 10,000 people, I’ve never recorded, of all those thousands of people we’ve watched swear and listen to swear in public, we’ve never seen it turn into anything ugly,” he says. “My conclusion is that most uses of swear words are inconsequential or virtually innocuous.” -Digg.com

I’m sure Timothy Jay did not follow these thousands of people around the rest of their lives to see if using curse words was good or bad one way or the other, so it appalls me that people are taking him seriously, but to each his own, right?

All I know is numbers don’t lie, but they can deceive if manipulated the right way!