Biol 101: Problem Solving Workshop #1

Summary

This workshop teaches non-scientists how to evaluate scientific claims using a checklist. It emphasizes the importance of determining if a claim is scientific, meets criteria of rationality, testability, and repeatability, and follows rigorous methods. Participants learn to assess evidence, including its collection, presentation, and whether it supports the claim. They also explore the difference between correlation and causation, and consider the credibility of the claimant and potential conflicts of interest. The workshop uses a case study on spa treatments to apply these concepts.

Takeaways

• 🔍 Evaluate scientific claims by using a checklist to consider various aspects of the inquiry.
• 🌐 Ensure the claim is focused on explaining something about the natural world and meets the criteria of being rational, testable, and repeatable.
• 🔬 Look for a logical flow in the explanation and a rigorous, methodical process in the scientific inquiry.
• 📊 Examine the type of evidence presented and its relevance, as well as the detail provided about how it was collected.
• 📈 Be critical of how evidence is presented, especially in graphical form, as it can skew conclusions.
• 🔢 Units are crucial for numerical evidence, as a number without units lacks meaning.
• ❓ Question whether conclusions logically follow from the evidence and if they are supported by multiple studies.
• 🔄 Be wary of overgeneralizations based on limited data or samples.
• ⚠️ Avoid confusing correlation with causation; just because two variables change together does not mean one causes the other.
• 👨‍🔬 Consider the expertise of the person making the claim and whether they have a conflict of interest or are promoting a conspiracy theory.
• 📚 Use a case study to apply these evaluation techniques, focusing on one aspect at a time rather than trying to address everything at once.

Q & A

What is the main focus of the workshop?

The main focus of the workshop is to teach non-scientists how to evaluate scientific claims by going through a checklist of considerations.

Why should one not attempt to answer all the questions on the checklist?

One should not attempt to answer all the questions on the checklist because it is not appropriate or necessary for every situation, and the checklist serves more as a source of ideas for evaluation rather than a comprehensive set of questions to be answered.

What are the three criteria that a claim must meet to be considered scientific?

A claim must be rational, testable, and repeatable to be considered scientific.

What types of scientific inquiry are mentioned in the script?

The types of scientific inquiry mentioned are observational, descriptive studies, hypothesis-driven experiments, and the development of models and theory.

Why is it important to understand the type of scientific inquiry when evaluating a claim?

Understanding the type of scientific inquiry is important because different types of inquiry use different rigorous methods, and knowing the type helps determine if the steps taken are appropriate for that type of inquiry.

What role does evidence play in evaluating scientific claims?

Evidence, which consists of the data collected during scientific inquiry, plays a critical role as it supports or refutes the claims being made.

Why is the presentation of evidence important?

The presentation of evidence is important because it can influence the conclusions drawn from the data, especially when graphical representations are used, which can skew interpretations if not appropriate.

What is the significance of units in numerical evidence?

Units are significant in numerical evidence because a number without units lacks context and meaning, affecting the interpretation and decisions based on that data.

How can one determine if a conclusion is valid based on the evidence presented?

One can determine if a conclusion is valid by checking if it logically follows from the evidence and does not make leaps that are not supported by the data.

What is the difference between correlation and causation as it relates to scientific claims?

Correlation refers to two variables changing in relation to each other, while causation implies one variable causes the change in the other. It's important not to confuse the two, as correlation does not necessarily imply causation.

Why is it important to consider the expertise of the person making the scientific claim?

The expertise of the person making the claim is important because an expert in the relevant field is more likely to make accurate and informed claims about a subject.

What are some red flags to look for when evaluating a scientific claim?

Red flags include conflicts of interest, economic motivations, and references to secrets, conspiracies, or special hidden information that require payment to access.

How should one approach the case study on spa treatment claims in the workshop?

One should approach the case study by reviewing each part, taking notes, and considering the questions and checklist items relevant to that part before moving on to the next.

Outlines

00:00

🔍 Evaluating Scientific Claims

The paragraph discusses a workshop aimed at teaching non-scientists how to evaluate scientific claims. It introduces a checklist to consider when assessing such claims, emphasizing that not all questions on the checklist need to be addressed for every situation. The focus is on determining whether a claim meets the criteria of science, including rationality, testability, and repeatability. The paragraph also highlights the importance of understanding the type of scientific inquiry (observational, descriptive, or hypothesis-driven) and whether the methods used were rigorous and appropriate. The workshop encourages participants to think critically about the evidence presented, the collection process, and the logical flow of the claims.

05:01

📊 Analyzing Evidence and Inferences

This section delves into the critical analysis of evidence and the inferences drawn from it. It stresses the need to evaluate the type and quality of evidence, such as its sufficiency and the rigor of its collection. The paragraph warns against hastily accepting conclusions that seem to skip logical steps or lack a clear connection to the evidence. It also addresses the importance of considering how evidence is presented, including the appropriateness of graphical representations and the clarity of data presentation. The summary points out the potential for overgeneralization and the common mistake of confusing correlation with causation, using historical examples to illustrate the pitfalls of these errors in reasoning.

10:01

🕵️‍♂️ Considering Expertise and Potential Biases

The final paragraph of the script focuses on additional factors to consider when evaluating scientific claims, such as the expertise of the claimant and potential conflicts of interest. It advises participants to be wary of claims made by individuals outside their field of expertise or those with ulterior motives, like economic gain. The paragraph also cautions against claims that rely on secrecy or conspiracy theories, suggesting that such elements are red flags for potentially dubious claims. The workshop concludes with instructions for participants to apply these considerations to a case study on spa treatments, encouraging a thoughtful and critical approach rather than a checklist式的 response to every aspect of the study.

Keywords

💡Scientific Claims

Scientific claims refer to statements or assertions made about the natural world that are based on empirical evidence and scientific reasoning. In the context of the video, evaluating scientific claims involves assessing whether they meet the criteria of scientific inquiry, such as being rational, testable, and repeatable. The video emphasizes the importance of scrutinizing these claims to ensure they are based on sound scientific methods and evidence.

💡Rational

Rational in this context means that a scientific claim should have a logical flow and not invoke supernatural entities. The video script discusses the need for a claim to be rational, suggesting that it should be based on logical reasoning and not on mysterious or unexplained steps. This is crucial for the claim to be considered scientifically valid.

💡Testable

A testable claim is one that can be subjected to empirical observation or experimentation to verify its accuracy. The video script mentions that scientific claims should be testable, meaning they should be able to withstand attempts to prove them wrong through rigorous methods, which is a hallmark of the scientific method.

💡Repeatable

Repeatability in science means that an experiment or observation can be conducted again by others to obtain the same results, ensuring the validity of the findings. The video script highlights the importance of repeatability as a criterion for scientific claims, emphasizing the need for a rigorous method that allows for replication by other researchers.

💡Scientific Inquiry

Scientific inquiry encompasses the various methods and processes used to investigate and understand the natural world. The video script discusses different types of scientific inquiry, such as observational studies, hypothesis-driven experiments, and the development of models and theories, all of which are valid approaches to gathering knowledge about the world.

💡Evidence

In the context of the video, evidence refers to the factual data collected during scientific inquiries. The script emphasizes the importance of evaluating the type, amount, and detail of evidence presented to support a scientific claim. It also cautions against drawing conclusions from insufficient or poorly collected evidence.

💡Inferences

Inferences are the conclusions or interpretations drawn from the evidence collected during scientific research. The video script advises viewers to critically evaluate the inferences made from evidence, ensuring that they logically follow from the data and do not overgeneralize or confuse correlation with causation.

💡Correlation vs. Causation

The video script distinguishes between correlation, which is a statistical relationship between two variables, and causation, which implies that one variable causes a change in another. It warns against the common mistake of assuming causation from mere correlation, using the example of automobile tire sales and birth rates in post-WWII Japan to illustrate the point.

💡Expertise

Expertise in the video refers to the qualifications and knowledge of the person making the scientific claim. The script suggests that the credibility of a claim can be assessed by considering whether the claimant is an expert in the relevant field, which is important for the claim's validity and reliability.

💡Conflicts of Interest

Conflicts of interest arise when personal or financial interests might compromise the objectivity of a scientific claim. The video script advises viewers to be wary of claims that may be influenced by the claimant's economic interests or other biases, as these can undermine the integrity of the scientific process.

💡Case Study

A case study, as mentioned in the video script, is a detailed analysis of a specific instance or situation, used to illustrate or explore a broader issue. In the video's context, a case study on a spa treatment is used to guide viewers through the process of evaluating scientific claims, allowing them to apply the concepts discussed in a practical scenario.

Highlights

Workshop focuses on evaluating scientific claims for non-scientists.

Participants are guided through a checklist for evaluating scientific claims.

The importance of determining if a claim meets the criteria of scientific inquiry is emphasized.

Claims should be rational, testable, and repeatable to be considered scientific.

The necessity of a logical flow in scientific explanations is discussed.

Participants are advised to consider the type of scientific inquiry and its methodology.

The presentation of evidence and its collection process is a critical aspect of evaluation.

The quantity and quality of evidence supporting a claim are important to assess.

The presentation of evidence, such as graphs and tables, should be appropriate and clear.

Units of measurement are crucial for understanding numerical evidence.

Inferences and conclusions should logically follow from the presented evidence.

Consistency with other studies strengthens the validity of a scientific claim.

Overgeneralization from limited evidence can lead to incorrect conclusions.

The difference between correlation and causation is a common pitfall in scientific claims.

Expertise in the relevant field is important when evaluating the credibility of a claim.

Conflicts of interest or economic motivations can influence the objectivity of a claim.

References to secrets, conspiracies, or hidden information should raise skepticism.

The case study method will be used to apply the evaluation process to a spa treatment claim.

Participants are encouraged to take notes and reflect on each part of the case study before proceeding.

A single discussion board post summarizing thoughts on the case study is required.

Transcripts

so in this workshop we're going to talk

about how you as non-scientists can

evaluate scientific claims we're going

to be going through a checklist of

things that you can think about but as

you work through the case study that

we're going to be using for this

workshop don't try to answer all of

these questions it's not even an

appropriate thing to try to do

one of the things that this checklist is

useful for is to give you a lot of ideas

to think about and evaluate and some of

these types of items are going to be

more relevant than others to specific

situations

so one of the first things that you

always want to do whenever you're

evaluating scientific claims is decide

is this science does it meet the

criteria

of a scientific inquiry or of a

scientific claim so is it focused on

explaining something about the natural

world remember that's what science is

trying to do and does it meet those

three criteria of being rational

testable and repeatable so is it trying

to invoke any kind of supernatural

entity not going to fit that rational

criteria is there a logical flow to the

explanation that you've been given or

does it seem to kind of skip some steps

and it goes from step a and then there's

something mysterious then it suddenly

reaches a conclusion you're not quite

sure where that conclusion came from

well then it's missing that rational

criterion as well

is the investigation following a

rigorous methodical process remember

there's three main types of scientific

inquiry we have observational

descriptive studies we have

hypothesis-driven experiments we have

development of models in theory and

those are all valid types of scientific

inquiry but they all are going to have a

rigorous method that they are using

where individuals are documenting the

steps that they're taking so that other

people can come in and try

replicating those steps and seeing if

the two sets of evidence that they

collect are compatible with each other

so

have you been presented with how

particular pieces of evidence were

gathered what steps were taken

to

perform this particular type of

scientific inquiry and what type of

scientific inquiry was it and does it

seem like the steps are appropriate for

that type of inquiry

then we're going to look at two

things that are both critical to

anything that you've decided yet seems

on the surface to be a valid type of

scientific inquiry we then need to look

at the evidence that that inquiry has

gathered the facts that have been

collected and then finally we'll look at

the inferences that are made from those

evidence

so the evidence that gets gathered is

facts this is the data that is collected

in the process of performing a

scientific inquiry and sometimes when

you're presented with a scientific claim

you're told what evidence supports that

claim and sometimes pieces of that

evidence might be missing so you want to

look at what type of evidence are you

given to support this claim is there

enough detail that you're given about

how it was collected or was an adequate

amount of evidence collected or did they

just go and talk to like one person and

get their opinion and that was their

entire bit of evidence

not going to be so

rigorous a claim in that case or are

there lots and lots and lots of

different types of evidence collected

from different sorts of sources and if

they all support a particular claim that

is going to strengthen the evidence for

that claim

and then you want to think about how

evidence is presented go back and look

at the topic lecture where it showed you

how

the way data is presented in graphical

form for example can very much skew the

conclusions that you're going to draw

even if it's relying on the same set of

data so you want to look at if a graph

is used is it appropriate for the type

of data a line graph for example isn't

appropriate if you're looking at

categorical data but if you're looking

at something where they've collected

like daily temperatures over a period of

time then a line graph might make some

more sense

if a table is used is it understandable

are you given units and clear labels

units are super super important because

a number without units doesn't really

have any meaning think about if someone

were to tell you you've won 10.

okay

i've won 10 is that 10

10 000 or 10 million dollars because the

decisions that i am going to make are

going to vary widely depending on

whether we're looking at 10 10 000 or 10

million dollars

so units are always a critical aspect of

any type of evidence that you are

presented that is numerical

and if you're looking at any kind of

graphic are the scale on those graphs

appropriate and are they labeled so

those are all the kind of things that

you might ask about the evidence but you

aren't necessarily going to ask every

single one of these questions you want

to think about what information you're

being given and what are the most

relevant questions for that and then in

contrast you want to then look at the

inferences the conclusions that are

being presented based on the evidence

that is available

and first thing you always want to ask

yourself is does it seem like that

conclusion

makes sense given the evidence you don't

want to have oh the daily temperature

has varied between

65 degrees fahrenheit and 90 degrees

fahrenheit so i think you should invest

a thousand dollars in this particular

stock

that is okay there's some evidence that

you've given me what the daily

temperatures were but where the heck did

that conclusion about why i should

invest money in this particular stock

come from that doesn't derive from that

set of evidence that you've been

presented

and then how well

do the particular claims from

one scientific

inquiry fit with other studies that are

on the same topic if you have lots of

studies done by different groups and

different people and they all are

kind of consistent with each other

that's going to be a stronger conclusion

than if there's one kind of random group

that has

a conclusion that's very very different

from what all the other people studying

that same issue have come up with

do we have inferences that have over

generalized so

if for example i look at the high

temperatures in orange county over the

past week i might come up with very very

different conclusions if i'm looking at

those numbers in january versus in june

versus in late august very very

different conclusions if i'm then saying

and this is what the

climate is like in southern california

well if it's one of those cold weeks in

january you're going to have a really

different conclusion about southern

california climate than if it's one of

those hundred degree weeks that we get

in the summer we're all buying

really really vastly different

conclusions just based on

over generalizing from a small sample

and that is a very easy example to think

of but that can often happen with

scientific claims that get presented to

you

and then finally we want to think about

is that inference confusing correlation

with causation and correlation is

two variables two things are

changing at the same time in ways that

seem to be related so maybe they're both

going up or they're both going down or

when one goes up the other one goes down

but there seems to be a pattern

and oftentimes when you have a

correlation like that people assume that

that means one of those things is

causing the other one to also move

and that rarely is the case

correlation

doesn't necessarily mean that those two

things are

related in a causal relationship where

one thing is causing the other thing to

go up or to go down and there's a very

classic case of this after world war ii

comparing

automobile tire sales with

number of births that were recorded in

japan

and they both tended to go up over a

consistent period of time in japan after

world war ii

and if you were going to

confuse correlation with causation here

you might say purchasing automobile

tires causes you to have a baby and all

of us can think about that and go

obviously not purchasing automobile

tires does not cause you to have a baby

and what it really was is that both of

those two factors were tied to a third

factor which is the war had ended the

economy was booming people were very

happy that the war had ended and we're

now getting to be back together so there

were many babies that were born that is

the source of all of those boomers that

we all have to deal with

and they were also

having a whole lot of economic growth

and so many people were able to purchase

automobiles and you need to have tires

in order to have an automobile so the

booming post-war economy was what was

leading to both births and purchase of

automobile tires going up and not

purchasing automobile tires causes you

to have a baby

so that's one thing to think about

correlation versus causation oftentimes

when you're looking at two things and

they're correlated

but they don't appear to be causing each

other there's some third thing that is

causing them both to change in whatever

way they are changing

and then there's some random things that

kind of collect together you can

evaluate whether something is scientific

you can evaluate the evidence you can

evaluate the inferences and then there's

this kind of random collection of stuff

is the person who is making the claims

an expert in the appropriate field so

is an expert in disease epidemiology for

example making claims about covid

transmission or is it somebody who was

an expert in say

astrophysics or

ophthalmology or something else that's

not related to disease transmission of

viral diseases

think about the expert and whether they

are an expert in the correct kind of a

field to make those claims that they are

making

are there any kind of conflicts or

interests or is there especially any

economic interest is someone trying to

get you to spend money

and that's why they're making a

particular claim and then finally

is there any reference to secrets or

conspiracies or special hidden

information especially something where

if you just give me some money then i

will be showing you that special secret

information that should be sending up

not just a red flag but all the red

flags that this claim might be a little

suspect

and so all of these things are the kind

of questions that you might ask yourself

again don't try to ask every single one

of these questions for every single

claim but that's giving you some ideas

about the kind of things you want to be

thinking about as you get presented with

different types of claims

and what you're going to do in this

workshop is use a case study and the

case study files are presented to you on

canvas in four parts

and it's a case study on a spa treatment

and the claims that are made about what

this treatment does and how it works

it's broken into four parts and i really

recommend that you do each part and take

notes on that part before moving on to

the next part

you're going to be creating a

single discussion board post for this

that's going to be about a paragraph so

again you're not going to be able to go

through every item on the checklist that

we just reviewed

and each part of the case study has some

kind of thought discussion questions for

you to think about

but you're not going to try to answer

all of those

the checklist that we've just gone

through and the questions that you'll

see on the case study are to kind of

help get you thinking about what you'd

like to write about that particular part

so don't worry about trying to answer

everything don't worry about trying to

go through the entire

set of checklist questions because that

would be pages and pages of material and

i want you to have a single fairly short

paragraph

and so

i now recommend that you either listen

to this recording again or that you go

on to canvas and pull down that first

case study file and complete that and

the instructions on canvas will give you

a little more guidance