+1443 776-2705 panelessays@gmail.com

Experimental Design lecture part 1.html

· Find at least 2 scholarly articles that illustrate an experiment somehow related to criminal justice. Read these articles and provide a reference/link/doi number/something in your write up so I can find that exact article myself.

· In your write up, you should identify all the major components of a classical experimental design in EACH of the 2 articles you found.

· Finally, create your own experimental design on a CRJ topic of your choosing. What are your variables? How will you differentiate your groups? What is the timing of your data collection? What I’m expecting here is enough detail to be similar to a Methods section in an academic paper where I could recreate exactly what you describe by the level of detail you include. Explain how each decision/choice you make works towards the overall experimental design and can illustrate causality in the results you could ultimately collect if you were to carry out the proposal study.

· Look over the weblink provided for the GSS (General Social Survey) – an example of a big and very common national survey (some of which we’ve used the data from already this semester). Think about how the questions are worded and notice any changes over the years of data collection. Put yourself in the shoes of the survey takers and think about what might be impacting you if you were to take the survey. Is there always an appropriate answer option for every person’s possible response? Just play around a bit and think critically as you investigate.



Experimental Design lecture part 2.html

Chapter 6: Causation and Experimentation

1

What Do We Mean by Causation?

Correctional institutions and their classification.

Classification of inmates and their behavior while incarceration.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

2

Satisfies Learning Objective 6.1: List the three criteria for establishing a causal relationship and the two cautions that can improve understanding of a causal connection.

Correctional institutions and its classification: Correctional institutions classified inmates into different security levels.

Classification of inmates and their behavior while incarcerated: If you compared rates of misconduct across prison settings, you would not have the answer because the inmates may have been very different at their time of incarceration. As such, any differences you observe in misconduct could be attributable to these “before incarceration” differences, not to the type of facility in which they are housed.

2

Causal Explanations (1 of 3)

Cause: Explanation for an event.

Researchers and testing hypothesis.

Independent variable is the presumed cause.

Dependent variable is the potential effect.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

3

Satisfies Learning Objective 6.1: List the three criteria for establishing a causal relationship and the two cautions that can improve understanding of a causal connection.

Cause: It is an explanation for some characteristic, attitude, or behavior of groups, individuals, or other entities (such as families, organizations, or cities) or for events.

Researchers and testing hypothesis: Researchers seek causal explanations that reflect tests of the types of hypotheses.

Independent variable is the presumed cause.

Dependent variable is the potential effect.

3

Causal Explanations (2 of 3)

Quantitative (Nomothetic) Causal Explanation

Nomothetic causal explanation.

Causal effect: Variations in both variables.

Counterfactual: No variation in the independent variable.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

4

Satisfies Learning Objective 6.1: List the three criteria for establishing a causal relationship and the two cautions that can improve understanding of a causal connection.

Nomothetic causal explanation: A type of causal explanation involving the belief that variation in an independent variable will be followed by variation in the dependent variable, when all other things are equal (Ceteris paribus) .

Ceteris paribus: Latin term meaning “all other things being equal”.

Causal effect: Variations in both variables: When variation in one phenomenon, an independent variable, leads to or results, on average, in variation in another phenomenon, the dependent variable. Example of a nomothetic causal effect: Individuals arrested for domestic assault tend to commit fewer subsequent assaults than do similar individuals who are accused in the same circumstances but not arrested.

Counterfactual: No variation in the independent variable: The outcome that would have occurred if the subjects who were exposed to the treatment actually were not exposed but otherwise had had identical experiences to those they underwent during the experiment.

4

Causal Explanations (3 of 3)

Qualitative (Idiographic) Causal Explanation

Concerned with context, deterministic.

Causal effect: Series of actions resulting in individual outcome.

Example of an idiographic causal effect.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

5

Satisfies Learning Objective 6.1: List the three criteria for establishing a causal relationship and the two cautions that can improve understanding of a causal connection.

Satisfies Learning Objective 6.4: Name two challenges to using experimental designs and two difficulties with identifying idiographic causal explanations

Concerned with context, deterministic: Idiographic explanation is deterministic, focusing on what caused a particular event to occur or what caused a particular case to change.

Idiographic causal explanation: An explanation that identifies the concrete, individual sequence of events, thoughts, or actions that resulted in a particular outcome for a particular individual or that led to a particular event; may be termed an individualist or historicist explanation.

Context: A focus of causal explanation; a particular outcome is understood as part of a larger set of interrelated circumstances.

Causal effect: Series of actions resulting in individual outcome: When a series of concrete events, thoughts, or actions result in a particular event or individual outcome, it is the causal effect from an idiographic perspective.

Example of an idiographic causal effect: An individual is neglected by his parents. He comes to distrust others, has trouble maintaining friendships, has trouble in school, and eventually gets addicted to heroin. To support his habit, he starts selling drugs and is ultimately arrested and convicted for drug trafficking.

5

Criteria for Nomothetic Causal Explanations

Criteria necessary for causality:

Empirical association.

Appropriate time order.

Nonspuriousness.

Conditions important for causality:

Mechanism.

Context.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

6

Satisfies Learning Objective 6.1: List the three criteria for establishing a causal relationship and the two cautions that can improve understanding of a causal connection.

Criteria necessary for causality: The first three criteria that are generally considered as necessary and most important basis for identifying a nomothetic causal effect are Empirical association, appropriate time order, and nonspuriousness.

Conditions important for causality: Identifying a causal mechanism and specifying the context in which the effect occurs are the two criteria that considerably strengthen the causal explanations, but are not considered as requirements for establishing a causal relationship.

6

Case Study: Media Violence and Violent Behavior (1 of 3)

Bushman’s experiment on media violence: Criteria for establishing causal relationships.

Research question: Aggression on viewing a violent videotape.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

7

Satisfies Learning Objective 6.1: List the three criteria for establishing a causal relationship and the two cautions that can improve understanding of a causal connection.

Bushman’s experiment on media violence: Criteria for establishing causal relationships: Brad Bushman’s experiments on media violence and aggression illustrate the five criteria for establishing causal relationships. Half of the participants watched a movie excerpt that was violent, and half watched a nonviolent movie excerpt. Then they were asked to compete with another participant, in a different room, on a reaction-time task.

Research question: Aggression on viewing a violent videotape: Do individuals who view a violent videotape act more aggressively than individuals who view a nonviolent videotape?

7

Case Study: Media Violence and Violent Behavior (2 of 3)

Association

Criterion for establishing a causal relationship.

Bushman experiment: Exposure to media violence and aggressive behavior.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

8

Satisfies Learning Objective 6.1: List the three criteria for establishing a causal relationship and the two cautions that can improve understanding of a causal connection.

A criterion for establishing a causal relationship: Association is a criterion for establishing a causal relationship between two variables; variation in one variable is related to variation in another variable as a condition to determine causality.

Bushman experiment: Exposure to media violence and aggressive behavior: Students who watched a violent videotape in his lab administered more intense noise to an opponent than those who watched a nonviolent videotape.

8

Case Study: Media Violence and Violent Behavior (3 of 3)

Time Order and experiment criterion.

Nonspuriousness.

Correlation does not prove causation.

Spurious relationship.

Mechanism: Cause of aggression.

Context and contextual effect.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

9

Satisfies Learning Objective 6.1: List the three criteria for establishing a causal relationship and the two cautions that can improve understanding of a causal connection.

Satisfies Learning Objective 6.2: Contrast the strengths and weaknesses of dealing with nonspuriousness through statistical control and through randomization.

Satisfies Learning Objective 6.3: Explain the meaning of the expression “correlation does not prove causation.”

Time order and experiment criterion: A criterion for establishing a causal relation between two variables; the variation in the independent variable must come before variation in the dependent variable. Criterion met by Bushman’s experiment: All the students saw the movie excerpts (which varied in violent content) before their level of aggressiveness was measured.

Nonspuriousness: A relationship that exists between two variables that is not due to variation in a third variable.

Spurious relationship: A relationship between two variables that is due to variation in a third variable.

Mechanism: Cause of aggression: A mechanism is a discernible process that creates a causal connection between two variables. People predisposed to aggression are more susceptible to the effects of violent media because they possess a relatively large network of aggressive associations that can be activated by violent cues.

Context and Contextual effect: Relationships among variables differ across geographic units such as counties or across other social settings, or even between different types of individuals.

9

Why Experiment? (1 of 3)

True Experiments

Experimental and control groups.

Random assignment of groups.

Posttest and pretest of dependent variable.

Matching: Individual and aggregate basis.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

10

Satisfies Learning Objective 6.4: Name two challenges to using experimental designs and two difficulties with identifying idiographic causal explanations.

Research design and causal conclusions: Research design influences ability to draw causal conclusions.

True experiment: Experiment in which subjects are assigned randomly to an experimental group that receives a treatment or other manipulation of the independent variable and a comparison group that does not receive the treatment or receives some other manipulation; outcomes are measured in a posttest.

Experimental group: In an experiment, the group of subjects that receives the treatment or experimental manipulation.

Control or comparison group: The group of subjects who are either exposed to a different treatment than the experimental group or who receive no treatment at all.

Random assignment (randomization): A procedure by which experimental and control group subjects are placed in groups randomly.

Posttest: Measurement of an outcome (dependent) variable after an experimental intervention or after a presumed independent variable has changed for some other reason. Pretest: Measurement of an outcome (dependent) variable prior to an experimental intervention or change in a presumed independent variable.

Matching: Individual or aggregate basis: A procedure for equating the characteristics of individuals in different comparison groups in an experiment. Matching can be done on either an individual or an aggregate basis.

10

Why Experiment? (2 of 3)

An Experiment in Action: Prison Classification and Inmate Behavior

Bench and Allen study: Prison classification affecting inmate behavior.

Result: Both groups received about the same number of disciplinary infractions.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

11

Satisfies Learning Objective 6.4: Name two challenges to using experimental designs and two difficulties with identifying idiographic causal explanations.

Bench and Allen study: Prison classification affecting inmate behavior: The hypothesis stated that the experimental group , those prisoners who were reclassified as medium security, would have a lower number of disciplinary infractions compared with the control group, the inmates who retained their maximum-security classification.

Results: Indicated that inmates reclassified to medium security did not receive a lower number of infractions; both groups received about the same number of disciplinary infractions, regardless of security classification.

11

Why Experiment? (3 of 3)

Field Experiments in Action: Determining the Effect of Incarceration on Employment

Field experiments: Conducted in real world.

Field experiment by Pager.

Effects of incarceration on the likelihood of obtaining employment.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

12

Satisfies Learning Objective 6.4: Name two challenges to using experimental designs and two difficulties with identifying idiographic causal explanations.

Field experiments: Conducted in real world: It is an experimental study conducted in a real-world setting.

Field experiment by Pager: Two pairs of applicants, one who had a criminal record and one who did not, applied for real jobs. One tester within each team was randomly assigned to have a criminal record and the other was not. The results indicated that the white testers with a criminal record were one-half to one-third less likely to receive a callback from employers, and the effect was even more pronounced for African American applicants.

12

What If a True Experiment Isn’t Possible? (1 of 7)

Quasi-experimental designs: Subjects are not randomly assigned.

Nonequivalent control group designs.

Before-and-after designs.

Ex post facto control group designs.

Comparison groups selected after treatment.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

13

Satisfies Learning Objective 6.5: Name and illustrate the three different quasi-experimental designs.

Quasi-experimental design: Subjects are not randomly assigned: A research design in which there is a comparison group that is comparable to the experimental group in critical ways, but subjects are not randomly assigned to the comparison and experimental groups. If testing a hypothesis with a true experimental design is not feasible due to various cost, time and ethical issues; Researchers may instead use quasi-experimental designs.

Nonequivalent control group designs: A quasi-experimental design in which there are experimental and comparison groups that are designated before the treatment occurs but are not created by random assignment.

Before-and-after designs: A quasi-experimental design may consist of before-and-after comparisons of the experimental group but has no control group.

Ex post facto control group designs: Nonexperimental design in which comparison groups are selected after the treatment, program, or other variation in the independent variable has occurred.

13

What If a True Experiment Isn’t Possible? (2 of 7)

Nonequivalent Control Group Designs

Comparison and treatment group should be comparable.

Selection methods:

Individual matching.

Aggregate matching.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

14

Satisfies Learning Objective 6.5: Name and illustrate the three different quasi-experimental designs.

Comparison and treatment group should be comparable: When random assignment is not possible, a nonequivalent control group design is often used. In this type of quasi-experimental design, a comparison group is selected to be as comparable as possible to the treatment group

Selection methods:

Individual matching: Individual cases in the treatment group are matched with similar individuals in the comparison group.

Aggregate matching: Involves identifying a comparison group that matches the treatment group in the aggregate rather than trying to match individual cases.

14

What If a True Experiment Isn’t Possible? (3 of 7)

Case Study: Parole Community Resource Centers and Recidivism

CRCs: Allow paroles live at their residences.

Hyatt and Ostermann’s nonequivalent control group experiment.

Determined effectiveness of CRCs.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

15

Satisfies Learning Objective 6.5: Name and illustrate the three different quasi-experimental designs.

Community resource centers (CRCs): These centers allow paroles to live at their own residences, but they are mandated to attend programs such as as GED and/or high school classes, employment preparation services, and life skills training.

Hyatt and Ostermann’s nonequivalent control group experiment:

Determined whether CRCs were more effective in reducing recidivism than traditional parole.

They matched individuals who had been under traditional parole to those who had been under CRC programs using such variables as age, gender, race/ethnicity, prior arrests, and so on.

The results indicated that CRC participants had higher rates of recidivism than those under traditional supervision.

15

What If a True Experiment Isn’t Possible? (4 of 7)

Before-and-After Designs

Do not have a comparison group.

Fixed-sample panel design.

Time series design.

Five criteria for establishing causality.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

16

Satisfies Learning Objective 6.5: Name and illustrate the three different quasi-experimental designs.

Do not have a comparison group: These type of designs do not have comparison group; all cases are exposed to the experimental treatment. The basis for comparison is instead provided by the pretreatment measures in the experimental group. These designs are thus useful for studies of interventions that are experienced by virtually every case in some population.

Fixed-sample panel design (panel study): A type of longitudinal study in which data are collected from the same individuals—the panel—at two or more points in time; in another type of panel design, panel members who leave are replaced with new members.

Time series design (repeated measures panel design): A quasi-experimental design consisting of many pretest and posttest observations of the same group.

Five criteria for establishing causality:

The before–after comparison enables us to determine whether an association exists between the intervention and the dependent variable (because we can determine whether there was a change after the intervention).

They also clarify whether the change in the dependent variable occurred after the intervention, so time order is not a problem. However, there is no control group, so we cannot rule out the influence of extraneous factors as the actual cause of the change we observe; spuriousness may be a problem.

16

What If a True Experiment Isn’t Possible? (5 of 7)

Case Study: The Effects of the Youth Criminal Justice Act

Carrington and Schulenberg’s study.

Effect of YCJ Act on police discretion.

Used time series design.

YCJA achieved intended outcome.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

17

Satisfies Learning Objective 6.5: Name and illustrate the three different quasi-experimental designs.

Effect of the YCJ Act on police discretion: Effect of the YCJ Act in Canada on police discretion with apprehended young offenders

Used time series design: This design typically includes many pretest and posttest observations that allow the researcher to study the process by which an intervention or a treatment has an impact over time. It is particularly useful for studying the impact of new laws or social programs that affect everyone and can be readily assessed by ongoing measurement.

17

What If a True Experiment Isn’t Possible? (6 of 7)

Ex Post Facto Control Group Designs

Not to be confused with quasi-experimental designs.

Disadvantage: Difficult to determine if association is spurious.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

18

Satisfies Learning Objective 6.5: Name and illustrate the three different quasi-experimental designs.

Ex post facto control group designs: It appears to be very similar to the nonequivalent control group design. However, the groups in ex post facto designs are designated after the treatment has occurred.

Disadvantage: The problem with this design is that if the treatment takes any time at all, people with particular characteristics may select themselves for the treatment or avoid it. This makes it difficult to determine whether an association between group membership and outcome is spurious.

18

What If a True Experiment Isn’t Possible? (7 of 7)

Case Study: Does an Arrest Increase Delinquency?

David P. Farrington’s study.

Deviance amplification process.

Hypothesis: Juveniles labeled as deviant, committed more delinquent acts.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

19

Satisfies Learning Objective 6.5: Name and illustrate the three different quasi-experimental designs.

David P. Farrington’s study:

Deviance amplification process: This is a classic study of how arrest sometimes increases delinquency, called the deviance amplification process, using ex post facto control group design.

Hypothesis: Juveniles labeled as deviant, committed more delinquent acts: It tested the hypothesis that juveniles who were publicly labeled as deviant through being convicted of a delinquent act would increase their deviant behavior compared with those who were not so labeled. The results indicated that youth who were labeled as delinquent (through conviction) subsequently committed more delinquent acts than similar youth who were not labeled in this way.

19

What Are the Threats to Internal Validity and Generalizability in Experiments (1 of 4)

True experiments: Good internal validity.

Quasi-experiments: Problems of internal invalidity.

Nonexperimental designs: Less certainty about internal validity.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

20

Satisfies Learning Objective 6.4: Name two challenges to using experimental designs and two difficulties with identifying idiographic causal explanations.

True experiments: Good internal validity: True experiments are particularly well suited to producing valid conclusions about causality (internal validity), but they are less likely to fare well in achieving generalizability.

Quasi-experiments: Problems of internal invalidity: Quasi-experiments may provide more generalizable results than true experiments, but they are more prone to problems of internal invalidity (although some design schemes allow the researcher to rule out almost as many potential sources of internal invalidity as does a true experiment).

Nonexperimental designs: Less certainty about internal validity: In general, nonexperimental designs (such as survey research and participant observation) permit less certainty about internal validity

20

What Are the Threats to Internal Validity and Generalizability in Experiments (2 of 4)

Causal (Internal) Validity

Selection bias.

Endogenous change.

External events/history effects.

Contamination.

Treatment misidentification.

21

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

Satisfies Learning Objective 6.4: Name two challenges to using experimental designs and two difficulties with identifying idiographic causal explanations.

Selection bias: A source of internal (causal) invalidity that occurs when characteristics of experimental and comparison group subjects differ in any way that influences the outcome. Differential attrition: A problem that occurs in experiments when comparison groups become different because subjects are more likely to drop out of one of the groups for various reasons.

Endogenous change: A source of causal invalidity that occurs when natural developments or changes in the subjects (independent of the experimental treatment itself) account for some or all of the observed change from the pretest to the posttest. It includes following three threats to internal validity.

Testing: Taking the pretest can in itself influence posttest scores.

Maturation: Changes in outcome scores during experiments that involve a lengthy treatment period may be due to maturation

Regression effect: A source of causal invalidity that occurs when subjects who are chosen for a study because of their extreme scores on the dependent variable become less extreme on the posttest due to natural cyclical or episodic change in the variable.

External events (history effect): A source of causal invalidity that occurs when something other than the treatment influences outcome scores; also called an effect of external events.

Contamination: A source of causal invalidity that occurs when the experimental and/or the comparison group is aware of the other group and is influenced in the posttest as a result.

Compensatory rivalry (John Henry effect): A type of contamination in experimental and quasi-experimental designs that occurs when control group members are aware that they are being denied some advantage and increase their efforts by way of compensation.

Treatment misidentification: A problem that occurs in an experiment when the treatment itself is not what causes the outcome, but rather the outcome is caused by some intervening process that the researcher has not identified and is not aware of. The Sources of treatment misidentification

Expectancies of the experimental staff (self-fulfilling prophecy): A source of treatment misidentification in experiments and quasi-experiments that occurs when change among experimental subjects is due to the positive expectancies of the staff who are delivering the treatment rather than to the treatment itself.

Double-blind procedure: An experimental method in which neither subjects nor the staff delivering experimental treatments know which subjects are getting the treatment and which are receiving a placebo.

Placebo effect: A source of treatment misidentification that can occur when subjects receive a treatment that they consider likely to be beneficial and improve because of that expectation rather than because of the treatment itself.

Hawthorne effect: A type of contamination in experimental and quasi-experimental designs that occurs when members of the treatment group change in terms of the dependent variable because their participation in the study makes them feel special.

21

What Are the Threats to Internal Validity and Generalizability in Experiments (3 of 4)

Interaction of Testing and Treatment

Problem in achieving external validity.

Treatment is effective only when particular conditions exist .

Solomon four-group design: Random assignment to the groups.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

22

Satisfies Learning Objective 6.4: Name two challenges to using experimental designs and two difficulties with identifying idiographic causal explanations.

Problem in achieving external validity: Occurs when the experimental treatment is effective only when particular conditions created by the experiment occur.

Solomon four-group design: Random assignment to the groups:

An experimental design in which there are four groups. Two of the groups represent a classic experimental design in which there is an experimental and a control group that each receives a pretest.

The final two groups represent an experimental and a control group but neither receives a pretest. This design helps to identify the interaction of testing and treatment.

22

What Are the Threats to Internal Validity and Generalizability in Experiments (4 of 4)

Generalizability

Sample generalizability:

Conclusion based on subset of a larger population.

Difficult to achieve.

Determined by characteristics of subjects.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

23

Satisfies Learning Objective 6.4: Name two challenges to using experimental designs and two difficulties with identifying idiographic causal explanations.

Sample generalizability: Exists when a conclusion based on a sample, or subset, of a larger population holds true for that population.

Difficult to achieve in absence of representative sample: Subjects in laboratory experiment, randomly assigned to a group, and kept under carefully controlled conditions are often not a representative sample of any large population of interest.

Determined by characteristics of subjects: The generalizability of …

Chapter 7: Survey Research

1

Survey Research in Action: Measuring Victimization

Measuring prevalence of rape is difficult.

Random sample surveys.

Tool for uncovering incidents of violence.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

2

Satisfies Learning Objective 7.1: Identity the circumstances that make survey research an appropriate methodology.

Measuring the prevalence of rape is difficult: It is difficult to measure the magnitude of rape, stalking, and intimate partner violence (IPV) for reasons, including the historical stigma attached to these crimes, fear of retaliation from their perpetrators, and safety concerns. Estimating incidence rates of these crimes is also difficult because fewer than 50% of these offenses are ever reported to police.

Random sample surveys: Random sample surveys of the population are used as a social science tool for uncovering incidents of violence. However, surveys employing diverse methodologies and different definitions of violence result in tremendously diverse estimates.

2

What Is a Survey? (1 of 2)

Survey research.

Systematic data collection

Data is from a broad spectrum of individuals.

Attractive Features of Survey Research.

Versatility.

Efficiency.

Generalizability.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

3

Satisfies Learning Objective 7.1: Identity the circumstances that make survey research an appropriate methodology.

Survey research: Research in which information is obtained from a sample of individuals through their responses to questions about themselves or others.

Attractive Features of Survey Research:

Versatility:

Survey research allows asking questions on any topic.

It enhances understanding of social issues

Computers can be used to ask different types of respondents different questions, present short videos or pictures, and record answers to sensitive personal questions.

It is not ideal for testing hypotheses.

Efficiency:

Survey research is efficient because variables can be measured without increasing the time or cost of data collection.

Generalizability.

Sample generalizability: Survey research is very appealing when sample generalizability is a central research goal. It helps in developing a representative picture of the attitudes and characteristics of a large population.

Cross-population generalizability: Surveys are used when cross-population generalizability is a primary concern. They allow a range of social contexts and subgroups to be sampled, and the consistency of relationships can be examined across the various subgroups.

3

What Is a Survey? (2 of 2)

The Omnibus Survey

Covers a range of topics.

Advantage: Generates data for broad segment.

Disadvantage: Offers limited depth.

Example: General Social Survey (GSS).

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

4

Satisfies Learning Objective 7.1: Identity the circumstances that make survey research an appropriate methodology.

Omnibus survey: A survey that covers a range of topics of interest to different social scientists.

Advantage: Generates data useful to a broad segment of the social science community rather than answer one particular research question.

Disadvantage: The disadvantage of the omnibus approach is the limited depth that can be achieved in any one substantive area.

Example: General Social Survey (GSS): An example of omnibus surveys is the General Social Survey (GSS) of the National Opinion Research Center at the University of Chicago. It administers, every two years, a 90-minute interview to a probability sample of almost 3,000 Americans.

4

Questionnaire Development and Assessment (1 of 3)

Questionnaire: A self-administered survey.

Interview schedule requires a interviewer.

Maintain focus: Defined inquiry and targeted population.

Questionnaire design determines what to include and exclude.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

5

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Questionnaire: A self-administered survey: The survey instrument containing the questions for a self-administered survey.

Interview schedule: requires a interviewer: The survey instrument containing the questions asked by the interviewer for an in-person interview or phone survey.

Maintain focus: Defined inquiry and targeted population: A survey should constitute a well-defined inquiry and a definitively targeted population.

Questionnaire design determines what to include and exclude: While designing a questionnaire it is important to determine what to include and exclude and what to emphasize or treat with less importance. Also, each section and every question should serve a clear purpose related to the study’s objective as well as complement other sections and questions.

5

Questionnaire Development and Assessment (2 of 3)

Build on Existing Instruments.

Use proven survey instruments.

Consider Translation.

Foreign born may not speak in English.

Translating questions requires expertise

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

6

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Build on Existing Instruments: If evidence from previous surveys indicates that a survey instrument provides a good measure of the concept or behaviors, then it should be used.

Consider Translation: It is important to consider translating the survey into one or more languages.

Foreign born may not speak in English: Many foreign born nationals may not be fluent with English. They can only be included in a survey if it is translated into their native language.

Translating questions requires expertise: While translating questions one must ensure that the concepts that are being measured have equivalence in different cultures. This requires involving teams of experts both in the language and in the culture for which the questionnaire is intended.

6

Questionnaire Development and Assessment (3 of 3)

Case Study: Measuring Violent Victimizations

National Crime Victimization Survey.

Assess experience of victimization.

Questions are specific.

CDC-sponsored NISVS.

Estimates prevalence of sexual violence.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

7

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

National Crime Victimization Survey (NCVS):

Assess experience of victimization: To assess whether respondents have experienced a victimization, the U.S. Department of Justice–sponsored NCVS asks a number of questions about experiences within the past six months.

Questions are specific: The questions are specific and help respondents think about many different contexts, including those instances that may not be considered as victimizations.

CDC-sponsored NISVS:

Estimates prevalence of sexual violence: The CDC-sponsored NISVS too estimates prevalence rates of sexual violence but the questions from which their estimates are based are very different indeed from NCVS.

7

Writing Survey Questions: More Difficult Than You Think! (1 of 7)

Survey questions: Straightforward.

Open-ended questions.

Advantage: Wealth of information.

Disadvantage: Difficult to summarize.

Interpretive questions.

Closed-ended questions.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

8

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Satisfies Learning Objective 7.3:: Discuss the advantages and disadvantages of including “don’t know” and neutral responses among response choices and of using open-ended questions.

Survey questions: Straightforward:

In principle, survey questions can be a straightforward and efficient means of measuring individual characteristics, facts about events, levels of knowledge, and opinions of any sort.

In practice, survey questions, if misleading or unclear, can result in inappropriate and unintended answers.

Structurally, questions on surveys generally fall into two categories: those with and those without explicit response choices.

Open-ended questions: Survey questions to which the respondent replies in his or her own words, either by writing or by talking.

Advantage and disadvantage of open-ended question: Open-ended questions provide wealth of information. This wealth of information, however, is exactly why many researchers do not use them. The verbatim text narratives obtained from open-ended questions take a great deal of time and energy to organize and summarize.

Interpretive questions: Questions included in a questionnaire or interview schedule to help explain answers to other important questions.

Closed-ended (fixed choice) question : A question format in which respondents are provided with explicit responses from which to select.

8

Writing Survey Questions: More Difficult Than You Think! (2 of 7)

Constructing Clear and Meaningful Questions.

Survey questions: Must be same for every one.

Must be understood in the same way.

Cannot be rephrased.

Respondents do not know the surveyor.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

9

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Survey questions: Must be same for every one:

The same survey questions must be used with each person, not tailored to the specifics of a given conversation.

Survey questions must be understood in the same way by people who differ in many ways.

You will not be able to rephrase a survey question if someone does not understand it because that would result in asking the person a different question from the one you asked the others in your sample.

Respondents do not know the surveyor: Survey respondents do not know the surveyor so cannot be expected to share the nuances of expression that the surveyor uses to communicate with the people.

9

Writing Survey Questions: More Difficult Than You Think! (3 of 7)

Constructing Clear and Meaningful Questions.

Avoid confusing phrasing and vagueness.

Avoid negative words and double negatives.

Avoid double-barreled questions.

Avoid making either disagreement or agreement disagreeable.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

10

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Avoid confusing phrasing and vagueness:

Clear, precise, and simply phrased question: Questions that are clear, precise, and simply phrased are most likely to have the same meaning for different respondents.

Avoid wordier questions: Wordier and longer the question, the more likely you are to lose a respondent’s attention and focus.

Identify reference period: Without identifying a reference period, or time frame around which a question is being asked, a researcher will not know how to interpret an answer.

Reference period: A time frame in which a survey question asks respondents to place a particular behavior (e.g., in the last six months).

Avoid negative words and double negatives:

Double-negative question: A question or statement that contains two negatives, which can muddy the meaning.

Avoid double-barreled questions:

Double-barreled question: A single survey question that actually asks two questions but allows only one answer.

It can also show up in response category.

Avoid making either disagreement or agreement disagreeable:

To reduce agreement bias present both sides of attitude scales in the question itself.

10

Writing Survey Questions: More Difficult Than You Think! (4 of 7)

Additional Guidelines for Closed-Ended Questions

Make response choices mutually exclusive.

Make the response categories exhaustive.

Utilize Likert-type response categories.

Minimize fence-sitting and floating.

Utilize filter questions.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

11

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Make response choices mutually exclusive: Mutually exclusive responses are response choices on a survey that do not overlap. They are required when researchers want respondents to make only one choice.

Make the response categories exhaustive: Exhaustive responses are response choices on a survey that allow an option for respondents who require another choice. This can easily be created if respondents are provided with a choice labeled “ Other, specify please”.

Utilize Likert-type response categories: Likert-type responses are survey responses in which respondents indicate the extent to which they agree or disagree with statements.

Minimize fence-sitting and floating:

Fence-sitters: Survey respondents who see themselves as being neutral on an issue and choose a middle (neutral) response that is offered.

Floaters: Survey respondents who provide an opinion on a topic in response to a closed-ended question that does not include a “don’t know” option but will choose “don’t know” if it is available.

Utilize filter questions.

Filter question: A survey question used to identify a subset of respondents who then are asked other questions.

Skip patterns: The unique combination of questions created in a survey by filter questions and contingent questions.

Contingent questions: Questions that are asked to only a subset of survey respondents.

11

Writing Survey Questions: More Difficult Than You Think! (5 of 7)

Combining Questions Into an Index

Single questions: Idiosyncratic variation.

Devise an index of multiple questions.

Use reverse code.

Reliability measures determine response consistency.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

12

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Single questions: Idiosyncratic variation: Variation in responses to questions that is caused by individuals’ reactions to particular words or ideas in the question instead of by variation in the concept that the question is intended to measure.

Devise an index of multiple questions: When scoring an index or scale made up of both favorable and unfavorable statements, one must remember to reverse code the unfavorable items.

Index: A composite measure based on summing, averaging, or otherwise combining the responses to multiple questions that are intended to measure the same variable, sometimes called a scale.

Reverse code: Recoding response choices that were originally coded to reflect both favorable and unfavorable attitudes toward a phenomenon as indicative of either all favorable or all unfavorable so the index is measuring the same thing.

Reliability measures determine response consistency: Reliability measures are special statistics that help researchers decide whether responses are consistent.

12

Writing Survey Questions: More Difficult Than You Think! (6 of 7)

Combining Questions Into an Index

Demographic Questions: Important independent variables.

Intrusive questions invade privacy.

Information should remain confidential.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

13

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Demographic questions: Important independent variables: Almost all questionnaires include a section on demographic information such as sex, age, race or ethnicity, income, and religion. For many research studies, these questions are important independent variables.

Intrusive questions invade privacy: It makes the questionnaire more intrusive than necessary. In fact, many respondents feel that questions about their income invade their privacy.

Information should remain confidential: Because demographic questions are usually perceived as private by respondents, some researchers place them in a section at the end of the questionnaire with an introduction reassuring respondents that the information will remain confidential.

13

Writing Survey Questions: More Difficult Than You Think! (7 of 7)

Don’t Forget to Pretest!

Pretest the questionnaire: External feedback.

Trying out on a small sample of individuals.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

14

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Pretest the questionnaire: External feedback: Simply asking what appear to be clear questions does not ensure that people will have a consistent understanding of what you are asking. One needs some external feedback, and thus one needs to pretest the questionnaire.

Pretested: When a questionnaire is taken by a small subsample of respondents to uncover any problems with the questions or response categories.

Trying out on a small sample of individuals: One needs to try out the questionnaire on a small sample of individuals.

14

Organization Matters (1 of 3)

Descriptive title for the questionnaire.

Cover letter for a mailed questionnaire

Introductory statement for an interview.

Should be credible, personalized, interesting, and responsible.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

15

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Descriptive title for the questionnaire: A descriptive title for the questionnaire is required because it indicates the overall topic. The title is essential because it sets the context for the entire survey.

Cover letter for a mailed questionnaire: Cover letter is the letter sent with a mailed questionnaire; it explains the survey’s purpose and auspices and encourages the respondent to participate.

Introductory statement for an interview: An introductory statement should read by interviewers in telephone or in-person interviews. Introductory statements should be credible, personalized, interesting, and responsible. Doing so sets the tone for the entire questionnaire.

15

Organization Matters (2 of 3)

Question Order Matters!

Question order: Influences responses.

Leads to context effects.

Split-ballot design: Administered to randomly selected subsets.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

16

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Question order: The question order influences responses. The potential impact of question order on the respondents’ answers is greatest when two or more questions concern the same issue or closely related issues so that asking one question affects reactions to the next question. This is known as the context effect.

Context effects: Occur in a survey when one or more questions influence how subsequent questions are interpreted.

Split-ballot design: Unique questions or other modifications in a survey administered to randomly selected subsets of the total survey sample so that more questions can be included in the entire survey or so that responses to different question versions can be compared.

16

Organization Matters (3 of 3)

Question Order Matters!

Questionnaire organizational guidelines.

Organize topics section wise.

Provide instructions.

Make it look attractive.

Enable easy coding of responses and data-entry.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

17

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Questionnaire organizational guidelines.

Organize major topic section wise and introduce each section with a brief statement.

Provide instructions to minimize respondent confusion, explain how each type of question is to be answered, and to guide respondents through skip patterns.

The questionnaire should look attractive, be easy to complete, and have substantial white space.

Response choices should be designated by numbers to facilitate coding and data entry after the questionnaire is completed.

17

Survey Designs (1 of 9)

Five survey designs.

Vary due to differences in:

Manner of administration.

Questionnaire Structure.

Setting.

Cost.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

18

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Satisfies Learning Objective 7.4: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.

Five survey designs: The five basic survey designs are the mailed (self-administered) survey, group-administered survey, phone survey, in-person survey, and electronic or web survey.

Vary due to differences in:

Manner of administration: Surveys can be administered through mails or any electronic medium. They can also be conducted on telephone or in person.

Questionnaire structure: Survey designs differ in the extent to which the content and order of questions are structured in advance by the researcher. Most mailed, group, phone, and electronic surveys are highly structured but may include some open-ended questions. However, in-person interviews are often highly structured, but they may include many questions without fixed-response choices.

Setting: Most mail and electronic questionnaires and phone interviews as well as in-person interviews are intended for completion by only one respondent. , Although sometimes researchers interview several family members at once.

Cost: In-person interviews are the most expensive type of survey. Phone interviews are much less expensive, but surveying by mail is cheaper still. Electronic surveys are now the least expensive method.

18

Survey Designs (2 of 9)

Mailed (Self-Administered) Surveys

Involves a mailed questionnaire.

Disadvantages:

Difficulty maximizing the response rate.

Hazard of incomplete response.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

19

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Satisfies Learning Objective 7.4: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.

Involves a mailed questionnaire: Mailed (self-administered) survey involves a mailed questionnaire to be completed by the respondent.

Difficulty maximizing the response rate: The principal drawback in using this method of survey administration is the difficulty maximizing the response rate because the researchers have to rely on people to voluntarily return the surveys.

Hazard of incomplete response: Related to the threat of nonresponse in mailed surveys is the hazard of incomplete response. Some respondents may skip some questions or just stop answering questions at some point in the questionnaire.

19

Survey Designs (3 of 9)

Group-Administered Surveys

Completed by individuals in group.

Issue of concern: Respondents may feel coerced.

Addressed by making elaborate introductory statement.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

20

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Satisfies Learning Objective 7.4: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.

Completed by individuals in group: Group-administered survey is completed by individual respondents who are assembled in a group.

Issue of concern: Respondents may feel coerced: Respondents may feel coerced to participate and as a result will be less likely to answer questions honestly. Group surveys are likely to be conducted on the organization’s premises, respondents may infer that the researcher is not at all independent of the sponsor.

Addressing the concerns: Make an introductory statement that:

Emphasizes the researcher’s independence.

Assures respondents that their survey answers will be completely anonymous.

Gives participants a chance to ask questions about the survey.

20

Survey Designs (4 of 9)

Surveys by Telephone

Interview over the phone.

Disadvantages:

Not reaching the proper sampling units.

Not getting enough complete responses to make the results generalizable.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

21

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Satisfies Learning Objective 7.4: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.

Interview over the phone: In a phone survey interviewers question respondents over the phone and then record their answers.

Disadvantages:

Not reaching the proper sampling units.

Not getting enough complete responses to make the results generalizable.

21

Survey Designs (5 of 9)

Surveys by Telephone

Reaching sampling units.

Use Random Digit Dialing (RDD).

Maximizing response to phone surveys.

Computer-assisted telephone interview (CATI).

Computer interactive voice response (IVR).

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

22

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Satisfies Learning Objective 7.4: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.

Reaching sampling units: Using random digit dialing (RDD): Random digit dialing (RDD) helps in drawing a random sample. A machine calls random phone numbers within designated exchanges regardless of whether the numbers are published.

Maximizing response to phone surveys.

Computer-assisted telephone interview (CATI): A telephone interview in which a questionnaire is programmed into a computer, along with relevant skip patterns that must be followed. It essentially combines the tasks of interviewing, data entry, and some data cleaning.

Computer interactive voice response (IVR): Software that uses a touch-tone telephone to interact with people to acquire information from or enter data into a database.

22

Survey Designs (6 of 9)

In-Person Interviews

Interviewer questions and records answers.

CAPI and CASI.

Maximizing response to interviews.

Contact rates lower in city centers.

Contact rate higher in households.

Bachman, Fundamentals of Research in Criminology and Criminal Justice, Fifth edition. © SAGE Publications, 2021.

23

Satisfies Learning Objective 7.2: List the different methods for improving survey questions along with the mistakes you do not want to make when writing questions.

Satisfies Learning Objective 7.4: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.

In-person interview: A survey in which an interviewer questions respondents and records their answers.

Computer-assisted personal interviewing (CAPI): An interview in which the interviewer carries a laptop computer programmed to display the interview questions and processes the responses that the interviewer types in as well as checking that these responses fall within the allowed ranges.

Computer-assisted self-interviewing (CASI): A system within which respondents interact with a computer-administered questionnaire by using a mouse and following audio instructions delivered via headphones.

Maximizing response to interviews:

Contact rates are lower in city centers:

Difficult to finding people at home.

Gain access to high-rise apartments.

Reluctance of interviewer to visit some areas at night.

Contact rate higher in households: Households with young children or elderly adults tend …