|
 |
|||||||||||
 |
|
|||||||||||
|
Computer Assisted Interviewing for SAMHSA's National Household Survey on Drug Abuse |
||||||||||||
In this chapter, we examine the effect of audio computer-assisted self-interviewing (ACASI) experimental factors on reported prevalence, inconsistencies in reporting, time required to complete the interview, and breakoff rates. In addition, we examine the effect that the experimental factors had on respondents' ability to complete the interview and their attitudes about the survey task. Sections 7.1 through 7.3 contain the descriptive results; in Sections 7.4 and 7.5, we present the results of the statistical tests.
Because all respondents did not answer the same questions, it was necessary to create a set of edited recency of use variables for constructing the lifetime, 12-month, and 30-day prevalence of use estimates. This was done in three stages. In the ACASI application, all respondents had multiple chances to report use. For example, even respondents who did not receive multiple use questions could report that they had used alcohol in the past 30 days and subsequently report that they had used it on 0 days in the past 30 days. And some respondents who were in the multiple use treatment category were asked twice about their use in a particular time period. Thus, we first created a series of recodes for 30-day and 12-month use that documented whether the respondent was consistent or inconsistent in reporting use in these time periods. For example, we created a variable MU30ALC that had the following four levels:
For the random half of the respondents who received consistency checks during the interview and who had inconsistent data (e.g., MU30ALC = 2 or 3), we created a variable that documented the outcome of the consistency check. This variable also had four levels based on the path through the questionnaire and classified the respondent as either a user or nonuser for the period in question. The edited recency variables were created from these intermediate recodes.
This process resulted in our inability to determine the recency of use of some respondents; therefore, we had a residual category of indeterminate answers. In the NHSDA series of surveys, a similar logical editing procedure is used; however, before the final estimates are made, all indeterminate recency of use answers are removed by statistical imputation. We did not use any statistical imputations for the field experiment analysis. Therefore, when we compared 1997 field experiment data with data from the 1997 NHSDA Quarter 4, we used the recency variable thatwas edited but not imputed because it was the most analogous paper-and pencil interview (PAPI) comparison variable.
For cigarettes, there is one final important difference between the field experiment and the 1997 NHSDA that affects the prevalence estimates. In the field experiment, we routed respondents who had only had one or two puffs of a cigarette to the next section after they had reported their age at first use.
In addition to the edited variables, we also produced tables of weighted estimates using raw variables that were created using the most direct question on recency of use. For example, respondents in Treatments 5, 6, 7, and 8 who answered "yes" to the question "During the past 30 days, have you _______?" were counted as 30-day users even if they later indicated having used 0 days in the past 30 days. For the comparison group, we used answers to the recency question "How long has it been since you last used ______?" to create the raw variables. Estimates using raw variables are given in RTI (1998).
Exhibit 7.1 summarizes the findings for the main effects of the ACASI experimental factors by showing the ratios of the prevalence estimates by the level of the main effects. In the following section, each factor is discussed separately, followed by the results of the modeling.
7.1 Structure of Contingent Questioning
As described in Chapter 5, a random half of the respondents received a single gate question and a random half received three gate questions. We hypothesized that using multiple gate questions would result in fewer inconsistent reports and higher reports of prevalence of use, particularly for lifetime and 12-month use. We speculated that lifetime prevalence might be higher because (a) respondents could first deny more proximate use, which other research (e.g., Turner, Lessler, & Devore, 1992a) has shown to be a more sensitive behavior, and (b) it would prevent routing respondents who inadvertently answered "no" to a single gate question to the next section, thereby missing a chance to gather information about their use. In addition, we felt that having multiple gate questions that permitted the respondent to focus on each reference period individually would reduce inconsistencies in reporting.
7.1.1 Prevalence Estimates
Exhibits 7.1.1 through 7.1.15 display the weighted estimates of prevalence by selected demographic variables. For cigarette use, a mixed pattern was observed in the overall estimates, with single gate questions sometimes yielding higher reports and sometimes yielding lower reports. Differences between the two approaches were not large in any case, and most ratios were close to one. A similar pattern was observed for alcohol use. For marijuana, cocaine, and any illicit drug use, the single gate questions tended to yield higher reports of use. These findings were not in the direction hypothesized prior to the field experiment.
7.1.2 Inconsistencies in Reporting
We also examined inconsistencies in reporting by identifying the number of people who had a chance to give an inconsistent report. We confined this detailed analysis to cigarettes, alcohol, and marijuana so that we would have enough people to make meaningful comparisons. Exhibits 7.1.16 to 7.1.20, contrary to our expectations, indicate that multiple gate questions resulted in a higher percentages of inconsistent reports in most of the cases compared to single gate questions. The denominators (N) of the percentages included people who were able to get to the questions (i.e., they were not routed to the next section at a previous question), where they recorded the number of days of use within the particular time period. For past 12-month use, respondents answering the treatment versions with no multiple use questions had no chance of recording an inconsistent response.
7.1.3 Operational Aspects of Reporting Under Alternative Contingent Questioning Strategies
Time required to complete the interview. The average length of time required to complete each drug section with multiple gate questions was slightly higher than the time required to complete each drug section with a single gate question. However, the differences were not large (see Exhibit 7.1.21).
Exhibit 7.1 Ratio of Prevalence Estimates for ACASI Experimental Factors Overall and by Age Group | |||||||||||
Overall |
18+ Years Old |
12 to 17 Years Old | |||||||||
Factor |
Lifetime |
12 Month |
30 Day |
Lifetime |
12 Month |
30 Day |
Lifetime |
12 Month |
30 Day | ||
Cigarettes |
|||||||||||
Multiple Gate Qs: Single/Multiple |
1.07 |
0.87 |
1.03 |
1.08 |
0.87 |
1.03 |
0.94 |
0.99 |
1.07 | ||
Multiple 30-Day and 12-Month Qs: Consistency Checks: |
0.94 |
0.84 |
0.77 |
0.93 |
0.82 |
0.76 |
1.05 |
1.09 |
0.92 | ||
Alcohol | |||||||||||
Multiple Gate Qs: Single/Multiple |
0.97 |
0.90 |
0.94 |
0.97 |
0.90 |
0.94 |
1.00 |
1.00 |
1.05 | ||
Multiple 30-Day and 12-Month Qs: Consistency Checks: |
0.97 |
0.91 |
0.85 |
1.03 |
1.03 |
0.89 |
0.97 |
0.90 |
0.85 | ||
Marijuana | |||||||||||
Multiple Gate Qs: Single/Multiple |
1.01 |
1.63 |
1.45 |
1.00 |
1.76 |
1.48 |
1.18 |
1.12 |
1.32 | ||
Multiple 30-Day and 12-Month Qs: Consistency Checks: |
0.99 |
0.94 |
0.64 |
0.99 |
0.94 |
0.59 |
1.02 |
0.94 |
0.85 | ||
Cocaine | |||||||||||
Multiple Gate Qs: Single/Multiple |
1.93 |
2.64 |
1.13 |
1.94 |
2.76 |
1.00 |
1.92 |
1.86 |
3.77 | ||
Multiple 30-Day and 12-Month Qs: Consistency Checks: |
0.62 |
0.75 |
1.09 |
0.63 |
0.74 |
0.95 |
0.74 |
0.96 |
4.68 | ||
Any Illicit Drug1 | |||||||||||
Multiple Gate Qs: Single/Multiple |
1.14 |
1.29 |
1.45 |
1.13 |
1.34 |
1.51 |
1.18 |
1.06 |
1.23 | ||
Multiple 30-Day and 12-Month Qs: Consistency Checks: |
0.99 |
0.92 |
0.64 |
0.99 |
0.93 |
0.60 |
0.98 |
0.84 |
0.76 | ||
1Any illicit drug includes marijuana, cocaine, crack, heroin, inhalants, hallucinogens, and nonmedical use of analgesics, sedatives, stimulants, and tranquilizers.
Source: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures: 1997 Field Experiment.
Breakoff rates. In Exhibit 7.1.22, we compared the percentage of cases that were reported as a breakoff (even if the interviewer went on to finalize the case as a completed interview) by the treatment versions (single gate vs. multiple gate questions). Then we compared the percentage of finalized breakoff cases among single rate and multiple gate questions. The data show no meaningful difference between the breakoff rates for single gate and multiple gate questions.
Respondent reactions as reported in the debriefing questionnaire. Exhibit 7.1.23 presents an analysis of items from the respondent debriefing questions to see whether multiple gate questions had any effect on respondent reactions to the computer-assisted interviewing (CAI) NHSDA interview. The exhibit includes data from five items. About 87% of the respondents who were administered the multiple gate questions indicated that they were able to enter answers easily into the computer compared to 90% of the respondents who were administered the single gate questions. There was no important difference between the responses for single gate and multiple gate questions for the other items, including respondents' difficulty in using the computer, their interest in the interview, and the accuracy and completeness of their answers.
Exhibit 7.1.1 Lifetime Use of Cigarettes, by Contingent Questioning Structure | ||||
Respondent Characteristics |
ACASI Treatment Groups |
Comparison Group | ||
Single Gate Questions |
Multiple Gate Questions |
Total ACASI |
1997 | |
Treatment Version: |
1, 2, 3, 4 |
5, 6, 7, 8 | ||
Total |
74.18% |
69.38% |
71.96% |
70.87% |
Age Group |
||||
12 to 17 |
43.20% |
46.04% |
44.48% |
36.05% |
18+ |
77.88% |
72.04% |
75.18% |
74.94% |
Gender |
||||
Male |
75.71% |
72.64% |
74.25% |
74.74% |
Female |
72.82% |
66.14% |
69.83% |
67.25% |
Race/Ethnicity |
||||
Hispanic |
64.34% |
63.15% |
63.68% |
55.82% |
Non-Hisp., Black |
62.65% |
66.11% |
64.30% |
61.62% |
Non-Hisp., All Other Races |
76.82% |
70.65% |
74.04% |
73.81% |
Education1 |
||||
< High School |
84.90% |
68.25% |
77.83% |
70.28% |
High School |
78.43% |
81.58% |
79.73% |
76.65% |
> High School |
75.01% |
67.77% |
71.35% |
75.27% |
1Education includes only individuals aged 18 or older.
Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.
Exhibit 7.1.2 Lifetime Use of Alcohol, by Contingent Questioning Structure | ||||
Respondent Characteristics |
ACASI Treatment Groups |
Comparison Group | ||
Single Gate Questions |
Multiple Gate Questions |
Total ACASI |
1997 | |
Treatment Version: |
1, 2, 3, 4 |
5, 6, 7, 8 | ||
Total |
82.01% |
84.50% |
83.16% |
82.15% |
Age Group |
||||
12 to 17 |
44.89% |
44.92% |
44.90% |
38.01% |
18+ |
86.45% |
89.01% |
87.63% |
87.31% |
Gender |
||||
Male |
80.10% |
88.20% |
83.95% |
87.07% |
Female |
83.70% |
80.81% |
82.41% |
77.55% |
Race/Ethnicity |
||||
Hispanic |
70.87% |
77.01% |
74.26% |
69.45% |
Non-Hisp., Black |
65.91% |
65.13% |
65.54% |
72.35% |
Non-Hisp., All Other Races |
85.51% |
88.92% |
87.04% |
84.98% |
Education1 |
||||
< High School |
84.79% |
78.92% |
82.30% |
77.22% |
High School |
81.23% |
89.67% |
84.73% |
85.83% |
> High School |
91.16% |
91.17% |
91.16% |
91.48% |
1Education includes only individuals aged 18 or older.
Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.
Exhibit 7.1.3 Lifetime Use of Marijuana, by Contingent Questioning Structure | ||||
Respondent Characteristics |
ACASI Treatment Groups |
Comparison Group | ||
Single Gate Questions |
Multiple Gate Questions |
Total ACASI |
1997 | |
Treatment Version: |
1, 2, 3, 4 |
5, 6, 7, 8 | ||
Total |
35.36% |
34.98% |
35.18% |
34.96% |
Age Group |
||||
12 to 17 |
22.08% |
18.70% |
20.56% |
16.10% |
18+ |
36.94% |
36.84% |
36.90% |
37.17% |
Gender |
||||
Male |
39.35% |
36.06% |
37.78% |
40.79% |
Female |
31.81% |
33.91% |
32.75% |
29.51% |
Race/Ethnicity |
||||
Hispanic |
38.98% |
39.12% |
39.06% |
24.59% |
Non-Hisp., Black |
27.72% |
34.49% |
30.95% |
31.43% |
Non-Hisp., All Other Races |
36.39% |
34.63% |
35.60% |
36.48% |
Education1 |
||||
< High School |
26.55% |
29.61% |
27.85% |
32.84% |
High School |
36.22% |
35.84% |
36.06% |
36.60% |
> High School |
41.12% |
39.19% |
40.15% |
38.91% |
1Education includes only individuals aged 18 or older.
Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.
Exhibit 7.1.4 Lifetime Use of Cocaine, by Contingent Questioning Structure | ||||
Respondent Characteristics |
ACASI Treatment Groups |
Comparison Group | ||
Single Gate Questions |
Multiple Gate Questions |
Total ACASI |
1997 | |
Treatment Version: |
1, 2, 3, 4 |
5, 6, 7, 8 | ||
Total |
19.18% |
9.93% |
14.91% |
10.03% |
Age Group |
||||
12 to 17 |
3.64% |
1.90% |
2.86% |
3.13% |
18+ |
21.04% |
10.84% |
16.32% |
10.84% |
Gender |
||||
Male |
26.68% |
9.70% |
18.60% |
12.47% |
Female |
12.53% |
10.15% |
11.46% |
7.76% |
Race/Ethnicity |
||||
Hispanic |
15.81% |
18.52% |
17.30% |
7.32% |
Non-Hisp., Black |
15.27% |
4.02% |
9.90% |
7.57% |
Non-Hisp., All Other Races |
20.08% |
10.12% |
15.59% |
10.70% |
Education1 |
||||
< High School |
36.36% |
8.47% |
24.51% |
9.52% |
High School |
17.54% |
9.64% |
14.27% |
10.37% |
> High School |
18.50% |
12.09% |
15.26% |
11.58% |
1Education includes only individuals aged 18 or older.
Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.
Exhibit 7.1.5 Lifetime Use of Any Illicit Drug, by Contingent Questioning Structure | ||||
Respondent Characteristics |
ACASI Treatment Groups |
Comparison Group | ||
Single Gate Questions |
Multiple Gate Questions |
Total ACASI |
1997 | |
Treatment Version: |
1, 2, 3, 4 |
5, 6, 7, 8 | ||
Total |
46.26% |
40.75% |
43.72% |
37.50% |
Age Group |
||||
12 to 17 |
33.48% |
28.32% |
31.15% |
20.45% |
18+ |
47.79% |
42.17% |
45.19% |
39.49% |
Gender |
||||
Male |
49.68% |
37.16% |
43.72% |
43.44% |
Female |
43.23% |
44.32% |
43.72% |
31.94% |
Race/Ethnicity |
||||
Hispanic |
46.31% |
42.58% |
44.25% |
28.63% |
Non-Hisp., Black |
34.56% |
38.82% |
36.59% |
33.40% |
Non-Hisp., All Other Races |
48.23% |
40.92% |
44.93% |
38.99% |
Education1 |
||||
< High School |
54.09% |
32.81% |
45.05% |
37.22% |
High School |
46.31% |
44.03% |
45.36% |
38.42% |
> High School |
46.79% |
43.49% |
45.12% |
40.92% |
1Education includes only individuals aged 18 or older.
Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.
Exhibit 7.1.6 Past 12-Month Use of Cigarettes, by Contingent Questioning Structure | ||||
Respondent Characteristics |
ACASI Treatment Groups |
Comparison Group | ||
Single Gate Questions |
Multiple Gate Questions |
Total ACASI |
1997 | |
Treatment Version: |
1, 2, 3, 4 |
5, 6, 7, 8 | ||
Total |
29.24% |
33.46% |
31.19% |
33.52% |
Age Group |
||||
12 to 17 |
22.74% |
23.05% |
22.88% |
25.18% |
18+ |
30.02% |
34.65% |
32.16% |
34.49% |
Gender |
||||
Male |
38.14% |
33.32% |
35.85% |
34.17% |
Female |
21.34% |
33.59% |
26.82% |
32.91% |
Race/Ethnicity |
||||
Hispanic |
34.51% |
23.70% |
28.55% |
32.69% |
Non-Hisp., Black |
24.35% |
36.07% |
29.94% |
35.19% |
Non-Hisp., All Other Races |
29.69% |
34.01% |
31.64% |
33.29% |
Education1 |
||||
< High School |
57.88% |
28.04% |
45.20% |
34.76% |
High School |
24.86% |
51.87% |
36.04% |
41.55% |
High School |
24.44% |
26.87% |
25.67% |
29.70% |
1Education includes only individuals aged 18 or older.
Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.
Exhibit 7.1.7 Past 12-Month Use of Alcohol, by Contingent Questioning Structure | ||||
Respondent Characteristics |
ACASI Treatment Groups |
Comparison Group | ||
Single Gate Questions |
Multiple Gate Questions |
Total ACASI |
1997 | |
Treatment Version: |
1, 2, 3, 4 |
5, 6, 7, 8 | ||
Total |
65.11% |
71.99% |
68.28% |
64.95% |
Age Group |
||||
12 to 17 |
36.42% |
36.49% |
36.45% |
32.74% |
18+ |
68.54% |
76.03% |
72.00% |
68.71% |
Gender |
||||
Male |
62.83% |
77.43% |
69.78% |
70.07% |
Female |
67.12% |
66.58% |
66.88% |
60.15% |
Race/Ethnicity |
||||
Hispanic |
54.39% |
71.17% |
63.65% |
57.10% |
Non-Hisp., Black |
44.86% |
56.29% |
50.31% |
53.54% |
Non-Hisp., All Other Races |
69.27% |
75.01% |
71.86% |
67.64% |
Education1 |
||||
< High School |
57.94% |
50.08% |
54.60% |
48.54% |
High School |
66.10% |
77.46% |
70.80% |
68.12% |
> High School |
74.15% |
81.74% |
77.99% |
75.47% |
1Education includes only individuals aged 18 or older.
Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.
Exhibit 7.1.8 Past 12-Month Use of Marijuana, by Contingent Questioning Structure | ||||
Respondent Characteristics |
ACASI Treatment Groups |
Comparison Group | ||
Single Gate Questions |
Multiple Gate Questions |
Total ACASI |
1997 | |
Treatment Version: |
1, 2, 3, 4 |
5, 6, 7, 8 | ||
Total |
12.47% |
7.64% |
10.24% |
9.36% |
Age Group |
||||
12 to 17 |
17.56% |
15.62% |
16.69% |
12.97% |
18+ |
11.86% |
6.73% |
9.49% |
8.94% |
Gender |
||||
Male |
17.80% |
9.62% |
13.91% |
13.08% |
Female |
7.75% |
5.68% |
6.82% |
5.88% |
Race/Ethnicity |
||||
Hispanic |
10.71% |
16.66% |
13.99% |
10.21% |
Non-Hisp., Black |
8.84% |
7.73% |
8.31% |
9.60% |
Non-Hisp., All Other Races |
13.21% |
6.67% |
10.26% |
9.25% |
Education1 |
||||
< High School |
10.89% |
4.19% |
8.05% |
14.87% |
High School |
9.05% |
7.95% |
8.59% |
10.10% |
> High School |
14.43% |
6.70% |
10.52% |
6.30% |
1Education includes only individuals aged 18 or older.
Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.
Exhibit 7.1.9 Past 12-Month Use of Cocaine, by Contingent Questioning Structure | ||||
Respondent Characteristics |
ACASI Treatment Groups |
Comparison Group | ||
Single Gate Questions |
Multiple Gate Questions |
Total ACASI |
1997 | |
Treatment Version: |
1, 2, 3, 4 |
5, 6, 7, 8 | ||
Total |
3.69% |
1.40% |
2.63% |
1.60% |
Age Group |
||||
12 to 17 |
2.23% |
1.20% |
1.77% |
2.54% |
18+ |
3.87% |
1.42% |
2.74% |
1.49% |
Gender |
||||
Male |
4.80% |
1.16% |
3.07% |
2.10% |
Female |
2.71% |
1.64% |
2.23% |
1.13% |
Race/Ethnicity |
||||
Hispanic |
6.72% |
2.91% |
4.62% |
1.56% |
Non-Hisp., Black |
4.31% |
2.41% |
3.40% |
2.98% |
Non-Hisp., All Other Races |
3.37% |
1.05% |
2.33% |
1.36% |
Education1 |
||||
< High School |
7.53% |
2.01% |
5.19% |
1.87% |
High School |
1.59% |
2.25% |
1.86% |
2.03% |
> High School |
4.41% |
0.82% |
2.59% |
1.01% |
1Education includes only individuals aged 18 or older.
Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.
Exhibit 7.1.10 Past 12-Month Use of Any Illicit Drug, by Contingent Questioning Structure | ||||
Respondent Characteristics |
ACASI Treatment Groups |
Comparison Group | ||
Single Gate Questions |
Multiple Gate Questions |
Total ACASI |
1997 | |
Treatment Version: |
1, 2, 3, 4 |
5, 6, 7, 8 | ||
