1999 National Household Survey on Drug Abuse Data Collection Report

2. SAMPLING AND COUNTING/LISTING OPERATIONS

A coordinated five-year sample design was developed for 1999 through 2003. The sample designs for both the 1999 main study and the supplemental sample, each a subsample of the five-year study, consisted of deeply stratified, multi-stage, area probability designs. Exhibit 2.1 presents details of the sample design.

The coordinated 1999-2003 design involved no overlap with the 1998 sample; however the design calls for 50 percent overlap in first stage units (area segments) between each successive year of the five-year study.

The first stage of the sample selection procedures began by geographically partitioning each state into roughly equal-sized field interviewer (FI) regions. These regions were formed as a means of stratification and so that each area yields roughly the same expected number of interviews during each data collection period. This partitioning divided the United States into 900 FI regions made up of counties or groups of counties.

These FI regions were subdivided into smaller geographic areas—called segments—that served as the primary sampling units. In general, segments consisted of adjacent Census blocks and were equivalent to area segments selected at the second stage of selection in previous NHSDAs. A total of 96 segments per FI region were selected (with probabilities proportional to size): 24 to field the five-year study and 72 to serve as backups in case of sample depletion or to field any supplemental studies SAMHSA may request. For the 1999 survey, a total of 7,200 segments within the 900 FI regions were selected.

After selecting these new areas, the process of counting and listing (C/L) the dwelling units (DUs) within each segment ensued. Segments to be used in 1999 were listed between June and November of 1998. Once all DUs for a particular quarter were listed, the second-stage selection process identified sample dwelling units (SDUs) for inclusion in the study.

Similar to previous NHSDAs, at the final stages of selection, five age group strata were sampled at different rates. These five strata were defined by the following age group classifications: 12-17, 18-25, 26-34, 35-49, and 50 years old and over. Race/ethnicity groups were not purposely over-sampled for the 1999 main study. However, consistent with previous NHSDAs, the 1999 NHSDA was designed to over-sample younger age groups.

The supplemental paper-and-pencil (PAPI) sample was designed to be roughly one-fourth the size of the main study sample at the last stage of selection; in reality, it was closer to one-fifth the size. To maximize precision between the main study CAI estimates and the PAPI supplemental sample estimates, the design of the supplemental sample closely mirrored that of the main study with an additional level of clustering imposed at the first stage of selection. This additional clustering lessened the costs associated with collecting the supplemental data by minimizing the number of interviewers needed to be trained on the PAPI instrument. Whether the units were for the main study or the PAPI supplemental sample, there were no differences in the field procedures used for counting and listing. See Exhibit 2.1 for more details about the PAPI supplemental sample design and selection.

In preparation for C/L field activities, the recruiting and hiring of Field Supervisors (FSs) began in April, 1998. It culminated in May, 1998, with the hiring and training of 23 staff recruited from the following groups:

• those with experience as supervisors on the study in previous years;

• staff with experience as supervisors on other RTI studies or with experience as data collection supervisors for other companies; and

• listers/interviewers from previous years of the study.

The FS training consisted of basic classroom exercises to familiarize them with the mechanics of the C/L task and the detailed steps used to subsegment when required. All C/L FSs were also certified or re-certified as listers. In addition to basic C/L concepts, the training covered identifying potential listing staff, hiring that staff, and the training and field supervision of listers. There were hands-on sessions with computer equipment, the Web-based Case Management System, and e-mail capabilities related to the management of listing staff.

Field Supervisors immediately started hiring their listing staff. The recruiting, hiring, training, and supervision of field listers occupied most of the FSs' time between May and September, 1998. In addition to finishing the listing operation by mid-November, after September these FSs (and approximately 50 additional FS recruits) began recruiting and hiring field staff for the screening and interviewing task.

A large pool of listing staff used in previous years filled many of the lister positions. However, about one-third of the listers for the 1999 study were new to the project in 1998. In general, for the states/locations where lister candidates were not available, a team of listers available for travel was assembled to complete the listings in those areas promptly.

A total of 364 listers were hired and worked from June through November, 1998, to complete counting and listing operations for the 1999 NHSDA. For training, three different home-study training packages were prepared for:

1. RTI-Certified and experienced listers from previous surveys;

2. experienced listers who had not worked with or been certified by RTI previously; and

3. individuals new to listing.

All listers received a memorandum and materials which included a C/L manual; C/L video tape; hire letter; Data Collection Agreement; 1999 NHSDA C/L Project Specification Sheet; Production, Time and Expense Reports; and general listing supplies.

To attain certification, those who were experienced but not RTI-Certified and those who were new to listing also received path-of-travel exercises and a certification packet to complete. Those new to listing had more time in which to complete these materials and received telephone training from their FSs. The telephone training supplemented the home-study before new listers completed their certification packages. Subject to Regional Supervisor (RS) approval, in the event a new lister needed additional training, the FS could conduct an in-person training with one or more listers or could pair a new lister with an experienced lister.

Once the listers successfully completed the required materials/training and returned signed Data Collection Agreements to RTI, they were authorized to begin their C/L assignments. For quality control, all listers were required to send their first three completed segments to their FS for review. Once approved, listers sent their completed assignments directly to RTI. (As needed, some listers were again asked to send their work to their FS either for monitoring of problems reported by RTI staff during processing or for particularly difficult segments.)

Prior to the start of actual C/L field work, segment kits were assembled at RTI. Each kit contained 1990 Census (TIGER) maps for the selected area, listing forms, and segment information sheets. A copy of the maps remained at RTI and another copy was given to the field supervisor for assisting with problems encountered in the field.

Beginning in late May, segment kits were assigned and sent to veteran listers to begin the listing process as other staff completed the certification process. Listers recorded the address or description of up to 400 dwelling units (DUs) in each segment.

To reduce the time required to count and list segments, several procedures were implemented to maximize efficiency. Since 1990 Census data was available for all segments, in many cases the "count" step was eliminated: the lister could immediately list the segment unless it was apparent the segment had experienced additional building or the lister determined that the segment was large (i.e., 400+ DUs) during the initial trip around the boundaries of the segment. As had been done on prior rounds of the NHSDA, a rough count procedure was allowed for segments containing large geographic land areas, large DU counts (400+ DUs), or significant growth in residential DUs (typically, 1,000+ DUs). This procedure permitted listers to obtain an approximate count of residential DUs in these segments from secondary sources—such as the post office, fire department, or county or city planning office—without having to conduct an exact count.

To reduce the turnaround time for sending a segment from the field to RTI to be subsegmented, FSs were trained to perform all routine subsegmenting activities. If a lister came across a segment that needed subsegmenting, the lister called in DU counts to the FS, who subsegmented it over the telephone (any segment with more than 400 DUs generally required subsegmenting). In many cases, this allowed the lister to—in one trip—count and list a segment with 400 or more DUs, rather than experiencing a delay of one or two weeks and necessitating a second trip to the segment. For unusual or very difficult subsegmenting tasks, the segments were not handled by FSs but were sent to RTI to be handled by sampling personnel.

The counting and listing of almost all of the segments was completed by the end of November 1998 (the exceptions involved a few access problems). Once the segments were listed and the completed segment kits were received at RTI, an editing process ensured that no DUs located outside segment boundaries were included, that listing sheets matched segment sketches/TIGER maps, and that proper listing order and related listing rules were observed. During this editing process, the sampling staff also checked all subsegmenting that occurred in the field to ensure it was done correctly.

Listed DUs were keyed into the computer control system. A selection algorithm then selected the specific sample dwelling units (SDUs) to be contacted for the study. FSs then assigned segments (or partial segments) to their interviewing staff. Interviewers received all assigned SDUs on their Newton handheld computer. Each selected unit and the next listed line (for use as a sample check during screening and interviewing) were also printed on Selected DU Lists. These lists, along with copies of the handwritten listing forms and maps, were placed in Segment Materials Envelopes and distributed to the assigned field staff just prior to the beginning of each quarter.

During the screening process, Field Interviewers (FIs) attempted to identify any unlisted DUs that existed within the SDU or within the interval between the SDU and the next listed DU. If the missed DUs were housing units, they were automatically entered into the Newton (up to established limits) and selected for participation. At most, the FI could independently enter three added DUs per SDU and a maximum of six missed DUs per segment. If the FI discovered more than these amounts, the FI called the FS first and then called RTI's Sampling Department for further instructions. If the missed DUs were group quarters units, the FI telephoned the FS first and then telephoned RTI for instructions.

While no upper-limit was placed on the total number of DUs that could be added to a segment, the FIs were instructed to notify RTI of any significant listing problems. A very small number of segments required re-listing during the screening and interviewing phase. Table 2.2 indicates the number of segments that incurred one to twenty-five added DUs, as well as the total number of added DUs for the 1999 NHSDA.

While monitoring data collection results during the first quarter, reports indicated the yield of PAPI interviews was much higher than anticipated. While having a sample that generates too many interviews (too rich) can be viewed less seriously than one that generates too few interviews, an investigation into the possible causes of this outcome was conducted. During the investigation, an error in the Newton's person-selection probabilities algorithm was discovered. After consultation with SAMHSA staff, corrective actions to adjust the sample selection parameters on the Newton for remaining cases were taken. While this adjustment helped decrease the interview yield for those lines not yet worked, it became evident that there were insufficient numbers of field interviewers to work those interviews already generated and those projected from sample housing units that had not been screened, and thus, the response rates would suffer. Therefore, a special subsampling procedure was implemented to randomly withdraw sample lines that had not been screened so that all cases remaining in the field could be worked within the allotted time period. Some Quarter 1 lines not yet worked were retrieved, and some lines that had been attempted but not completed were re-fielded in Quarter 2. See section 2.6.2 for more information on the subsampling that occurred in Quarter 1.

For Quarter 3, refinements were made to the PAPI sample allocation by imposing a stratification at the segment level (as opposed to the FI region level) similar to that used prior to the first stage of selection. This helped define a post-stratum with less than 10 percent Hispanicand non-Hispanic black combined; this stratum also served as the over-sampling cutoff group where no over-sampling of minorities was implemented. By adjusting the allocation to these post-strata, there was better control of the sample allocation by household type and provisions for an efficient screening ratio for the PAPI sample.

During Quarter 1 of the 1999 NHSDA, it became evident that the response rates were not comparable to those achieved in prior years. The principal cause for the reduction in response rates was an insufficient number of field interviewers combined with a majority of them being inexperienced. One action taken to overcome the response problem was to subsample from all pending cases so that cases retained could be worked more thoroughly. This special subsampling was conducted in two phases. During the first phase, the CAI sample had a total of 8,640 of the 13,161 unfinished dwelling units (i.e., pending screeners) pulled out of the sample and the PAPI sample had 5,484 of the 8,301 unfinished dwelling units pulled out of that sample. In the second phase, dwelling units eligible to be sampled included those that were unfinished and those with pending person interviews. For the CAI sample, a total of 3,958 such units were removed in the second phase, and for the PAPI sample, 2,116 such units were removed. In order to reduced the effect of unequal weights, all pending dwelling units (all units from round 1 and 1,827 units from round 2) for the CAI sample only were put back into the sample in Quarter 2. The sample weights were adjusted to reflect the subsampling and putting back of cases for both the CAI and PAPI samples.

In many of the major urban areas, field staff had some difficulties gaining access to locked buildings, and listers in particular had some trouble listing very large public housing complexes. Access in some suburban areas proved problematic as well; more and more planned communities have intercoms, guarded gatehouses or entryways outfitted with cameras and scrambled buzzer systems. Access to military bases, college dormitories, and large retirement communities also sometimes proved problematic. With the large sample expansion in 1999, the increase in the number of access problems was expected. Special mechanisms or protocols were set up to handle them promptly and in some cases avoid them entirely.

Access problems were typically resolved through effective follow-up efforts of supervisory staff, including situation-specific letters of request and in-person visits by the Field and/or Regional Supervisors. In particularly difficult situations, SAMHSA offered additional support via special refusal conversion letters or telephone follow-ups by the Project Officer.

In previous years, the often problematic access to military bases had been resolved by dealing directly with the base commanders. With the new sample design, all land area in every state was subject to selection and military bases were prone to selection because of the younger populations typically located in and around the base. Therefore, a more formal and standard approach to gaining access was necessary. Through joint RTI/SAMHSA efforts, a contact person within the Pentagon for each branch of the service was established. These individuals were advised in advance of base selections for the year. They then notified the base commanders regarding RTI's need to access these bases for both listing and screening/interviewing work. Additionally, standard letters and informational packages were developed by RTI staff to help obtain access to all selected bases. These efforts were effective: access to all but one selected base was secured (the base in question was in the process of going from active to inactive status).

In previous surveys, colleges and universities were also only occasionally problematic. The same sampling design parameters that resulted in more frequent selection of military bases also had the same effect on college and university housing. For that reason, several standard approaches were developed to accommodate the concerns of school administrators. Although similar letters were sent in past years, having standardized letters available that addressed reoccurring issues with a variety of attachment options was new and very effective.

Most schools requested or required only a letter stating the sponsor and the purpose of the study, and identifying the lister or data collection staff. However, some schools wanted more complete information and the right to approve the field data collection procedures and personnel working in and around their campuses. Most of these situations resulted in packages being sent that contained:

1. RTI IRB information;

2. OMB approval information;

3. descriptive information about the procedures and data collection plan; and

4. various descriptive study materials used with respondents during data collection.

In the end, only two educational institutions denied the request for cooperation for the counting and listing phase of the 1999 NHSDA.

A small number of segments were identified during the counting and listing phase as difficult to access during months with unusual weather. Most involved roads made impassable by snow during the winter months. Others involved roads inaccessible due to rain, and one or two isolated locations involved water-only access that often froze during the winter months. If segments with weather or geographic access problems were selected for a quarter in which the access would be a problem (generally Quarters 1 or 4), the segment was switched with a segment in the same region for an appropriately paired time period. For example, inaccessible first quarter segments were switched with second quarter segments in the same region that would be more accessible during the first quarter; fourth quarter segments were switched with more easily traveled third quarter segments. Generally the "switched" segment was selected because it had more accessible road surfaces, was more urban, or had fewer inaccessible roads.

In a few locations, such as some areas in Alaska, there were no segments that were better for reassignment during the problematic time period. When that happened, staff made prompt assignments, emphasized early completion of the work, and tried to plan around good weather forecasts to accomplish the field work as early in the period as possible.

1999 NHSDA Sample Design Summary

First Stage of Selection for the Main Study: Segments

The 1999 design provided for estimates-by-state in all 50 states and the District of Columbia. States should therefore be viewed as the "first level" of stratification as well as a reporting variable. Eight states, labeled the "big" states in Table 2.1, had a sample designed to yield 3,600-4,630 respondents per state. The remaining 43 "small" states¹ had a sample designed to yield 900-1,030 respondents per state.

The larger sample sizes obtained at the state level, along with small area estimation techniques refined under previous NHSDA contracts, enabled the development of estimates for all states, for several demographic subgroups within each state (i.e., age group and race/ethnicity group), and for some Metropolitan Statistical Areas and a few small areas in the "big" states.

The "second level" of stratification defined contiguous geographic areas within each state and also corresponded in size to the annual assignment for a single field interviewer (FI). These FI regions were of approximately equal size in terms of allocated sample.

Additional implicit stratification was achieved by sorting the first-stage sampling units by an MSA/SES (socioeconomic status) indicator² and by percentage of non-Hispanic white. The first stage sample units for the 1999 NHSDA were selected from this well-ordered sample frame.

For the first stage of sampling for the 1999 NHSDA, each of the FI regions was partitioned into noncompact clusters of dwelling units by aggregating adjacent Census blocks. Consistent with the terminology used in previous NHSDA studies, these geographic clusters of blocks were referred to as segments. On average, segments were formed so that they contained at least 150 dwelling units and were constructed using 1990 Decennial Census data supplemented with revised population counts obtained from outside sources. A sample dwelling unit in the NHSDA refers to either a housing unit or a group quarters listing unit (such as a dormitory room or a shelter bed).

A sample of segments was selected within each FI region, with probabilities proportionate to a composite size measure and with minimum replacement. Segments were formed so that they contained sufficient numbers of dwelling units to support three annual NHSDA samples. This allowed half of the segments used in any given year's main sample to be used again in the following year as a means of improving the precision of measures of annual change. This alsoallows for any special supplemental sample or field test that SAMHSA may wish to conduct in any given NHSDA year within the same segments (e.g. the 1999 supplemental sample that is discussed below).

In order to coordinate the sample selection for 1999 through 2003, 96 segments were selected within each FI region. An equal probability subsample of eight segments was used for the 1999 NHSDA. These eight segments were randomly assigned to quarters and to two waves within each quarter. The waves used in the 1999 NHSDA were designated as Waves 1 and 2. Wave 1 segments were used only for the 1999 survey. Wave 2 segments were used for the 1999 and 2000 surveys. New dwelling units (i.e. those not previously selected for the 1999 study) will be selected from the Wave 2 segments in 2000. An additional sample of Wave 3 segments will be selected from the remaining pool of 88 (96-8=88) preselected segments within each FI region in order to meet total sample size requirements for the 2000 survey.

Data from roughly one-fourth of the final sample of respondents was collected during each calendar quarter. This important design feature helped control any seasonal bias that might otherwise exist in drug use prevalence estimates and other important NHSDA outcome measures of interest.

Second Stage of Selection for the Main Study: Listed Lines

Before any sample selection within selected segments began, specially-trained staff listed all dwelling units and potential dwelling units within each selected area segment. A dwelling unit is either a housing unit for a single household or one of the eligible noninstitutional group quarters that are part of the defined target population. The listings were based primarily on observation of the area segment and could include vacant dwelling units and units that appeared to be dwelling units but were actually used for nonresidential purposes. The objective of the listing was to attain as complete a listing as possible of eligible residential addresses; any false positives for residences were eliminated during the household screening process after the sample was selected.

The sampling frame for the second stage of sample selection was the lines of listed dwelling units and potential dwelling units. After accounting for eligibility, nonresponse, and the third-stage sample selection procedures, it was determined that roughly 280,273 lines were needed to obtain a sample of 70,000 responding persons distributed by state and age-group. During the study's implementation, however, a total of 223,868 lines were selected and yielded a final respondent sample of 66,706 (as shown in Table 2.1).

As in previous years, if an interviewer encountered any new dwelling unit in a segment or found a dwelling unit missed during the counting and listing activities, the new/missed dwellings wereselected into the 1999 NHSDA using a half-open interval selection technique.³ That selection technique eliminated any frame bias that might have been introduced because of errors and/or omissions in counting and listing activities and also eliminated any bias that might have been associated with using "old" segment listings.

Third Stage of Selection for the Main Study: Persons

After dwelling units were selected within each segment, an interviewer visited each selected dwelling unit to obtain a roster of all persons aged 12 and over residing in the dwelling unit. This roster information was then used to select zero, one, or two persons for the survey. Sampling rates were pre-set by age group and state. Roster information was entered directly into the electronic screening instrument (the Newton) which automatically implemented this third stage of selection based on the state and age group sampling parameters.

Using an electronic screening instrument also provided the ability to impose a more complicated person-level selection algorithm at the third stage of selection. As a result of this unique design feature, any two survey-eligible people within a dwelling unit had some chance of being selected—i.e., all survey eligible pairs of people had some non-zero chance of being selected. This 1999 design feature is of interest to NHSDA researchers because it allows analysts to examine how the drug use propensity of one individual in a family relates to that of other family members residing in the same dwelling unit (e.g., the relationship of drug use between a parent and child).

As illustrated in Table 2.1, at the third stage of selection, 89,883 people were selected from 169,166 screened and eligible dwelling units. A total of 66,706 completed interviews were obtained from these 89,883 selected persons.

PAPI Supplemental Sample Selection

The supplemental sample was selected in four stages. For the first stage, a sample of 250 FI regions was randomly subselected from the 900 FI regions formed for the main study. These 250 FI regions were selected randomly within strata defined to isolate relatively high-concentration Hispanic areas, high-concentration black areas, high-concentration white areas, and remaining areas. This race/ethnic-based stratification was imposed to optimally sample Hispanics and blacks at the last stage of selection. Unlike the main study, the supplementalsample was designed to oversample Hispanic and blacks (as well as younger individuals) in order to maximize contrast estimates for these important subpopulations of interest.

Within the 250 subselected FI regions, all the segments selected for the main study sample were selected for the supplemental sample. The main study probabilities of selection applied since segments were randomly selected from within the main study FI region strata. This complete segment overlap between the main study and supplemental sample within the 250 FI regions provided for the maximal amount of precision in contrast estimates between the two samples.

At the third stage of selection, within each segment a sample of dwelling units was selected from among those dwelling units not selected for the main study. Similar to the main study, at the fourth stage of selection, either zero, one, or two people were selected from within each successfully screened dwelling unit. As with the main study, any pair of survey-eligible residents within the dwelling unit had some known, non-zero chance of being selected for the survey.

A decision was made to decrease the sample size for the PAPI study midway through the survey year. Therefore, a total of 46,328 lines were selected. Of those households, 35,635 were eligible and screened; they yielded a total of 13,809 completed interviews.

Expected Precision of NHSDA Estimates

The multi-stage, stratified 1999 NHSDA design was been optimally constructed to achieve specified precision for various person subpopulations of interest. These SAMHSA-specified, precision requirements call for the expected relative standard error on a prevalence of 10% not exceed the amounts listed below.

For the main study:

• 3.00% for total population statistics;

• 5.00% for statistics in four age group domains: 12-17, 18-25, 26-34, 35 and over;

• 11.00% for statistics computed among Hispanics in four age group domains: 12-17, 18-25, 26-34, 35 and over;

• 11.00% for statistics computed among non-Hispanic blacks in four age group domains: 12-17, 18-25, 26-34, 35 and over; and

• 5.00% for statistics computed among non-Hispanic, non-blacks in four age group domains: 12-17, 18-25, 26-34, 35 and over.

And for the supplemental sample:

• 3.40% for total population statistics;

• 6.75% for statistics in four age group domains: 12-17, 18-25, 26-34, 35 and over;

• 11.25% for statistics computed among Hispanics in four age group domains: 12-17, 18-25, 26-34, 35 and over;

• 11.25% for statistics computed among non-Hispanic blacks in four age group domains: 12-17, 18-25, 26-34, 35 and over; and

• 7.50% for statistics computed among non-Hispanic, non-blacks in four age group domains: 12-17, 18-25, 26-34, 35 and over.

To achieve these precision requirements and meet state sample-size requirements, the optimal person-level sample distribution by strata was determined that minimized data collection costs while simultaneously meeting the above-specified precision requirements for several critical NHSDA outcome measures. Separate models were examined for the main study and the supplemental sample since, as noted above, both samples were designed to produce independent estimates of the respondent universe with differing levels of precision.

The precision constraints in the 1999 design optimization models were set up using local area predictions of drug use from a recent project involving small area estimation techniques used to generate local area estimates using 1991-1993 NHSDA data. Drug use estimates across strata were appropriately scaled to reflect the generic 10% prevalence.

Sampling Summary of 1999 Main Study NHSDA

Note:
"Small" states refers to states where the design yielded 914 respondents on average. "Big" states refers to states where the
design yielded 3,425 respondents on average.

Segments with Added Dwelling Units
1999 NHSDA

Number of Added DUs per Segment (X)			Number of Segments with X Added DUs			Cumulative Number of Added DUs*
	1			517			517
	2			188			893
	3			102			1,199
	4			51			1,403
	5			32			1,563
	6			26			1,719
	7			11			1,796
	8			10			1,876
	9			10			1,966
	10			4			2,006
	11			4			2,050
	12			7			2,134
	13			1			2,147
	14			3			2,189
	16			1			2,205
	25			1			2,230

*Total number of added DUs = 2230

¹ For reporting and stratification purposes, the District of Columbia is treated the same as a state and no distinction is made in the discussion.

² The four categories are defined as: (1) MSA/low SES, (2) MSA/high SES, (3) NonMSA/low SES, and (4) NonMSA/high SES.

³ In summary, this technique states that if a dwelling unit is selected for the 1999 study and an interviewer observes any new or missed dwelling units between the selected dwelling unit and the dwelling unit appearing immediately after the selection on the counting and listing form, then all new/missed dwellings between the selection and the next one on the listing form will be selected. If a large number of new/missed dwelling units are encountered (generally greater than six) then a sample of the missing dwelling units will be selected.

This page was last updated on June 03, 2008.

SAMHSA, an agency in the Department of Health and Human Services, is the Federal Government's lead agency for improving the quality and availability of substance abuse prevention, addiction treatment, and mental health services in the United States.