Oregon Survey Research Laboratory Page 1 University Health Center Survey 2002 Survey Methodology UNIVERSITY HEALTH CENTER SURVEY APRIL-MAY 2002 SURVEY METHODOLOGY SURVEY INSTRUMENT DEVELOPMENT SAMPLING DATA COLLECTION, DATA PROCESSING, AND QUALITY CONTROL DATA REDUCTION AND CODING OREGON SURVEY RESEARCH LABORATORY 5245 UNIVERSITY OF OREGON EUGENE, OR 97403-5245 TELEPHONE: 541-346-0824 FACSIMILE: 541-346-0388 E-MAIL: OSRL@OREGON.UOREGON.EDU WWW: OSRL@OREGON.UOREGON.EDU/~ By Patricia A. Gwartney, Ph.D., Founding Director and Co-Project Directors Vikas Gumbhir, Juyeon Son INTRODUCTION This report summarizes the University of Oregon Survey Research Laboratory?s (OSRL?s) survey methodology for its ninth annual University Health Center (UHC) Survey. Working closely with UHC representatives, Gerald Fleischli, Anne Mattson, and Paula Staight, OSRL planned, pre-tested, and implemented a telephone survey of 405 randomly selected University of Oregon (UO) students. BACKGROUND UHC intends for the annual survey to assess its services and to track health-related trends in student knowledge, attitudes, and behavior. UHC and OSRL have developed three broad groups of survey questions for the annual studies: 1. core questions asked annually, 2. periodic questions asked in either odd or even years, and 3. topical, once-only questions, intended to tap issues of the moment. SURVEY METHODOLOGY This section describes OSRL?s procedures for developing and implementing the telephone survey instrument and sample to conduct this representative study. SURVEY INSTRUMENT DEVELOPMENT In spring of this year UHC and OSRL staff collaborated in meetings and discussions to identify the periodic questions that would be included in the 2002 instrument, to ascertain and rectify problems with any core or periodic questions, and to distinguish key concepts for topical questions. The team endeavored to operationalize survey questions that are appropriate to the UO?s needs, UHC?s needs, comparable to other major Oregon and national surveys, and as valid and reliable as possible at a reasonable price. Oregon Survey Research Laboratory Page 2 University Health Center Survey 2002 Survey Methodology The UHC Survey 2002 included the following topics: 1. Physical and mental health and wellness, including overall assessments, pain, illness, height, weight, suicide thoughts, suicide attempts, stress, experience of discrimination, and ADHD; 2. Health maintenance activities, including pap smear checks, exercise, and athletic participation; 3. Tobacco, alcohol, and drug use, with special attention to marijuana, as well as herbal remedies; 4. Safety behaviors, including drunk driving, sex under the influence of alcohol or drugs, car safety belt use, motorcycle and bicycle helmet use, and the carrying of weapons; 5. Sexual activity, including use of contraception, condom use, pregnancy, rape, sexual orientation, and sexually transmitted diseases; 6. UHC use and knowledge, specifically student satisfaction, reasons for non- use, suggestions for improving services, and cost comparisons; 7. Health insurance coverage, who pays for it, medical expenses, and insurance opinions; 8. Opinions about fee increases and improvements to the interior architecture of the remodeled UHC; and 9. Basic background and demographic characteristics, such as age, sex, race/ethnicity, residence, GPA, marital status, and parental status. All survey questions underwent OSRL's standard three-pronged pretest procedure, involving: (a) members of the survey population, (b) OSRL's Questionnaire Review Committee, comprised of survey experts from our staff and university-wide advisory committee, and/or (c) potential users of the data from the UHC. Individual questions were pretested for clarity, accuracy, validity, and variability of response. The entire instrument was pretested for flow, comprehensiveness, length, and factors which affect respondents' cooperation and attention. Section 2 of this report provides a facsimile of the telephone survey instrument, with embedded ?topline? results and all skip logic. SAMPLING OSRL randomly selected 645 currently enrolled UO graduate and undergraduate students from the Registrar?s records as the sample for this survey. As in prior years, we excluded Continuing Education Program students from the sample. One week before data collection commenced, OSRL mailed a pre-contact letter to all randomly sampled students (see Section 3 for a facsimile of the letter). In order to obtain human subjects approval for this study, and due to certain survey questions? sensitive Oregon Survey Research Laboratory Page 3 University Health Center Survey 2002 Survey Methodology subject matter, UO?s Committee for the Protection of Human Subjects requires OSRL to send this letter. The pre-contact letter introduced the study?s goals and purpose, explained its importance, described how respondents were selected, identified the potentially sensitive subject areas in the survey interview, assured confidentiality and voluntariness, and provided contact names, email addresses, and telephone numbers for respondents who might have questions. Each precontact letter contained an original signature of OSRL?s Director, signed on a soft surface with a ballpoint pen, in order to achieve indentation. Sampling error for a study of this size is moderate to small. Survey sampling errors assist data users in assessing how much confidence to place in a particular survey result. Moderately large random samples, as in this study, reduce sampling error. Survey results with low variability also produce less sampling error; e.g., a variable with a 5/95 proportional split has narrower confidence intervals than a variable with a 50/50 proportional split. For this study, the confidence interval is +4.8 percentage points on variables with a 50/50 proportional split (at the 95% confidence level). This means analysts can be 95% sure that the true population figure lies between 45.2% and 54.8% (i.e., 50% +4.8 percentage points). For variables with a 5/95 proportional split, the confidence interval is +2.1, which means analysts can be 95% sure that the true population figure lies between 92.9% and 97.1% (i.e., 95% +2.1 percentage points). For detail, see OSRL?s ?Sampler? at http://darkwing.uoregon.edu/~osrl/miscpapers/sampler.html. DATA COLLECTION, DATA PROCESSING, AND QUALITY CONTROL OSRL timed the survey to fall more than four weeks after the end of Spring Break (since students? Spring Break activities could artificially inflate reports of certain types of behavior, such as alcohol consumption). OSRL conducted interviewer training on Thursday, April 25th, 2002; see Section 3 for interviewer instructions. Interviewing commenced on Saturday, April 28th and continued until Thursday, May 10th when the target sample size was achieved, n=405. Interviewers called at all times of the day and all days of the week, with the exception of Sunday morning. Interviews averaged 15.5 minutes. On average, over 11.3 telephone dial attempts were required for each completed interview, but up to 25 calls were made. All interviews were conducted in English. Only experienced interviewers were employed for this study. Altogether, OSRL interviewers made 4,590 telephone calls to complete 405 interviews. Among the original 645 telephone numbers chosen, 65 were unusable because the number was wrong, disconnected, non-working, nonresidential, or a fax/modem telephone number. In addition six randomly-chosen students were gone for the study dates and, therefore, could not be interviewed. OSRL routinely reports a CASRO-type response rate, according to the highest industry standards (source: Robert M. Groves, Survey Errors and Survey Costs, 1989). The formula for calculating this response rate requires that each telephone dial attempt be Oregon Survey Research Laboratory Page 4 University Health Center Survey 2002 Survey Methodology assigned a call disposition code. At the completion of the survey project, the final disposition code for each telephone number is used for response rate calculation. The overall survey response rate was 73%, and the refusal rate was 4%1. Section 4 provides the study?s complete sample, call disposition, and response rate report. The survey was conducted using OSRL?s WinCATI system, in which sampling, interviewing, and data entry is accomplished interactively and seamlessly. Interviews are pre-programmed and appear automatically at each workstation. The programmed survey instrument contains all survey questions, interviewer probes for consistency, and pre- coded answer categories. Skip logic is programmed into the system, preventing inappropriate or incorrect questions from being asked. The WinCATI system eliminates out-of-range responses and wild codes by validating each response interactively and not allowing inappropriate responses to be entered. In administering the survey instrument, trained interviewers use telephone headsets in sound-reduced carrels at computer workstations connected by an NT network. Randomly distributed telephone numbers appear automatically at each workstation and are mated to the survey instrument. Interviewers place telephone calls with a computer keystroke, preventing dialing errors. As respondents answer questions, interviewers enter the data into the WinCATI data file. Telephone numbers and names are automatically stripped from the interview data to ensure confidentiality. Thus, the WinCATI system eliminates many routine and error-prone coding and data entry tasks and enables OSRL to maintain the highest standards of quality control. Interviewer training is a key aspect of quality control at OSRL. We employ only highly trained, skilled and motivated interviewers. Interviewer training begins with an extensive program of general interviewing skills, neutral probing, bias-free responses, telephone etiquette, practice interviews, role-playing and testing by supervisors. We also completely train and test interviewers in WinCATI so that interviewers and the data collection system work together flawlessly. General training is followed by several hours of project-specific training for each survey. Project-specific training includes an overview of the project goals and sample, unusual features of the study, respondents? commonly-asked questions revealed in pretesting and interviewers? scripted responses, as well as role-playing using both paper and WinCATI versions of the survey. Finally, at the beginning of each interviewing shift, OSRL?s Interviewer Supervisors hold a 5-10 minute mini-training to review and refresh interviewing techniques and to address any new developments in the survey process. Project management and supervision is another key element to OSRL?s quality control. OSRL Supervisors continuously monitor the interface between respondents, interviewers, and the computer systems that record respondents? answers. Interviewers are routinely evaluated, tested and provided with constructive criticism. Interviewers are provided pre- scripted answers to respondents? common objections or questions as part of their training, 1 Response rate was calculated in following manner. Completed interview / (Eligible sample + ((Eligible sample / (Eligible sample + Ineligible sample)) * Sample with unknown status)) Oregon Survey Research Laboratory Page 5 University Health Center Survey 2002 Survey Methodology but supervisors also are always available to help should the need arise. Finally, OSRL?s laboratory setting has created a valuable sense of teamwork among our interviewers, which in turn provides peer monitoring and mutual helpfulness dedicated to quality. DATA REDUCTION AND CODING After the completion of data collection,, a Project Director transferred the raw data from WinCATI into SPSS and Excel with appropriate variable and value labels, and made data corrections recorded by interviewers. This instrument included several open-ended survey questions. OSRL Interviewers are trained to record open-ended responses exactly as spoken by respondents, word for word. After the completion of data collection, highly trained and experienced coders transformed these open-ended responses into numerical categories to aid survey analysis. The coded open-ended answers are then be merged with the survey database.2 Coders used the open-ended code categories developed in previous years for this year?s study to maximize compatibility with the previous study results. Only very slight modifications to the code categories were required to accommodate this year?s results. Section 7 provides these answer categories, along with the percentage distributions of answers. 2 Due to unavoidable emergency circumstances, the merging of open-ended codes with the database will be accomplished later in summer 2002 and OSRL will send UHC fresh copies of the datasets.