MEASURING FACULTY ATTITUDES AND PERCEPTIONS TOWARD DISABLITY AT A FOUR-YEAR UNIVERSITY: A VALIDITY STUDY by ALLISON LOMBARDI A DISSERTATION Presented to the Department of Special Education and Clinical Sciences and the Graduate School of the University of Oregon in partial fulfillment of the requirements for the degree of Doctor ofPhilosophy March 2010 11 University of Oregon Graduate School Confirmation of Approval and Acceptance of Dissertation prepared by: Allison Lombardi Title: "Measuring Faculty Attitudes and Perceptions toward Disability at a Four-year University: A Validity Study" This dissertation has been accepted and approved in partial fulfillment of the requirements for the Doctor of Philosophy degree in the Department of Special Education and Clinical Sciences by: K Brigid Flannery, Chairperson, Special Education and Clinical Sciences Deborah Olson, Member, Special Education and Clinical Sciences Joe Stevens, Member, Educational Methodology, Policy, and Leadership Douglas Blandy, Outside Member, Arts and Administration and Richard Linton, Vice President for Research and Graduate Studies/Dean of the Graduate School for the University of Oregon. March 20, 2010 Original approval signatures are on file with the Graduate School and the University of Oregon Libraries. © 2010 Allison Renee Lombardi 111 Allison Renee Lombardi An Abstract of the Dissertation of for the degree of iv Doctor of Philosophy in the Department of Special Education and Clinical Sciences to be taken March 2010 Title: MEASURING FACULTY ATTITUDES AND PERCEPTIONS TOWARD DISABILITY AT A FOUR-YEAR UNIVERSITY: A VALIDITY STUDY Approved: _ Dr. K. Brigid Flannery The primary purpose of this study was to develop and field test the ExCEL survey instrument, intended to measure faculty attitudes toward and perceptions about students with disabilities at four-year colleges and universities. A secondary purpose was to examine how demographic characteristics of the sample and prior disability-focused training experiences influenced these attitudes and perceptions. A cross validation study was conducted using exploratory and confirmatory factor analysis. The exploratory factor analysis (EFA) resulted in an eight factor structure: Knowledge of Disability Law, Fairness, Invitation to Disclose, Willingness to Invest Time, Accessibility of Course Materials, Willingness to Adjust Course Assignments, Satisfaction with Campus Resources, and Willingness to Make Testing Accommodations. The confirmatory factor analysis (CFA) was intended to confirm the factor structure but resulted in an inadmissible vsolution, potentially due to model misspecification. These findings show initial reliability and validity of the ExCEL survey and suggest item development as a crucial next step in the development process. Follow-up analyses showed minimal differences between faculty according to gender but significant differences according to faculty rank (e.g., tenured, tenure-track, and non-tenure) and department. In addition, group differences emerged according to prior disability-focused training, where faculty who had previous disability- focused training had greater positive attitudes and perceptions toward students with disabilities than did faculty without such training. These results will inform the field about important constructs to consider in instrument development for measuring faculty attitudes and perceptions toward disability, including knowledge of legal mandates and accommodations and lmowledge of and willingness to adjust instruction according to the tenets of Universal Design. As well, the findings will inform four-year college and university administrations when planning targeted interventions for faculty members in disability awareness training. VI CURRICULUM VITAE NAME OF AUTHOR: Allison R. Lombardi PLACE OF BIRTH: Pittsburgh, Pennsylvania DATE OF BIRTH: November 1, 1977 GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon, Eugene University of California, Berkeley DEGREES AWARDED: Doctor ofPhilosophy, Special Education and Clinical Sciences, 2010, University of Oregon Master of Arts, Education, 2003, University of California, Berkeley Bachelor of Arts, 2000, University of California, Berkeley AREAS OF SPECIAL INTEREST: Secondary to Postsecondary Transition Quantitative Measurement PROFESSIONAL EXPERIENCE: Research assistant, Secondary Special Education and Transition, University of Oregon, Eugene, 2008-present Research specialist, Educational Policy Improvement Center, Eugene, Oregon, 2007-present Learning specialist, Services for Student Athletes, University of Oregon, Eugene, 2004-2007 VB GRANTS, AWARDS AND HONORS: Demonstration Projects to Ensure Students with Disabilities Receive a Quality Higher Education, Office ofPostsecondary Education, 2008 PUBLICATIONS: Murray, C., Lombardi, A., & Wren. C. (in press). The effects of disability-focused training on the attitudes and perceptions of university staff. Remedial and Special Education. Murray, C., Lombardi, A., Wren, C. , & Keys, C. (2009). Associations between prior disability-focused training and disability-related attitudes and perceptions among university faculty. Learning Disability Quarterly, 32(2), 89-100. Lombardi, A. (2008). "Student Athlete Learning Issues" chapter in NACADA Monograph Series #18 Advising Student Athletes: A Collaborative Approach to Success. Vlll TABLE OF CONTENTS Chapter Page I. LITERATURE REVIEW 1 Statement of the Problem 1 The Prevalence of Students with Disabilities at Four-year Colleges and 2 Universities . Systemic Differences between High School and College......... 4 Differences in Legal Rights 4 Instructional Climate.................. 5 Student Perceptions of Faculty Attitudes....................................................... 6 Inclusive Instructional Practices 8 Faculty Attitudes and Perceptions 9 Type of Accommodation Request 12 Faculty Training and Professional Development.. 13 Universal Design......................................................................................... 14 Summary of the Problem 16 Measuring Faculty Attitudes and Perceptions toward Disability........... 18 Establishing Validity...................................................................................... 22 Factor Analysis 23 Research Questions........................................................................................ 25 II. NIETHODS........................................................................................................... 26 Instrument......................................................................................................... 26 Participants........................................................................................................ 28 Procedures......................................................................................................... 29 Sample and Response Rate 30 Data Analyses 33 IX Chapter Page III. RESlTLTS 37 Analyses............................................................................................................. 37 Exploratory Factor Analysis 39 Descriptive Statistics and Factor Reliability............ 51 Confirmatory Factor Analysis........................................................................ 53 Group Comparisons 56 IV. DISCUSSION..................................................................................................... 68 Factor Structure 68 Willingness to Accommodate 70 Universal Design............................................................................................ 73 Reliability 75 Validity............................................................................................................... 75 Group Differences.............................................................................................. 77 Limitations..... 82 Implications for Practice..... 84 APPENDICES 87 A. EXCEL SURVEY 87 B. FIRST RECRUITMENT EMAIL TO FACULTY PARTICIPANTS 96 C. LETTER OF CONSENT TO PARTICIPATE: FACULTy......................... 98 D. SECOND EMAIL TO FACULTYPARTICIPANTS 100 E. THIRD EMAIL TO FACULTY PARTICIPANTS....................................... 101 F. FOURTH EMAIL TO FACULTY PARTICIPANTS 102 G. SEClTRITY QUESTIONS..... 103 H. ITEM MEANS AND STANDARD DEVIATIONS..................................... 104 1. ITEM CORRELATION MATRIX. 106 REFERENCES 109 LIST OF TABLES Table Page 1. Seven Principles of Universal Design 16 2. Sample Characteristics and Response Rates ofFaculty by College Membership................................................................................................ 31 3. Communalities of ExCEL Survey Items 40 4. Total Variance Explained ofUnrotated Eigenvalues and Oblique Rotation Eigenvalues.................................................................................. 43 5. Pattern Coefficients Resulting from EFA with PAF and Oblique Rotation.. 45 6. Summary of Factor Intercorrelations 52 7. Descriptive Statistics and Reliability Coefficients for Full ExCEL Survey and Subscales................ 53 8. Standardized Discriminant Function Coefficients (SDFCs) and Structure Coefficients (SCs) for Each Subscale by Teaching Status 58 9. Standardized Discriminant Function Coefficients (SDFCs) and Structure Coefficients (SCs) for Each Subscale by the Interaction of Gender by Teaching Status.................................................. 60 10. Standardized Discriminant Function Coefficients (SDFCs) and Structure Coefficients (SCs) for Each Subscale by College Membership 64 11. Standardized Discriminant Function Coefficients (SDFCs) and Structure Coefficients (SCs) for Each Subscale by Prior Training 67 x Xl LIST OF FIGURES m~re p~ 1. Representation of all possible faculty participants. 31 2. Representation of faculty sample included in study..................................... 32 3. CFA model of ExCEL survey instrument..................................................... 55 1CHAPTER I LITERATURE REVIEW Statement of the Problem College student diversity has dramatically increased over recent decades. Shifts in student demographics continue, resulting notably in an increase in enrollment for students with disabilities (Scott, McGuire & Shaw, 2003). Over time this population has continued to increase on four-year college and university campuses nationwide (Mull, Sitlington, & Alper, 2001; U.S. Department of Education, 1999), as students with disabilities now represent approximately nine percent of the total college population (Hom, Peter, & Rooney, 2002). As a result, there are new challenges for postsecondary students and faculty alike, particularly regarding accommodation requests for students with either physical or learning disabilities. Despite these challenges, it is important to increase the number of students with disabilities attending and completing four-year colleges and universities in order to ensure that all students not only have an equal opportunity to a quality higher education but are equipped to compete in an advanced and global economy. A reported barrier to postsecondary success of students with disabilities is negative faculty attitudes and perceptions about disability (Leyser, Vogel, Wyland, Brulle, 1998; Reed, Lund-Lucas & O'Rourke, 2003; Vogel, Leyser, Wyland & Brulle, 1999). In ranking students with and without disabilities according to qualification for 2admission into various postsecondary institutions, more students with disabilities were less qualified for admittance than their non-disabled counterparts (U.S. Department of Education, 1999). Because of this, faculty members might assume all students with disabilities do not meet the minimum standard admission guidelines for the institution and therefore are not as qualified as their non-disabled peers. This assumption may lead to discriminatory attitudes and behaviors of faculty members when approached with accommodation requests, attitudes that directly impact the retention rates of students with disabilities. Four-year colleges and universities should assess their faculty members' attitudes and perceptions of disability in order to determine targeted interventions for professional development training in disability awareness. In order to do this, a reliable and valid measure must be developed that adequately captures the constructs associated with faculty attitudes and perceptions toward students with both physical and learning disabilities. The prevalence of students with disabilities at four-year colleges and universities. Although enrollment rates of students with disabilities have increased over time, discrepancies persist between students with and without disabilities. For example, students with disabilities are less likely than their non-disabled peers to attend a 4-year college or university after completing high school. Postsecondary enrollment and completion rate discrepancies exist for students with both learning and physical disabilities (U.S. Department of Education, 1999). Murray, Goldstein, Nourse, and Edgar (2000) found that only 9% of young adults with learning disabilities had attended four-year colleges or universities five years after 3high school graduation; in comparison, 62% of young adults without learning disabilities continued their education at the same type of postsecondary institutions. Further, only 2% of the adults with learning disabilities had graduated from four-year postsecondary institutions a full ten years following high school graduation, whereas 46% of the nondisabled comparison group had completed undergraduate programs (Murray et aI., 2000). More recently, Wagner, Newman, Cameto, Levine, and Garza (2006) found that 7% of students with learning disabilities reported future plans to complete a four-year degree, and approximately 3 in 10 out-of-school youth with either physical or learning disabilities have enrolled in some type of postsecondary education, as compared to a 41 % postsecondary enrollment rate of their non-disabled peers. Further, Wagner, Newman, Cameto, Levine, and Marder (2007) found that although most high school students with either physical or learning disabilities reported feeling confident they would earn their diploma, they felt less confident that they would attend some level of postsecondary schooling. These findings show that both students with physical and/or learning disabilities are a minority population on four-year college and university campuses. There is evidence that students with learning disabilities are less likely to attend four-year colleges and universities than students with Speech Language Impairments, Hearing Impairments, Visual Impairments, and Orthopedic Impairments (U. S. Department of Health and Human Services, 2003; U.S. Department of Labor, 2007; Wagner et aI., 2006). Although these findings suggest students with physical disabilities are more likely to pursue and complete a postsecondary degree program than students 4with learning disabilities, regardless of disability type, discrepancies persist between the postsecondary enrollment and completion rates of students with and without disabilities. Together, these findings support why students with disabilities continue to be an underrepresented group at four-year college and university settings. Systemic differences between high school and college. Due to differences between IDEA and Section 504 of the Rehabilitation Act of 1973 and the Americans with Disabilities Act, students with disabilities may experience a rough transition particularly in the areas oflegal rights, curriculum, and instructor training (Scott et aI., 2003). While many high school students with disabilities report that they intend to pursue some type of employment, fewer report that they will pursue postsecondary schooling, particularly at a 4-year college or university (Wagner et aI., 2006). Students' lack of confidence to continue their education may be the result of two major differences between K-12 and postsecondary settings: legal rights and instructional climate. Differences in legal rights. Stark differences in support services between K-12 and postsecondary settings are particularly true for students with learning disabilities. For example, postsecondary students with disabilities are not legally entitled to specialized instruction, as they may be at the secondary level. Instead, college students with disabilities are fully integrated in the general student population, and are expected to self-advocate for their needs. Further, K-12 curricula are more likely to be formed based on federal and state standards and policy mandates, and therefore more standardized and consistent within states. Postsecondary curricula encompass a wide range of information and objectives, and are not subject to state standards (Scott et aI., 2003). This lack of 5standardization allows college faculty members a vast amount of freedom when curriculum planning. Too often, this freedom leads faculty members to adopt instructional methods that are more similar to their own learning preferences, and less responsive to student needs (Ouellett, 2004). Although differences between the two systems, and their subsequent legal mandates, obviously exist, systemic differences are rarely considered when rationalizing the discrepancy in postsecondary achievement between students with and without disabilities. Typically, blame is placed on the individual student's lack of preparation rather than acknowledgment of systemic differences. Further study of the discrepancies between students with disabilities and their non-disabled peers at four-year college settings is warranted, particularly from a systems-level perspective. Instructional climate. Federal and state regulations and laws are not the only differences between the K-12 and postsecondary systems. It is common for most postsecondary faculty to have expertise in specialized content areas, not in effective pedagogical methods (Ouellett, 2004; Scott et aI., 2003). In addition, there are typically little to no required training programs on disability awareness. While some voluntary training programs exist, published descriptions of these efforts are less common (Murray, Wren, Stevens, & Keys, 2009). As a result, faculty members are often unaware of acceptable accommodation requests and effective instructional practices that can enhance the learning environment for all students, including students with disabilities (Burgstahler, Duclos, & Turcotte, 1999; Ouellet, 2004). 6At the same time, effective teaching has become an area of professorial merit. It is becoming more common for faculty members to undergo perfoffi1ance reviews that include the quality of instruction delivered to students (Scott et aI., 2003). This shift is perhaps the result of the increasing perception of college students as consumers of education. Therefore, it is in the best interest of postsecondary faculty members to participate in professional development, particularly in the area of disability awareness and the use of inclusive instructional principles that will benefit all students. However, professional development interventions are difficult to plan and implement without an initial assessment of the faculty attitudes and perceptions toward students with disabilities at a given university. Student perceptions of faculty attitudes. Students with disabilities enrolled in colleges and universities have expressed that faculty attitudes toward individuals with disabilities and their understanding of disabilities has a direct impact on their motivation and determination to succeed at the postsecondary level (Dowrick, Anderson, Heyer, & Acosta, 2005; Farone, Hall, & Costello, 1998; Hartman-Hall & Haaga, 2002; Parker, Embry, Scott, & McGuire, 2003). Research findings from the students' perspective indicate that some faculty, staff, and administrators lacked information on disability rights, generally displayed poor attitudes toward students with disabilities, and did not always show willingness to accommodate (Dowrick et aI., 2005; Farone et aI., 1998). In a study by Hartman-Hall and Haaga (2002), students were asked to react to hypothetical scenarios that involved faculty expressing positive and negative attitudes toward accommodation requests. Findings show that negative responses from faculty -------- ----- 7 influenced students' decisions in terms of further pursuing support for their disability; in other words, students were more likely to shy away from disability support if they experienced negative responses from faculty members when approaching them with accommodation requests. However, positive responses to accommodation requests from faculty members led students to further pursue and utilize support services (Hartman-Hall & Haaga, 2002). In this case, faculty attitudes had a direct impact on students' motivation to pursue their degrees, as faculty attitudes directly influenced students' decision to seek out and utilize support services. These findings suggest that faculty attitudes greatly influence retention rates of students with disabilities. Dowrick et aI., (2005) found that students with disabilities experienced difficulty obtaining basic accommodations and supports in various postsecondary settings. Using focus groups, samples of students with disabilities from varying postsecondary settings were gathered, including large four-year institutions and two-year community colleges. These institutions represented different geographic locations within the United States, including large urban, suburban, and rural areas. A major finding was the students' perception that disability policy often does not lead to practice; in short, legislation does not ensure that students will receive the accommodations they rightfully deserve. Students expressed the need to self-advocate for basic accommodations and described encounters with faculty members unwilling to accommodate or unknowing of the disability law. Conversely, students reported faculty mentor relationships were among the most valuable types of support. 8Inclusive instructional practices. Much of the research has addressed student perceptions of faculty members' willingness to provide requested accommodations. However, when asked, students reported that a barrier to learning (and potentially retention) was more about the instructional practices of faculty members, and less about their willingness to accommodate (Madaus, Scott & McGuire, 2003a). Madaus, Scott, and McGuire (2003a, 2003b) conducted a study focused on student perspectives of effective instructional practices. Students were asked to identify instructional strategies that had positive effects on them in their college courses, as well as barriers that had negative effects. Findings show that students feel most successful in courses where clear, consistent expectations are set from the beginning of the course, learning was treated as a process, and a variety of instructional strategies were employed by the professor. In contrast, students reported barriers to learning when professors gave inconsistent assignments or exams related to material presented in lecture (Madaus et aI., 2003a). Importantly, these aspects are not specific to accommodation requests; in other words, clear and consistent expectations, and the use of various instructional practices will benefit all students, not only students with disabilities. Further, the study found the main difference between positive and negative experiences of students with disabilities in college courses was dependent on the attitudes of individual professors (Madaus et aI., 2003a; 2003b). Professors who promoted positive learning, were approachable and available, were consistent in communication of course material and expectations, used engaging instructional strategies (e.g. hands-on and/or small group activities, scaffolded assignments), made connections to prior 9knowledge or other, more understandable sources, challenged students to think and learn differently, and positively recognized individuality within students (Madaus et aI., 2003b). All of these qualities are centered on inclusive instructional practices and are not necessarily based on accommodations. While some may be linked to accommodations, it is important to recognize that none of these instructional characteristics are actually accommodations. Further, all of these qualities could be included as components to professional development training for faculty. Evidence shows a primary reason for the low retention rates of postsecondary students with disabilities is negative attitudes expressed by faculty members (Beilke & Yssel, 1999; Deshler, Ellis, & Lenz, 1996). These findings indicate that discriminatory faculty attitudes and assumptions about students with disabilities are a major barrier to student persistence and retention in four-year colleges and universities. Therefore, it is imperative to further study faculty attitudes toward and perceptions of students with disabilities in order to improve retention of students with disabilities at four-year colleges and universities. In particular, an examination of faculty willingness to provide requested accommodations and use inclusive instructional practices is warranted. Faculty attitudes and perceptions. Faculty attitudes and perceptions toward students with disabilities make a difference because they directly affect the campus climate and personal experiences of all students. As more students with disabilities enroll in four-year colleges and universities, it is becoming increasingly urgent to examine these attitudes in order to design and implement effective interventions that will build supportive campus climates for all students. 10 Research findings show mixed results of faculty attitudes toward and perceptions of students with disabilities. Some faculty, staff, and administrators have negative attitudes and/or lack knowledge about the characteristics of students with disabilities (Leyser et aI., 1998; Reed et aI., 2003; Vogel et aI., 1999). Parker, Embry, Scott, and McGuire (2003) conducted a study in which disability services personnel were asked to describe typical questions from faculty members about teaching students with disabilities. Responses fell into six major categories of concern: (a) how students with disabilities could be "otherwise qualified" to participate in higher education; (b) providing effective instruction to students with disabilities; (c) providing accommodations to students with disabilities; (d) maintaining academic standards when students with disabilities participate in higher education, including issues of fairness; (e) characteristics of students with disabilities; and (f) compliance with legal obligations to assure equal access to higher education for students with disabilities. Further, study participants represented a range of faculty knowledge and attitudes about students with disabilities. Some faculty members had no knowledge of testing or teaching accommodations and were not aware of existing disability legislation; whereas other faculty members expressed a tendency to "over-accommodate", asking the disability services personnel what he/she needs to do to get the student to "pass" (Parker et aI., 2003, p. 7). Much of the research on faculty attitudes has focused on accommodation requests from students with disabilities. Vogel et al (1999) found that faculty attitudes can be negative depending on the type of accommodation request. For example, faculty responded they were more willing to provide teaching accommodations than exam 11 accommodations, and less willing to provide supplementary materials, such as lecture outlines or alternative assignments. Faculty members indicated they were most willing to allow students to record lectures, and to allow extended exam time and to take proctored exams in another location (such as an Office of Disability Services), but they were least willing to alter the format of exams (Vogel et aI., 1999). A study by Leyser et ai. (1998) showed similar results, as well as revealed differences in attitudes and perceptions according to other variables such as gender, personal experience, faculty rank and departmental affiliation. Further, and consistent with previous research, Reed et aI., (2003) found that faculty showed less support for accommodations and support services that required additional funding, staffing, and resources. Other findings indicate that faculty members generally have positive attitudes toward granting accommodation requests to students with disabilities (Murray, Wren, & Keys, 2008; Skinner, 2007). In these studies, faculty willingness to accommodate correlated with several variables, including gender, personal experience, faculty rank, and departmental affiliation. Murray, Wren and Keys (2008) found that faculty members are generally willing to make accommodations for students with disabilities, and that faculty perceptions differ according to faculty rank, gender, and departmental affiliation. Skilmer (2007) found that faculty willingness to provide accommodations varied according to departmental affiliation, faculty rank, and type of requested accommodation. Vogel et ai. (1999) found that many faculty members had limited contact and experience with students with disabilities in their classes, as well as limited training and knowledge of disability legislation. Despite these limitations, most faculty members stated they were 12 willing to make accommodations for students with disabilities as requests arose, and many reported they had previously made such accommodations (Vogel et al. 1999). Although it is important to consider the mixed results in the research findings - some studies show negative attitudes and some generally more positive attitudes toward granting accommodation requests for students with disabilities - it is also necessary to further examine the differences in terms of departmental affiliation, gender, faculty rank, and type of requested accommodation. As well, it is important to consider the effect of professional development training on faculty attitudes and perceptions toward disability, and confirm whether such training efforts can alter instructional practices to improve the experiences of students with disabilities in postsecondary settings. Type ofaccommodation request. Disability type (physical vs. learning disability) may influence faculty member attitudes toward students requesting accommodations. Some studies show that faculty members will reasonably accommodate students with physical disabilities (Beilke & Ysse1, 1999; Blaqua, Rapaport, & Kruse, 1996; Nelson, Dodd, & Smith, 1990). Oftentimes, these types of accommodations include ensuring accessibility of buildings, classrooms, and general facilities on campus. However, students with learning disabilities require very different accommodations; typically, these students request exam and teaching accommodations, such as extended exam time, a private exam room, tape-recorded lectures, outlines oflecture notes, and even altered exam and/or assignment formats. There is evidence that faculty show prejudicial attitudes towards students with non-visible disabilities and may even, at times, suspect 13 them of "using their disability as a way to gain preferential treatment" (Beilke & Yssel, 1999, p. 365). More research is needed to further explore differences in disability type and the effects on faculty willingness to make accommodations. Perhaps, as noted above, faculty are more willing, in general, to make accommodations for physically disabled students, and less willing to make accommodations for learning disabled students. In particular, an instrument is needed that includes distinct items measuring faculty willingness to accommodate specific types of requests. For example, faculty should be asked if they would agree to provide lecture materials in large print for visually impaired students, as well as if they would agree to provide extended exam time to a learning disabled student. Some faculty members may agree to the former accommodation request but disagree with the latter, and vice versa. A valid and reliable instrument that makes these distinctions between accommodations associated with physical or learning disabilities will help us further understand the nature of faculty attitudes and perceptions toward accommodation requests. Faculty training and professional development. Faculty attitudes and perceptions are influenced by the amount of information they learn about students with disabilities (Murray, Lombardi, Wren, & Keys, 2009). Typically, faculty may not know much about the rights of students with disabilities, including the legislation of Section 504 and the Americans with Disabilities Act. Many faculty members rely on the campus Office ofDisability Services (ODS) to provide verification as to whether a student is required to receive accommodations due to a disability. Some may argue this is precisely 14 the role of ODS; others may make the case that faculty ought to become familiar with the law, and at the very least, keep up with recent rulings of federal courts in order to make the most appropriate accommodation decisions for students in their classrooms (Cope, 2005). In addition, faculty training can serve to foster a solid working relationship between ODS and faculty, where clear lines of communication are established and faculty then feel comfortable using ODS as a resource from semester to semester. Finally, faculty training can increase knowledge of disabilities in general, as some faculty members have had little to no previous experience with disability in their academic careers. Regardless, knowledge and awareness of disability law have become a necessary training component for university faculty at four-year college campuses due to the rising enrollment of students with disabilities. Possibly, with more training in instructional strategies, faculty attitudes toward students with disabilities can become more positive. If this is the case, training programs play an integral role in enhancing the quality of instruction of postsecondary faculty and benefitting all students, including students with disabilities. The outcome could be a more positive postsecondary learning experience and greater success in postsecondary degree programs for students with disabilities. Universal design. Some postsecondary institutions have used the concept of Universal Design (UD) to spearhead faculty training programs. This framework is used as an instructional backbone to faculty professional development training. UD principles help encourage faculty members to utilize and embed more inclusive instructional practices into their course(s). As previously stated, research has shown that students feel 15 most successful in courses where clear, consistent expectations are set from the beginning of the course, learning was treated as a process, and a variety of instructional strategies were employed by the professor (Madaus et aI., 2003a). UD principles may help encourage faculty members to utilize these aspects. There are seven principles ofUD that were developed by the North Carolina State University Center for Universal Design, a group of architects, product designers, engineers, and environmental design researchers (Connell et aI, 1997). The principles were meant to provide guidance in the usability of environments for a wide range of learners. The seven principles are listed in Table 1. Although, little empirical evidence exists to support the effectiveness of Universal Design (UD), as it is a rather new concept in the field of postsecondary faculty and support for students with disabilities, the framework has the potential to influence instructors to use inclusive teaching methods that reach a broad range of learners. There is an underlying connection between UD and disability accommodations; both encourage accessibility to more diverse groups of people. While UD is not as specific as accommodations, it is likely that if the UD principles are followed, the need for accommodations may decrease (Ketterlin-Geller & Johnstone,2007). In other words, by encouraging faculty to embed the UD principles within their course framework, they may experience a decrease of specific accommodations from students with disabilities. In essence, the accommodations would come naturally along with the inclusive learning environment promoted by UD. 16 Table 1 Seven Principles ofUniversal Design Principle Equitable use Flexibility in use Simple and intuitive use Perceptible information Tolerance for error Low physical effort Size and space for approach and use Definition The design is useful and marketable to people with diverse abilities. The design accommodates a wide range of individual preferences and abilities. Use of the design is easy to understand, regardless of the user's experience, knowledge, language skills, or current concentration level. The design communicates necessary information effectively to the user, regardless of ambient conditions or the user's sensory abilities. The design minimizes hazards and the adverse consequences of accidental or unintended actions. The design can be used efficiently, comfortably, and with a minimum of fatigue. Appropriate size and space is provided for approach, reach, manipulation, and use regardless of the user's body size, posture, or mobility. Summary of the problem. The number of students with disabilities entering postsecondary institutions continues to increase. As this population grows, it is imperative that retention rates improve so that students with disabilities receive a quality higher education and have equal opportunities entering the work force as adults. Although many factors could be identified as barriers to the success of students with disabilities at four-year colleges and universities, one known barrier is negative faculty attitudes and perceptions of disability, which is often specifically related to 17 accommodation requests (Leyser et aI., 1998; Reed et aI., 2003; Vogel et aI., 1999). Some research exists regarding the prevalence of negative and positive attitudes toward students with either physical or learning disabilities (Leyser et aI., 1998; Reed et aI., 2003; Vogel et aI., 1999), and some research exists about the differences in faculty attitudes and perceptions of disability type and willingness to provide requested accommodations (U. S. Department of Health and Human Services, 2003; U.S. Department of Labor, 2007; Wagner et aI., 2005). There is some correlational evidence linking faculty attitudes to other variables such as gender, faculty rank:, and departmental affiliation (Murray et aI., 2008; Skinner, 2007). Finally, there is evidence that faculty members who receive training in disability awareness are more likely to demonstrate positive attitudes and perceptions toward students with disabilities (Murray et aI., 2009). Possibly, training faculty members in disability awareness and inclusive instructional principles, promoted by such frameworks as Universal Design, may help to create more positive attitudes and perceptions of students with disabilities as well as enhance instructional practices that reach a broad range of student learners. Training should also include a general overview of disabilities and specifics about the role of ODS, including strategies for faculty to foster collaborative working relationships with ODS staff. If faculty training opportunities are proven to be effective in influencing positive attitudes then appropriate training interventions should be designed and implemented. Despite increased understanding about university faculty attitudes and perceptions towards students with disabilities, prior research suffers from several measurement and 18 methodological limitations. For example, many researchers have not used reliable and valid measures of faculty attitudes and perceptions when assessing these perceptions and most analyses focus on item level results rather than reliable latent constructs. Thus, there is a need to develop and field test reliable and valid measures of faculty attitudes and perceptions towards university students with disabilities. Measuring Faculty Attitudes and Perceptions toward Disability Among the research studies previously described, many different methods were employed to measure faculty attitudes and perceptions. This methodological variation indicates a lack of measurement consistency across four-year college and university settings. In order to further investigate the role of faculty attitudes on the enrollment and retention rates of students with disabilities at four-year colleges and universities, a reliable and valid instrument should be developed. The instrument must measure the identified constructs, based on the literature, that capture faculty attitudes and perceptions. Also, this instrument should measure the effects of training on faculty attitudes so that group comparisons can be made between faculty who have and have not received training. Once developed and validated, the survey instrument can be used across four-year college and university settings to give administrators a sense of where and how to target interventions for faculty professional development in the area of disability awareness. Some research studies have examined existing measures of attitudes toward students with disabilities. Not all studies addressed validity and reliability of the specified measure in the study (e.g., Chubon, 1992; Rao, 2004), such as the Attitudes 19 toward Disabled Persons Scale (ATDP) by Yuker, Block, and Campbell (1960). Developed in the late 1950's, the ATDP is a measure of attitudes toward persons with physical disabilities. Although the measure is over 40 years old, recent research studies have used the instrument and found evidence of adequate reliability, where Cronbach's a is equal to or greater than .80 (Rao, 2002). An instrument developed by Askamit, Morris, and Luenberger (1987) included subscale items that measured comfort levels when working with students with learning disabilities, reactions to curricular and instructional accommodations, perception of success rates of students with disabilities, familiarity with disability legislation, characteristics of leaming disabilities, support services available on campus, and ability to respond to the needs of students with disabilities (as cited in Rao, 2002). A Cronbach's a of .82 was reported for the attitude subscale. Although this measure provides evidence of reliability, it is limited because it only addresses learning disabilities. Ideally, a measure would include items that measure attitudes toward learning and physical disabilities so that comparisons can be made as to whether faculty members treat various accommodation requests in the same manner. Rao (2002) conducted a literature review of studies on faculty attitudes of students with disabilities in higher education, in which attitudinal measures were included. Common variables that were included across studies were (a) gender, (b) age, (c) experience, (d) rank, (e) departmental affiliation, (f) knowledge of disability laws, and (g) disability type (Rao, 2002). Rao found that gender had an effect on willingness to accommodate in five research studies included in the literature review, and no effect on 20 willingness in four studies. Only one study (Fonosch & Schwab, 1981) showed faculty rank as a significant factor in faculty willingness to accommodate; other studies showed no difference in faculty rank (e.g. assistant, associate, or full professor, and instructor). Departmental affiliation, however, did make a difference in faculty attitudes in five of the studies, which included findings that faculty from "soft sciences" (e.g. sociology, psychology) were more likely to have positive attitudes than faculty from the "hard sciences" (e.g. math, biology), and faculty from the Education Department tended to show more willingness to accommodate than other departments (Rao, 2002). Murray, Wren and Keys (2008) developed the Project PLUS survey to measure faculty attitudes and perceptions of students with learning disabilities. The 38-item instrument was given to faculty at a private Midwestern university located in an urban setting. An exploratory factor analysis yielded 12 reliable factors, which included (a) willingness to provide major accommodations, (b) willingness to provide exam accommodations, (c) fairness and sensitivity, (d) knowledge oflearning disabilities, (e) willingness to personally invest, (f) willingness to provide teaching accommodations, (g) resource constraints, (h) performance expectations, (i) disclosure and believability, CD inviting disclosure, (k) knowledge to make accommodations, and (1) providing accommodations. Although the 12 factors addressed a range of constructs, the instrument is limited to measuring faculty attitudes toward learning disabilities; willingness to provide accessibility-related accommodations is not included as a construct in the measure. 21 In general, these measures have some commonalities. Even though some measures focused on physical disabilities and some on learning disabilities, the measures had constructs in common, including (a) perceptions and knowledge of disability (visible and non-visible), (b) willingness to invest time supporting students with disabilities, (c) fairness and sensitivity, (d) performance expectations of students with disabilities, (e) knowledge of disability law, (f) willingness to provide teaching, exam, and accessibility accommodations, and (g) knowledge of support services and campus resources targeted toward students with disabilities. Further development of an instrument that measures faculty attitudes and perceptions should include items that capture these general constructs and address different disability types. More research is needed to further determine and develop adequate, reliable, and valid instruments. Since evidence exists that students with disabilities face additional challenges and low retention and completion rates at the postsecondary level, it is crucial to further explore the possible barriers to success faced by this population. Since faculty attitudes affect students with disabilities in their pursuit for a college degree, the ability to measure these attitudes will help colleges and universities pinpoint further interventions to solve the problem, including directed training and professional development efforts. Also, once training is implemented, measures ought to capture the effects of training on faculty attitudes and perceptions toward students with disabilities in order to evaluate interventions. 22 Establishing validity. An important step of developing an adequate survey instrument is establishing construct validity, which is the degree to which the instrument measures what it purports to measure (Haynes, Richard, & Kubany, 1995; Kane, 2001; Messick, 1980). Content validity, a facet of construct validity, centers on developing and refining the constructs of the instrument by reviewing the measure blueprint, specifications, and the measure itself, in order to provide evidence based on test content as well as contribute to improved clarity of the measure (AERA, APA, & NCME, 2009; Haynes, Richard, & Kubany, 1995). In this case, an adequate instrument that measures faculty attitudes should include items meant to target and measure specific underlying theoretical constructs. Specifically, the theoretical constructs include (a) perceptions and knowledge of disability (visible and non-visible), (b) willingness to invest time supporting students with disabilities, (c) fairness and sensitivity, (d) performance expectations of students with disabilities, (e) knowledge of disability law, (f) willingness to provide teaching, exam, and accessibility accommodations, and (g) knowledge of support services and campus resources targeted toward students with disabilities. An additional factor could be (h) willingness to utilize inclusive instructional practices, as Madaus et al. (2003a) found that students with disabilities cited the most success with these types of courses. These eight theoretical constructs must be observed and tested in order to determine ifthey adequately represent faculty attitudes. Validating these eight theoretical constructs through analysis of an assessment instrument is essentially the process of building construct validity. Most importantly, validation of the interpretation of the test results is crucial; not validation of 23 a test score itself, but the interpretation and utilization of that test score, and the decisions that will be made based on that test score that will affect future outcomes (Kane, 2001). In this case, the results will help inform postsecondary institutions of faculty attitudes and perceptions, and appropriate targeted interventions, such as professional development training in disability awareness, will be the outcome. In developing an assessment instrument, Clark and Watson (1995) provide recommendations for procedures to follow in establishing construct validity. These steps include (a) the identification of a theoretical model, (b) building substantive validity, which includes the development of an initial item pool, and (c) building structural validity, which includes item selection and psychometric evaluation (Clark & Watson, 1995). As previously mentioned, theoretical constructs have been identified after review of the literature, and item construction is based on eight hypothesized factors. A complete validity study of a survey instrument that measures faculty attitudes and perceptions will include the creation of an initial item pool, item selection, and psychometric evaluation using methods such as factor analysis. Factor analysis. In order to establish content and structural validity, the items that are meant to measure these constructs may be psychometrically evaluated with the methods exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). Applying such methods will confirm or disconfirm the instrument constructs, and inform research as to whether they need to be refined or revised (Clark & Watson, 1995). Most importantly, this process will help to ensure the instrument measures what it purports to measure, the basic definition of construct validity (Kane, 2001; Messick, 1980). Both 24 EFA and CFA are more robust that other methods (MANOVA, for example) because they allow the researcher to detemline goodness of model fit using theoretical constructs measured by a group of items, rather than one single item. Groups of items (called factors) are written to capture one holistic construct and then tested for validity in the factor analysis process. EFA helps detemline the nature oflatent variables that explain the variation and covariation in a set of measured variables by revealing the sources of common variation underlying the measured variables within the data (Preacher & MacCullum, 2003). The results of the EFA will determine the appropriate item groupings and develop the unknown constructs of the instrument that gauge attitudinal changes in faculty toward students with disabilities. It is imperative to adequately conduct the EFA on the survey instrument otherwise erroneous conclusions may be drawn regarding faculty attitudes about students with disabilities. Factor analysis can only be a useful tool if used correctly and appropriately; unfortunately some researchers end up using results foolishly by misinterpreting item groupings and misrepresenting constructs (Clark & Watson, 1995). In order to ensure adequate interpretation of item groupings and constructs, researchers must carefully consider (a) extraction, (b) the number of factors to retain, and (c) rotation (Costello & Osborne, 2005). Confirmatory factor analysis (CFA) is a method where the factor structure resulting from an EFA can be tested and confirmed. Researchers use CFA instead of EFA when there is a strong rationale or hypothesis as to why the items are grouped or correlated. Some studies will employ both EFA and CFA methods by splitting the 25 sample size in half and then assigning the first half to EFA and the second half to CFA (Johnson, Stevens & Zvoch, 2007: Johnson & Stevens, 2001). The factors extracted in the EFA can be confirmed in the CFA for the remaining portion of the sample. The CFA models are then estimated using structural equation modeling (SEM) methods. In measurement studies, SEM is more ideal than standard statistical procedures (such as ANOVA) because SEM involves a priori hypotheses and offers the convenience of differentiating between observed and latent variables (Kline, 1998). Because research exists on some measures of faculty attitudes and perceptions of students with disabilities, and preliminary constructs have been identified based on these pre-existing measures, this is a case where hypotheses have been made. It is appropriate, in this case, to employ both EFA and CFA because the constructs are still exploratory in nature and the use of both methodologies will offer more robust results. Research questions. The primary purpose of this investigation was to develop and field test a survey instrument intended to measure faculty attitudes and perceptions toward students with disabilities at four-year colleges and universities. A secondary purpose was to examine how demographic characteristics of the sample and prior training experiences influenced faculty attitudes and perceptions. The research questions were as follows: (a) Is there evidence of reliability and validity for the ExCEL survey? (b) Are there group differences by gender, faculty rank, and departmental affiliation? (c) Are there group differences by prior disability awareness training? 26 CHAPTER II METHODS This study explored the validity and reliability of the Expanding Cultural Awareness of Exceptional Learners (ExCEL) survey instrument in order to determine ifit can be used across four-year college and university settings. Along with evaluation of psychometric properties, group differences were examined according to gender, teaching status, and college affiliation. As well, group differences were examined to determine if training has an effect on faculty attitudes and perceptions toward disability. Instrument A pre-existing instrument, the PLUS survey, (Murray et aI., 2008) was adapted and revised into a new instrument, the Expanding Cultural Awareness ofExceptional Learners (ExCEL) survey. It is important to note that the PLUS survey was used to measure faculty attitudes and perceptions toward students with learning disabilities, whereas the ExCEL survey included items measuring faculty attitudes and perceptions toward all students with disabilities. Therefore, the initial item pool for the ExCEL survey included extracted items from the PLUS survey and newly drafted items based on the literature. The ExCEL survey is included in Appendix A. Important decisions were made during the development of the ExCEL survey. The original twelve reliable factors from the pre-existing PLUS survey were retained, but 27 some modifications were made. These changes were agreed upon after a consultation meeting with the Director of Disability Services. In an effort to create a more parsimonious instrument, three factors related to accommodations were collapsed into one "Willingness to Make Major Accommodations", "Willingness to Make Exam Accommodations", and "Willingness to Make Teaching Accommodations". One factor was added to the hypothesized list: Willingness to Create More Inclusive Learning Environments. Nine items were written to capture this construct. This decision was made in particular because Disability Services had been working to educate faculty members on the principles of Universal Design, and many of these principles include the concept of delivering instruction in an inclusive fashion. As such, faculty members have been encouraged to embed more inclusive instructional strategies into their course curricula. They are not required to do so, but Disability Services would like to have a sense of how many faculty members are willing to make these curricular changes. The instrument included 57 selected response questions with ordered response categories meant to measure level of agreement or opinion (Dillman, 2007). This format was chosen because the ExCEL survey is meant to uncover the attitudes and beliefs of the faculty members. By selecting a vague quantifier to encompass their opinion of a topic, participants may quickly respond to opinion questions on topics they mayor may not have previously considered (Patton, 1982). There were six response options - strongly agree, agree, somewhat agree, somewhat disagree, disagree, and strongly disagree. 28 Participants Participants included teaching faculty at a medium-sized public research university located in the Pacific Northwest. As of 2007, there were 1,714 faculty members and 1,267 graduate assistants at this university, a total of2,98l teaching faculty. Eighty-two percent of the faculty members were white, seven percent were Asian/Pacific Islander, three percent were Hispanic, one percent was African American, one percent was Native American, and one percent was Multi-ethnic. Five percent declined to report racial identity. Fifty-four percent of faculty members were male and forty-six percent were female. The average faculty-to-student ratio for the university was 1: 18. At this university, the number of students with disabilities has grown approximately 20% over the past five years. As of2007, there were 20,376 students emolled, including 763 students with disabilities who comprise 4% of the student population. Of the students with disabilities population, 63% are diagnosed with either a learning disability or Attention Deficit Hyperactivity Disorder. The remaining 37% are diagnosed with another disability type, such as mobility, hearing, visual, speech impairments, psychological or health disability, brain injury, or seizure. Due to budgetary constraints with the survey incentives, the decision was made to recruit only those faculty members employed at .5 full-time employment (FTE) or greater. This decision was made with the assumption that these faculty members would have the greatest impact on the students they teach. After selecting only those faculty members employed at .5 FTE or greater, the survey was sent to 1,084 faculty members. As a result, no graduate assistants were recruited for the study. 29 I met individually with each Dean from the eight separate colleges. After giving an overview of the project, I asked for their help in providing me with access to the email listserv used to send information to faculty members employed .5 FTE or greater within their respective college. After meeting with the Dean of the Law School and the Director of Disability Services, it was determined the Law School faculty would not receive the surveyor participate in the study. This decision was made because different procedures in providing accommodations to students with disabilities are followed at the Law School, and essentially the Disability Services office handles their requests in a much different manner. For example, Law School students do not submit any accommodation requests directly to faculty. Instead, Disability Services staff work with administrative staff at the Law School to accommodate students with disabilities. Because of these differences, many of the items on the survey would not be relevant to Law School faculty. After all meetings with the Deans occurred, the finalized participant population included faculty from all of the colleges (with the exception of the Law School) who were employed at .5 FTE or greater. This population included 1,084 potential participants. Procedures Faculty members received an email including an introductory recruitment letter that described the research project and a link to the ExCEL survey (see Appendix B), which was presented in an online format through Survey Monkey. They were asked to complete the survey on a voluntary basis. All participants were offered a $3 coupon to a popular local cafe and bakery, regardless ofwhether they completed the surveyor not. 30 Participants completed an online consent form prior to participating in the actual survey (see Appendix C). Ifparticipants selected "no" to the consent form, they were not allowed to advance to the beginning of the online survey. Participants were sent three follow-up emai1s encouraging them to take the survey after the initial email invitation was sent (see Appendices D, E, and F). These emai1s were sent two, four, and six weeks after the initial email invitation was sent, respectively. In order to track faculty responses over time, a series of security questions were added at the beginning of the survey (see Appendix G). Sample and Response Rate At the conclusion of the data collection period, the final sample consisted of 295 participants. The overall response rate was 27%. Although far from ideal, it is consistent with response rates of previous studies of similar nature involving university faculty and disability awareness (Bourke et al., 2000; Murray et al., 2008; Murray et al., 2009; Vogel et al., 1999, Vogel e1 al., 2008). The highest response rate was from the Honors College (71 %) although this particular college had the smallest population. The lowest response rate was from Journalism (18%). Table 2 lists the response rate by college membership. Figures 1 and 2 illustrate the differences between the population and representative sample. 31 Table 2 Sample Characteristics and Response Rates ofFaculty by College Membership All Faculty Sample Response Rate College N % N % Arts and Sciences 490 45% 137 47% 28% Architecture and Allied Arts 148 14% 32 11% 22% Business 81 7% 32 11% 40% Journalism 60 6% 11 4% 18% Honors 7 < 1% 5 1% 71% Education 243 22% 48 17% 20% Music and Dance 55 5% 25 9% 45% Total Participants III Arts dnd Sciences mAAA Business till Journdlism Honor:. Education Musicand Ddnce Figure 1. Representation of all possible faculty participants 32 Sample mil Arts and Sciences flJAAA fIJ Journalism Iionors EdLJl:ation Mw;ic and Dance Figure 2. Representation of faculty sample included in the study After subjective inspection of pie charts representing the population and sample, I determined, for the most part, the sample was representative of the faculty population, with some overrepresentation of Business, Honors College, and Music faculty. In addition to college membership, participants were asked to report gender, age, teaching status, years teaching, and type of courses taught. For gender, 44% ofthe faculty were male and 56% were female. The mean reported age was 48.3 (SD = 10.2). For teaching status, 64% of the faculty reported they were tenure-line, indicating they were tenured or on a tenure-track, whereas the remaining 36% reported they were nontenure, indicating they were nontenure track, visiting, or post-retire. The mean reported years teaching at the postsecondary level was 15.86 (SD = 10.1). When asked which type of courses they primarily taught, 55% of participants indicated a major- specific course, 25% indicated a graduate-level course, 12% indicated a general 33 education course, 6% indicated an elective course, and less than 1% indicated a lab or discussion section. Additionally, participants were asked a series of questions about prior experience in disability awareness and teaching students with disabilities. When asked how many students with disabilities they had taught in the past five years, 37% of participants reported 1-5 students, 22% reported 6-10 students, 12% reported 10-20, 12% reported more than 20, 13% reported "don't know or not sure", and 4% reported "none". Participants were asked if they had a disability, and 9% responded "yes"; meanwhile 49% indicated a family member, friend, or other personal contact had a disability, 72% indicated they had taught at least one student with a disability at some point in their teaching career, and 14% indicated they had no personal experiences with disability. Finally, participants were asked if they had ever received training related to disability or working with college students with disabilities, where 29% reported "Yes" and 71 % reported "No". Of the 29% who had received some type of disability-focused training (n = 86),64% reported they attended a workshop, 27% reported they took one or more courses, 60% reported they read books or articles, 38% reported they visited website(s), and 27% reported "other" (please note: participants were asked to check all that apply for type of training received). Data Analyses Examination of reliability and validity addressed research question one. Reliability was examined using Cronbach's u, a measure of internal consistency, on the entire instmment and within subscales. Criterion of.70 or higher was considered 34 adequate reliability, and .80 or higher was considered preferable (Nunnally, 1975). Validity was examined through the psychometric evaluation of the instrument using exploratory and confirmatory factor analysis. In exploratory factor analysis (EFA), there are three important decisions to consider: (a) the extraction method (b) the number of factors to retain, and (c) the rotation method to be utilized (Costello & Osborne, 2005; O'Connor, 2000; Preacher & MacCullum, 2003). In the first decision, Principal Axis Factoring (PAF) was selected as the extraction method. In the second decision, Kaiser's rule and scree plots were used to determine the number of factors to retain. Kaiser's rule states that only eigenvalues greater than 1.0 shall be retained, as these components tend to account for more variance of a single measured variable. As well, parallel analysis and Velicer's minimum average partial (MAP) test were utilized; these procedures have been validated, unlike Kaiser's rule and scree plots (O'Connor, 2000), and therefore using a combination of these procedures to determine the number of factors to retain is optimal. The third decision involved choosing between orthogonal and oblique rotation methods. Essentially, orthogonal rotation requires the factors to be uncorrelated, whereas oblique methods do not employ such a restriction. In this study, it was most appropriate to employ oblique rotation methods since this method allows the estimation of factor cOlTelations, and I suspected the constructs would be interrelated. In carrying out the psychometric evaluation of the instrument, there were two possible plans for data analyses presented in my proposed dissertation, which I referred to as Plan A and Plan B. Plan A was the ideal data analyses plan, and included conducting a 35 crossvalidation study with exploratory and confirmatory factor analysis. The sample was to be randomly split so that half was subj ect to EFA and the other half subj ect to the confirmatory factor analysis (CFA). This process would test the validity of the instrument and therefore address the second part of research question one. Research questions two and three would be addressed through invariance testing, where the factor structure of the instrument would be compared according to the stated predictor variables (gender, teaching status, college membership, previous training). Plan A was to be utilized ifthe participant response rate of the survey was reasonable and a minimum of a 2: 1 case-to- variable ratio was attained (Costello & Osborne, 2005; Fabrigar, Wegener, MacCallum, & Strahan, 1999; Stevens, 2002). Like Plan A, Plan B included exploratory factor analysis (EFA). However, Plan B was to be utilized if the desired 2: 1 case-to-variable ratio was not attained due to a low response rate, and the sample size was not large enough to conduct a CFA (Kline, 1998). Invariance testing is not possible without conducting a CFA, and therefore Plan B was to address research questions two and three by conducting Multivariate Analysis of Variance (MANOVA) to determine the group differences in responses on the factor scores (taken from the EFA results) according to the stated predictor variables (gender, teaching status, college membership, previous training). Essentially, since Plan B addressed the situation where sample sizes would be too small to conduct CFA, MANGVA was substituted for the group comparisons, and would address research questions two and three. 36 The sample size was large enough to conduct both an EFA and CFA; however, it was not large enough to conduct invariance testing. A minimum sample size of 200 is required per group to conduct invariance testing (Kline, 1998). Therefore, a combination of Plan A and B was followed, where the EFA and CFA were conducted and one-way between-subjects MANOVAs were conducted to examine group differences. In the MANOVAs, factor scores were the outcome variables and the categorical predictor variables were gender, faculty rank, college membership, and training status (whether or not the participant received previous training on disability awareness). In total, three MANOVAs were conducted. The a priori criterion for declaring statistical significance was .05. As previously stated, reliability was examined using Cronbach's a on the entire instrument and within subscales. Subscales were determined based on the EFA results, and the subscale scores were used as the outcome variables for the MANOVAs. The MANOVAs were conducted using the CFA sample (n = 200). 37 CHAPTER III RESULTS The purpose of this study was to test the initial reliability and validity of an instrument intended to measure the attitudes and perceptions of university faculty members toward disability. The measure was a 57-item survey that was designed with eight subscales. In this study, I sought to answer the following research questions: Research Question 1: Is there evidence of reliability and validity for the ExCEL survey? Research Question 2: Are there group differences by gender, faculty rank, and departmental affiliation? Research Question 3: Are there group differences according to the amount of disability awareness training received? Analyses As mentioned in the previous chapter, there were two potential data analysis plans outlined (Plan A and Plan B), and the final decision between them was dependent on sample size. The sample size was large enough for Plan A to be carried out; however, the sample size was not large enough to conduct invariance testing, which would have addressed research questions two and three in Plan A. Due to the insufficient sample size, multivariate analysis of variance (MANOVA) was utilized to examine group ------------- 38 differences in order to answer research questions two and three. It is important to note that while MANOVA is a robust methodology in testing for group differences, it is particularly different from invariance testing because means are compared in MANOVA, whereas with invariance testing the factor structure of the instrument is compared according to group membership. Therefore, invariance testing provides more information on the ways in which structure differs by group, such as correlations among factors, coefficients, and residuals) Two types of software were used for analyses, SPSS 15.0 (SPSS, Inc., 2006) and AMOS 7.0 (Arbuckle, 2006). Prior to conducting analyses, data were screened for model assumptions of multivariate normality and homoscedasticity. Model assumptions were met, and there were some items with non-nonnal distributions. Because of this, principal axis factoring (PAF) was chosen as the factor extraction method over maximum likelihood (ML) (Costello & Osborne, 2005). Outliers were present, and mean differences were examined before and after removing the outliers from the data. Some outliers were more than 3 standard deviations from the mean, and were subsequently flagged as potential influential cases (Stevens, 2002). Approximately 8% of the data were missing. Missing data were treated with imputation using the estimation maximization (EM) algorithm (Dempster, Laird, & Rubin, 1977). Psychometric properties of the instrument were examined in the exploratory and confirmatory factor analysis results. Because this was a crossvalidation study, the initial total sample (n = 295) was randomly split so that approximately one third of the sample was used for the EFA (n = 95), and approximately two thirds were used for the CFA (n = 39 200). The choice to split the sample accordingly was based on a minimum sample size recommendation of 200 for confirmatory factor analysis (Kline, 1998). Exploratory factor analysis. In total, three EFAs were conducted to determine the best factor structure to use in the CFA model. For each EFA, PAF was used with oblique rotation and the same decision rules were applied concerning item deletion. According to these decision rules, an item was removed if: (a) it cross-loaded on more than one factor at .32 or greater, (b) it failed to load onto any factors, (c) it had a communality less than .20, or (d) the item loaded onto a factor, but after close examination, it was determined the item did not group well conceptually with other items in the factor, and therefore the construct was not theoretically plausible (Costello & Osborne, 2005; Tabachnick & Fidell, 2001). The initial EFA included all items in the original survey (n = 57, see Appendix A). After applying the aforementioned decision rules, eighteen items were removed and another EFA was conducted with the remaining items (n = 39). After applying the decision rules again, twelve items were removed and another EFA was conducted with the remaining items (n = 27). The 27-item version of the survey instrument was the final EFA conducted. More items were removed prior to conducting the CFA based on the same decision rules and the final model used for the CFA included a reduced number of items (n = 19). The means and standards deviations of each item are included in Appendix H. The correlation matrix for the items is shown in Appendix 1. Item intercorre1ations were low to moderate. The item communalities were moderate to strong, with values ranging 40 from the highest item "I am confident in my understanding of the Americans with Disabilities Act" (.899) and the lowest item "I routinely build in some flexibility in my evaluation methods, such as dropping the lowest quiz score or providing different exam formats (essay, multiple choice)" (.120). This item was the least correlated with the other items, and was so low it was not retained. There was one other item that fell below the 0.20 criterion, and therefore was not retained ("I refer students to academic support services on campus, such as Academic Learning Services (ALS), when they need help") (.196). The item communalities are shown in Table 3. Immediately following, factor eigenvalues are summarized in Table 4. Table 3 Communalities ofExCEL Survey Items Item I am confident in my understanding of section 504 of the Rehabilitation Act of 1973 I am confident in my understanding of the legal definition of disability I am confident in my understanding of the Americans with Disabilities Act (1990) I am willing to provide copies of my lecture notes or outlines to students with documented disabilities I am willing to provide copies of my overheads and/or PowerPoint presentations to students with documented disabilities Providing accommodations to students with disabilities is a way to ensure equal opportunity and access to learning in higher education settings I am willing to spend extra time (i.e., in addition to typical office hours) meeting with students with documented disabilities to clarify and/or review course related content or prepare for an upcoming exam Extraction 0.684 0.797 0.899 0.606 0.706 0.384 0.776 Item Extraction I am willing to allow a student with a documented disability to 0.858 complete extra credit assignments for academic success even when this option is not stated on the course syllabus I am willing to allow any student to complete extra credit assignments 0.561 in my course(s) Providing teaching accommodations to students with documented 0.592 disabilities is unfair to students without disabilities I believe that students with disabilities use the disability as an excuse 0.692 when they are not doing well in my class I am willing to arrange extended time on exams for students who have 0.861 documented disabilities I am willing to spend extra time (i.e., in addition to normal office 0.774 hours) helping any student prepare for an exam or review course material I feel very comfortable meeting with students to discuss their 0.724 disability-related accommodation needs I am willing to allow students with documented disabilities to take 0.434 proctored exams in a supervised location outside of the normal exam location I feel very comfortable discussing with students barriers they may 0.454 experience given the design or evaluation methods used in my course(s) I routinely build in some flexibility in my evaluation methods, such as 0.120* dropping the lowest quiz score or providing different exam formats (essay, multiple choice, papers) for all students Students with documented disabilities who request support from 0.610 Disability Services receive adequate services from that office I receive adequate support from Disability Services to make 0.786 appropriate accommodations for students with documented disabilities Making adequate accommodations for students with documented 0.601 disabilities in my courses is unrealistic given time constraints and other job demands I include a statement in my syllabus inviting students with disabilities 0.541 41 Item to discuss their needs with me I make a statement in class inviting students with disabilities to discuss their needs with me I would like to modify curricular aspects of my course(s) in order to create a more inclusive learning environment for all students I prefer to use a variety of instructional formats in my class, including small group and hands on activities I believe it is my responsibility as an instructor to provide consistent information and expectations on all assignments and exams I refer students to academic support services on campus, such as Academic Learning Services (ALS), when they need extra help Students with disabilities should be able to perform just as well as students without disabilities in my course(s) *communality < .20 and item removed from CFA Extraction 0.456 0.588 0.375 0.386 0.196* 0.405 42 43 Table 4 Total Variance Explained of Unrotated Eigenvalues and Oblique Rotation Eigenvalues Unrotated Oblique Rotation Eigenvalue Percent of Cumulative Eigenvalue Factor vanance percent Knowledge of Disability Law 5.722 21.2% 21.2% 3.166 Fairness 2.594 9.6% 30.8% 3.449 Invitation to Disclose 2.227 8.2% 39.0% 2.975 Willingness to Invest Time 1.848 6.8% 45.9% 2.985 Accessibility of Course 1.658 6.1% 52.0% 2.361 Materials Willingness to Adjust Course 1.470 5.4% 57.4% 1.723 Assignments Satisfaction with Campus 1.398 5.2% 62.6% 1.854 Resources Willingness to Provide Testing 1.142 4.2% 66.9% 2.511 Accommodations Eight factors emerged from the final EFA. The decision to retain eight factors was based on the more conventional methods of the Kaiser-Guttman rule (eigenvalues> 1) and examination of scree plots (Preacher & MacCallum, 2003), as well as Velicer's MAP test and Parallel Analysis (O'Connor, 2000), all of which showed slightly different results. According to Kaiser's rule and examination of the scree plot, eight factors emerged. However, the results of Velicer' s MAP test revealed seven factors to retain, and the results of the Parallel Analysis test revealed nine factors. Since Velicer's MAP 44 test tends to underestimate the number of factors, and Parallel Analysis tends to overestimate (O'Connor, 2000), I made the decision to retain eight factors based on the combination of results. The factors were labeled: (a) "Knowledge of Disability Law", (b) "Fairness", (c) "Invitation to Disclose", (d) "Willingness to Invest Time", (e) "Accessibility of Course Materials", (f) "Willingness to Adjust Course Assignments", (g) "Satisfaction with Campus Resources", and (h) "Willingness to Provide Testing Accommodations". With these eight factors, approximately 67% of the total variance of the original items was accounted for. Prior to rotation, eigenvalues were unequally distributed, but after applying rotation, smoothing occurred from uillotated to rotated solutions and eigenvalues were redistributed. Due to moderate intercorrelations between items, pattern coefficients seemed most appropriate to report and interpret. Results of the pattern matrix of coefficients are shown in Table 5. Table 5 Pattern Coefficients Resulting/rom EFA with PAF and Oblique Rotation Item Q I: understanding of section 504 Q2: understanding of disability Knowled Fairness Invitation Willing to Accessib- Willing to Satisfaction Willing to ge of to Invest ilityof Adjust with Provide Disability Disclose Time Course Course Campus Testing Law Materials Assignments Resources Accom- modations 0.795 0.859 Q3: understanding of ADA Q13: accommodations ... ensure equal opportunity and access Q22: ...providing accommodations ...unfair to students without disabilities Q24: students with disabilities ... use the disability as an excuse Q41: Making adequate accommodations .,. is unrealistic given time constraints . Q46: statement in my syllabus inviting students 0.942 0.425 -0.835 -0.868 -0.488 0.726 ..j::>. Vl Item Q47: statement in class inviting students Q14: willing to spend extra time ... meeting with students with disabilities Q27: willing to spend extra time ... helping any student QlO: willing to provide copies of my lecture notes Q12: willing to provide ... overheads and/or PowerPoint Q15: willing to allow students with disabilities complete extra credit assignments Q16: ... allow any student to complete extra credit assignments Q38: Students with disabilities ... receive adequate services ... Disability Services Knowled ge of Disability Law Fairness Invitation to Disclose 0.584 Willing to Invest Time 0.837 0.867 Accessib- ilityof Course Materials 0.790 0.790 Willing to Adjust Course Assignments 0.925 0.690 Satisfaction with Campus Resources 0.790 Willing to Provide Testing Accom- modations ~ 0\ Item Q39: I receive adequate support from Disability Services Q25: willing to arrange extended time on exams Q29: willing to allow ... proctored exams in a supervised location Knowled ge of Disability Law Fairness Invitation to Disclose Willing to Invest Time Accessib- ilityof Course Materials Willing to Adjust Course Assignments Satisfaction with Campus Resources 0.886 Willing to Provide Testing Accom- modations 0.941 0.562 Note: Coefficients < .30 were removed from the table for ease of interpretation .j::>. -..l 48 Based on the results of the pattern matrix, I carefully examined the conceptual grouping of items within factors. From this process, I determined eight factor labels that seemed to express the most logical communality of the items as theoretical constructs. The pattern coefficients showed that item 1 ("I am confident in my understanding of Section 504 ofthe Rehabilitation Act of 1973") (.795), item 2 ("I am confident in my understanding of the legal definition of disability") (.859), and item 3 ("I am confident in my understanding of the Americans with Disabilities Act of 1990)" (.942) had large coefficients on factor 1. After inspection of these items, I interpreted factor 1 to represent the construct "Knowledge of Disability Law". Item 22 ("Providing teaching accommodations to students with documented disabilities is unfair to students without disabilities") (-.835), and item 24 ("I believe that students with disabilities use the disability as an excuse when they are not doing well in my class") (-.868) had large coefficients on factor 2. In addition, item 13 ("Providing accommodations to students with disabilities is a way to ensure equal opportunity and access to learning in higher education settings") (.425) and item 41 ("Making adequate accommodations for students with documented disabilities in my courses is unrealistic given time constraints and other job demands") (-.488) had moderate coefficients on factor 2. After inspection of these items, I interpreted factor 2 to represent the construct "Fairness". Item 46 ("I include a statement in my syllabus inviting students with disabilities to discuss their needs with me") (.726) had a large coefficient on factor 3, and item 47 ("I make a statement in class inviting students with disabilities to discuss their needs with 49 me") (.584), item 51 ("I would like to modify curricular aspects of my course(s) in order to create a more inclusive leaming environment for all students") (.565), item 52 ("I prefer to use a variety of instructional formats in my class, including small group and hands on activities") (.385), item 53 ("I believe it is my responsibility as an instructor to provide consistent information and expectations on all assignments and exams") (.458), and item 57 ("Students with disabilities should be able to perform just as well as students without disabilities in my course[s]") (.416) all had moderate coefficients on factor 3. Although initially six items loaded onto factor 3, four items were removed after applying the decision rules. One item was removed because it cross loaded onto another factor, and three others were not theoretically plausible with the other items. That is, when grouped together, the items did not make logical sense as a measured construct. Therefore, items 46 and 47 were retained for factor 3. After inspection of these items, I interpreted factor 3 to represent the construct "Invitation to Disclose". Items 14 and 27 ("I am willing to spend extra time (i.e., in addition to typical office hours) meeting with students with documented disabilities to clarify and/or review course related content or prepare for an upcoming exam" (.837) and "I am willing to spend extra time (i.e., in addition to normal office hours) helping any student prepare for an exam or review course material" (.867) respectively) had large coefficients on factor 4. After inspection of these items, I interpreted factor 4 to represent the construct "Willingness to Invest Time". Items 10 and 12 ("I am willing to provide copies of my lecture notes or outlines to students with documented disabilities" (.790) and "I am willing to provide copies of my 50 overheads and/or PowerPoint presentations to students with documented disabilities" (.790), respectively) had coefficients on factor 5. After inspection of these items, 1 interpreted factor 5 to represent the construct "Accessibility of Course Materials". Item 15 ("1 am willing to allow a student with a documented disability to complete extra credit assignments for academic success even when this option is not stated on the course syllabus") (.925) had a large coefficient on factor 6, and item 16 ("1 am willing to allow any student to complete extra credit assignments in my course(s)") (.690) had a coefficient on factor 6. After inspection of these items, 1 interpreted factor 6 to represent the construct "Willingness to Adjust Course Materials". Items 38 and 39 ("Students with documented disabilities who request support from Disability Services receive adequate services from that office" (.790) and "1 receive adequate support from Disability Services to make appropriate accommodations for students with documented disabilities" (.886), respectively) had large coefficients on factor 7. After inspection of these items, 1 interpreted factor 7 to represent the construct "Satisfaction with Campus Resources". Finally, item 25 ("1 am willing to arrange extended time on exams for students who have documented disabilities") (.941) had a strong coefficient on factor 8, and item 29 ("1 am willing to allow students with documented disabilities to take proctored exams in a supervised location outside of the nOffi1al exam location") (.562) had a moderate coefficient on factor 8. After inspection of these items, 1 interpreted factor 8 to represent the construct "Willingness to Provide Testing Accommodations". 51 Although 27 total items were included in the EFA, eight of these items were removed due to one or more of the stated decision rules. Therefore, the final model used in the CFA included 19 items. An oblique analysis was conducted and there was no decision made to rerun the analysis with Varimax because there was a moderate amount of intercorrelation between the factors. The highest intercorrelation was between factors 2 ("Fairness") and 8 ("Willingness to Provide Testing Accommodations"), which was .491. The next highest intercorrelation was between factors 2 ("Fairness") and 4 ("Willingness to Invest Time"), which was .374. Finally, the weakest intercorrelation was between factors 2 ("Fairness") and 6 ("Willingness to Adjust Course Assignments"), which was .004. Table 6 shows the factor intercorrelations. Descriptive statistics and factor reliability. Descriptive statistics and Cronbach's a were examined for reliability, which addressed part of the first research question. The response scale was a six-point level of agreement scale, ranging from 1 (strongly disagree) to 6 (strongly agree). Descriptive statistics for average scores are reported for the entire survey and by subscale for ease ~f interpretation, as the eight subscales were comprised of different numbers of items. The mean scores for the subscales ranged from a high of 5.38 ("Willingness to Make Testing Accommodations") to a low of2.99 ("Willingness to Adjust Course Assignments"). Table 6 Summary ofFactor Intercorrelations Factor 1 2 3 4 5 6 7 8 1) Knowledge ofDisability Law 1.000 2) Fairness 0.243 1.000 3) Invitation to Disclose 0.335 0.185 1.000 4) Willingness to Invest Time 0.251 0.374 0.279 1.000 5) Accessibility of Course 0.158 0.205 0.291 0.293 1.000 Materials 6) Willingness to Adjust Course 0.063 0.004 0.109 0.143 0.156 1.000 Assignments 7) Satisfaction with Campus 0.118 0.233 0.173 0.149 0.067 -0.135 1.000 Resources 8) Willingness to Provide Testing 0.125 0.491 0.164 0.296 0.141 -0.155 0.159 1.000 Accommodations VI tv 53 A summary of reliability coefficients and descriptive statistics is shown in Table 7. Cronbach's a was examined for factor reliability. All factors had coefficient alphas of .70 or higher, with the exception of factor 8, "Willingness to Provide Testing Accommodations" (.64). Table 7 Descriptive Statistics and Reliability Coefficients for Full ExCEL Survey and Subscales Average Score Measure Item n a Mean SD ExCEL survey 19 .79 4.34 0.53 Knowledge of Disability Law 3 .86 3.01 1.31 Fairness 4 .77 5.02 0.76 Invitation to Disclose 2 .72 4.08 1.42 Willingness to Invest Time 2 .73 4.75 1.00 Accessibility of Course Materials 2 .74 5.09 1.00 Willingness to Adjust Course Assignments 2 .77 2.99 1.32 Satisfaction with Campus Resources 2 .80 4.36 0.85 Willingness to Make Testing Accommodations 2 .64 5.38 0.65 Confirmatory factor analysis. After conducting the EFA, the eight-factor structure was tested with a confirnlatory model. As previously mentioned, eight items were removed after the final EFA but prior to conducting the CFA due to cross loadings of .32 or greater, failure to load onto any factor, or loading onto a low factor that was later eliminated due to theoretical implausibility. The CFA model structure is shown in Figure 54 3. The results of the CFA yielded an inadmissible solution, which indicates that some variance estimates are negative and suggests the model is inadequate or that the sample is too small. More specifically, six ofthe eight factors in the model had only two items, which typically indicates an underidentified model (Kline, 1998). There were two cases of negative residual variance on two different factors: "Satisfaction with Campus Resources" and "Willingness to Invest Time". Specifically, the items on these two factors with the negative residual variance were item 14 ("1 am willing to spend extra time (i.e., in addition to typical office hours) meeting with students with documented disabilities to clarify and/or review course related content or prepare for an upcoming exam") and item 39 ("1 receive adequate support from Disability Services to make appropriate accommodations for students with documented disabilities"). The model was tested again after eliminating these two factors, however, an inadmissible solution was again the result. This time a negative residual variance occurred on another factor. These findings show the factor structure ofthe instrument is not ready to be confirmed yet. The results of the EFA are best utilized to inform decision making on item review and development. Figure 3. CFA model of ExCEL survey instrument 55 56 Group comparisons. To answer research questions two and three, I conducted several Multivariate Analyses of Variances (MANOVAs) using the same sample from the CFA model testing (n = 200). Specifically, research question two asked ifthere were group differences according to gender, teaching status, and departmental affiliation. For this analysis, gender and teaching status were specified as grouping variables and the factor scores from the eight attitude and perception subscales obtained in the last EFA were used as outcome variables. Due to insufficient sample size in some categories, teaching status was converted from a variable with six categories into a dichotomous variable. The variable was grouped dichotomously so that faculty were classified as (a) tenure-line, or (b) nontenure, adjunct, visiting, or graduate assistant. I chose this particular division because the original intent was to determine if there were differences in attitudes and perceptions toward disability between faculty who were tenure-line and those who were not. Boxplots and histograms were examined for univariate and multivariate nonnality, and no subscales had a significant deviation from normality. Because outliers were present, the MANOVAs were conducted with and without the outliers. For the gender and teaching status MANOVAs, there were no differences between the two results, and therefore the outliers were not considered influential cases and left in the analysis. For the interaction of gender by teaching status, there were differences between results with and without outliers. Because of this difference, MANOVA results are presented with the outliers excluded for the interaction of gender by teaching status. Homogeneity of variance was tested using Box's Test of Equality of Covariance Matrices. Examination of Box's M shows there was heterogeneity of variance, indicating 57 the observed covariance matrices of the dependent variables differ to a statistically significant degree across groups, F(108, 38158) = 1.50,p = .001. Since heterogeneity of variance was found, a more conservative a of .025 was used in the MANGVAs. An estimate of effect size (TJp2) is also included in the presentation of results. Partial eta squared (TJp2) is typically interpreted as the proportion of the total variance attributable to a factor (Cohen, 1973; Levine & Hullett, 2002). The results indicated a statistically significant difference in the multivariate combination of the subscale scores based on teaching status, Wilks' A = .896, F(8, 189) = 2.74,p < .025, 'lp2 = .10. Results of the univariate tests indicated a statistically significant difference in faculty attitudes and perceptions according to teaching status on two ofthe eight subscales, specifically "Invitation to Disclose" (p = .002), and "Willingness to Invest Time" (p = .011). However, there were no statistically significant differences in the multivariate combination of subscale scores based on gender, Wilks' A = .928, F(8, 189) = 1.82, p = .075, 'lp2 = .07, or the interaction of gender by teaching status, Wilks' A = .923, F(8, 189) = 1.92, p = .052, 'li = .07 (with outliers included in the analysis, results to follow for analysis excluding outliers). In conjunction with conducting the MANGVAs, I examined the discriminant function used to maximally differentiate between the groups for each subscale by teaching status. The standardized discriminant function coefficients (SDFCs) and structure coefficients (SCs) are reported in Table 8. There were only two levels of gender and teaching status, and therefore only one discriminant function computed. The main effect of teaching status on the eight subscales was significant and the canonical 58 correlation was .32, indicating that approximately 10% the variance was explained by the discriminant function. Table 8 Standardized Discriminant Function Coefficients (SDFCs) and Structure Coefficients (SCs) for Each Subscale by Teaching Status Subscale Knowledge of Disability Law Fairness Invitation to Disclose Willingness to Invest Time Accessibility of Course Materials Willingness to Adjust Course Assignments Satisfaction with Campus Resources Willingness to Provide Testing Accommodations Teaching Status SDFC SC .335 .166 -.158 .445 -.468 -.172 .073 .138 .642 .512 -.606 -.439 .170 .324 .487 .564 The SDFCs indicated the "Accessibility of Course Materials" and "Willingness to Adjust Course Assignments" subscales had the largest contribution to the formation of the discriminant function, whereas the "Willingness to Invest Time", "Fairness", and "Satisfaction with Campus Resources" subscales had the smallest contributions to the formation of the discriminant function. The SCs indicated that "Willingness to Provide Testing Accommodations", "Accessibility of Course Materials", and "Fairness" subscales had the strongest correlations and were moderately related to the discriminant function, 59 demonstrated by correlations of.564 and .512. The subscales of "Willingness to Invest Time", "Knowledge ofDisability Law", and "Invitation to Disclose", however, were weakly related to the discriminant function, with correlations of .138, .166, and -.172 respectively. These results indicate tenure-line and nontenure faculty vary the most in providing course materials in a variety of formats and allowing adjustments to course assigmnents. As verified by the means, nontenure faculty are more likely to be flexible in providing course materials in a variety of formats and allowing adjustments to course assignments than tenure-line faculty. However, there is little variation between tenure- line and nontenure faculty in attitudes regarding investing time toward helping students outside of class time or posted office hours, ensuring equal treatment of students with and without disabilities, and satisfaction with campus resources, such as the Disability Services Office. As mentioned, MANOVAs were conducted with and without the outliers in order to determine if outliers should be considered influential cases. There were no differences in results except for the interaction of gender by teaching status. With the outliers excluded, results were statistically significant, Wilks' A = .919, F(8, 181) = 2.00,p .048, 11p2 = .08. Results of the univariate tests indicated a statistically significant difference according to the interaction of gender by teaching status on one subscale, "Accessibility of Course Materials" (p = .028). These results should be interpreted with caution, as they depend on the presence or absence of outliers. In conjunction with conducting the MANOVA, I examined the discriminant function used to maximally differentiate between the groups. The standardized 60 discriminant function coefficients (SDFCs) and structure coefficients (SCs) are reported in Table 9. Table 9 Standardized Discriminant Function Coefficients (SDFCs) and Structure Coefficients (SCs) for Each Subscale by the Interaction ofGender by Teaching Status Subscale Knowledge of Disability Law Fairness Invitation to Disclose Willingness to Invest Time Accessibility of Course Materials Willingness to Adjust Course Assignments Satisfaction with Campus Resources Willingness to Provide Testing Accommodations Gender*Teaching Status SDFC SC -.335 -.315 -.526 -.062 .412 .401 .120 .089 .776 .544 -.237 -.127 -.485 -.435 .148 .156 There were only two levels of gender and teaching status, and therefore only one discriminant function computed. The main effect of gender by teaching status on the eight subscales was significant and the canonical correlation was .28, indicating that approximately 8% the variance was explained by the discriminant function. The SDFCs indicated the "Accessibility of Course Materials" subscale had the largest contribution to the formation of the discriminant function at .776, with sizable contributions from the "Fairness", "Satisfaction with Campus resources" and "Invitation to Disclose" subscales 61 as well. The "Willingness to Invest Time" and "Willingness to Provide Testing Accommodations" subscales had the smallest contributions to the formation of the discriminant function. The SCs indicated the "Accessibility of Course Materials", and "Satisfaction with Campus Resources" subscales were moderately related to the discriminant function, demonstrated by correlations of .544 and -.435. The subscales of "Willingness to Invest Time", "Fairness", "Willingness to Adjust Course Assignments" and "Willingness to Provide Testing Accommodations", however, were weakly related to the discriminant function, with correlations of .089, -.062, -.127, and -.127 respectively. These results indicate the most variation occurred between faculty according to gender and teaching status on the subscale "Accessibility of Course Materials". In order to completely address research question two, a second MANOVA was conducted using college membership as the grouping variable. There were eight colleges at the university, however two of the eight colleges had insufficient data, as MANOVAs require at least as many participants per cell as there are dependent variables (Tabachnick & Fidell, 2001). These two colleges, which were Honors College (n = 3) and Journalism (n = 8), were eliminated from the MANOVA analysis. Additionally, although the Law School was considered one of the eight colleges at the university, law school faculty were not sent the survey because of procedural differences when handling disability accommodation requests (see Chapter 2). Because of this, five separate colleges were used for the college membership grouping variable, which included Arts and Sciences (n = 90), Business (n = 22), Education (n = 35), Architecture and Allied Arts (n = 20), 62 Music and Dance (n = 21), and the eight attitude and perception subscales served as outcome variables. Boxplots and histograms were examined for univariate and multivariate normality. No subscales had a significant deviation from normality. Outliers were present, and MANOVAs were conducted with and without the outliers. There were no significant differences between the results, and therefore the outliers were left in the analysis and were not considered influential cases. Homogeneity of variance was tested using Levene's Test of Equality of Error Variances and Box's Test of Equality of Covariance Matrices. Examination ofBox's M shows there was homogeneity of variance, indicating the observed covariance matrices of the dependent variables do not differ to a statistically significant degree across groups, F(144, 18285) = l.34,p > .OOL Levene's Test of Equality of Error Variances yielded non-significant results for all subscales, indicating error variance of the dependent variables (subscales) does not differ to a significant degree across groups. The results indicated statistically significant differences in the multivariate combination of the subscale scores based on college membership, Wilks' A = .528, F(8, 176) = 3.84,p < .001, 1lp2 = .14. Results of the univariate tests indicated a statistically significant difference in faculty attitudes and perceptions according to college membership on six of the eight subscales, specifically "Knowledge of Disability Law" (p < .001), "Fairness" (p = .023), "Invitation to Disclose" (p <.001), "Accessibility of Course Materials" (p = .002), "Willingness to Adjust Course Assignments" (p = .037), and "Willingness to Make Testing Accommodations" (p = .023). 63 In conjunction with conducting the MANGVA, I examined the discriminant function used to maximally differentiate between the groups. There were four mathematically possible discriminant function roots to differentiate between the five groups, but only two ofthe four roots were statistically significant, which means there were two possible weighted linear combinations that describe the differences among faculty according to college membership. The canonical correlation for the first root was .59, indicating that approximately 35% ofthe variance in college membership was explained by the discriminant function, and the canonical correlation for the second root was .38, indicating approximately 14% of the variance in college membership was explained by the second root. The standardized discriminant function coefficients (SDFCs) and structure coefficients (SCs) are reported in Table 10. The SDFCs indicated the "Knowledge ofDisability Law" and "Invitation to Disclose" subscales had the largest contribution to the formation of the first root, whereas the "Willingness to Invest Time", "Fairness", and "Willingness to Adjust Course Assignments" subscales had the smallest contributions to the formation of the first root. The SCs indicated that the "Knowledge of Disability Law" and "Invitation to Disclose" subscales were strongly related to the first root demonstrated by correlations of .742 and .633. The subscales of "Willingness to Provide Testing Accommodations" and "Satisfaction of Campus Resources", however, were weakly related to the first root, with correlations of .041 and -.106, respectively. This finding indicates the most variation in faculty attitudes across colleges occurred in their ways of inviting students to disclose disabilities, as well as understanding and knowledge oflegal mandates related to disability. 64 Table 10 Standardized Discriminant Function Coefficients (SDFCs) and Structure Coefficients (SCs) for Each Subscale by College Membership Root 1 Root 2 Subscale SDFC SC SDFC SC Knowledge of Disability Law .716 .742 .335 .166 Fairness .117 .221 -.158 .445 Invitation to Disclose .547 .633 -.468 -.172 Willingness to Invest Time -.072 .255 .073 .138 Accessibility of Course Materials .272 .278 .642 .512 Willingness to Adjust Course Assignments .125 .192 -.606 -.439 Satisfaction with Campus Resources -.205 -.106 .170 .324 Willingness to Provide Testing Accommodations -.151 .041 .487 .564 For the second root, the SDFCs indicated that the "Accessibility of Course Materials" and "Willingness to Adjust Course Assignments" subscales had the largest contribution, whereas the "Willingness to Invest Time", "Fairness" and "Satisfaction of Campus Resources" subscales had the smallest contributions to the formation of the second root. The SCs indicated that the "Willingness to Provide Testing Accommodations" and "Accessibility of Course Materials" subscales were strongly related to the second root with correlations of .564 and .512, respectively. The subscales of "Willingness to Invest Time" and "Knowledge ofDisability Law", however, were weakly related to the second root, with correlations of .138 and .166, respectively. This finding indicates the most variation in faculty attitudes across colleges occurred in their 65 willingness to use inclusive instructional practices, which entail flexibility in required course assignments and the use of accessible course materials. The third research question asked whether faculty members with prior experience in disability awareness training have more positive attitudes and perceptions than those who have not received training. To answer this question, another MANOVA was conducted where I grouped faculty dichotomously according to whether or not they had participated in prior disability-focused training experiences. "Training" was determined based on respondents' answers to a series of questions in the beginning of the survey. First, respondents were asked if they had received any prior disability-focused training, to which they answered Yes or No. A follow-up question asked what type of training, and the options included (a) attended a workshop (b) took one or more courses, (c) read books or articles, (d) visited website(s), and (e) other. Overall, 29% of faculty had participated in prior training (n = 57) and 71 % had not participated in any form of prior training (n = 143). The eight attitude and perception subscales were again used as dependent variables for this analysis. Boxplots and histograms were examined for univariate and multivariate normality. No subscales had a significant deviation from normality. Outliers were present, and MANOVAs were conducted with and without the outliers. There were no significant differences between the results, and therefore the outliers were not considered influential cases and left in the analysis. Homogeneity of variance was tested using Box's Test of Equality of Covariance Matrices. Examination of Box's M shows there was heterogeneity of variance, indicating the observed covariance matrices ofthe dependent variables differ to a statistically significant degree across groups, F(36, 41276) = 1.99,p 66 < .001. Since heterogeneity of variance was found, a more conservative a of .025 was used in the MANGVA. The between subjects multivariate results indicated a statistically significant difference in the multivariate combination of the subsca1e scores based on prior training experiences, Wilks' A = .800, F(8, 191) = 5.96,p < .001, 11p2 = .20. Results of the univariate tests indicated that faculty who had experienced prior training had higher scores on the subsca1es measuring Knowledge of Disability Law (p < .001) and providing students with Invitation to Disclose (p < .001) than did faculty who had not previously attended disability-focused training. In conjunction with conducting the MANGVA, I examined the discriminant function used to maximally differentiate between the groups. The SDFCs and SCs for each subsca1e are included in Table 11. There were only two levels of prior training experience and therefore only one discriminant function. The canonical correlation was .44, indicating that approximately 19% of the variance was explained by the discriminant function. The SDFCs indicated that the "Knowledge of Disability Law" (-.837) had the largest contribution to the formation of the discriminant function, and "Invitation to Disclose" also had a moderately large contribution (-.438), However, the "Accessibility of Course Materials", "Willingness to Adjust Course Assignments", "Willingness to Provide Testing Accommodations" and "Satisfaction with Campus Resources" subscales had the smallest contributions to the formation of the discriminant function. The SCs indicated that the "Knowledge of Disability Law" and "Invitation to Disclose" subsca1es were strongly related to the discriminant function demonstrated by correlations of -.877 and -.559. The subsca1es "Accessibility of Course Materials"," Willingness to Adjust 67 Course Assignments"," Willingness to Provide Testing Accommodations" and "Satisfaction with Campus Resources", however, were weakly related to the discriminant function, with correlations of -.075, .069, -.036, and -.100, respectively. This finding indicates the most variation occurs between faculty with and without prior training in their current practices inviting students to disclose disabilities and their knowledge of legal mandates related to disability. Table 11 Standardized Discriminant Function Coefficients (SDFCs) and Structure Coefficients (SCs) for Each Subscale by Prior Training Subscale Knowledge of Disability Law Fairness Invitation to Disclose Willingness to Invest Time Accessibility of Course Materials Willingness to Adjust Course Assignments Satisfaction with Campus Resources Willingness to Provide Testing Accommodations Training SDFC SC -.837 -.877 -.121 -.193 -.438 -.559 .181 -.121 -.006 -.075 .156 .069 -.107 -.100 .057 -.036 68 CHAPTER IV DISCUSSION In this study, I sought to examine the psychometric properties of the ExCEL Survey, an instrument that purports to measure the attitudes and perceptions of university facuIty toward students with all disability types. These attitudes and perceptions were centered on the willingness to accommodate students with disabilities and to adjust instruction and course assignments to incorporate the tenets of Universal Design. In this chapter, I will summarize and interpret the findings of this study that pertain to each research question. As well, I will discuss limitations, address implications of the findings as they relate to applied settings, and discuss pertinent directions of future research. Factor Structure Prior to conducting the study, items were written intending to capture eight hypothesized factors. The final subscales that emerged from the EFA differed somewhat from the hypothesized factors and included: "Knowledge of Disability Law", "Faimess", "Invitation to Disclose", "Willingness to Invest Time", "Accessibility of Course Materials", "Willingness to Adjust Course Assignments", "Satisfaction with Campus Resources", and "Willingness to Make Testing Accommodations". This factor structure explained 67% of the total variance of the original items. Item pattem coefficients on each subscale ranged from .94 to .38. The mean scores for the subscales ranged from a high of 5.38 ("Willingness to Make Testing Accommodations") to a low of 2.99 69 ("Willingness to Adjust Course Assignments"), indicating that participants were most likely to agree to meet requests from students regarding testing accommodations, but least likely to adjust course assignments at the request of students. After three EFAs were conducted, 30 items were removed. The decision to remove items from each EFA was based on the decision rules outlined in Chapter 3. The most common reason for removing items was cross loading onto two or more factors. In fact, of the 30 removed items, 23 (77%) were removed because they cross loaded onto two or more factors. This finding suggests the hypothesized constructs overlapped too much; in other words, items that were written with the intent to capture one construct overlapped with another construct, so that the items were not worded specifically enough for each construct. These results suggest the factor structure of the instrument is not ready to be confirmed, and the focus should be on further item development. Through the results of this study, it is now clear which items load onto multiple factors and therefore did not function as intended. These items need to be re-written so they are more focused and specific to the intended construct and less general to other constructs, and tested again so cross loadings will be avoided in future analyses. The factor structure of the instrument was not confirmed, as verified by the inadmissible solution resulting from the CFA. Although it is adapted from a previous instrument (Murray et aI., 2008), it is important to consider the preliminary stages of development for this instrument. In this stage of development, some items were removed, rewritten, and one construct was added (including the addition of9 new items). These are major changes to the instrument and more than one field-test will be necessary to confirm the factor structure. Further item development is needed prior to conducting 70 another CFA. Additionally, there is no evidence a CFA was conducted on the previous instrument, so the inadmissible solution was not necessarily due to the new items. There were some differences between the hypothesized and actual factors that warrant further discussion. Specifically, these differences involve three of the eight hypothesized factors, measuring knowledge of disabilities (both visible and non-visible), willingness to provide accommodations, and willingness to create more inclusive learning environments (essentially to adapt instruction based on the tenets of Universal Design). Of the 30 removed items, 14 (47%) were items intended to capture these three hypothesized constructs. Further, these are three crucial facets in measuring faculty attitudes and perceptions toward disability. Because these findings indicate these constructs were not measured in the ExCEL survey, it is particularly important to further examine the removed items and determine how they can be revised in future attempts to measure these constructs. Willingness to accommodate. The "Willingness to Accommodate" construct is a cornerstone of the instrument, as much of postsecondary students' experiences involve their interaction with faculty members when requesting accommodations. For most students with disabilities, accommodations area crucial component of success, and being denied accommodations could be a major factor in student outcomes. In previous studies where similar instruments were examined, this construct was present (Murray et aI., 2008; Vogel et aI., 2008). Murray et ai. (2008) found their instrument had three different factors directly related to accommodations: "Willingness to Provide Major Accommodations", "Willingness to Provide Teaching Accommodations", and 71 "Willingness to Provide Testing Accommodations". Vogel et al. (2008) found one general construct that related to knowledge of accommodations. Similar to Murray et al. (2008), the results of this study indicated the presence of one factor specifically related to providing accommodations, which was "Willingness to Provide Testing Accommodations". There were two other factors related to accommodating students, "Willingness to Adjust Course Assignments" and "Accessibility of Course Materials", both of which included items related to providing accommodations. For example, one of the items of "Accessibility of Course Materials" is "I am willing to provide copies of my lecture notes to students with documented disabilities", a typical accommodation request from students with learning disabilities, Attention Deficit/Hyperactivity Disorder, or students with visual impairments. Therefore, three factors emerged representing the "Willingness to Accommodate" construct, which specifically addressed faculty attitudes and perceptions of (a) providing testing accommodations to students, such as extended time on exams, (b) changing course assignments, such as adding an extra credit assignment, or (c) allowing access to course materials that are not typically shared, such as lecture notes. All of these actions, if taken by a faculty member, would be seen as a type of accommodation for students with disabilities, and it is therefore important to emphasize that three of the eight factors on the instrument measure "Willingness to Accommodate". In my hypothesized factors, I intended to capture attitudes and perceptions toward students with different disability types, specifically visible and non-visible disabilities. Particularly, there were two items written to measure this distinction, which were (a) "I have provided recommended accommodations (e.g. extended time, private testing room, 72 etc.) for students with leaming disabilities in my course(s)" and (b) "I have provided recommended accommodations (e.g. large-print materials, captioned videos, etc.) for students with disabilities in my course(s)". The first item lists examples of typical accommodation requests from students with leaming disabilities, which are considered non-visible. The second item lists examples of typical accommodation requests from students with more obvious, visible disabilities. Both of these items were deleted from the instrument after conducting the first EFA because they cross-loaded onto two factors with coefficients of .32 or greater. This finding suggests the items may not have been written specifically enough to capture the different attitudes and perceptions faculty members have toward students with visible and non-visible disabilities. However, there are several problems with these items, which may have contributed to their removal from the instrument. First, both items began with the stem "I have provided accommodations", which measures whether faculty members have done this in the past, not necessarily whether they would provide the accommodation if a student requested it. In hindsight, the stem should have been written as "I am willing to provide" which captures all faculty members' willingness, essentially the attitude I intended to measure. Second, because the items loaded onto two factors, the hypothesized constructs were not captured specifically enough so that they represented different, although relative, facets of faculty attitudes and perceptions toward disability. This finding may suggest that since disability types are so dynamic and accommodation requests typically differ to a great degree, one construct to encompass all is not theoretically plausible. 73 Universal design. Little to no research exists on quantitative attempts to measure faculty attitudes and perceptions toward adopting the tenets of Universal Design into their own instructional practices. While faculty attitudes and perceptions toward disability have been previously measured, none of those instruments included a component to address tenets of Universal Design. This particular portion of the instrument tested uncharted territory. The tenets of Universal Design were mentioned in Chapter 1 (see Table 1, page 15). Nine items were written to measure the construct "Willingness to Create Inclusive Learning Environments", a construct intended to capture the instructional aspects of Universal Design. Seven ofthe nine items were removed after the first or second EFA because they either (a) cross loaded on two or more factors with pattern coefficients of .32 or greater, (b) did not load onto any factors, or (c) did not load along with other items to create a theoretically plausible construct. Like the "Willingness to Accommodate" construct, these results indicate the items were not written specifically enough to uniquely capture inclusive instruction and therefore Universal Design. These results are not unexpected for such a newly developed instrument, especially given the little empirical evidence behind the theoretical framework of Universal Design. Two items, on the other hand, were retained after the final EFA was conducted, but these items did not load onto one factor together that represented "Willingness to Create Inclusive Instructional Environments". Instead, the two items loaded onto different factors, which were "Accessibility of Course Materials" and "Willingness to Adjust Course Assignments". Recall these same two factors mentioned in the discussion on the "Willingness to Accommodate" construct. These findings show a large amount of 74 overlap between the intended constructs of "Willingness to Accommodate" and "Willingness to Create Inclusive Learning Environments", a finding that illustrates the inherent overlap between accommodating students and adopting the principles of Universal Design. Ketterlin-Geller and Johnstone (2007) surmised that faculty members are less likely to make individual accommodations for students should they choose to embed LID principles into their course framework. These findings support this theory because so many items cross loaded, showing the degree to which both constructs overlap. More research must be conducted to determine greater distinctions between a faculty member's willingness to accommodate and willingness to modify instruction. It seems there would be different attitudes toward them, as willingness to modify instruction could be more time-consuming for faculty members. However, further research must be done confirm and disconfirm these theories. Overall, these findings show measuring faculty attitudes and perceptions of disability accommodations and Universal Design is important. The accommodations- related constructs overlapped with the Universal Design constructs, and some items intended for one construct resulted on another construct. This finding suggests there is a good amount of overlap between attitudes and perceptions toward accommodation requests and Universal Design principles. Further research of this nature is needed where items are more fully developed that represent each construct more holistically and discriminantly. In particular, future studies should measure both constructs and compare faculty attitudes toward making accommodation requests and modifying instruction according to the tenets of Universal Design. 75 Reliability Reliability of the instrument was assessed using Cronbach's a, which was examined within factors and the entire instrument. The overall reliability was strong, a = .79, approaching the preferable .80 criteria and exceeding the acceptable .70 criteria (Nunnally, 1975). Reliability within subscales ranged from highest "Knowledge of Disability Law" (a = .86) to lowest "Willingness to Make Testing Accommodations" (a = .64). "Willingness to Make Testing Accommodations" was the only subscale that fell below the acceptable.70 criteria. However, it is important to note the only measure of reliability was internal consistency, and no alternate forms or test-retest design was utilized simply due to the sampling plan. Measures of internal consistency (such as Cronbach's a) provide the degree to which the items on the instrument combine to measure a single trait; however, Cronbach's a is not the most robust form ofreliability (Henson, 2001). Validity The recommended steps to establishing construct validity were followed in this study, including the identification of a theoretical model, development of an initial item pool, and item selection and psychometric evaluation (AERA, APA, & NCME, 2009; Clark & Watson, 1995). Partial evidence of construct validity was obtained through the findings in this study, particularly evidence of content, convergent, and discriminant validity. Content validity was established in several ways: (a) many of the items were taken from a pre-existing survey instrument that showed good promise of reliability and validity (Murray et aI., 2008), and (b) of the items that were newly added, all content was 76 based on an extensive literature review of Universal Design, specifically the tenets related to instruction. In addition, the items were reviewed by the Director of Disability Services to ensure consistency with their understanding of Universal Design. Content validity of the instrument will be strengthened by using the outcomes of the item analysis to drive future instrument development. As previously mentioned, many items were omitted after three rounds ofEFA. A logical next step is to carefully examine those deleted items and detennine if the problem lies in the wording of the item, or if the content is not necessary for the intended constructs. The eight factor structure of the ExCEL survey shows evidence of convergent and discriminant validity. More specifically, items grouped together represent a set of indicators with moderate to strong intercorrelations within each construct, which is evidence of convergent validity. Because the factor correlations were weak to moderate, there is evidence of discriminant validity (Kline, 1998). As previously stated, because so many items cross loaded on one or more factor, future item development must focus on the distinction between the constructs. For example, in drafting new items, researchers should ask: what distinguishes a faculty member's willingness to provide testing accommodations from willingness to adjust course assignments? The results of the CFA provide evidence against construct validity, as the factor structure could not be confirmed. Additionally, the presence of two-item subscales suggests the model may be incorrect or flawed, and more items are needed to adequately measure these constructs. Therefore, the results of this study show partial evidence for and against the validity of the instrument. As mentioned, the instrument has substantial reliability evidence. A single administration of a version of the instrument cmmot 77 provide overall definitive evidence the ExCEL survey measures precisely what it purports to measure (Messick, 1995). Therefore, it is especially important to focus on item development as the most immediate next step in future research. These findings show the instrument has great promise, but must undergo multiple iterations in order to show strong evidence of construct validity. In evaluating evidence for external validity, it's important to consider several limitations to the study. One aspect of external validity is the extent to which results can be generalized from sample to population, and therefore the nature of the sample is important to consider. The response rate was lower than anticipated (27% overall) although adequately representative of the faculty population at this particular university setting. Findings should be interpreted with caution due to the low response rate. Also, the survey was voluntary. Only faculty who desired to volunteer their time responded, potentially leading to a sampling bias. Group Differences Research questions two and three specifically addressed group differences on the measure. To examine whether there were group differences on the measure, based on gender, teaching status, and college membership, two separate MANOVAs were conducted using the eight subscales as dependent variables. To examine whether there were group differences in the measure based on prior disability-focused training experiences, I conducted another MANOVA using the eight subscales as dependent variables. Please note, two separate MANOVAs were conducted in order to answer research question two; the first MANOVA examined group differences based on gender and teaching status, and the second MANOVA was based on college membership. I 78 chose this design because the sample size of each group in college membership was not adequate enough to include three predictors in the same MANOVA. The results of these analyses indicated there were no statistically significant differences in subscale score based on gender, which contradicts earlier findings that female faculty members are more likely to meet accommodation requests from students with disabilities (Leyser et aI., 1998; Murray et aI., 2008; Skinner, 2007). On the other hand, there were statistically significant differences in subscale score based on teaching status, where tenure-line faculty members scored significantly lower overall than nontenure faculty members. These findings suggests that nontenure faculty members are more likely to meet accommodation requests and adopt inclusive instructional practices than tenure-line faculty members, which is inconsistent with previous findings, where faculty rank did not have a significant effect on attitudes and perceptions (Rao, 2002). There is no specific explanation why these results differed from previous studies, except to consider the different university contexts as a major factor in influencing faculty attitudes and perceptions toward disability. University context may be more influential than gender or teaching status. Univariate tests in teaching status revealed statistically significant differences in tenure-line faculty and nontenure faculty on two of the eight subscales, which were "Invitation to Disclose" and "Willingness to Invest Time". These findings suggest that tenure-line faculty are less likely to make an announcement in class or include a statement of their syllabus that invites students with disabilities to disclose to them at the beginning of the academic term. Also, they are less likely to spend extra time, outside of typical office hours, helping students with disabilities with class assignments or exam preparation. 79 These results were paralleled by the examination of which factors were correlated with the discriminant function. These findings suggest the greatest amount of variation in attitudes and perceptions between tenure-line and nontenure faculty members is on the "Accessibility of Course Materials" and the "Willingness to Adjust Course Assignments" subscales, which is interesting given these subscales consist of items that measure flexibility and accessibility, more of the inclusive instructional practices and less of the legal mandates associated with accommodations. These subscales show the most amount of variation, indicating nontenure faculty are more likely to be flexible and adopt inclusive instructional practices than tenured/tenure track faculty. In order to test whether the outliers should be considered influential cases, the MANGVA was conducted again with the outliers excluded. Results showed statistically significant subscale scores based on the interaction of gender by teaching status. After examination of the univariate tests and paralleled by the results of the discriminant function, the most amount of variation between groups occurred on the "Accessibility of Course Materials" subscale, where nontenure male faculty were most likely to provide accessible course materials than nontenure female, tenure-line male, and tenure-line female faculty. However, these results should be interpreted with caution because they depend on the presence or absence of outliers, and were not significant when outliers were included in the analysis. The second MANGVA examined group differences according to college membership, and answered the second part of research question two, specifically addressing group differences in subscale scores based on membership in the various colleges within the university. As previously mentioned, this analysis was limited to five 80 of the eight colleges (two colleges had inadequate sample sizes and one college was not included in the study). The results of this analysis showed statistically significant differences in subscale score based on college membership. This finding suggests faculty attitudes and perceptions toward disability varied between the colleges on campus, and attitudes and perceptions may be influenced depending on which college a faculty member belongs to, a finding consistent with previous studies (Leyser et aI., 1998; Murray et aI., 2008; Rao, 2002; Skinner, 2007). More specifically, on the "Knowledge of Disability Law" subscale, faculty members in Education scored significantly higher than faculty from the other four colleges, indicating a greater degree of knowledge oflegal mandates around disability. Education faculty scored the highest on the "Fairness", "Invitation to Disclose", "Accessibility of Course Materials" subscales as well. However, faculty from the College of Business scored the highest on the "Willingness to Make Testing Accommodations" subscale. These findings are generally consistent with previous findings that faculty attitudes and perceptions differ according to college or departmental affiliation, and that Education faculty tend to be the most accommodating and exhibit the most amount of fairness toward students with disabilities (Leyser et aI., 1998; Murray et aI., 2008; Skinner, 2007). However, there are no previous findings that Business faculty members are particularly accommodating, and this finding is therefore inconsistent with the literature. More investigation is warranted to further understand why Business faculty members scored particularly high in their attitudes and perceptions toward providing exam accommodations to students with disabilities, and why Education faculty members 81 have a greater knowledge of disability legal mandates, have accessible course materials, exhibit fairness toward students with disabilities, and make an outward invitation to students to disclose their disability. Potentially, this finding is a result ofpractices specific to the Colleges ofBusiness and Education, where perhaps College-wide policies and procedures about disability contributed to the differences, but more investigation is needed in order to clarify. The second root, or weighted linear combination, showed "Accessibility of Course Materials", "Willingness to Adjust Course Assignments", and "Willingness to Provide Testing Accommodations" had the largest contribution to the formation of the discriminant function, indicating the most amount ofvariation in attitudes and perceptions between faculty from the five colleges occurred in these subscales. Again, as in the weighted linear combination of the tenure-line versus nontenure faculty MANOVA, the most amount of variation occurred in subscales measuring flexibility in instructional methods and course assignments. This finding shows that faculty members in the five colleges differ in their attitudes and perceptions to a significant degree when it comes to flexibility and adopting inclusive instructional practices, which is not particularly surprising considering there are no policies in place where faculty are required to adopt such practices. Finally, a third MANOVA was conducted in order to address research question three, specifically regarding differences between faculty who have and have not received prior disability-focused training. The results of this analysis showed statistically significant differences in subscale score based on prior disability-focused training or experiences. Essentially, faculty who receive training opportunities are more likely to 82 understand and carry out legal responsibilities affiliated with disability and are more likely to make a statement in class or on their syllabus that encourages students to disclose their disability. These findings suggest that university faculty training could playa crucial role in ensuring students with disabilities receive a quality higher education experience. Further, these findings suggest that as long as faculty are made aware of the laws and strategies to accommodate students with disabilities, they are likely to comply and willing to help. Faculty members who received prior disability-focused training, however, were not necessarily likely to adopt inclusive instructional practices or provide accessible course materials. A potential next step is to consider the content or structure of training opportunities to determine if they centered on laws and accommodations. If this is the case, future research studies should examine the effect of training opportunities based on the tenets of Universal Design in order to determine if faculty members tend to modify their instruction after learning how to create accessible and inclusive learning environments. Limitations There are several limitations to this study that should be considered in the interpretation of these findings. First, there is potential sampling bias. The factor structure was examined using a sample of faculty from a single university in the Pacific Northwest. Further, the survey was voluntary so the sample included only those faculty members willing to invest their time. All faculty members from the initial sample (N = 1084) were given an incentive, a $3 coffee coupon, but this may not have been all that influential. The response rate was far from ideal, at 27%, although it is consistent with other studies involving university faculty and disability awareness (Bourke et al., 2000; 83 Murray et aI., 2008; Murray et aI., 2009; Skinner, 2007; Vogel et aI., 1999, Vogel el aI., 2008), and suggests that, perhaps, there are challenges to sampling university faculty. Regardless, factor analysis is sample dependent, and it is possible a different factor structure would have resulted had the response rate been higher. Also, it's important to note that this particular university has a rather high percentage of white faculty (82%) compared to all ethnic minority faculty (13%) with 5% of the faculty declining to report. A second limitation is potential respondent bias. The instrument was a self-report survey, which allows the potential for dishonest responses. Due to the sensitive nature of the item content, it is likely participants desired to give the most politically correct response, even if it was not the most honest response. This bias was avoided by assuring confidentiality to participants, however, the potential for respondent bias is important to consider in interpreting study results. A third limitation is the case-to-variable ratio. In EFA, the recommended case-to- variable ratio is 5:1 (Costello & Osborne, 2005; Fabrigar et aI., 1999; Stevens, 2002), and this study had a 2:1 ratio. In the effort to complete the cross-validation study with both an EFA and CFA, the ratio was compromised so there would be a larger sample of 200 for the CFA. In the end, I determined further item development is needed prior to conducting another CFA. A fourth limitation, and essentially the reason why the CFA solution was inadmissible, was the presence of two-item subscales. Six of the eight subscales included only two items and while two items per factor might be acceptable, a minimum of three is recommended, four is more desirable, and five or more strong items (loadings .50 or higher) indicate a strong factor (Costello & Osborne, 2005; Kline, 1998). Oftentimes, 84 two-item subscales are unstable, and may lead to problems such as underidentification and nonconvergence of the measurement model. Two-item subscales were not intended, and all hypothesized factors had a minimum of 6 items prior to conducting analyses. However, so many items were removed after the three rounds ofEFA, resulting in item removal and subsequent two-item subscales. This limitation shows the importance of item development as the crucial next step in further development of the ExCEL survey. Careful time must be spent on item development prior to conducting another CFA. Factors should include more than enough items so that items can be removed but will not result in two-item subscales. More discrimination between factors is necessary to avoid cross loadings. Finally, the quantitative instrument poses some limitations. Although the results provided an overall sense of faculty members' perceptions and attitudes toward disability, much ofthe detail of these attitudes was not captured. It is important to further explore dominant themes found in the findings through qualitative research methods. Focus groups and individual interviews with faculty members, students with disabilities, and Disability Services personnel on very targeted and focused topics may provide rich and detailed findings that will not be captured with a quantitative instrument. This instrument provides the specific areas on which to focus, but qualitative research is needed for further, more in-depth exploration of these areas. Implications for Practice Despite these limitations, the ExCEL survey shows evidence of initial reliability and validity and therefore can be used as a tool for assessing university faculty attitudes and perceptions of disability. Specifically, university administrators may use subsca1e 85 scores to examine willingness to provide accommodations and to adjust instruction to include more inclusive principles associated with Universal Design. As previously mentioned, it is becoming more common for effective teaching to be an area of evaluation included in faculty performance reviews. This shift may result in an increased demand for postsecondary faculty members to participate in professional development, particularly in the areas of disability awareness and inclusive instruction that will benefit all students. Survey results can be used to plan for targeted faculty professional development trainings at a college or departmental level. Mean scores on subscales will inform university administrators of areas of weakness in particular departments, among certain faculty ranks, or within the entire university. For example, the survey results showed a particularly high level of awareness of "Knowledge of Disability Law" among Education Faculty. In recruitment efforts, university administrators know that the Education faculty need less training about legal mandates, and more training in inclusive instructional principles. Trainings could be targeted by department, and Department Heads and Deans may use this information to plan for the most effective training in areas of weakness particular to their department or college. Further, the ExCEL survey has the potential to be used by faculty as a self- assessment of disability awareness. Faculty members could take the ExCEL survey online and receive immediate feedback on results, highlighting strengths and weaknesses and recommendations for improvement. These recommendations may include tips for providing accommodations and incorporating more inclusive instructional principles into course planning and delivery. Ideally, faculty should be able to use their results to improve their instruction in order to reach a wider range of diverse learners. 86 It is most important to recall the larger picture: more students with disabilities pursue postsecondary education now than ever before. Although few pursue education at four-year universities, the number continues to increase. With this change, disability awareness must be increased among university faculty, who tend to have the most amount of direct contact with and influence on the overall postsecondary experiences of students with disabilities. Further, following the tenets of Universal Design will promote the use of inclusive instructional practices among all faculty. This change will benefit not only students with disabilities, but all university students, especially given the increase in diverse learners on college campuses nationwide. This study shows the crucial first step to assessing any institutional climate is to reliably and validly measure faculty attitudes and perceptions toward students with disabilities. Such assessment can and will lead to targeted faculty interventions that will enhance the overall quality of education received by all students, with and without disabilities. APPENDIX A EXCEL SURVEY Project Excel: Faculty Survey a. Gender 87 Female 0 Male 0 b. Teaching Status (check one) Tenured 0 Tenure Track 0 Non-tenure Track 0 Adjunct 0 Visiting 0 Post-retire 0 Graduate assistant 0 c. Rank (check one) Instructor 0 GTF 0 Assistant Professor 0 Associate Professor 0 Full Professor 0 d. University of Oregon College (check one) Arts and Sciences 0 Business 0 Architecture and Allied Arts 0 Honors CollegeD Education 0 Music and Dance 0 Law 0 Journalism 0 e. Arts and Sciences only (please check division): Humanities 0 Social Sciences 0 Sciences 0 88 f. How many years have you been teaching at the postsecondary level? _ g. I teach primarily (check one) o General education courses o Elective courses o Discussion/Lab section (subsection oflecture) o Major-specific courses o Graduate level courses h. In the past five years, how many college students with disabilities have you taught or worked with? o None o 1-5 o 6-10 o 10-20 o Over 20 o Don't know/not sure i. What is your age? _ j. Have you had any personal experiences with disability? (please check all that apply) D I have a disability D Family member, friend or other personal contact has a disability D I have worked with or taught students with disabilities D No, I have not had any personal experiences with disability k. If you were to attend a training session at University of Oregon, which topics would you find most relevant and/or interesting? Rank order topics (1= most important, 5=least important) D Increasing my understanding of disability issues in the college setting Dlncreasing my understanding of student experiences DLeaming more about inclusive instructional approaches that would reduce the need for student-specific accommodations DBetter understanding of Disability Services and how they can support me and my students DIn-depth understanding of specific types of disabilities Directions: The following questions pertain to training you may have received in supporting students with disabilities. la. Have you ever received training related to disability or working with college students with disabilities? Yes No o D If no, please skip to Survey Directions on the next page. If yes, please answer the following: lb. What type of training? (please check all that apply) D Attended a workshop D Took one or more courses o Read books or articles D Visited Website(s) D Other (please describe or list below): I c. How much training? D Less than 1 hour D between 1 and 3 hours D between 4 and 6 hours D between 7 and 9 hours o 10 hours or more Continue survey on next page .... 89 Survey Directions: Please rate the following 60 items from 1 (Strongly Disagree) to 6 (Strongly Agree) Response Format 90 1 = Strongly Disagree 2 = Disagree 3 = Somewhat 4 = Somewhat Disagree Agree 5 = Agree 6 = Strongly Agree 1. I am confident in my understanding of section 504 102030405060 ofthe Rehabilitation Act of 1973. 2. I am confident in my understanding of the legal 102030405060 definition of "disability" 3. I am confident in my understanding of the 102030405060 Americans with Disabilities Act (1990). 4. I believe that students with learning disabilities can 102030405060 be successful at the university level. 5. Students with non-visible disabilities (e.g. learning 102030405060 disability, psychological condition, etc.) are often reluctant to disclose their disability. 6. I would like more information about the needs of 102030405060 students with disabilities at the University of Oregon. 7. I am sensitive to the needs of students with 102030405060 disabilities at the University of Oregon. 8. Students with disabilities are able to compete 102030405060 academically at the university level. 9. Students with disabilities attend postsecondary 102030405060 schools at rates proportionate to students without disabilities. 91 10. I am willing to provide copies of my lecture notes 102030405060 or outlines to students with documented disabilities. 11. I am willing to provide additional time to complete 10 20 30 40 50 60 assignments in my course(s) to any student who expresses a need. 12. I am willing to provide copies of my overheads 102030405060 and/or PowerPoint presentations to students with documented disabilities. 13. Providing accommodations to students with 102030405060 disabilities is a way to ensure equal opportunity and access to learning in higher education settings. 14. I am willing to spend extra time (i.e., in addition to 102030405060 typical office hours) meeting with students with documented disabilities to clarify and/or review course related content or prepare for an upcoming exam. 15. I am willing to allow a student with a documented 102030405060 disability to complete "extra credit" assignments for academic success even when this option is not stated on the course syllabus. 16. I am willing to allow any student to complete 102030405060 "extra credit" assignments in my course(s). 17. I am willing to reduce the overall course reading 102030405060 load for a student with a documented disability even when I would not allow a reduced reading load for other students. 18. I make individual accommodations for students 10 20 30 40 50 60 who have disclosed their disability to me. 19. I believe that my overall teaching style permits all 102030405060 students to learn the course material regardless of their individual needs. 92 20. I am willing to allow students with documented 102030 40 50 60 disabilities to record class sessions when necessary. 21. I am willing to extend the "due dates" of 102030 405060 assignments to accommodate the needs of students with documented disabilities when necessary. 22. Providing teaching accommodations to students 10 20 3D 40 50 60 with documented disabilities is unfair to students without disabilities. 23. When students with disabilities are having 102030 405060 difficulties in my course(s), I am uncertain about where I can find additional support at the University of Oregon. 24. I believe that students with disabilities use the 10 20 3D 40 50 60 disability as an excuse when they are not doing well in my class. 25. I am willing to arrange extended time on exams for 102030 40 50 60 students who have documented disabilities. 26. I am willing to change the method of responding on 102030 40 50 60 exams (e.g., from written to oral) for students with documented disabilities. 27. I am willing to spend extra time (i.e., in addition to 102030 405060 normal office hours) helping any student prepare for an exam or review course material. 28. I feel very comfortable meeting with students to 10 20 3D 40 50 60 discuss their disability-related accommodation needs. 93 29. I am willing to allow students with documented 102030405060 disabilities to take proctored exams in a supervised location outside of the normal exam location. 30. I am willing to allow students with documented 102030405060 disabilities to use technology (e.g., laptop, calculator, spell checker) to complete tests even when such technologies are not permitted for use by students without disabilities. 31. Providing testing accommodations (such as 102030405060 extended exam time) to students with documented disabilities is unfair to students without disabilities. 32. I feel very comfortable discussing with students 102030405060 barriers they may experience given the design or evaluation methods used in my course(s). 33. Typically, students with disabilities do not perform 102030405060 as well as the rest of the students in my course(s). 34. I am aware of assistive technology that students 102030405060 with disabilities can use to aid their understanding of course material. 35. Disability Services will provide exam proctoring 102030405060 for students with documented disabilities. 36. I incorporate disability-related topics into the 102030405060 content of my course(s). 37. I routinely build in some flexibility in my 102030405060 evaluation methods, such as dropping the lowest quiz score or providing different exam formats (essay, multiple choice, papers) for all students. 38. Students with documented disabilities who request 102030405060 support from Disability Services receive adequate services from that office. 94 39. I receive adequate support from Disability Services 102030405060 to make appropriate accommodations for students with documented disabilities. 40. Currently, I do not have sufficient knowledge to 102030405060 make adequate accommodations for students with disabilities in my course(s). 41. Making adequate accommodations for students 102030405060 with documented disabilities in my courses is umealistic given time constraints and other job demands. 42. As a faculty member, I am legally required to 102030405060 provide requested accommodations to students with disabilities. 43. I have provided recommended accommodations 102030405060 (e.g. extended time, private testing room, etc.) for students with learning disabilities in my course(s) 44. I have provided recommended accommodations 102030405060 (e.g. large-print materials, captioned videos, etc.) for students with disabilities in my course(s). 45. When students approach me with accommodation 102030405060 requests they typically show me a letter that lists recommended accommodations. 46. I include a statement in my syllabus inviting 102030405060 students with disabilities to discuss their needs with me. 47. I make a statement in class inviting students with 102030405060 disabilities to discuss their needs with me. 95 48. If students with documented disabilities wait to talk 102030405060 to me until they are not doing well in my class then I feel it's too late to provide appropriate accommodations. 49. Typically, I use scaffolding methods to assist 10203040 5060 students with assignments in my course(s), such as using past assignments as examples or requiring students submit an outline and/or annotated bibliography weeks before submitting a final paper. 50. I put my lecture notes online for all students (on 102030405060 Blackboard or another website). 51. I would like to modify curricular aspects of my 102030405060 course(s) in order to create a more inclusive learning environn1ent for all students. 52. I prefer to use a variety of instructional formats in 1020 30 40 50 60 my class, including small group and hands on activities. 53. I believe it is my responsibility as an instructor to 10 20 30 40 50 60 provide consistent information and expectations on all assignments and exams. 54. At times, I feel burdened when students with 102030405060 disabilities approach me with accommodation requests. 55. I refer students to academic support services on 102030405060 campus, such as Academic Learning Services (ALS), when they need extra help. 56. I am willing to use technology so that my course 10203040 5060 material can be available in a variety of formats (e.g. podcast of lecture available for download, course readings available as mp3 files). 57. Students with disabilities should be able to perform 102030405060 just as well as students without disabilities in my course(s). 96 APPENDIXB FIRST RECRUITMENT EMAIL TO FACULTY PARTICIPANTS Dear Faculty Member, Recently, the UO received grant funding from the Office of Postsecondary Education to implement Project ExCEL- UO, Expanding Cultural Awareness of Exceptional Learners at the University of Oregon. The goal of Project ExCEL is to improve the experience of students with disabilities at the UO by providing all faculty members with additional information and training related to understanding and teaching students with disabilities. This professional development model will be implemented collaboratively by faculty in Special Education and Clinical Sciences and Disability Services. The model includes three interrelated training components that together are designed to impact the overall culture ofthe university in ways that make it more responsive to the needs of students with disabilities. In order to effectively use the grant funding, a survey has been designed for all teaching faculty. If you are currently teaching, or have taught classes in the past, please take the time to complete the survey. Your input is valuable; it will help us assess the current campus climate and plan future instructional opportunities for the duration of the grant. The actual survey should take about 20 minutes to complete. In addition, please accept an Allann Brothers discount coupon as a token of our appreciation for your time. Both the survey and the coupon are available at the link: http://www.uoregon.edu/~allisonl/ There is a written consent form at the beginning of the survey. By indicating "yes" you are giving consent to participate. Your participation is voluntary. Thank you in advance for your time and input. If you have any question, please contact any of the project staff listed below. The Project ExCEL team Dr. Christopher Murray, Principal Investigator 346-1445; cjmurray@uoregon.edu Dr. Hilary Gerdes, Director of Disability Services 346-1063; hgerdes@uoregon.edu Allison Lombardi, Proj ect Coordinator 255-9405; allisonl@uoregon.edu Leslie Gilbert, Project Assistant 346-3173; 19i1bert@uoregon.edu 97 98 APPENDIXC LETTER OF CONSENT TO PARTICIPATE: FACULTY You are invited to participate in a research study conducted by Dr. Christopher Murray, a faculty member, and Allison Lombardi, a doctoral student, from the Special Education and Clinical Sciences Department, as well as Hilary Gerdes, Director ofDisability Services, at the University of Oregon. We hope to learn about faculty attitudes and perceptions toward disability in order to implement a professional development model that enhances disability awareness. In addition, Ms. Lombardi will conduct a validity study on the survey instrument as part of a dissertation study. You were selected as a possible participant in this study because you are a member of the teaching faculty at the University of Oregon. Ifyou decide to participate, please indicate "yes" to the question below to begin the survey. The survey should take about 20 minutes to complete. Your responses will be completely anonymous to the researchers. Your participation is voluntary. The survey questions will ask about your attitudes and perceptions toward students with disabilities as an instructor. We cannot guarantee that you personally will receive any benefits from this research. In addition, each participant will be offered a discount coupon to Allann Brothers. If you have any questions about the research study, please contact a member of the Project Excel team: Dr. Christopher Murray, Principal Investigator 346-1445 £i!purray@uoregon.edu Dr. Hilary Gerdes, Director of Disability Services 346-1063 hgerdes~uoregon.edu Allison Lombardi, Project Coordinator 255-9405 allisonl@uoregon.edu Do you agree to participate in the study? o Yes o No 99 100 APPENDIXD SECOJ\ID EMAIL TO FACULTY PARnCIPANTS Dear UO Faculty Member: This email is a reminder to all teaching faculty to please complete the Project ExCEL Faculty Survey at your earliest convenience. If you have already completed the survey, thank you very much for your input and please disregard this email. If you have not yet completed the survey, it should take about 20 minutes to complete. We realize you are very busy and we appreciate your time. Your input is invaluable to us and the success of our project. In addition, please accept an Allann Brothers discount coupon as a token of our appreciation for your time. Both the survey and the coupon are available at the link: http://www.uoregon.edu/~allisonl/ There is a written consent form at the beginning of the survey. By indicating "yes" you are giving consent to participate. Your participation is voluntary. The Project ExCEL team Dr. Christopher Murray, Principal Investigator 346-1445 cjmurray@uoregon.edu Dr. Hilary Gerdes, Director of Disability Services 346-1063 hgerdes@uoregon.edu Allison Lombardi, Project Coordinator 255-9405 allisonl@uoregon.edu 101 APPENDIXE THIRD EMAIL TO FACULTY PARTICIPANTS Dear UO Faculty Member: This is another friendly reminder to please complete the Project ExCEL Faculty Survey at your earliest convenience. If you have already completed the survey, thank you very much for your input and please disregard this email. If you have not yet completed the survey, it should take about 20 minutes to complete. Unfortunately, our response rate has been lower than anticipated so far and your input is crucial to the success of our project. We realize you are very busy, but we do hope that you could find some time to complete the survey. Thank you again! The link is: http://www.uoregon.edu/~allisonl/ There is a written consent form at the beginning of the survey. By indicating "yes" you are giving consent to participate. Your participation is voluntary. The Project ExCEL team Dr. Christopher Murray, Principal Investigator 346-1445 cjmurrayra1uoregon.edu Dr. Hilary Gerdes, Director of Disability Services 346-1063 hgerdes@uoregon.edu Allison Lombardi, Project Coordinator 255-9405 allisonl@uoregon.edu ----~------ 102 APPE1\TDIX F FOlJRTH EMAIL TO FACULTY PARTICIPANTS Dear UO Faculty Member: This is the final reminder to all teaching faculty to please complete the Project ExCEL Faculty Survey at your earliest convenience. If you have already completed the survey, thank you very much for your input and please disregard this email. We realize you are very busy, but we do hope that you could find some time to complete the survey. Your input will greatly impact the success of our proj ect and help UO students with disabilities. Also, we offer a discount coffee coupon to all teaching faculty as a token of our appreciation for your time. You will find both the survey and coupon at the link below: http://www.uoregon.edu/~allisonl/ There is a written consent form at the beginning of the survey. By indicating "yes" you are giving consent to participate. Your participation is voluntary. The Project ExCEL team Dr. Christopher Murray, Principal Investigator 346-1445 cjmurray@uoregon.edu Dr. Hilary Gerdes, Director ofDisability Services 346-1063 hgerdes@uoregon.edu Allison Lombardi, Project Coordinator 255-9405 allisonl@uoregon.edu 103 APPENDIXG SECURITY QUESTIONS Please answer the following security questions below prior to beginning the Project Excel faculty survey. Your answers to these questions will generate a unique code in our system, which will allow your responses to be confidential. 1. What are the first two letters of your mother's maiden name? --- 2. What is your birth month? (If single digit, place a 0 in front, such as 01 for January) __ 3. What are the last two letters ofthe city where you were born? _ 4. What are the last two digits ofthe year you were born? _ APPENDIXH ITEM MEANS AND STANDARD DEVIATIONS Item M SD N Q1: I am confident in my understanding of section 504 2.57 1.46 95 of the Rehabilitation Act of 1973 Q2: I am confident in my understanding of the legal 3.28 1.49 95 definition of disability Q3: I am confident in my understanding of the 3.34 1.50 95 Americans with Disabilities Act (1990) QlO: I am willing to provide copies of my lecture notes 5.16 0.93 95 or outlines to students with documented disabilities Q12: I am willing to provide copies of my overheads 5.43 0.73 95 and/or PowerPoint presentations to students with documented disabilities Q13: Providing acconmlOdations to students with 5.33 0.74 95 disabilities is a way to ensure equal opportunity and access to learning in higher education settings Q14:1 am willing to spend extra time (i.e., in addition to 5.10 0.92 95 typical office hours) meeting with students with documented disabilities to clarify and/or review course related content or prepare for an upcoming exam Q15: I am willing to allow a student with a documented 3.45 1.52 95 disability to complete extra credit assignments for academic success even when this option is not stated on the course syllabus Q 16:1 am willing to allow any student to complete extra 3.02 1.52 95 credit assignments in my course(s) Q22:Providing teaching accommodations to students 1.94 0.79 95 with documented disabilities is unfair to students without disabilities Q24: I believe that students with disabilities use the 2.01 0.83 95 disability as an excuse when they are not doing well in my class Q25: I am willing to arrange extended time on exams 5.48 0.66 95 for students who have documented disabilities Q27: I am willing to spend extra time (i.e., in addition 4.57 1.29 95 to normal office hours) helping any student prepare for an exam or review course material 104 105 Item M SD N Q28: I feel very comfortable meeting with students to 5.26 0.77 95 discuss their disability-related accommodation needs** Q29: I am willing to allow students with documented 5.55 0.58 95 disabilities to take proctored exams in a supervised location outside of the normal exam location Q32: I feel very comfortable discussing with students 4.96 0.91 95 barriers they may experience given the design or evaluation methods used in my course(s)** Q37: I routinely build in some flexibility in my 4.17 1.50 95 evaluation methods, such as dropping the lowest quiz score or providing different exam formats (essay, multiple choice, papers) for all students** Q38: Students with documented disabilities who request 4.39 0.78 95 support from Disability Services receive adequate services from that office Q39: I receive adequate support from Disability 4.35 0.97 95 Services to make appropriate accommodations for students with documented disabilities Q4l: Making adequate accommodations for students 2.18 0.92 95 with documented disabilities in my courses is umealistic given time constraints and other job demands Q46: I include a statement in my syllabus inviting 4.30 1.72 95 students with disabilities to discuss their needs with me Q47: I make a statement in class inviting students with 3.91 1.60 95 disabilities to discuss their needs with me Q5l: I would like to modify curricular aspects of my 4.18 1.22 95 course(s) in order to create a more inclusive learning environment for all students** Q52: I prefer to use a variety of instructional formats in 4.92 1.22 95 my class, including small group and hands on activities** Q53: I believe it is my responsibility as an instructor to 5.44 0.66 95 provide consistent information and expectations on all assignments and exams** Q55: I refer students to academic support services on 4.84 1.06 95 campus, such as Academic Learning Services (ALS), when they need extra help** Q57: Students with disabilities should be able to 4.87 0.96 95 perform just as well as students without disabilities in my course(s) APPENDIX I ITEM CORRELATION MATRIX 106 Item Q1 Q2 Q3 Q\O Q12 Q13 Q14 Q15 Q16 Q22 Q24 Q25 Q27 Q28 Q29 Q32 Q37 Q38 Q39 Q41 Q46 Q47 Q51 Q52 Q53 Q55 Q57 Q1 1.000 Q2 0.729 1.000 Q3 0,752 0.825 1.000 QIO 0.081 0.082 0.082 1.000 Q12 0078 0.Q38 0.090 0646 1.000 Q13 0.015 0.172 0.141 0.137 0.214 1.000 Q14 0.241 0.178 0.154 0.156 0.160 0.236 1.000 Q15 0097 0058 -0.006 0.029 0.058 -0.047 0.213 1.000 Q16 0.194 0.122 0.079 0.144 0.138 -0.005 0.099 0.622 1.000 Q22 -0.168 -0.168 -0.134 -0.190 -0.180 -0.462 -0.176 -0.184 -0.047 1.000 Q24 -0.142 -0.101 -0. [66 -0.043 -0.075 -0.365 -0.360 -0071 0.\00 0.547 1.000 Q25 0.019 0.024 0.072 0.107 0.182 0.385 0.329 0.041 -0089 -0.210 -0.299 1.000 Q27 0.2[7 0.240 0.170 0.257 0.239 0.145 0.747 0.132 0.133 -0050 -0.157 0.240 1.000 Q28 0.232 0.199 0.160 0.131 0.4\4 0.385 0.302 0.008 0.027 -0.250 -0.269 0.353 0293 1.000 Q29 0.204 0.162 0229 0.112 0.177 0.265 0.226 -0125 -0.124 -0.139 -0.272 0.56[ 0.171 0.242 1.000 Q32 0.187 0229 0.089 0.067 0.180 0.272 0.342 0.219 0.215 -0.299 -0.310 0160 0.290 0.484 0167 1.000 Q37 0.102 0.070 -0.042 0.164 0.148 0.022 0075 0100 0.239 -0.093 0.002 0.024 0.188 0.156 0.042 0.186 1.000 Q38 0.003 0.112 0.075 -0.052 ·0.101 0.162 0.073 -0.058 0.063 -0.015 -0062 0.032 0037 -0.011 0.027 -0.032 0053 1.000 Q39 0.037 0.074 0.109 -0.001 -0.03 [ 0.128 0.100 -0.156 -0.015 -0.100 -0.125 0.153 O. [36 0.082 0.167 -0.032 -0023 0.676 1.000 ........ 0 -.....l Item QI Q2 Q3 QIO Q12 Q13 QI4 QI5 Q16 Q22 Q24 Q25 Q27 Q28 Q29 Q32 Q37 Q38 Q39 Q41 Q46 Q47 Q51 Q52 Q53 Q55 Q57 Q41 -0.279 -0.252 -0.302 -0.224 -0.310 -0.432 -0.353 -0.065 -0.112 0.402 0.589 -0.375 -0.363 -0.297 -0.348 -0.392 -0029 -0.136 -0.218 1.000 Q46 0.202 0329 0.283 0.068 0.022 0.002 0.187 0.136 0.175 -0.058 -0.020 0.175 0.105 0.135 0.009 0.011 -0.021 0.129 0.076 -0034 1.000 Q47 0.323 0.337 0.302 0.084 0.153 0.088 0.207 0.034 0.081 -0.084 -0.024 0.1 I 1 0.188 0.268 0.128 0.148 0.099 0.033 -0.061 -0.040 0.592 1.000 Q51 0.175 0.147 0.183 0.377 0.277 0.041 0.211 0.269 0.255 -0.244 0026 0.179 0.210 0.155 0.148 0.154 0.109 0.045 0.048 -0.107 0.412 0.278 1.000 Q52 0.234 0.263 0.236 0.230 0.260 0.116 0.236 0.040 0.147 -0.001 -0.004 -0.027 0.281 0.114 0.082 0.047 0.068 0.050 -0.019 -0.076 0.252 0.221 0.514 1.000 Q53 0.137 0.139 0.106 0.106 0.080 0.210 0.349 -0.063 0.023 -0.187 -0.269 0.299 0.272 0.332 0.254 0.200 0.150 0.110 0.074 -0.184 0.293 0.266 0.244 0.267 1.000 Q55 0.094 0.181 0.225 -0.001 0.035 0.186 0.056 0.013 -0.054 -0.204 -0.233 0.159 0.028 0.266 0.269 0.051 0.021 0.094 0.126 -0.073 0.099 0.252 0.209 0.01\ 0243 1.000 Q57 0.168 0152 0.200 0.162 0.142 0.250 0.247 -0.126 0.113 -0.261 -0.221 0.165 0.204 0.340 0.181 0.160 -0.017 0.230 0.307 -0.182 0.294 0.221 0.297 0.268 0.390 0.167 1.000 >---' o 00 109 REFERENCES American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2009). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association. Arbuckle, 1. L. (2006). AMOS 7. O. Amos Development Corporation. Askamit, D., Morris, M., & Luenberger, 1. (1987). Preparation of student services professionals and faculty for serving learning-disabled college students. Journal ofCollege Student Personnel, 28(3),53-59. Beilke, 1. R & Yssel, N. (1999). The chilly climate for students with disabilities in higher education. College Student Journal, 33(3), 364-372. Blacqua, T. c., Rapaport, R J., & Kruse, B. G. (1996). Perceptions ofclassroom accommodations among college students with disabilities. ERIC Document Reproduction Service No. ED 400 640. Burgstahler, S. (2008). Universal design ofinstruction: Definition, principles, and examples. Retrieved June 16,2008, from http://www.washington.edu/doitlBrochures/Academicslinstruction.html Burgstahler, S., Duclos, R, & Turcotte, M. (1999). Preliminary findings: Faculty, teaching assistant and student perceptions regarding accommodating students with disabilities in postsecondary environments. Seattle, WA: University of Washington, DO-IT. Chubon, R. A (1992). Attitudes toward disability: Addressing fundamentals of attitude theory and research in rehabilitation education. Rehabilitation Education, 6, 301- 312. Clark, L., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3),309-319. Cohen, J. (1973). Eta-squared and partial eta-squared in fixed factor ANOVA designs. Educational and Psychological Measurement, 33, 107-112. Connell, B. R, Jones, M., Mace, R, Mueller, 1., Mullick, A, Ostroff, E. (1997). 171e principles ofuniversal design. Retrieved August 23, 2008, from http://www.design.ncsu.edu/cud/about_ud/udprincipleshtmlformat.html 110 Cope, D. (2005). Disability law and your classroom. Academe Online, 91(6). Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(7). Available online: http://pareonline.net/pdf/v1On7.pdf Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion). Journal ofthe Royal Statistical Society, Series B, 39, 1-38. Deshler, D. D., Ellis, E. S., & Lenz, B. K. (1996). Teaching adolescents with mild disabilities. Denver, CO: Love Publishing Company. Dillman, D. A. (2007). Writing questions. In D.A. Dillman (ed.) Mail and Internet Surveys: The Tailored Design Method (2nd Ed.). (pp. 32-78). New York: John Wiley. Dowrick, P. W., Anderson, 1., Heyer, K., & Acosta, J. (2005). Postsecondary education across the USA: Experiences of adults with disabilities. Journal of Vocational Rehabilition, 22,41-47. Fabrigar, L. R., Wegener, D. T., MacCallum, R. c., & Strahan, E. R. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3),272-299. Farone, M. C., Hall, E. W., & Costello, J. J. (1998). Postsecondary disability issues: An inclusive identification strategy. Journal ofPostsecondary Education and Disability, 13, 35-45. Fonosch, G. G., & Schwab, L. O. (1981). Attitudes of selected university faculty members toward disabled students. Journal and College Student Personnel, 22, 229-235. Hartman-Hall, H. M., & Haaga, D. A. (2002). College students' willingness to seek help for their learning disability. Learning Disability Quarterly, 25, 263-274. Haynes, S. N., Richard, D. C. S, & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3),238-247. Henson, R. K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation, 34, 177- 189. 111 Johnson, B., & Stevens, J. J. (2001). Exploratory and confinnatory factor analysis of the School Level Environment Questionnaire (SLEQ). Learning Environments Research, 4, 325-344. Johnson, B., Stevens, J. J., & Zvoch, K. (2007). Teachers' perceptions of school climate: A validity study of scores from the revised school level environment questionnaire. Educational and Psychological Measurement, 67(5),833-844. Ketterlin-Geller, L. & Johnstone, C. (2008). Accommodations and universal design: Supporting access to assessments in higher education. Journal ofPostsecondary Education and Disability, 19(2), 163-171. Kline, R (1998). Principles and practice ofstructural equation modeling. New York: Guilford Press. Levine, T. R., & Hullett, C. R (2002). Eta squared, partial eta squared, and misreporting of effect size in communication research. Human Communication Research, 28(4),612-625. Madaus, J. W., Scott, S. & McGuire, J. (2003a). Addressing student diversity in the classroom: The approaches ofoutstanding university professors (Universal Design for Instruction Project Technical Report No. 02). Retrieved from University of Connecticut, Center on Postsecondary Education and Disability website: http://www.facultyware.uconn.edu/TechnicalReports.cfm Madaus, J. W., Scott, S. & McGuire, J. (2003b). Barriers and bridges to learning as perceived by postsecondary students with learning disabilities (Universal Design for Instruction Project Technical Report No. 01). Retrieved from University of Connecticut, Center on Postsecondary Education and Disability website: http://www.facultyware.uconn.edu/TechnicalReports.cfm Matthews, P. R, Anderson, D. W., & Skolnick, B. D. (1987). Faculty attitudes toward accommodations for college students with learning disabilities. Learning Disabilities Focus, 3, 46-52. Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35(11),1012-1027. Messick, S. (1995). Validity of psychological assessment. American Psychologist, 50(9), 741-749. Mull, c., Sitlington, P. L., & Alper, S. (2001). Postsecondary education for students with learning disabilities: A synthesis of the literature. Exceptional Children, 97-118. 112 Murray, C., Goldstein, D. E., Nourse, S. & Edgar, E. (2000). The postsecondary school attendance and completion rates of high school graduates with learning disabilities. Learning Disabilities Research & Practice, 15, 119-127. Murray, C., Lombardi, A., Wren, C., & Keys, C. (2009). The effects of disability-focused training on attitudes and perceptions of university staff. Learning Disability Quarterly, 32(2),87-102. Murray, C., Wren, c., & Keys, C. (2008). University faculty perceptions of students with learning disabilities: Correlates and group differences. Learning Disability Quarterly, 31, 1-19. Murray, C., Wren, c., Stevens. E., & Keys, C. (2009). Promoting university faculty and staff awareness of students with learning disabilities: An overview of the productive learning u strategies (PLUS) project. Journal ofPostsecondary Education and Disability, 22(2), 117-129. Nelson, J. R, Dodd, J. M., & Smith, D. J. (1990). Faculty willingness to accommodate students with learning disabilities: A comparison among academic divisions. Journal ofLearning Disabilities, 23(3), 185-189. Nunnally, J. C. (1975). Psychometric theory: 25 years ago and now. Educational Researcher, 4(10), 7-21. O'Connor, RP. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer's MAP test. Behavior Research Methods, Instruments, and Computers, 32(3),396-402. Ouellett, M. L. (2004). Faculty development and universal instructional design. Equity and Excellence in Education, 37, 135-144. Parker, D. R, Embry, P. R, Scott, S. S., & McGuire, J. M. (2003). Disability service providers I perceptions about implementing Universal Design for Instruction (UDI) on college campuses (Universal Design for Instruction Project Technical Report No. 03). Retrieved from University of Connecticut, Center on Postsecondary Education and Disability website: http://www.facultyware.uconn.edu/TechnicaIReports.cfin Patton, M. Q. (1982). Thoughtful questionnaires. In M.Q. Patton (ed.) Practical Evaluation. (pp. 139-159). Newbury Park, CA: SAGE. Preacher, K. P., & MacCullum, R C. (2003). Repairing tom swift's electric factor analysis machine. Understanding Statistics, 2(1), 13-43. 113 Rao, S. (2004). Faculty attitudes and students with learning disabilities in higher education: A literature review. College Student Journal, 38(2), 191-198. Reed, M. J., Lund-Lucas, E., & O'Rourke, K. (2003). Standards of practice in postsecondary special needs programming: Student and administrator opinion. The Canadian Journal ofHigher Education, 2, 27-56. Scott, S. S., McGuire, J. M., & Shaw, S. F. (2003). Universal design of instruction: A new paradigm for adult instruction in postsecondary education. Remedial and Special Education, 24(6),369-379. Shaw, S. F., & Dukes, L .L. (2001). Program standards for disability in higher education. Journal ofPostsecondary Education and Disability, 14(2), 81-90. Skinner, M. E. (2007). Faculty willingness to provide accommodations and course alternatives to postsecondary students with learning disabilities. International Journal ofSpecial Education, 22(2), 32-45. SPSS, Inc. (2006). SPSS 15.0 for Windows Graduate Student Version. Lead Technologies. Stevens, J. P. (2002). Applied Multivariate Statistics for the Social Sciences. (4th ed.) Mahweh, NJ: Lawrence Erlbaum Associates, Publishers. Stevens, J. J., & Zvoch, K. (2007). Confirmatory factor analysis and the terra nova comprehensive tests of basic skills. Educational and Psychological Measurement, 67(6), 976-989. Tabachnick, B. G., Fidell, L. S. (2001). Using multivariate statistics (4th Ed.). New York: Harper & Row. U.S. Department ofEducation, National Center for Education Statistics. (1999). Students with disabilities in postsecondary education: A profile ofpreparation, participation, and outcomes. NCES 1999-187. Washington, DC: U.S. Government Printing Office. U.S. Department of Health and Human Services. (2003). Trends in the well-being of America's children and youth. Washington. DC: U.S. Government Printing Office. U.S. Department of Labor (2007). College enrollment and work activity of2006 high school graduates (USDL 07-0604). Retrieved March 12,2008 http://www.bls.gov/news.release/hsgec.nrO.htm. 114 U.S. Department of Labor (2008). Education pays in higher earnings and lower unemployment rates. Retrieved May 1, 2008, http://www.bls.gov/emp/emptab7.htm University of Oregon, Office of Affirmative Action and Equal Opportunity. (2007). Affirmative Action Plan. Retrieved October 27,2008 from: http://aaeo.uoregon.edu/plan.html University of Oregon, Office ofInstitutional Research. (2007). Faculty counts by type. Retrieved October 23,2008 from: http://ir.uoregon.edu/data Vogel, S. A., Leyser, Y, Wyland, S., & Brulle, A. (1999). Students with learning disabilities in higher education: Faculty attitudes and practices. Learning Disabilities Research & Practice, 14, 173-186. Vogel, S. A., Holt, J. K., Sligar, S., & Leake, E. (2008). Assessment of campus climate to enhance student success. Journal ofPostsecondary Education and Disability, 21(1),15-31. Wagner, M., Newman, L., Cameto, R., Levine, P, and Garza, N. (2006). An Overview of Findings From Wave 2 ofthe National Longitudinal Transition Study-2 (NLTS2). Menlo Park, CA: SRI International. Retrieved October 20, 2007 from: www.nlts2.org/reports/2006_08/ Wagner, M., Newman, L., Cameto, R., Garza, N., and Levine, P. (2005). After High School: A First Look at the Postschool Experiences of Youth with Disabilities. A • Report from the National Longitudinal Transition Study-2 (NLTS2) Menlo Park, CA: SRI International. Wagner, M., Newman, L., Cameto, R. (2004). Changes over time in the secondary school experiences ofstudents with disabilities. A report offindings from the National Longitudinal Transition Study (NLTS) and the National Longitudinal Transition Study-2 (NLTS-2). Menlo Park, CA: SRI International. Yuker, H. E., Block, J. R., & Campbell, W. J. (1960). A scale to measure attitudes toward disabled persons. Albertson, NY: Human Resources Center.