MINUTE

SPECIAL COMMISSION ON 21ST CENTURY COLLEGES AND UNIVERSITIES

Tuesday, May 9, 2000
10:30 a.m.
Stephen F. Austin State University
Nacogdoches, Texas

*****

Pursuant to a notice posted in accordance with Senate Rules, a public hearing of the Special Commission on 21st Century Colleges and Universities was held on Tuesday, May 9, 2000.

*****

MEMBERS PRESENT
Chairman Jim Adams
Senator Teel Bivins
Commissioner Tony Garza
Dr. Betsy Jones
Elaine Mendoza
Jeff Sandefer
Karen Shewbart
Danny Vickers
Senator Royce West

MEMBERS ABSENT
Kirbyjon Caldwell
James Hooten
Margie Kintz
Nancy Marcus
Senator Bill Ratliff
Senator Carlos Truan
Pam Willeford (ex-officio)

*****

Chairman Jim Adams called the meeting to order at 10:30 a.m. There being a quorum present, the following business was transacted.

Chairman Adams thanked Stephen F. Austin State University for hosting the meeting.

Chairman Adams announced that the next meeting of the Commission will be held on June 12, 2000 in El Paso, Texas. He informed the members that Commission staff would be sending information on scheduling future meetings and explained the format for the meeting.

Chairman Adams introduced Dr. John Yopp, Vice President of Graduate and Professional Education for the Educational Testing Service.

Dr. Yopp's presentation focused on standardized testing and its role in the admissions process and educational assessment. He said Texas is one of the top users of ETS tests and scores. He testified that standardized tests are used to provide institutions, students, parents, accrediting agencies, employers, faculty, and the public with valid, reliable assessments of students' abilities and to ensure accountability in the educational system.

Fourteen higher education institutions in Texas use academic profiling to measure outcomes to assess the effectiveness of the first two years of higher education. Academic profiling is a test of college-level reading, writing, mathematics, and critical thinking in the context of content areas. It is used by 2-year and 4-year institutions in 50 states to measure the skills gained through the general education core curriculum. The tests are used to compare peer institutions, are not used to hold students back from upper level courses and are given to all students rather than to a random sample.

Dr. Yopp went on to say that ETS major field tests are also used in Texas to measure effectiveness in certain subject areas. They provide for national, peer, and student sub-group comparisons. Thirty-five Texas institutions use the ETS major field tests. They are very popular nationally and in other countries.

Jim Adams stressed the Commission's interest in the fairness issue. Dr. Yopp said ETS places a great deal of emphasis on making sure tests are not biased and said that the tests go through many stages of sensitivity analysis.

Historically, tests are written to assess median performance. Students are given tests with questions that have been pre-tested for the average ability of students to get the right answer. With new technology, Computer Adaptive Tests measure student performance for each question and tracks each student's precise ability level. CATs allow for year-round and continuous testing. Students can see their scores right away. Students are monitored by two monitors and with cameras during the test.

To support the validity of the tests, ETS has audit standards for equality and fairness. The tests are reviewed by outside organizations, and ETS continually monitors to make sure the test is measuring what it was meant to measure. ETS provides information to the score users and test takers about the appropriate uses of test scores. It is easy for universities to rely heavily on test scores to sort students. The test scores are not intended to be relied upon as a sole means of making decisions.

In response to Senator West's question of how much weight universities should give test scores, Dr. Yopp said that institutions should give test scores enough weight to yield the students who best match the mission of the institution. Scores are especially useful when all the applicants have high GPAs.

Senator Bivins noted that there is an inherent conflict between schools that use test scores as a sorting mechanism and ETS statements that schools should not rely so much on the test for that purpose even though the test scores can often help at schools where GPAs tend to be nearly identical.

Jeff Sandefer said some schools like Harvard give lower weight to standardized scores because tests measure only part of what is needed to be successful, but that test scores, especially in mathematics, can be very predictive of performance.

Senator West asked how ETS conducts sensitivity review, noting that ETS has been heavily criticized on this point. Dr. Yopp said that ETS looks at the whole test creation process to ensure fairness, equity and that the tests meet the needs of multiple cultures. Each stage of the question-writing process is subject to review.

They look at the performance of subgroups on each item. If an item functions poorly in certain cultures, ETS eliminates it from the test. Senator West expressed interest in meeting with Dr. Yopp at another time in the future to further explore test development.

Admissions tests measure the outcome of previous educational experience. There are several factors that influence success that are not measured by standardized tests. Senator West asked why the tests should not be constructed to measure those non-cognitive skills which can be used to predict the same potential for success as cognitive skills. Yopp responded that the tests are devised to test for cognitive skills, and that certain amounts of information from cognitive skills tests are needed, such as in math for engineering.

Chairman Adams introduced Dr. David England, President of North Lake Community College Performance Measures Committee, who gave an overview of how community colleges manage performance measures and accountability. Community colleges currently operate under nine legislatively-mandated performance measures: course completers, number of hours taught by full time faculty, percentage of economically disadvantaged students, percentage of students passing licensure exams, number of transfers to a university, percentage of minority students enrolled, percentage of remedial students who pass the TASP, percentage of academically disadvantaged students, and the number of degrees or certificates awarded.

Dr. England testified that the Texas Association of Community Colleges recommends a more extensive system. He noted that community colleges have a multi-faceted mission and that they have value not reflected in graduation or transfer rates. He cited a need to find a way to track outcomes to find out if community colleges are succeeding in their mission.

Dr. England discussed a proposal that would identify all possible positive outcomes, such as marketable skill achievers and students who leave institutions in good standing but before graduation. He said that many students come to school without the intent of graduating. They only seek to get certain skills to qualify for better jobs. Corporations will hire away students before they graduate.

The institutional effectiveness system at the Higher Education Coordinating Board would provide follow-up. Ineffective programs can be eliminated. The idea is that there would be a system of continuous improvement based on requirements for strategic planning, etc.

Dr. Daniel Bonevac and Dr. Robert Koons, both Professors of Philosophy at the University of Texas at Austin, provided testimony on the proposed Texas Excellence through Assessment of Teaching in Higher Education Examination, or TEATH testing. They testified that TEATH would be an important way to measure student learning so that good schools could be rewarded and used as models. In the absence of such measures, legislative funding decisions tend to be political.

Currently, faculty are evaluated by publications, and student surveys that measure their opinion of the process and professors' personalities rather than on how much they learned. TEATH would be a way to hold institutions accountable rather than students, though the test scores would appear on students' transcripts.

TEATH scores would be matched to SAT scores to evaluate the value added rather than absolutes and the information would be made available to the public. This type of information would provide consumers with unbiased information on institutional effectiveness, focus university administrators on the educational bottom line, foster collaboration among institutions, and give students an opportunity to document performance, among other benefits.

Senator West asked how testing would fit the overall objectives regarding questionable validity, reliability of tests and still use it as assessment instruments. Dr. Koons responded that the TEATH test factors out equity problems because it measures value added.

Senator Bivins asked how TEATH would work at schools that do not require the SAT for admission. Dr. Koons said there would have to be an admission test. Senator Bivins expressed concern that the TEATH could stigmatize the graduate who doesn't test well.

Elaine Mendoza raised the concern that an entire industry such as the prep courses and preparatory materials for SAT and GRE would be the result. Dr. Yopp responded that students should not need more than familiarization in theory for the TEATH. The difference is that a student's future is placed on the SAT. TEATH is a low stakes test based on classroom experience.

Dr. England pointed out that it takes a lot of resources to do standardized tests and noted that TASP has not changed the bottom line after millions of dollars have been spent on it.

Danny Vickers said you have to measure somehow and that any test is flawed. He liked the idea of putting scores on a transcript and said TAAS forces accountability, however flawed. Senator Bivins said maybe the assessment should be between TASP and TEATH, but he questioned how TEATH would be applied to music students.

Dr. Yopp agreed that would be problematic. Major field tests could be used as achievement tests, but it would be difficult to get a delta. Academic profile tests are taken after general studies, whereas TEATH would be taken at graduation.

Betsy Jones questioned the use of a standardized course of study beyond general studies. Dr. Yopp said major field tests tend to be successful because there is some consensus on what every student in a particular field should know.

Ms. Mendoza questioned how institutions can be prevented from teaching to the TEATH as some public schools have come to do with TAAS. Mendoza said that would be missing the point of a college education.

Karen Shewbart asked how accepted the TEATH proposal was. Koons acknowledged that the small sphere of endorsements was limited to colleagues. No empirical advantage is built in for various institutions. General faculty groups do not want to be evaluated.

Senator Bivins asked Dr. Yopp what was most predictive of performance at a 4-year institution. Yopp replied that it depends on the institution and said that GPAs balanced with standardized scores should be considered in context with the rigor of the institution.

Chairman Adams introduced Representative Todd Staples, R-Palestine, who addressed the Commission on Stephen F. Austin State University's past role and future potential in providing higher education opportunities in East Texas.

Chairman Adams introduced the following students from the George Bush School of Government and Public Service at Texas A&M University: Mr. Dan Ray, Ms. Brenda Aguilar, Ms. Mary Helen Harris, Mr. Daniel Morales, Ms. Rebecca Norris, and Ms. Jill Shaunfield. Chairman Adams presented the students with certificates signed by Lieutenant Governor Rick Perry. The students discussed the report, Beyond the Barriers: Issues and Ideas in Improving Access, which they prepared for the Commission.

The Bush School report focuses on the following areas: definition and integration of access goals, a higher education campaign, higher education preparation, financial aid information and disbursement, and partnerships with employers. The students compiled data from high school counselors and research, and from interviews with the Texas Workforce Commission, institutions with retention programs, and secondary education experts. All Texas Education Agency regions are represented in the survey statistics. 97% are from public schools; 78% are from schools where the average family income is less than $30,000.

In response to Ms. Shewbart's question about the reports recommendation that TAAS and TASP be linked and that a preparatory academy be created for remedial coursework, Mr. Ray responded that community colleges are penalized for students going on to college. Ms. Shaunfield added that students do not know they need remediation until it is too late. She suggested that if students took the TASP immediately after the TAAS, any need for remediation could be identified, and students could take those classes while still in high school. Senator Bivins commented that Senate Bill 103 of the most recent legislative session may accomplish that.

Ms. Aguilar commented that the longer it takes a student to get through the process, the more likely something else will interfere with completion of the student's education.

Chairman Adams brought up the minutes from the April 11, 2000 meeting for approval. Senator Bivins asked that approval of the minutes be deferred to the next meeting so that the minutes can be amended.

There was no public testimony.

There being no further business, at 2:25 p.m. Chairman Adams moved that the Commission stand adjourned. Without objection, it was so ordered.

Jim Adams, Chair
Kimberly Berry, Commission Clerk

Top