ALASKA STATE LEGISLATURE SENATE HEALTH & SOCIAL SERVICES COMMITTEE  February 12, 2001 1:35 p.m. MEMBERS PRESENT Senator Lyda Green, Chair Senator Loren Leman Senator Gary Wilken Senator Jerry Ward Senator Bettye Davis MEMBERS ABSENT  All Members Present   COMMITTEE CALENDAR  Discussion of SB 36 from the 20th Legislative Session, benchmark assessments and the High School Graduation Qualifying Exam. WITNESS REGISTER  Ms. Shirley Holloway, Commissioner Department of Education & Early Development th 801 W 10 St. Juneau, AK 99801-1894 Mr. Eddy Jeans, Manager School Finance and Facilities Section Department of Education & Early Development th 801 W 10 St. Juneau, AK 99801-1894 Mr. Bruce Johnson, Deputy Commissioner Department of Education & Early Development th 801 W 10 St. Juneau, AK 99801-1894 Mr. Phillip Reeves, Assistant Attorney General Human Services Section Department of Education & Early Development 801 W 10 St. Juneau, AK 99801-1894 ACTION NARRATIVE TAPE 01-10, SIDE A  Number 001 CHAIR LYDA GREEN called the Senate Health & Social Services Committee meeting to order at 1:35 p.m. Present were Senator Leman, Senator Wilken, Senator Davis and CHAIR Green. Senator Ward arrived at 1:36 p.m. MS. SHIRLEY HOLLOWAY, Commissioner, Department of Education and Early Development (DOEED), said Mr. Eddie Jeans would cover SB 36, from the 20th Legislative Session, and then begin the first of five work sessions on the benchmark assessment exam and the High School Graduation Qualifying Exam (HSGQE). MR. EDDIE JEANS, School Finance Manager, DOEED, said his presentation would be based on the executive summary of the report to the state legislature. The report was a result of reporting requirements under SB 36. SB 36 required DOEED to prepare three reports on the following topics: 1) district cost factors, 2) comparison of old and new funding formula, and 3) education adequacy report. MR. JEANS said DOEED tried to use the methodology that CTB McGraw Hill and the McDowell Group used to determine district cost factors that were adopted by SB 36. After running an analysis, it was apparent that the McDowell Group methodology was no longer appropriate or amenable to update. DOEED then entered into a contract with the McDowell Group to have them look into the analysis to see if they came to the same conclusion. The McDowell Group did find that the methodology used in 1998 to develop the cost factors that are currently in statute could not be used. They recommended that DOEED seek funding for a new cost study based on the actual cost of providing the services around the region as opposed to what districts are spending. DOEED recommends that an appropriation be made for a contract to develop a new methodology that can be updated on an ongoing basis. Number 307 SENATOR LEMAN asked if the money the districts are spending would not confirm the existing distribution because they are probably spending close to what is appropriated. He asked if there should be some type of common standard to compare against. MR. JEANS agreed. In the last few studies, the McDowell Group has based the cost factor on what districts are currently spending, not what it costs them to provide the service. SENATOR LEMAN noted that districts should be compared on a reasonable standard. CHAIR GREEN asked if the chart of accounts reconciliation addresses this problem. MR. JEANS said the state board had just adopted a revised chart of accounts that would go into effect July 1, 2001, which would provide more detail on what districts are spending. The revised chart also gives districts additional guidance on how to classify expenditures. DOEED does not believe the audited financial statements are the appropriate tools for developing a cost differential. MR. JEANS said the next report comparison of the old funding formula to the new formula. DOEED had two findings for this section of the report. One is for the erosion of the supplemental funding floor. The other is a provision under one of the old funding formulas that was an oversight in the rewrite of SB 36, which provided a hold-harmless type of funding for districts that experience a rapid decrease in enrollment. This provided a stair- step approach so the reduction did not have to be absorbed all in one year. The erosion of the funding floor takes money away from districts whose funding is based on cost differentials that may not be appropriate. Until DOEED has developed a new methodology to determine what the differentials are, it is recommending the repeal of the erosion of the supplemental funding floor. The Governor's funding task force has also recommended the suspension of the erosion of the supplemental floor until new cost factors are developed and adopted by the legislature. MR. JEANS said the third section of the report is on education adequacy. The report recommends that all of the DOEED findings be given to the Governor's education funding task force. Number 651 MR. BRUCE JOHNSON, Deputy Commissioner, DOEED, said content standards started in the early 1990s. They were voluntary and designed to give school districts targeted help in the development and alignment of their curricula. Hundreds of Alaskans were involved in the construction of the content standards in 10 areas, which has now evolved to 12 areas, and also in public hearings. The passage of the competency exam requirement in 1997 is what moved this effort forward. CHAIR GREEN asked for the definition of content standards. MR. JOHNSON said content standards identify contents that Alaskans felt were important in 10 areas. Performance standards are a subset of content standards and they have to be measurable. Alaska moved from content standards to performance standards in reading, writing, and mathematics, which hold young people accountable through benchmark assessments and the HSGQE. Some content standards do not lend themselves well to performance standards because of things like appreciation - "How do you measure appreciation with a paper and pencil exam?" MR. JOHNSON said the process was done by Alaskans and guided by McGraw Hill publishing. When the competency law was passed, Alaska did not have the expertise to develop its own exams. McGraw Hill guarantees legal defensibility such as, this test question measures this standard, and they also go to court to help with this aspect. At the same time, they allow Alaska to direct the process. Alaska determines the standards, chooses the test questions from McGraw Hill's item pool, and Alaska revises some of the test questions in order to fit urban and rural life equally well. The first exam was field tested in 1998, both the benchmark exam and the HSGQE. After the field test, groups of Alaskans came together again to look at how well the questions worked on the field test. The questions they asked were: Did an urban student answer a question as well as a rural student or was there something wrong with the question; Did the question favor one group over another, such as girl versus boy, or one race over another? When a question was found that did not work well statewide, that question was thrown out. About 40 percent of the field questions were eliminated. Number 1022 CHAIR GREEN asked if the test changes each year. MR. JOHNSON replied that the test does change from year to year. From edition to edition, some of the same questions are used but there are always new questions. There can never be a new question unless it has been field tested first. McGraw Hill guarantees, from version to version, that there is equivalency and the difficulty level is the same. CHAIR GREEN asked if a field test could be given to students in other states. MR. JOHNSON said not in a standard assessment because Alaska's standards are not the same as other states' standards. MR. JOHNSON said field testing led to the development of the first three versions of Alaska's HSGQE, which were administered in the spring of 1999. The benchmark assessment was also developed at this time for grades 3, 6 and 8. For four or five years, a single test would be used for the benchmark assessments and then the test would be revised. MR. JOHNSON said that after the exams were administered in the spring of 1999, groups of Alaskans convened to establish the passing score for the HSGQE. The passing score was determined and the HSGQE is a pass, no pass exam. There are five opportunities to take the exam during normal high school years and six additional times following the normal date of graduation. Number 1225 SENATOR LEMAN asked if the pass, no pass scores are different for each of the three sections. MR. JOHNSON answered yes. There were committees of 18 to 20 people, including parents, business people and teachers, in each of the three disciplines, all working independently of each other. There was no connection between the passing score from reading to math or from reading to writing. SENATOR LEMAN asked what the passing scores are for each of the sections. MR. JOHNSON said scores ranged from a low in reading of 305 to a high in mathematics of 383, writing fell in between at 340. Number 1333 SENATOR LEMAN said some of his constituents do not understand how the tests were developed. MR. JOHNSON said four levels were established for the benchmark test score levels - advanced, proficient, below proficient, and not proficient. Federal requirements require information be reported to the federal government on the scores - it cannot be pass, no pass. One of the objectives of the benchmark test is to provide families, teachers, and schools with feedback on how well the students are doing. CHAIR GREEN asked if the federal government would accept results from any test. MR. JOHNSON said it is up to each state to determine how it will assess, although most states are moving toward standard based testing. MR. JOHNSON said results of the benchmark assessments were distributed in October of 1999. DOEED then distributed the results to school districts and the school districts are required to distribute the results to families, students and teachers in a way they determine appropriate. MR. JOHNSON said teachers need to know what students are being held accountable for - DOEED does this in two ways; first with the establishment of practice exams. A contract last spring with the Anchorage school district developed practice exams in reading, writing, and mathematics for the high school level. The practice exam did not go through all the iterations of the actual test but it gave a good indication of the types of questions. Second, DOEED published a HSGQE content guide. The guide went to all high school teachers who had responsibilities related to teaching students in these areas. The guide is an attempt to break down the standards so they were more understandable, as well as provide guidance on how the standards might be tested on the exam. Right now, a guide is being developed for the benchmark exam. CHAIR GREEN asked when the HSGQE guide was released. MR. JOHNSON replied last year. He said the guide was an effort to take away the mystique of a highly confidential test. CHAIR GREEN noted that a teacher could use the guide to develop skills to work from. She asked if there would be a time when more than one practice test would be developed for use by teachers. MR. JOHNSON said that would be ideal but it would cost a lot of money. One way to keep the cost down is to not own the test. Presently, tests are a joint effort between McGraw Hill and Alaska. McGraw Hill maintains the copyright on the test because some of the questions are being used elsewhere. Versions of a test could be released in the future but only because they are no longer being used anywhere else. For the first time, a practice exam is now on line for the HSGQE. SENATOR LEMAN asked if the other test is still on line. MR. JOHNSON replied yes, but it will probably be discontinued. CHAIR GREEN said a teacher could take the model questions and insert different numbers and still be able to use the test for practice, particularly for math. MR. JOHNSON said it is hoped that teachers will become more skilled at using similar types of assessment tools in their regular classroom instruction. CHAIR GREEN said as long as this falls within performance and content standards, a teacher would not be teaching to the test, a teacher would be teaching information that would be tested. MR. JOHNSON said DOEED learned that some of the test questions and how they were asked was perplexing to students. MR. JOHNSON brought the continuous renewal chart to the committee's attention. He said the content team met in Anchorage for two days and liked the standards used for reading, writing, and mathematics. They then looked at the test itself in regard to specific questions. One question that was quickly determined to be necessary was: Is this skill really something necessary for success in later life? In other words, the operational definition became: If the student does not get the question right, should he or she be denied a diploma? This definition is different from anything that has been used up to this point. MR. JOHNSON passed out material that showed a circle indicating performance standards - the targeted and taught skills and content. Mr. Jeans said the committee is leaning toward the notion that not all performance standards are key for later success in life. CHAIR GREEN asked if certain questions could be targeted. MR. JOHNSON said it could be done, it would be much like the benchmark assessments with four scores determining four levels. Number 1953 CHAIR GREEN asked if there would be a problem with a lack of base from which to build in the lower grades. She asked if there were things students would not have to perform perfectly in order to pass to another grade MR. JOHNSON responded yes, some are foundational. If a foundational performance standard is missing, then it will be difficult for a child to be successful at the next level - this is where the energy should be targeted. SENATOR WILKEN said his calculations show a student would have 15 chances to take the tests instead of 11. MR. JOHNSON said tests could be taken starting in the spring of a sophomore year, fall and spring of the junior year, and fall and spring of the senior year. This would total five attempts. Then, a student could take it again during the fall and spring for three years after the normal date of graduation. SENATOR WILKEN agreed with the first five assessments but said a student could be enrolled in high school until 20 years of age, for two more years or four more opportunities. There is also a trip date of 23; a student could take the test up to their 24th birthday, which gives them six more opportunities. MR. JOHNSON said children with disabilities could stay in K-12 up to the age of 23. SENATOR WILKEN noted the traditional student could take the test 10 times. He asked for the confidence interval the test is designed to be valid within MR. JOHNSON responded 95. SENATOR WILKEN explained the confidence interval is 95 - on any given day, out of 100 students, 5 students that should have passed did not, or five students that should not have passed did. SENATOR WILEN asked if 99 was the original competency level goal. MR. JOHNSON said there was not a target for establishing that error of measurement. Phasing the cut scores was considered at one time so there would be one passing score and after that there would be a higher level passing score. But it was found that very few students benefited from the phased approach. SENATOR WILKEN asked if the 95 percent competency interval was accepted across the United States or whether it is something Alaska determines and then builds around. He also asked if other states have tests designed for higher than 95 percent MR. JOHNSON said other states do have higher percents but DOEED's approach was to look at where the scores should be set and where the standard error of measurement should be. DOEED wanted a broader standard and to accomplish this, the standard had to be set at the lower end of the range. For Alaska's first test, the 95 percent confidence interval was desirable so that a diploma would not be denied to a student who knew the information. Number 2295 SENATOR WILKEN asked what the committee did in August and how that differs from what will be done at the next meeting. MR. JOHNSON said the same thing was done but a broader question was necessary. "Is this standard essential for later life for all kids?" was determined to not be a narrow enough definition. To set a new passing score, with a different emphasis, another question had to be asked: If a student does not know the standard will they be denied a diploma? The standard may sound like a necessary standard for a foundational education but when looking at the range of difficulty within that standard, the standard can only be accomplished with individual test questions. CHAIR GREEN asked for an example. MR. JOHNSON said in a writing example a student may be asked to explain an answer in two or three sentences. There will be both highly complex responses as well as basic responses. The committee recognized that both could be right, depending on what the target is for a foundational education. TAPE 01-10, SIDE B  SENATOR WILKEN asked if the same committee would be asked a different question or whether a different question would be asked of a different committee. MR. JOHNSON replied a different committee that contains some of the original members to allow for representation from prior efforts would look to see if the test emphasis is correct. After this is completed, the item pool of questions would be looked at to see if there are sufficient questions within the pool that have been field tested in Alaska and which are appropriate for constructing a new version of the test. Once the new version of the test has been given, a new passing score would be set. SENATOR WARD asked how group representatives were found. MR. JOHNSON said volunteers were solicited from around the state and people were also nominated. A pool was then created and a representation was selected. McGraw Hill set a minimum of 15 content experts to be within the discipline, which left six slots open. SENATOR DAVIS asked how the February test would differ from the last test. MR. JOHNSON said there would be no difference. It will be an identical test but questions will be imbedded for field test purposes. SENATOR DAVIS asked what the time line is for the new version. MR. JOHNSON replied that it hinges upon whether there are enough items in the current pool to construct a new exam around the emphasis the group wants. If another field test is needed, the next opportunity is in the spring of 2002. A new exam would then be constructed with new questions and it would be administered in the spring of 2003 and then the passing scores would be set. The new passing scores would be for the graduating class of 2004. On the other hand, if there are enough items within the pool, the time could be shortened, but not before 2003. CHAIR GREEN noted that the school designator committee would begin a series of meetings in February of 2001 and recommend school designators. She asked, under this time line, how many years would this require for the establishment of a record? MR. JOHNSON said this is a question being considered by the committee and it will be brought before the board in March. MR. PHILLIP REEVES, Assistant Attorney General, DOEED, said there is not much direct information available on the defensibility of the Alaska exam because there is no case law looking directly at the exam. There are two federal decisions that provide criteria that a court would apply in reviewing the validity of an exam. These cases are from Florida and Texas and, in both cases, a challenge was brought under Title VI of the Civil Rights Act based on a claim of disproportionate failure rates by minority students. Title VI would be a likely avenue for a challenge in Alaska because if a claim were made under the Civil Rights Act and the plaintiff showed a significant percentage difference in the passing rates of a minority group versus a majority, the burden would shift to the state to defend its program. CHAIR GREEN asked for the broad definition of a minority. MR. REEVES said the Civil Rights Act refers to racial minorities and possibly religious minorities. MR. REEVES said in the Texas case, the baseline question of the state's burden was that the state had to show it had an educational necessity for its examination requirement. However, the court explained that while this sounds like a high burden, the state would be charged with showing that the challenged practice serves as a legitimate educational goal to the institution. The courts are unlikely to modify a state's policy decision on what the standards should be. Courts would instead focus on whether the state has reasonably implemented that standard through an educational program that gives opportunities to the students to learn what is necessary to pass the test. The two main areas a court would focus on are content validity and instructional validity. Content validity could be phrased in the question: Does the test accurately measure the test taker's knowledge in the content area being tested? Alaska has contracted with a corporation, a testing expert, to develop test questions and assist DOEED in utilizing resources from across the state. The test is based on the state board adopted performance standards. There was a focus on ridding the test of questions that had a cultural bias and there is a continuous renewal process to evaluate and improve the exam. MR. REEVES said instructional validity, curricular validity or opportunities to learn, as a legal issue, are all the same concept. A way to phrase this issue is: Does the curriculum and the total educational program offer each student a reasonable opportunity to gain the knowledge and skills that are tested? This question looks at the educational program in determining if schools are teaching to this standard. In order to hold a student accountable, courts want to determine whether the student has been given the opportunity to learn. The Texas and Florida cases provide a useful outline of criteria that courts apply. In each case there was a multi-year test implementation program. The court imposed the multi-year program in Florida, which was first utilized in 1978, and in 1979 the test had to be passed to gain a diploma. The court imposed a four year injunction and enjoined the state from withholding diplomas for four years after the requirement was put in. In Texas, the challenge came eight years after the test was made a requirement for graduation. The court looked at the eight- year track record and found that the state-mandated remedial programs were a success. In each case there was a time span, which showed the test was having the effect of improving the passage rates of the group that was contesting the test. CHAIR GREEN asked if the Texas decision was handed down January 7, 2000. MR. REEVES said to his recollection the initial exam was put in place in the late 1980s and was then changed in 1990. The 1990 exam was the contested exam. CHAIR GREEN asked how long the exam had been under review. MR. REEVES noted the challenge was held in 1999. The test became mandatory in 1990 and Mr. Reeves was not aware of when the case was originally filed, but the court heard evidence on the defensibility of the test in 1999. Mr. Reeves said that much of the case argument was over issues, such as whether the court should look at improvement over the years or disparity in the failing rates between the plaintiffs and the majority group. CHAIR GREEN asked if failure of the Texas Assessment of Academic Skills (TAAS) would keep a student from graduating. MR. REEVES answered yes. TAAS is the current exam and passage of it is required to receive a diploma. CHAIR GREEN asked if Texas offers other graduation certificates. MR. REEVES said TAAS is a mandatory requirement, but there were other opportunities. The court only looked at the requirement that the test be passed in order to receive a diploma. CHAIR GREEN said she has seen information indicating that the TAAS failure rate was about 2 percent. Truancy is also very low in Texas because the police are "rounding them up" because schools are being held accountable and teacher reviews depend on student test results. CHAIR GREEN asked for the final verdict in the Texas decision. MR. REEVES said the court upheld the validity of the diploma requirement. CHAIR GREEN asked about the Florida cases. MR. REEVES said there are three cases in Florida. In the first case, the district court issued a four-year injunction. The second case was upheld and the court upheld the examination in the third case. CHAIR GREEN asked if the test in Florida was mandatory. MR. REEVES said yes. CHAIR GREEN said she does not think standards should ever be lowered but she wondered if modest requirements for graduation could be put in statute. She wondered if the Texas and Florida cases asked the question: Could a student be tested for something more challenging than what the state or district required as subject content? Number 1517 MR. REEVES said it is clear under Title I and the federal law on special education requirements that there needs to be a system of assessments - there cannot be one factor that denies a student a diploma. However, the courts have upheld the idea that a multiple assessment system can be one in which a student has to pass the exam, complete course work and meet attendance requirements. CHAIR GREEN asked if a student could be tested on something greater than what he or she is required to learn? MR. REEVES said the state would fail under instructional validity if it could be shown there was not an educational program to provide training for passage of the test. SENATOR WARD asked Mr. Reeves if he had looked at the Alaska test to see if it was legally defensible. MR. REEVES said he has not done an analysis on the Alaska testing program under the instructional validity program. He has not looked at the content validity, which is what he was referring to when he said Alaska was relying upon the expertise of the contractor and its process for putting the test together. SENATOR WARD said he wants to know if the scoring criteria is legally defensible. He talked with some of the people on the committee who indicated they did not participate in the final product. Alaskan teachers who were on the committee have also said this is not their test. He would like to know if the test is testing subject matter that has not been taught to all students and, if so, who did this and when. Senator Ward asked Mr. Reeves to look into his questions to see if there is a problem so that it can be fixed. Number 1209 MR. REEVES went through other components of instructional validity because this is where the greatest potential exposure is in applying these concepts to Alaska's program. The main way Alaska has addressed the instructional validity criteria is through the implementation of a comprehensive testing program including benchmark examinations at third, sixth, and eighth grade. The benchmark exams are designed to identify students who are falling behind the curve at a certain point. The tests allow for focused instruction and remediation throughout a student's educational career. The tests allow districts to identify and correct problems with curriculum starting at earlier grades and they provide data to DOEED that help the department assist districts in curriculum development. The assessment programs depend on the benchmark exam process to identify and respond to those instructional problems on the individual and curriculum levels. The tests would also defend a challenge to the instructional validity of the graduation exam by pointing to the interventions the benchmark exams provide showing a legitimate educational program. This leads to the potential problem based on timing. The benchmark exams were first given in the spring of 2000, which means the first sixth grade class that participated in the exam graduates in 2006. The class of 2002 was not involved in the benchmark exam process so those students can legitimately claim they did not have specific notice of what the requirements were until the exam was given. Timing is the biggest challenge in defense of the program. MR. REEVES referred to a resource guide published in December of 2000 by the Office of Civil Rights of the United States Department of Education. The article provides an expansive discussion of high stakes testing across the country and problems that have arisen. There is a common problem in states that attempt to use a single exam for the dual purposes of driving curriculum development and holding students accountable. If the dual purposes are pursued, inevitably, by design, a gap is created between what the test measures and what the student has been taught. It is only with the result of the first test that the curriculum can be developed to ensure the right educational program is given. To accomplish both things from the exam, a lag is needed between the first exam and when students are held accountable so that it can be said the state provided students with an educational program that allowed them to pass the exam. MR. REEVES said that as far as special education is concerned, there are two federal laws that protect students with disabilities. The first is the Rehabilitation Act, which requires that any state program that accepts or expends federal funds not discriminate on the basis of a person's disability. The second is the Individuals with Disabilities Education Act (IDEA-97), which has more specific protections. Some of the protections that directly impact the testing are that children with disabilities must, to the maximum extent practical, be included in general state assessment programs with appropriate accommodations. These cases have uniformly held that the fact a standard is set beyond the capacity of certain children with disabilities does, in itself, violate federal laws on disabilities. Essentially, legislatures have the discretion to decide whether to set an alternative standard. This does not mean that special education children do not have viable grounds for a challenge. Their challenge would challenge either the test validity or instructional validity, applied only to their specific educational program. The state would be required to show that the Individual Education Program (IEP) teams have the information and are working to provide the best educational program to assist these children in passing the test. The accommodation requirement is that as long as the accommodation does not affect the validity of the test, for example, as long as it is not being said that someone can use a calculator to calculate simply to show calculation ability, the accommodation should be allowed. This issue has to be dealt with on a student-by-student basis. CHAIR GREEN suggested the law that was passed was too brief and did not allow enough direction nor did the interpreters take the liberty of broadening the accommodation. CHAIR Green asked if Mr. Reeves could defend the test. MR. REEVES said he would be ready to defend the test today. The test is not indefensible, but the timing element needs to be identified as the most difficult defensible piece. SENATOR WILKEN said regulations were circulated having to do with special education and asked if they have something to do with the exit exam and, if so, what is the status of the regulations. MR. REEVES was not sure which regulations Senator Wilken was referring to. There are extensive regulations on special education out for public comment at this time. Over the last 12 or 18 months regulations have been advanced regarding accommodations. The board through reference to DOEED's participation guideline booklet has adopted these. The biggest change in the participation guidelines for this coming spring testing is that there are audiotapes for students who do not read. These students can take the mathematics and writing test using an audiotape. SENATOR WILKEN asked what state has done the best with the high stakes test. MR. REEVES said he is not the best person to ask from the educational perspective on who has the best testing system. It does appear from case law that the court was impressed by Texas, particularly with the success of its remediation system and the fact that state mandated remediation showed a gain. There being no further questions, CHAIR GREEN adjourned the meeting at 3:05 p.m.