SB 250-SCHOOL PERFORMANCE REPORTS  DR. ED MCLAIN, Deputy Commissioner of DOEED, introduced Mr. Mark Leal, Director of Assessment at DOEED. He explained that SB 250 extends the implementation date for the assignment of performance designators and ties monitoring based on those designations into the reports. It also provides for a new effective date. DR. MCLAIN explained that enactment of SB 250 will enable DOEED to: · incorporate data from the fall of 2004 test into the report by requiring the first designation to occur in September of 2004; · align the state program with the new requirements of the federal Elementary and Secondary Education Act, which recently passed and DOEED is in the process of reviewing; · collect data that will enable DOEED to designate schools based on current performance and the progress they make from year to year; · build capacity in Alaska for tighter data collection, which requires skills, understanding, protocols and procedures; and · clarify that a focus of the designation will be student achievement of essential skills. SENATOR WILKEN asked why, on page 2, line 12, the review timeframe was changed from the school year to the calendar year. DR. LEAL explained the date change will align monitoring requirements with the current school report card date, which is January 15. SENATOR WILKEN asked how Alaska's school designator program corresponds with the federal legislation that recently passed. DR. LEAL said he believes there are a lot of parallels. Alaska is moving in the direction of being able to calculate growth by testing students from grades 3 through 10. DOEED wants to make sure that the types of growth goals that it addresses are aligned with the federal requirements. It does not want to find a school is deficient for Title 1 purposes yet proficient in Alaska's model. DOEED is waiting to see if the tests it is using are acceptable to the federal government. He clarified that if Alaska's program differs, it wants to do so purposely. DOEED does not want to accept the federal model wholesale but it does want to be as closely aligned as possible. SENATOR WILKEN asked if the federal law does not derail what Alaska has started and that DOEED will be able to comply with it. DR. MCLAIN said that's essentially correct but some key pieces have DOEED concerned and raise specific questions, i.e., Alaska's system is dependent on both criteria and norm referenced testing and DOEED has done some linkage studies. It is not clear whether that will be fully accepted by the federal government. CHAIRWOMAN GREEN asked if the language in paragraph 3 (page 2) implies that DOEED will use the last year's data only rather than data accumulated over the years. DR. MCLAIN explained that the most recent designation will be based on both status and growth, which contains data from current and past records. CHAIRWOMAN GREEN commented that theoretically a school could receive a good designation but have declined from the past. She then asked for a definition of "multiple student measures" on page 2, line 8. 1:45 p.m. DR. LEAL indicated the basis of the designator system will be the student assessments: both the norm-referenced and the benchmark and high school qualifying exams. DOEED is looking at the status score and the growth score as being multiple measures. It is also looking at the dropout rate for high schools, percent of students graduating, and local assessments. CHAIRWOMAN GREEN asked if those definitions were expanded and provided in regulation by the state board of education. DR. MCLAIN said he would have to check on the definitions but noted the definition of "multiple student measure" has been the subject of a lot of discussion within the profession. The designator committee has had a healthy, ongoing discussion about what to include because there is some argument for including other inputs as measures, i.e. parental involvement, class offerings, etc. However, the bottom line would continually come back to whether students are learning. CHAIRWOMAN GREEN said she wants to make sure that the language in the bill is adequate for the designators to do their job. SENATOR LEMAN referred to the language on page 2, line 8, and asked if it has the same meaning as the language in the Governor's transmittal letter that reads, "... school performance designations are based on student achievement, including the results of the competency exam." He maintained that both phrases, taken together, mean that the results of the competency exam will be used to define student achievement of essential skills. DR. MCLAIN said that is correct and that DOEED will be looking at status as well as growth. SENATOR LEMAN said if the legislature does not pass SB 250, DOEED will have to produce a report by August of this year. He asked whether DOEED has enough information now to produce a draft proposal for what the designations might be for various schools and questioned whether DOEED has identified some that are outstanding and others that are in peril. DR. LEAL replied that the committee has not yet designated schools but it has narrowed down the type of information it would use. The committee is preparing a model and regulations to present to the state board of education in March. That model shows what DOEED would be able to implement by August, if necessary. DOEED would use the current status scores from the assessments taken by students this spring. SENATOR LEMAN read from a letter from the Anchorage School District, "In some of our schools, we have almost a 100 percent turnover of students." He said he assumes that means in one year and he finds that hard to believe. He asked if DOEED has ever seen a turnover rate that high. DR. MCLAIN replied: Through the Chair, Senator Leman, high turnover rate is one piece that we are cognizant of as we look, and the committee's been looking at that as we incorporate the model - as we looked at how various states have handled the high turnover - and there are districts and there are schools that report very high turnover. I'm not prepared to tell you where those are but we do routinely hear of those and our state has generally a fairly high turnover - and how those kids get accounted is to the committee's thinking a critical consideration for us. There are states that will only include in the accountability kids who have been in the school who take the test the last year and have been with us a full year. There [are] other models that look at 90 days, etcetera. So we're aware of the impact of high turnover. There are schools that have high turnover. I can't speak to 100 percent or whatever but I would not be automatically surprised at that. CHAIRWOMAN GREEN asked whether course grades will be considered in the student measure. DR. LEAL said they will not. He explained that one problem the committee ran into when trying to find other indicators of student achievement was finding indicators that are not comparable across the state because there is no accountability in course grades among schools or districts. CHAIRWOMAN GREEN asked for confirmation that DOEED is comfortable with the language in SB 250 and that it contains the necessary tools to put in place a system from which to begin reporting in September of 2004. DR. MCLAIN clarified that the first report will be published in January of 2005 and he believes DOEED can comply. CHAIRWOMAN GREEN asked what the legislature can expect to see in the annual report in the meantime. DR. MCLAIN said DOEED would be very pleased to keep the committee and public informed about the process as it is developed. DOEED has some data to build on because it is required to report on Title 1 schools to the federal government. CHAIRWOMAN GREEN asked if the bill needs additional language to make sure that interim reports are forthcoming. DR. LEAL said that DOEED can report on the schools identified as school improvement sites for the Title 1 program - schools that are not meeting the adequate yearly progress measures. DOEED will be refining that data as it deals with the new federal act. The 1994 federal act required DOEED to identify and report schools that are not making adequate yearly progress to the federal government. Those schools will move into the sanctions under the new act. DOEED also publishes a yearly school report card on the web. Up until last year, DOEED published school district report cards; last year DOEED began publishing the information by school including the attendance rate, graduation rate, dropout rate, enrollment change, grade 10 and 11 high school graduation qualifying exam results, grade 4 and 7 CAT 5 test results and the benchmark exam results at grades 3, 6 and 8. CHAIRWOMAN GREEN asked if the school improvement sites are pre- designated based on a population or funding source. DR. LEAL clarified those are only the schools that receive additional funding through Title 1. CHAIRWOMAN GREEN asked whether all school districts receive that funding. DR. MCLAIN said to the best of his knowledge, every school district receives some Title 1 monies but not all schools receive those funds. He explained that one of the criteria for Title 1 funds is both economic and educational disadvantage. To the degree that Title 1 schools are generally representative of schools with economic and educational disadvantage, they identify a pool that will warrant more attention. CHAIRWOMAN GREEN asked if the yearly school report card draws the same conclusions. DR. MCLAIN said as it becomes known that DOEED has moved from district-based to site-based reporting, people will begin to look back at dropout rates and other key indicators. A lot of the pieces that go into the determination will be available as public information, which may help some schools get a handle on what to work on. 2:05 p.m. SENATOR WILKEN noted the Fairbanks North Star Borough School District passed a resolution in support of SB 250 at a meeting on January 20 and that he will place a copy in the record. CHAIRWOMAN GREEN referred to a letter from the Anchorage School District and read, "However, designating a school as deficient or in-crisis on the basis of test scores alone ignores certain factors." She asked if the current law is based on test scores alone. DR. MCLAIN indicated the designation was intended to focus on achievement and test scores are a prime way to measure that. He pointed out that a number of models group schools, for example, schools of a comparable socio-economic base, to see how students are doing within that group. That approach brings good news, but also difficulties. DOEED believes the intent of the law is to measure achievement. DOEED can recognize different factors and show schools ways to modify their instruction without lowering expectations. DR. LEAL stated that he believes that is the critical reason to count growth rather than just status. By making growth two- thirds of the measure, DOEED will be saying that even though a school has a transient population, they must be included. DOEED wants to refine its ability to collect the growth data so that it can make statements about schools based on growth in test scores. CHAIRWOMAN GREEN pointed out the growth in test scores is not mentioned anywhere in statute. She repeated that she wants to be sure the instructions are clear, broad and specific enough that in the future these measures are done in the same way. She asked if that requirement is covered in regulation. DR. MCLAIN explained the charge of the designator committee is to develop those. The committee has received a lot of national help and has local and instate expertise. DOEED can share the records of that development. CHAIRWOMAN GREEN took public testimony. DR. NICK STAYROOK, Fairbanks North Star Borough School District, said the Fairbanks School Board supports the delay of school designators. Regarding multiple student measures, a committee he was a member of looked at including post-graduation success rate in addition to achievement scores. His group did not consider, because of the wording of the bill, parent satisfaction with the school. Dr. Stayrook said one goal of the bill is to assign and look at the quality of schools throughout Alaska. Educational research shows that schools will pay attention to the things they are held accountable for so if Alaska's designator system strictly looks at student performance in essential skills in reading, writing and math, schools will concentrate on those skills. He believes it is very important to provide more time to consider ways to look at other kinds of student achievement. DR. STAYROOK pointed out that Alaska has many types of specialized schools, i.e., a school for incarcerated youth in Fairbanks, alternative high school programs, and schools primarily for special education students. Many of those schools will be in crisis because that is where students who are low performing and at risk are placed. It is important to consider how to fairly and validly rate the quality of those schools in a way that goes beyond looking at scores on the benchmark, high school, and norm referenced exams. In addition, Alaska has many schools that are extremely small. Designators can change significantly in schools when only two or three students take the exams. A statistical analysis of test scores of two or three students is generally not valid as an indicator of how well a school is doing. 2:10 p.m. CHAIRWOMAN GREEN asked Dr. Stayrook if, as a member of the designator committee, he feels the language in the bill is adequate. DR. STAYROOK said the language is fairly good. One of the things the committee has struggled with over the last couple of years is multiple student measures; it felt restricted to only look at student measures. He suggested revising the language to read, "multiple student and parent measures." MR. JIM HOLT, Superintendent of Schools, Fairbanks North Star Borough School District, said he shares the concern that delaying the date alone will not satisfy the Fairbanks School Board. A majority of states now have a quality schools initiative similar to Alaska's. Many of those states exempt certain schools. Alaska has created schools for specific populations; those schools are in crisis. The Fairbanks district is trying to deal with the achievement gap: closing that gap would be a good indicator of success. DR. BRUCE JOHNSON, Association of Alaska School Boards, stated support for delaying the implementation of the school designator system. Two members from the Association serve on the designator committee. They are very pleased with the work to date. That committee is working with a highly qualified consulting firm that has experience in other states. From the beginning of the discussion on school designators, the most important piece of the puzzle was how to get the growth piece into the equation. It is important to not design a system that encourages students who are not achieving to drop out. School districts are responsible for their students. They cannot take responsibility for the home environment but their accountability should begin when students arrive at school. Figuring out how to do that is a highly complex issue in a state as diverse as Alaska. He pointed out in California, schools with a population of fewer than 200 students do not participate in the designator program. That would amount to 75 percent of Alaska's schools so that approach would not get at the intent of the legislation. Alaska's challenge is greater but the people working on it are paying a lot of attention to the nuances in those issues and are trying to be true to the law. MR. BYRON RICE, representing the Glennallen-Copper River School District, said to his understanding, school designator labels will be based solely on the results of assessments conducted in the spring of 2002. That is a far cry from the original intention of the designator committee, which called for a weighted review of test scores. The committee called for test scores to be weighted at one-third, with the remaining two-thirds weighted on transient rates, attendance, numbers of special education students, size of school and analysis of historic assessment scores. He has asked participants in this process whether basing a school label on one assessment score is appropriate. Some people believe that would be unethical. MR. RICE noted that although alternate assessments are available for special education students, that option will only impact a very small number of the special education population. He said he questions the belief that providing special education students with more time to take assessments will invalidate those assessments. He also noted that an unfair system could have a devastating effect on schools that are labeled unsatisfactory in terms of recruitment and retention. MR. RICE'S final comment was in regard to how cutoff scores will be determined. If a delay is not granted, the August 2002 score will be determined through a computation based on cutoff scores formulated from metropolitan areas of the state. He said he has no problem being held accountable for students' progress when they are present at the school but when he has exhausted every means available to get that child to school and still comes up short, he has concerns about being held accountable for their lack of progress. He said that he concurs with the conclusions of the designator committee that without more time and data to professionally deal with the aforementioned questions, the state will walk into a disaster. TAPE 02-4, SIDE B CHAIRWOMAN GREEN asked Dr. McLain to respond to Mr. Rice's concerns. DR. MCLAIN reiterated some of his earlier points by saying that the intent is to look at both status as well as growth as the designator committee feels very strongly that growth is an integral part of any sort of plan. By looking at growth, some of Mr. Rice's concerns will be addressed. He agrees this is a complex issue and it is one that is being looked at in every state and territory. The Administration has asked for a delay because DOEED wants to do a good job. The current law requires a report in August 2002. If that law is not changed, DOEED will comply but the report will not provide as satisfactory of a designation. DR. LEAL added that, in regard to the special education students, an essential piece of the Elementary and Secondary Education Act is the inclusion of all students in accountability systems and assessment. Some accommodations could go too far and invalidate a test but extra time is not one of them. The designator committee is focused on coming up with a fair and equitable way to score schools, which is why it has requested a delay. CHAIRWOMAN GREEN announced that she will bring the bill back before the committee in about one and one-half weeks. There being no further business to come before the committee, she adjourned the meeting at 2:23 p.m.