Audit Release: ODE and PPS Must Do More to Monitor Spending and Address Systemic Obstacles to Student Performance, Particularly at Struggling Schools


Report Highlights

At the Oregon Department of Education (ODE) and Portland Public Schools (PPS), monitoring of spending and evaluation of performance is limited and inconsistent, even as student performance continues to lag. This lack of sustained focus limits achievement gains and affects students at high-poverty schools the most. Oregon’s K-12 education funding has fallen into the lower half among states and retirement costs are rising quickly, increasing the importance of ensuring effective education spending.

Background

Since the 1990s, K-12 education funding has shifted substantially to the state, which spends about $4.3 billion a year in addition to local property taxes collected for education. Despite some recent improvement, the state as a whole and PPS specifically still face low graduation rates and large achievement gaps, underscoring the need to improve returns on this investment. However, improving K-12 education requires navigating multiple layers of government, from local to state to federal. PPS’s new leadership has taken important steps toward developing a uniform core curriculum, improving training, and adding student support, but still has substantial work to do. The state is also transitioning to the latest in a series of accountability systems. These challenges increase the risk that a sharp, unified focus on improving student achievement will be further delayed.

Purpose

The purpose of this audit was to examine spending at ODE and PPS, the state’s largest school district, including transparency, controls, and priorities, and determine additional steps these agencies could take to improve returns on the state’s education investments and increase achievement. As a component of the audit, we honored a request from the Oregon Legislature to audit ODE grant management.

Key Findings

  1. ODE does relatively little to support and monitor efficient district spending. The Legislature has not required detailed reviews of school district efficiency outside of narrowly focused initiatives.
  2. PPS has more money to spend per student than many of its state and national peer districts, but the district is spending proportionately less on instruction and more on support services than many of its peers. Potential savings areas we identified include executive administration, substitute teacher use, health benefits, bus service, and legal costs.
  3. PPS needs to develop a more transparent budget, publicly report on the results of its investments in student achievement, and detail how its revenues and spending compare to peer districts. The district also needs to better control increasing employee use of purchasing cards.
  4. ODE and PPS are not adequately evaluating whether grants, contracts, and other dollars, often intended for Oregon’s most vulnerable students, are improving student performance. At PPS, contract issues include poor oversight of alternative education contracts and limited scrutiny of non-competitive contracts. ODE has not adequately evaluated its school improvement efforts under the federal Title I program, focused on some of the most vulnerable children in the state. The lack of sustained focus at PPS and statewide has the most detrimental effect on schools serving high numbers of African-American, Latino, and economically disadvantaged students.
  5. At PPS, inequities affecting these students include relatively high rates of teacher turnover and absences at high-poverty schools, a disconnect between teachers and administrators on managing student conduct, and a teacher hiring and transfer system that contributes to high-poverty schools having less experienced teachers. The district has also not prioritized principal or teacher stability at high-poverty schools, adequately supported principals, or developed a consistent and effective performance evaluation system.
  6. In comparison to students in peer districts, PPS does relatively poorly with African-American, Latino, and economically disadvantaged students. Conversely, the district does relatively well compared to peers with white students and students who are not economically disadvantaged.
  7. ODE’s limited enforcement of district standards, a new K-12 accountability system at risk of delays, a reliance on short-lived improvement initiatives, and a disjointed system of education funding all increase the risk that the state’s student performance will continue to lag. ODE enforcement of standards designed to improve student achievement is limited, and education leaders and the Legislature have not resolved how best to improve districts without infringing on local control.

Recommendations

We made 26 recommendations to help improve return on education investments at ODE and PPS.

Among the key recommendations for ODE:

  • Coordinate with the Governor’s Office, the State Board of Education, the Legislature, and districts to develop a plan to align education investments for the long-term.
  • Work with the State Board and stakeholders to evaluate Division 22 district standards for clarity and enforceability and ensure that ODE has adequate resources to review compliance and enforce standards when districts fall short.
  • For key grants, incorporate best-practice performance management, including setting quantitative and qualitative performance expectations in contracts, establishing baseline measurements, and providing timely and constructive feedback to grantees.

Among the key recommendations for PPS:

  • Investigate and report on potential savings areas in depth, including the level of executive administration, use of substitute teachers and educational assistants due to educator absences, health benefits, bus services, legal services, and building utilization.
  • While working to improve instructional quality, address other obstacles that create inequities at high-poverty schools. Strategies include changes to attendance rules, boundary changes, and practices that could encourage retention of high-quality principals and teachers at high-poverty schools, such as additional pay, enhanced training, and additional classroom support.

We also made a recommendation to the PPS school board. In general, the board should ensure that district administrators prioritize key steps to improve the efficiency and effectiveness of district operations. These steps include developing a strategic plan that focuses on long-term investment and measurement of results, and addressing inequities at high-poverty schools.

ODE and PPS agreed with all of our recommendations, with agency leaders detailing steps the agencies have already taken and providing timelines for completion. The auditees’ responses are at the end of the report.

Read the full report here.

Featured New Audit Release Performance Audit

Audit Release: Stronger Accountability, Oversight, and Support Would Improve Results for Academically At-Risk Students in Alternative and Online Education

Report Highlights


The Secretary of State’s Audits Division found that the Oregon Department of Education (ODE) has not focused on improving education for at-risk students in alternative and online schools and programs, though they account for nearly half the state’s high school dropouts. Sharpening Oregon’s focus would improve accountability, district oversight, and school and program performance, and would benefit at-risk students and the state’s economy.

Background

Many vulnerable students attend Oregon’s alternative schools and programs and online schools. Responsibility for improving education for those students is shared by ODE, school districts, and others.

Audit Purpose

To determine how ODE and school districts can help increase the success of academically at-risk students in alternative and online education. Online and alternative education schools and programs also serve students who are not academically at-risk. The audit did not focus on their effectiveness with these students.

Key Findings

  1. ODE has not adequately tracked and reported on the performance of alternative schools and programs. As a result, the state lacks critical information about school and program effectiveness.
  2. Enhanced state monitoring and support, and more robust district oversight could improve results for at-risk students in alternative schools and programs, and in online schools.
  3. Some states have held districts, alternative schools, and programs to high standards and provided more support to help at-risk students succeed.
  4. Other states have also increased oversight of fast-growing online schools. In contrast to these states, Oregon’s laws allow online schools to increase enrollment rapidly regardless of their performance.
  5. To reach our findings, we interviewed multiple stakeholders, reviewed documents, analyzed school performance data, researched practices in other states, visited schools, and surveyed all of Oregon’s school districts. Our office also released an audit of graduation rates recently that focuses on students in traditional high schools.

Key Recommendations

This audit includes recommendations designed to improve results for at-risk students in alternative and online schools and programs. ODE should develop a more meaningful accountability system for alternative and online education. The agency should establish and monitor standards for crucial practices, such as annual district evaluations of these schools and programs. ODE should also strengthen state attendance and funding standards for online schools.

ODE generally agreed with our recommendations. The agency’s response can be found at the end of the report.

Read full report here.

Featured New Audit Release

Audit Release: The Oregon Department of Education Should Take Further Steps to Help Districts and High Schools Increase Oregon’s Graduation Rate

Report Highlights


The Oregon Department of Education (ODE) has prioritized improving four-year graduation rates in recent years. The Secretary of State Audits Division found ODE could make further progress by helping schools and districts focus on specific groups, such as students who transfer between districts, low-income students, and middle school students. ODE can also better help districts and schools use effective improvement tools, use data to identify students in danger of not graduating, and communicate the importance of graduation to parents and the community.

Background

One in four Oregon public high school students does not graduate on time. Current steps to boost on-time graduation rates include plans to reduce chronic absenteeism, prevent students from dropping out, increase access to college-level courses in high school, and increase career-technical education.

Audit Purpose

The purpose of the audit was to determine how ODE and school districts could increase four-year graduation rates in Oregon’s public high schools.

Key Findings

Through school visits, interviews, data analysis, and document reviews, we found that:

  • Students who changed districts during high school – more than a quarter of all high school students – had graduation rates roughly 30% lower than students who did not. ODE does not analyze or report graduation performance for these students.
  • Schools with mid-range graduation rates – 67%-85% – receive limited improvement support from ODE, though most non-graduates attend these schools.
  • More than 70% of students who do not graduate on time are low-income. ODE should assess the need for services to help those students succeed.
  • The Legislature and ODE has not emphasized middle school performance or student transitions from middle school to high school, though students who struggle in middle school are already at risk of not graduating.
  • ODE does not track student grades or specific credits attained, data the agency could use to help more students graduate.
  • ODE should improve its internal communications and help districts and schools communicate the importance of graduation to parents and the community.

Recommendations

The report includes recommendations to the Oregon Department of Education on additional efforts it could take to increase on-time graduation rates. Among them: focusing on specific student groups, supporting coordination between middle schools and high schools, collecting more detailed student data, and helping districts better use improvement tools.

The Oregon Department of Education generally agreed with our findings and recommendations. The agency’s response can be found at the end of the report.

Read the full report here.

Featured New Audit Release

Oregon Department of Education: Computer Systems Ensure Integrity of Data, But Other Processes Need Improvement

Executive Summary


The Oregon Department of Education (department) oversees the education of over 560,000 students in Oregon’s public K-12 education system. The annual distribution of the State School Fund of $3 billion and federal funding of about $750 million help fund Oregon’s public education.
The department’s computer systems reasonably ensure the integrity of data used to distribute the State School Fund and appropriately process school district claims for federal funding. However, improvements are needed to provide better security for computer systems and student data, manage changes to computer systems, and ensure systems can be restored in the event of a disaster.

Read full report here.

Computer systems ensure integrity of student and school data

Department staff use the Consolidated Collection System to analyze and aggregate school and student data. They use information from this system to allocate monies to Oregon’s schools and education service districts. Computer systems reasonably ensured the integrity of student and school information through automated processes that accurately identify students and detect potential data errors. In addition, department analysts use system information to validate student and school data.

Computer systems appropriately receive and process School district claims for federal funding

The department uses the Electronic Grant Management System and the Federal Cash Ordering System to receive and process requests for federal program expenditure reimbursements. We found that computer controls reasonably ensure that these systems could appropriately receive and process school district claims for federal funding. These systems ensure grant claims do not exceed available balances and reject claims that otherwise would be ineligible for reimbursement.

Security measures for computer systems were insufficient

Although the department provides important protection measures for security, improvements are needed to better secure their computer systems and data. Weaknesses we identified relate to the department’s processes for planning, configuring, managing, and monitoring information technology security components. As such, the department does not provide an appropriate layered defense to protect agency computer applications. Thus, confidential student level information is at increased risk of disclosure or compromise.

Management of changes to computer systems needs improvement

The department has formal processes and tools for managing changes to their systems, but staff do not always fully utilize them. Independent and technical reviews of computer code changes did not always occur and processes were not in place to ensure only approved code could be placed in production. These weaknesses increase the risk that developers could introduce unauthorized or untested changes to the systems.

System files and data are appropriately backed up but procedures for timely restoration after a disaster are absent

The department has processes in place to back up critical data and can restore individual files as needed. However, department management and staff have not fully developed and tested a comprehensive disaster recovery plan capable of restoring critical systems and data in the event of a disaster or major disruption. Without a disaster recovery plan, the department cannot ensure it can timely restore operations in the event of a disaster.

Recommendations

We recommend that Department of Education management ensure resolution of identified security weaknesses, improve processes for changing computer code, and fully develop and test processes for restoring computer systems after a disaster.

Agency Response

The full agency response can be found at the end of the report.

Featured IT Audit New Audit Release

Oregon Department of Education: Clearer Communication, Consistent Use of Results and an Ongoing Commitment to Improvement Could Help Address Testing Concerns

Executive Summary


Our audit responds to House Bill 2713 (2015), developed with input from the State Auditor. It called for a Secretary of State audit to review the impacts of the statewide summative assessment on Oregon’s public schools, and make recommendations for improvement.

Through a series of surveys, site visits and interviews, we learned many schools faced challenges in the first year of administering the new Smarter Balanced test, including adjusting to the demands on staff and school resources. Some reported fewer challenges in the second year.

Some educators are concerned that certain student populations may experience more negative impacts than others. Some also told us that a more comprehensive assessment system would be useful.

Read full report here.

Oregon introduced Smarter Balanced in 2015

The Smarter Balanced assessment is a new test introduced by the Oregon Department of Education to all public schools in the spring of 2015. Smarter Balanced tests 3rd – 8th graders and 11th graders in math and English language arts near the end of the school year. The test assesses students’ progress toward meeting Oregon’s college- and career-ready standards, the Common Core State Standards. Smarter Balanced requires more time and depth of knowledge than the previous test.

There is not clear agreement on the test’s purpose

The Smarter Balanced test is intended to provide a measure for accountability, data to identify achievement gaps, and information about whether students meet standards overall, and many value these purposes. We also heard from educators who feel the test should be more useful in the classroom. However, other tools may be better suited for that purpose. The Oregon Department of Education could take a more active role in communicating about the test’s purpose.

The results of the test are not used consistently

Schools, school districts and the state use Smarter Balanced test results inconsistently, and sometimes not at all. Educators told us that it would be easier to use results if they received them sooner. Many reported that additional guidance on how to use results would be helpful. Some also reported that a more comprehensive assessment system would be useful.

Many reported test administration challenges

Educators described schoolwide challenges in the first year of administering Smarter Balanced. Testing did not just affect the classrooms that were actively testing, but could also place additional staffing and resource demands on the entire school. However, some said there were fewer challenges in the second year.

Testing took away from other duties of school and school district personnel. Some schools hired additional staff or substitutes specifically for testing. Testing also tied up computer labs for months at some schools. Time spent taking and preparing for the test took away from instruction time.

Some student populations may experience more negative impacts than others

Standardized testing may affect certain student groups more than others. Despite having accommodations, we heard concerns that the test’s greater use of technology and language may increase the risk that some students will not be able to demonstrate their abilities accurately. Students who take longer to complete the assessment may miss more instruction time.

Students in special education, English Language Learners, and students with less exposure to technology and typing may be particularly affected.

Recommendations

We recommend that the Oregon Department of Education improve communication, foster consistent use of results and continue its commitment to improve test administration. Our specific recommendations can be found on page 18 of the report.

Agency Response

The full agency response can be found at the end of the report.

Featured New Audit Release Noteworthy