Audit Release: Progress has been Made to Address Security Weaknesses at the State Data Center, but Improvements are Still Needed


Report Highlights

Security at the Enterprise Technology Services State Data Center (data center) has improved due to organizational and staffing changes and the increased role of the Enterprise Security Office. Several longstanding security challenges have been addressed, yet more work remains to further refine and improve security capabilities and to address other areas where roles are not sufficiently defined. The operating environment for the data center remains stable and appropriately controlled. Disaster recovery capabilities have improved, although prioritization of recovery order needs to occur to ensure that the most critical state systems can be restored timely in the event of a major disaster.

Background

The data center is comprised of an extensive inventory of computer operating system platforms and networks. It provides centralized computer services such as networking, email, backup, and server services for more than 100 state agencies, boards, and commissions. Since the creation of the data center in 2006, numerous prior audits have identified significant security weaknesses. Starting in 2015, organizational changes moved overall responsibility for the data center to the Office of the State Chief Information Officer (OSCIO) and expanded the staffing and role of the Enterprise Security Office.

Purpose

Because of the critical services the data center provides, we audit it every two to three years. This audit followed up on the status of prior audit findings and evaluated the current security framework and stability of the operating environment.

Key Findings

We found:

  1. The OSCIO has made significant progress in improving security at the data center through security planning and staffing, vulnerability assessments, security event monitoring, and anti-malware and patching processes. Further progress is needed to refine these processes and better track vulnerability remediation.
  2. Some security areas require improvement, including privileged access, asset and configuration management, and security incident response. Work is underway to improve Windows privileged access.
  3. Day-to-day computing remains stable and disaster recovery capabilities have improved. While additional disaster recovery capabilities are being built, data center customers need to prioritize which systems should be recovered first in the event of disaster.

Recommendations

We recommend improvements in defining roles and responsibilities, refining vulnerability scanning and security event monitoring, monitoring privileged access, and disaster recovery prioritization.

The Department of Administrative Services and the OSCIO agreed with all of our recommendations. The agency’s response can be found at the end of the report.

Read the full report here.

Featured New Audit Release

Audit Release: Energy Trust Administrative Costs are Generally Reasonable, but the Public Utility Commission Can Improve Oversight of These Costs


Report Highlights

The Oregon Public Utility Commission (PUC) has designed controls to ensure administrative and program support costs at Energy Trust of Oregon are reasonable. Energy Trust is a nonprofit organization and is not subject to state administrative cost requirements. However, PUC could strengthen its oversight of Energy Trust administrative costs by more clearly defining what constitutes reasonable costs, revising key performance metrics, and clarifying financial reporting requirements.

Background

Energy Trust is a nonprofit organization funded by a grant agreement with PUC to develop and administer energy efficiency and renewable energy programs in certain utility service territories in Oregon. The grant funding comes from three separate charges on bills of customers of electric and natural gas utilities regulated by PUC.

Purpose

The purpose of the audit was to determine whether Energy Trust administrative costs are reasonable and whether PUC has reasonable controls in place to oversee Energy Trust’s administrative costs.

Key Findings

  1. Energy Trust complies with PUC’s administrative cost control requirements. We found these controls to be reasonable, and Energy Trust has consistently spent below the established administrative cost cap of 8% of revenue per year. However, Energy Trust’s administrative costs increased from $1.6 million to $10.1 million between 2002 and 2017, as its annual revenues increased from $30.6 million to $194.2 million during the same period. Improved oversight could help PUC better ensure that Energy Trust makes reasonable administrative spending decisions.
  2. We determined Energy Trust’s administrative costs are generally reasonable. However, we identified a small percentage of questionable administrative costs that do not align with state agency standards or the grant guidelines that govern Energy Trust operations. PUC could improve its oversight by providing guidance for acceptable administrative costs.
  3. Increased clarity and detail in financial reporting would improve transparency and stakeholder oversight. PUC monitors Energy Trust’s administrative costs through an enforced spending cap and public budget and reporting processes. Revised reporting methodologies would increase the transparency of Energy Trust’s administrative costs and spending trends.

Recommendations

Our report includes recommendations to PUC regarding the clarity of its grant agreement with Energy Trust, revision of performance metrics, and reporting of administrative costs.

PUC generally agreed with our recommendations. The agency’s response can be found at the end of the report.

Read the full report here.

Featured New Audit Release

Methods (to our madness): Complex analysis in the public eye

The Secretary of State recently released a performance audit on the Oregon Health Authority: Oregon Health Authority Should Improve Efforts to Detect and
Prevent Improper Medicaid Payments. This audit received a lot of media exposure, in part due to an Audit Alert released in May, some months before the scheduled audit release date. Unsurprisingly, this led to more than a little pressure. How did our 4 person audit team (Ian Green, Wendy Kam, Kathy Davis, and Eli Ritchie) approach this audit, stay cool under fire and make sure their conclusions were sound? I sat down with the lead auditor, Ian Green, to find out more.

You led the OHA, Improper Medicaid Payments audit. What are your strategies when you’re faced with a complex agency and a complex topic?

When we started this audit, we knew we’d be looking at improper payments. Even that’s such a big topic, we knew we’d need to scope it down where we could. So we got as much information as we could from all levels – hundreds of interviews with officials and analysis, looking at agency documentation, research on best practices, all of that.

What methods did you use to identify improper payments?

Our primary focus was to look at process issues, but we did attempt to find some improper payments. We used audit software to analyze large data sets. We did a lot of data matching and looked for results that were outliers. For instance, we checked to see if providers were getting duplicate reimbursements. It’s a complex system, so providers and billers might make errors that should be caught before payments are set out. Another example was checking to see if there were people enrolled in the Oregon Health Plan who shouldn’t be – like if someone had moved out of state.

 What challenges did you face doing this audit, and what strategies did you use to address them?

One challenge was the sheer amount of data. We looked at over two hundred million records.  There was so much data that running tests could take a very long time. My team would run a script and leave it overnight to finish. We had to be very careful about how we set up our tests. Since we kept everything scripted out, each time we got new information, we could just update that script. That kept the testing sustainable, which is very important given all the last minute information we received.

To address the complexity of the topic, we separated our approach into three subtopics: prevention, detection, and recovery. Each person on the team focused on one area, and we’d meet to discuss weekly. That helped make sure we covered all the information while still working together closely.

Another challenge was trying to get complete data. We’d request data and be told we had it all. And then we’d find out it was incomplete. That meant we had to continue reworking our analysis constantly. Without scripting, it would have been extremely time-consuming to perform this work manually.

What was the hardest thing about completing this audit in the public eye?

It’s a very sensitive topic. We knew that we’d get a lot of scrutiny. But we did what we always do, which is to work really hard to make sure all our conclusions are accurate and well-supported, and put all our work through a thorough quality assurance process.

 Is there anything you wish non-audit folks knew about the audit process?

Generally, there’s a public perception that an audit should find everything that might be going wrong. Auditors look at a higher level to see if there are controls in place to prevent something from going wrong. If we’re concerned, we may do deeper testing to see what’s actually happening. For instance, we looked at processes to manage improper payments. Our goal wasn’t to find all of the improper payments being made. Our testing helped measure the effect of the processes that are currently in place.

Anything else?

It was a big audit. We’ve been excited to see important changes happening, even while we were still working on the audit. The Oregon Health Authority is working to address weaknesses in their processes and being more transparent. That’s a really good outcome, from our perspective.

 

Check out the audit here: http://sos.oregon.gov/audits/Documents/2017-25.pdf

Auditing and Methodology Auditors at Work Featured Performance Audit

Audit Release: Stronger Accountability, Oversight, and Support Would Improve Results for Academically At-Risk Students in Alternative and Online Education

Report Highlights


The Secretary of State’s Audits Division found that the Oregon Department of Education (ODE) has not focused on improving education for at-risk students in alternative and online schools and programs, though they account for nearly half the state’s high school dropouts. Sharpening Oregon’s focus would improve accountability, district oversight, and school and program performance, and would benefit at-risk students and the state’s economy.

Background

Many vulnerable students attend Oregon’s alternative schools and programs and online schools. Responsibility for improving education for those students is shared by ODE, school districts, and others.

Audit Purpose

To determine how ODE and school districts can help increase the success of academically at-risk students in alternative and online education. Online and alternative education schools and programs also serve students who are not academically at-risk. The audit did not focus on their effectiveness with these students.

Key Findings

  1. ODE has not adequately tracked and reported on the performance of alternative schools and programs. As a result, the state lacks critical information about school and program effectiveness.
  2. Enhanced state monitoring and support, and more robust district oversight could improve results for at-risk students in alternative schools and programs, and in online schools.
  3. Some states have held districts, alternative schools, and programs to high standards and provided more support to help at-risk students succeed.
  4. Other states have also increased oversight of fast-growing online schools. In contrast to these states, Oregon’s laws allow online schools to increase enrollment rapidly regardless of their performance.
  5. To reach our findings, we interviewed multiple stakeholders, reviewed documents, analyzed school performance data, researched practices in other states, visited schools, and surveyed all of Oregon’s school districts. Our office also released an audit of graduation rates recently that focuses on students in traditional high schools.

Key Recommendations

This audit includes recommendations designed to improve results for at-risk students in alternative and online schools and programs. ODE should develop a more meaningful accountability system for alternative and online education. The agency should establish and monitor standards for crucial practices, such as annual district evaluations of these schools and programs. ODE should also strengthen state attendance and funding standards for online schools.

ODE generally agreed with our recommendations. The agency’s response can be found at the end of the report.

Read full report here.

Featured New Audit Release

Secretary of State’s 2017-2018 Audit Plan

Overview of the Audit Plan

The Audits Division of the Secretary of State’s Office adheres to an overall audit strategy that a high-quality and transparent annual audit plan is critical for meeting our mission.

The Division follows professional standards and guidelines for the development of the Annual Audit Plan.

These guidelines recognize that an annual audit plan and work schedule benefit the organization by establishing which agencies, programs, contracts, or other areas will be prioritized for audits on an annual basis.

Including performance, IT, and financial topics, the Oregon Audits Division will tackle 30 audits in the upcoming year, with several more possibilities lined up on the 2018-2019 horizon.

Read the Plan here.

 

Accountability and Media Auditors at Work Featured

State of Oregon Financial Condition Report: Fiscal Year 2016

The Oregon Secretary of State Audits Division releases an annual financial condition report for the previous fiscal year, which covers revenues and expenditures and reports on the State’s overall fiscal health. Enjoy our summary below, or read the full report here.

Featured Financial Audit New Audit Release