Audit Release: OLCC Cannabis Information Systems are Properly Functioning but Monitoring and Security Enhancements are Needed

Report Highlights

Although the Oregon Liquor Control Commission (OLCC) has taken positive steps to establish information systems for recreational marijuana regulation, we identified several weaknesses associated with OLCC’s new IT systems used for marijuana licensing and tracking. They include data reliability issues and insufficient processes for managing marijuana applications and vendors. In addition, OLCC has not implemented an appropriate agency-wide IT security management program. We identified eight IT security issues that significantly increase the risk that OLCC’s computer systems could be compromised, resulting in a disruption of OLCC business processes.


In 2014, voters approved Measure 91, which legalized the production, sale, and use of recreational marijuana in Oregon. To help regulate and support this new industry, OLCC implemented the Marijuana Licensing System and the Cannabis Tracking System.

Audit Purpose

The purpose of our audit was to review and evaluate key general computer controls governing OLCC’s IT security management program, and application controls over the Cannabis Tracking and Marijuana Licensing Systems.

Key Findings

Within the context that legal marijuana is an emergent and unique public policy and the state is understandably still in the process of implementing governance programs, regulations, controls, and resources, we found:

  1. Data reliability issues with self-reported data in the Cannabis Tracking System (CTS) and an insufficient number of trained compliance inspectors inhibit OLCC’s ability to monitor the recreational marijuana program in Oregon.
  2. OLCC should improve processes for ensuring the security and reliability of data in the CTS and the Marijuana Licensing System. In addition, better processes are needed to monitor vendors that host and support these applications.
  3. OLCC has not implemented an effective IT security management program for the agency as a whole.
  4. OLCC has not formally developed a disaster recovery plan and has not tested backup files to ensure they can be used to restore mission-critical applications and data.


The report includes 17 recommendations to the Oregon Liquor Control Commission focused on addressing the weaknesses in the CTS data reliability, management of software as a service, IT security management, and disaster recovery and backup processes.

The Commission generally agreed with our recommendations.  The Commission’s response can be found at the end of the report.

Read full report here.


Featured New Audit Release

The Balance ReBlog: Communication skills for workplace success (w/ Weird Al bonus video!)

The ability to communicate effectively with superiors, colleagues, and staff is essential, no matter what industry you work in. Workers in the digital age must know how to effectively convey and receive messages in person as well as via phone, email, and social media. Good communication skills will help get hired, land promotions, and be a success throughout your career.

Alison Doyle with outlines the communication skills that serve both job applicants and workplace peers. Can you guess what the #1 most important skill is?


Communication tips not enough? Let Weird Al guide you toward full enlightenment with the following ballad:


Accountability and Media Featured

Audit Release – Foster Care in Oregon: Chronic management failures and high caseloads jeopardize the safety of some of the state’s most vulnerable children

Report Highlights

Oregon’s most vulnerable children are being placed into a foster care system that has serious problems. Child welfare workers are burning out and consistently leaving the system in high numbers. The supply of suitable foster homes and residential facilities is dwindling, resulting in some children spending days and weeks in hotels. Foster parents are struggling with limited training, support and resources. Agency management’s response to these problems has been slow, indecisive and inadequate. DHS and child welfare managers have not strategically addressed caseworker understaffing, recruitment and retention of foster homes, and a poorly implemented computer system that leaves caseworkers with inadequate information.


Since 2011, there have been over 11,000 children in the Oregon foster care system each year. These children are vulnerable and are often the victims of child abuse and neglect.

Audit Purpose

The purpose of the audit was to determine what changes and improvements DHS can make to better promote the wellbeing of children in foster care and ensure they are better protected and cared for.

Key Findings

  1. DHS and Child Welfare struggle with chronic and systemic management shortcomings that have a detrimental effect on the agency’s ability to protect child safety. Management has failed to address a work culture of blame and distrust, plan adequately for costly initiatives, address the root causes of systemic issues, use data to inform key decisions, and promote lasting program improvements. As a result, the child welfare system, which includes the foster care program, is disorganized, inconsistent, and high risk for the children it serves.
  2. DHS does not have enough foster placements to meet the needs of at-risk children, due in part to a lack of a robust foster parent recruitment program. The agency struggles to retain and support the foster homes it does have within its network. The agency also lacks crucial data regarding how many foster placements are needed and the capacity of current foster homes, inhibiting the agency’s ability to fully understand the scope of the problem.
  3. A number of staffing challenges compromise the division’s ability to perform essential child welfare functions. These challenges include chronic understaffing, overwhelming workloads, high turnover, and a large proportion of inexperienced staff in need of better training, supervision, and guidance.


We make 24 recommendations that address the agency’s management challenges, foster parent recruitment and retention, and child welfare staffing. Our recommendations also affirm the foundational recommendations Public Knowledge LLC made in September 2016.

The Department generally agrees with our recommendations. The Department’s response can be found at the end of the report.

Read the full report here.

Featured New Audit Release

Audit Release: The State Must Do More to Prepare Oregon for a Catastrophic Disaster

Report Highlights

Oregon is at risk of a major Cascadia earthquake and tsunami that will threaten infrastructure, cost potentially billions of dollars, and result in numerous deaths. The state must do more to prepare for such a disaster, including completing and implementing critical plans, fulfilling minimum standards for an effective emergency management program, and adequately staffing the agency charged with coordinating emergency management efforts.


The emergency management system encompasses local governments and almost all of state government. The Office of Emergency Management (OEM) is charged with coordinating Oregon’s emergency management efforts, including mitigation, preparedness, response, and recovery.

Audit Purpose

The purpose of this audit was to determine the status of state agency and local emergency management efforts to prepare for a catastrophic event, such as a Cascadia earthquake and tsunami.

Key Findings

  1. Oregon does not meet key emergency management program standards. These national baseline standards are a tool to strengthen preparedness and response, demonstrate accountability, and identify resource needs.
  2. Planning efforts across all levels of Oregon’s emergency management system are lacking. Critical continuity plans that ensure functional government services in the wake of a disaster are either missing or incomplete. Additionally, insufficient staff resources put the state at risk of losing potentially millions of dollars in federal grant funding for future disasters.
  3. Current statewide staffing is inadequate to reduce Oregon’s vulnerability to disasters. OEM in particular is understaffed, despite repeated budget requests to the Legislature, which inhibits the agency’s capacity to coordinate emergency management efforts in the state.
  4. More accountability, such as public reporting and tracking, is needed to ensure progress on long-term resilience goals and projects and to enhance public awareness.

To reach our findings, we conducted a survey of state agencies and local emergency management programs. We also interviewed staff at OEM, other executive branch agencies, and the legislative and judicial branches of state government. We researched programs in other states and assessed emergency management program standards.


This audit includes 11 recommendations, five to OEM and six to the Governor’s Office. These recommendations include such actions as completing, implementing, and exercising emergency and continuity plans; meeting minimum emergency management program standards; reporting on efforts to improve state resilience; defining roles and responsibilities and assessing and filling resource gaps.

OEM agreed with all the recommendations we made to them. The Governor’s Office agreed with all but one of our recommendations. That recommendation, they believe they have already implemented. Both OEM and the Governor’s Office’s responses can be found at the end of the report.

Read the full report here.

Emergency Management Framework

Featured New Audit Release Performance Audit

Audits in the News: Air quality permitting

Audits in the News: Air quality permitting

We here in the audits division are proud that the work we do makes a difference. Our work attracts the attention of the legislature, statewide news sources, and even local media outlets. Local media coverage of our audits is just another way we communicate with the people of Oregon about the work that we’re doing on their behalf to make government better. This is part of an ongoing series of posts rounding up recent instances in which the Oregon Audits Division makes a cameo in the local news.

In our first performance audit released in 2018, auditors examined the air quality permitting process at the Department of Environmental Quality. The team found a significant backlog among permit renewals and identified strategies the agency can use to evaluate staffing and workload, provide better guidance, and reduce the backlog and better safeguard Oregon’s air.

You can read the entire audit here.

The Oregonian/OregonLive – Audit: Oregon regulators face ‘unmanageable workload,’ potentially harming air quality

Read the story here.

“The state employees overseeing Oregon’s air quality program are overworked, understaffed and face “unmanageable workloads” that have led to substantial permit and inspection backlogs, potentially increasing the chances those businesses aren’t complying with the latest pollution laws.

Those are the key takeaways of an audit of the Department of Environmental Quality’s air pollution programs. The Secretary of State’s office audit released Wednesday notes that the agency’s budget, adjusted for inflation, shrank 8 percent since the 2001-03 biennium. Dozens of job vacancies left unfilled means the agency is effectively down 250 employees, or roughly 29 percent of its authorized workforce, during the same period.”

Statesman Journal – DEQ permit and inspection backlog endangers air quality, audit finds

Read the story here.

“Forty-three percent of Oregon’s largest air polluters are operating with expired permits — some as long as five years past due.

That means those businesses aren’t required to meet the most current environmental standards, the Oregon Audits Division said in a report issued Wednesday.

The Oregon Department of Environmental Quality told auditors it’s behind on inspecting air polluters as well, but can’t quantify the problem because it has no system for tracking inspection due dates.”

Oregon Public Broadcasting – Human Health Put At Risk In Oregon By Air Pollution Permit Backlog: Audit

Read the story here.

“A backlog of outdated air pollution permits is endangering public health and frustrating business owners, according to a newly released audit by the Oregon Secretary of State’s Office.

About 40 percent of air quality permits for major industrial sources of pollution are overdue for renewal by the Oregon Department of Environmental Quality, according to the audit. Oregon is also behind on timely inspections for air quality permits, but it doesn’t know by how much: the agency has no system to track when facilities are due for inspection, according to the audit.”

Portland Tribune – Auditors: DEQ backlogs ‘endanger state’s air quality and health’

Read the story here.

“State auditors say that a backlog in permits and inspections at the Oregon Department of Environmental Quality “endanger the state’s air quality and the health of Oregonians.”

The secretary of state’s Audits Division released the audit Wednesday, Jan. 3. The audit is available here.

DEQ is responsible for monitoring and regulating emissions from industrial sources and enforcing violations. The department also oversees land and water quality.”

Audits in the News Featured

Internal Auditor ReBlog: Let’s talk about feedback

Through frequent conversations with practitioners who are relatively new to the internal audit profession — including both people within and outside my organization — it seems there is a disconnect when it comes to feedback. Manager-level employees tell me they often provide informal feedback to the staff and senior auditors who work with them. Meanwhile, those same managers’ staff and seniors say they don’t receive enough feedback, don’t know if they are “on track,” and don’t know what they are doing well and what they can im​prove. This is where a few simple conversation areas can reap great benefits.

Laura Soileau, a director in Postlethwaite & Netterville’s Consulting Department in Baton Rouge, Louisiana, discusses the importance of ongoing communication and relationship building in the workplace when delivering – and receiving – feedback (both in formal performance evaluations and day-to-day). She provides a list of useful questions for supervisors and staff to ask each other, and to ask themselves. Maintaining healthy working relationships and keeping the lines of communication open, professional, and productive “should be a shared responsibility.” Feedback is crucial to keeping performance on track, but in this case, the ‘who and how’ is almost as important as the ‘why.’

Read more here!

Accountability and Media Featured

Methods (to our madness): Complex analysis in the public eye

The Secretary of State recently released a performance audit on the Oregon Health Authority: Oregon Health Authority Should Improve Efforts to Detect and
Prevent Improper Medicaid Payments. This audit received a lot of media exposure, in part due to an Audit Alert released in May, some months before the scheduled audit release date. Unsurprisingly, this led to more than a little pressure. How did our 4 person audit team (Ian Green, Wendy Kam, Kathy Davis, and Eli Ritchie) approach this audit, stay cool under fire and make sure their conclusions were sound? I sat down with the lead auditor, Ian Green, to find out more.

You led the OHA, Improper Medicaid Payments audit. What are your strategies when you’re faced with a complex agency and a complex topic?

When we started this audit, we knew we’d be looking at improper payments. Even that’s such a big topic, we knew we’d need to scope it down where we could. So we got as much information as we could from all levels – hundreds of interviews with officials and analysis, looking at agency documentation, research on best practices, all of that.

What methods did you use to identify improper payments?

Our primary focus was to look at process issues, but we did attempt to find some improper payments. We used audit software to analyze large data sets. We did a lot of data matching and looked for results that were outliers. For instance, we checked to see if providers were getting duplicate reimbursements. It’s a complex system, so providers and billers might make errors that should be caught before payments are set out. Another example was checking to see if there were people enrolled in the Oregon Health Plan who shouldn’t be – like if someone had moved out of state.

 What challenges did you face doing this audit, and what strategies did you use to address them?

One challenge was the sheer amount of data. We looked at over two hundred million records.  There was so much data that running tests could take a very long time. My team would run a script and leave it overnight to finish. We had to be very careful about how we set up our tests. Since we kept everything scripted out, each time we got new information, we could just update that script. That kept the testing sustainable, which is very important given all the last minute information we received.

To address the complexity of the topic, we separated our approach into three subtopics: prevention, detection, and recovery. Each person on the team focused on one area, and we’d meet to discuss weekly. That helped make sure we covered all the information while still working together closely.

Another challenge was trying to get complete data. We’d request data and be told we had it all. And then we’d find out it was incomplete. That meant we had to continue reworking our analysis constantly. Without scripting, it would have been extremely time-consuming to perform this work manually.

What was the hardest thing about completing this audit in the public eye?

It’s a very sensitive topic. We knew that we’d get a lot of scrutiny. But we did what we always do, which is to work really hard to make sure all our conclusions are accurate and well-supported, and put all our work through a thorough quality assurance process.

 Is there anything you wish non-audit folks knew about the audit process?

Generally, there’s a public perception that an audit should find everything that might be going wrong. Auditors look at a higher level to see if there are controls in place to prevent something from going wrong. If we’re concerned, we may do deeper testing to see what’s actually happening. For instance, we looked at processes to manage improper payments. Our goal wasn’t to find all of the improper payments being made. Our testing helped measure the effect of the processes that are currently in place.

Anything else?

It was a big audit. We’ve been excited to see important changes happening, even while we were still working on the audit. The Oregon Health Authority is working to address weaknesses in their processes and being more transparent. That’s a really good outcome, from our perspective.


Check out the audit here:

Auditing and Methodology Auditors at Work Featured Performance Audit