Methods (to our madness): Municipal audits at the state

Each summer, the state’s municipal audit manager, Amy Dale, heads out to perform field reviews of accounting firms around Oregon, as well as a few in Idaho. The firms she visits perform financial audits for Oregon’s municipalities. It’s part of Amy’s unique role as a municipal auditor manager for the state of Oregon to help ensure local governments provide annual financial reports. I sat down with Amy between some of her field reviews this month to learn more about it.

Oregon’s municipalities are any cities, counties, school districts, special districts, or corporations subject to local government control that receive tax dollars from Oregon residents. A municipal audit law, passed in the early 1900s and updated most recently in 2015, requires that municipalities file a financial report each year. These are made public on the Secretary of State’s website.  Annual filing helps promote transparency and accountability regarding the money they collect from residents, and the summary of audit reports completed by the state’s municipal program provides an overview of the status of larger municipalities.

Of the state’s 35 financial auditors, only two work fulltime in the municipal audit program. In addition to supporting the firms who provide the audits, Amy works directly with the municipalities throughout their submission process, oversees the review of a sampling of financial reports each year, and, with her team, provides feedback on whether the reports meet financial reporting requirements.

Amy Dale, Municipal Audit Manager for the Secretary of State Audits Division

“I love this role because it’s interesting,” says Amy. “There’s always something different happening, and there’s a strong sense of helping residents and local governments, which is something I value.”

Amy’s role may have been a bit simpler in the early days, when there were a lot fewer of these municipalities. Now, around 1800 municipalities of a variety of size and purpose must file financial reports. Oregon’s cities, counties, and school districts make up about a third of them. The remaining 1000+ districts are special districts. They perform specialized services for the area where they operate, such as water, irrigation, fire, emergency management, pesticide, parks and recreation, and many more. To fund these services, they collect money through separate taxes, assessments, or fees. A county or city can have several special districts in its geographical area. For instance, in Marion County alone there are almost a hundred special districts.

Amy and her team help the many municipalities submit accurate and timely reports. “Our reviews shouldn’t be seen as punitive for those who didn’t meet standards, but rather to help them get it right,” Amy explains. “Reviewing audit reports is great practice for our financial auditors, too, especially as audit and reporting standards continue to change.”

If the municipal audits team finds reporting errors they’ll let the organization know what didn’t go right. Says Amy, “The municipalities don’t always like getting a letter from us with corrections, but the goal is to be helpful. Sometimes we end up getting thanks.”

The state takes this reporting requirement seriously; as of 2015, the municipal audit program will place cities and counties who haven’t filed a report on time on a withholding list. Until they file their reports, these municipalities will not receive 10% of state revenue contributions they’d otherwise collect.

Reports go up on the Secretary of State’s website so any resident can read them. The Secretary of State’s summary reports of the previous fiscal year’s audits filed are also on the website.

Want to know even more about municipal audits and last year’s summary report? Check out the full report on the Oregon Audit Division website from FY 2016 HERE.

Auditors at Work Featured Financial Audit

Methods (to our Madness): How IT audits help keep your $$$ safe

Recently, the Secretary of State Oregon Audits Division released an IT audit of GenTax, the software system that Oregon’s Department of Revenue uses to process tax payments and returns. This month, I sat down to talk to Erika Ungern, an 18 year veteran of the Audits division and the lead for the audit.

Why was the GenTax system selected for an audit?

A lot of the work we do on the IT team supports financial auditors. They need to know that the information they use for their audits is reliable. GenTax is a fairly new system – the Department of Revenue completed the last of four rollouts in November 2017 – so it was a good time to take a look.

What was the goal of this audit?

We were auditing to answer the question: Does the system do what it needs to do? That meant primarily looking to see if there are application controls in place so data remains complete, accurate, and valid during input, processing and output. In this case, GenTax is the software DOR uses to process tax returns and payments – which is something all taxpayers may be interested in.

What sort of criteria do you use to assess how well the controls are in place?

We currently use the Federal Information System Controls Audit Manual, or FISCAM. It’s a standard methodology for auditing information system controls in federal and other governmental entities. It provides guidance for evaluating the confidentiality, integrity, and availability of information systems. The information included in FISCAM ties back to National Institute of Standards and Technology (NIST) publications.

How did you go about gathering information?

This audit, like all IT audits, started with interviews and a review of agency policies and procedures. We need to know how agencies have implemented the technology and how staff are using it. We test different pieces of the technology depending on the answers we get. For instance, if we hear that the agency has specific controls in place, we’ll test those controls. If they tell us they don’t have controls, then that’s our finding. For instance, a lot of agencies don’t have strong disaster recovery controls in place for IT systems. That was the case for this one. We check back on their progress in follow-up audits.

Was there anything unique about this audit?

It was somewhat unique in that we were looking at a system that DOR purchased, and both DOR and the vendor are actively involved in supporting the software. Agencies used to build their systems all in-house, and when we would do an audit, we would only talk to agency personnel. When we do an audit of purchased software, system changes are sometimes made exclusively by the vendor, and our audit questions focus on how the agency makes sure those changes are correct, since we are not auditing the vendor’s change management procedures. In this case, DOR and the vendor both make changes to the system, so we asked both agency and vendor personnel about their processes to ensure the changes were correct.

Another new thing was reporting some results that didn’t hit the materiality threshold. This audit reported on a few things that only affect a small percentage of returns the software processes, like the fact the software doesn’t currently provide notification when taxpayers make a mistake in reporting withholding on their returns that causes them to overpay taxes. These results may end up going hand in hand with the performance audit of DOR’s culture that’s going on right now.

Any other thoughts on auditing for IT auditors, or auditors in general?

You know, IT audits are like a lot of other audits. Getting good results is all about asking the right questions. You don’t always know what they are when you start, but do your best to figure them out!

Read the full audit HERE

Members of the audit team included:
Will Garber, CGFM, MPA, Deputy Director
Teresa Furnish, CISA, Audit Manager
Erika Ungern, CISSP, CISA, Principal Auditor
Sherry Kurk, CISA, Staff Auditor
Sheila Faulkner, Staff Auditor

Accountability and Media Auditors at Work Featured

Methods (to our madness): Complex analysis in the public eye

The Secretary of State recently released a performance audit on the Oregon Health Authority: Oregon Health Authority Should Improve Efforts to Detect and
Prevent Improper Medicaid Payments. This audit received a lot of media exposure, in part due to an Audit Alert released in May, some months before the scheduled audit release date. Unsurprisingly, this led to more than a little pressure. How did our 4 person audit team (Ian Green, Wendy Kam, Kathy Davis, and Eli Ritchie) approach this audit, stay cool under fire and make sure their conclusions were sound? I sat down with the lead auditor, Ian Green, to find out more.

You led the OHA, Improper Medicaid Payments audit. What are your strategies when you’re faced with a complex agency and a complex topic?

When we started this audit, we knew we’d be looking at improper payments. Even that’s such a big topic, we knew we’d need to scope it down where we could. So we got as much information as we could from all levels – hundreds of interviews with officials and analysis, looking at agency documentation, research on best practices, all of that.

What methods did you use to identify improper payments?

Our primary focus was to look at process issues, but we did attempt to find some improper payments. We used audit software to analyze large data sets. We did a lot of data matching and looked for results that were outliers. For instance, we checked to see if providers were getting duplicate reimbursements. It’s a complex system, so providers and billers might make errors that should be caught before payments are set out. Another example was checking to see if there were people enrolled in the Oregon Health Plan who shouldn’t be – like if someone had moved out of state.

 What challenges did you face doing this audit, and what strategies did you use to address them?

One challenge was the sheer amount of data. We looked at over two hundred million records.  There was so much data that running tests could take a very long time. My team would run a script and leave it overnight to finish. We had to be very careful about how we set up our tests. Since we kept everything scripted out, each time we got new information, we could just update that script. That kept the testing sustainable, which is very important given all the last minute information we received.

To address the complexity of the topic, we separated our approach into three subtopics: prevention, detection, and recovery. Each person on the team focused on one area, and we’d meet to discuss weekly. That helped make sure we covered all the information while still working together closely.

Another challenge was trying to get complete data. We’d request data and be told we had it all. And then we’d find out it was incomplete. That meant we had to continue reworking our analysis constantly. Without scripting, it would have been extremely time-consuming to perform this work manually.

What was the hardest thing about completing this audit in the public eye?

It’s a very sensitive topic. We knew that we’d get a lot of scrutiny. But we did what we always do, which is to work really hard to make sure all our conclusions are accurate and well-supported, and put all our work through a thorough quality assurance process.

 Is there anything you wish non-audit folks knew about the audit process?

Generally, there’s a public perception that an audit should find everything that might be going wrong. Auditors look at a higher level to see if there are controls in place to prevent something from going wrong. If we’re concerned, we may do deeper testing to see what’s actually happening. For instance, we looked at processes to manage improper payments. Our goal wasn’t to find all of the improper payments being made. Our testing helped measure the effect of the processes that are currently in place.

Anything else?

It was a big audit. We’ve been excited to see important changes happening, even while we were still working on the audit. The Oregon Health Authority is working to address weaknesses in their processes and being more transparent. That’s a really good outcome, from our perspective.

 

Check out the audit here: http://sos.oregon.gov/audits/Documents/2017-25.pdf

Auditing and Methodology Auditors at Work Featured Performance Audit