Methods (to our Madness): How IT audits help keep your $$$ safe

Recently, the Secretary of State Oregon Audits Division released an IT audit of GenTax, the software system that Oregon’s Department of Revenue uses to process tax payments and returns. This month, I sat down to talk to Erika Ungern, an 18 year veteran of the Audits division and the lead for the audit.

Why was the GenTax system selected for an audit?

A lot of the work we do on the IT team supports financial auditors. They need to know that the information they use for their audits is reliable. GenTax is a fairly new system – the Department of Revenue completed the last of four rollouts in November 2017 – so it was a good time to take a look.

What was the goal of this audit?

We were auditing to answer the question: Does the system do what it needs to do? That meant primarily looking to see if there are application controls in place so data remains complete, accurate, and valid during input, processing and output. In this case, GenTax is the software DOR uses to process tax returns and payments – which is something all taxpayers may be interested in.

What sort of criteria do you use to assess how well the controls are in place?

We currently use the Federal Information System Controls Audit Manual, or FISCAM. It’s a standard methodology for auditing information system controls in federal and other governmental entities. It provides guidance for evaluating the confidentiality, integrity, and availability of information systems. The information included in FISCAM ties back to National Institute of Standards and Technology (NIST) publications.

How did you go about gathering information?

This audit, like all IT audits, started with interviews and a review of agency policies and procedures. We need to know how agencies have implemented the technology and how staff are using it. We test different pieces of the technology depending on the answers we get. For instance, if we hear that the agency has specific controls in place, we’ll test those controls. If they tell us they don’t have controls, then that’s our finding. For instance, a lot of agencies don’t have strong disaster recovery controls in place for IT systems. That was the case for this one. We check back on their progress in follow-up audits.

Was there anything unique about this audit?

It was somewhat unique in that we were looking at a system that DOR purchased, and both DOR and the vendor are actively involved in supporting the software. Agencies used to build their systems all in-house, and when we would do an audit, we would only talk to agency personnel. When we do an audit of purchased software, system changes are sometimes made exclusively by the vendor, and our audit questions focus on how the agency makes sure those changes are correct, since we are not auditing the vendor’s change management procedures. In this case, DOR and the vendor both make changes to the system, so we asked both agency and vendor personnel about their processes to ensure the changes were correct.

Another new thing was reporting some results that didn’t hit the materiality threshold. This audit reported on a few things that only affect a small percentage of returns the software processes, like the fact the software doesn’t currently provide notification when taxpayers make a mistake in reporting withholding on their returns that causes them to overpay taxes. These results may end up going hand in hand with the performance audit of DOR’s culture that’s going on right now.

Any other thoughts on auditing for IT auditors, or auditors in general?

You know, IT audits are like a lot of other audits. Getting good results is all about asking the right questions. You don’t always know what they are when you start, but do your best to figure them out!

Read the full audit HERE

Members of the audit team included:
Will Garber, CGFM, MPA, Deputy Director
Teresa Furnish, CISA, Audit Manager
Erika Ungern, CISSP, CISA, Principal Auditor
Sherry Kurk, CISA, Staff Auditor
Sheila Faulkner, Staff Auditor

Accountability and Media Auditors at Work Featured

Audit Release – Oregon Department of Revenue: GenTax Accurately Processes Tax Returns and Payments, but Logical Access and Disaster Recovery Procedures Need Improvement

Report Highlights


The Oregon Department of Revenue (DOR) designed and implemented controls in their GenTax system to provide reasonable assurance that tax return and payment information remains complete, accurate, and valid from input through processing and output. Logical access controls and change management controls are generally sufficient, but some areas need improvement. In addition, existing controls ensure the creation of appropriate backup of GenTax system files, though DOR does not have assurance they could timely restore the system in the event of a disaster or major disruption.

Background

The Oregon Department of Revenue replaced its legacy tax systems with GenTax, an integrated tax processing software package. This system processed about $10.3 billion in payments and $1.2 billion in refunds for tax periods ending in 2016.

Audit Purpose

The purpose of our audit was to review and evaluate key application and general computer controls governing DOR’s GenTax system. We focused on personal income, withholding, and corporate income and excise tax programs.

Key Findings

  1. GenTax controls ensure accurate input of tax return and payment information for personal income, withholding, and corporate income and excise tax programs. Additional processing and output controls provide further assurance that GenTax issues appropriate refunds and bills to taxpayers for taxes due.
  2. Logical access controls are generally sufficient, but DOR needs to make improvements to ensure managers have enough information to request appropriate access. DOR should also ensure that access remains appropriate for users who change jobs and is removed for users who are terminated.
  3. DOR monitors and tracks changes to GenTax to ensure system developers implement only approved program modifications, but better guidance is needed for testing procedures to ensure program modifications meet business needs.
  4. DOR does not have sufficient assurance that it could timely restore GenTax in the event of a disaster or major disruption.
  5. DOR has not obtained independent verification that the GenTax vendor has implemented appropriate controls over servers at an external data center to provide additional assurance that Oregon data is secure.

Recommendations

The report includes 11 recommendations to DOR regarding needed improvements to logical access procedures, disaster recovery plans and tests, and independent assurance of controls over servers at an external data center. DOR generally agreed with our recommendations. DOR’s response can be found at the end of the report.

Read the full report here.

Featured IT Audit New Audit Release

Audits in the News: May

Audits in the News: Prior audit work on delinquent debt appears in an article

We here in the audits division are proud that the work we do makes a difference. Our work attracts the attention of the legislature, statewide news sources, and even local media outlets. Local media coverage of our audits is just another way we communicate with the people of Oregon about the work that we’re doing on their behalf to make government better. This is part of an ongoing series of posts rounding up recent instances in which the Oregon Audits Division makes a cameo in the local news.

The Audits Division’s work continues to garner mentions by Oregon media, as reporters continue to explore issues covered in previous audits.

Capital Bureau/Portland Tribune – State vendors who owe taxes, other debt continue to be paid

Read the story here.

“Auditors in the secretary of state’s office already found instances in which the state could have recovered debt by intercepting state payments to some of these vendors. In a state audit released last fall, the auditors found “more than 9,000 state debtors were on the state vendor list and had received payments or were authorized to receive payments.” Auditors found specific instances in which the state could have recouped some of the debt owed, by intercepting payments to the vendors.”

Revisit that audit here.

Audits in the News Featured

Methods (to our Madness): Using Data to Tell the Story of a Debt Problem

Periodically, we will highlight some of the methods used in a recently released audit. Every performance audit is unique and can require creative thinking and methodologies to answer our audit objective. Some of these methods could be replicated or present valuable lessons for future projects.

Sometimes it takes a number to get the point across. And sometimes it takes actually doing the work to show that the work can be done.

These were two of the big takeaways from a recent conversation I had with Jamie Ralls, principal auditor at the OAD and project lead for a recently released performance audit on debt collection: Oregon Needs Stronger Leadership, Sustained Focus to Improve Delinquent Debt Collection.

Vendor offset analysis showed potential savings of at least $750,000 a year

Jamie conducted an analysis for the audit on vendor offset. Vendor offset is when a state matches a list of debtors that owe the state money to a list of vendors that the state pays money to for services. Then instead of paying money to the vendors for services, the state intercepts those payments and applies it to the debt. This is something that 40 other states do, but Oregon did not do at the time of the audit.

Jamie looked at what Oregon could have collected had it used vendor offset. The result: At least $750,000 a year.

Limitations in the data resulted in a cautious estimate

The $750,000 a year estimate was likely low considering that the list of debtors was incomplete from a statewide perspective. The Department of Revenue maintained the list and it did not include debt held at other agencies. Additionally, due to the complexity of the analysis, the team only calculated debt and payments by year. An ongoing monthly calculation would have produced a greater collection amount.

Lessons learned: document along the way

Jamie said that if she could go back she would have been better about documenting all of the steps she took in the analysis as she went along. She was so caught up in the excitement of the work that she did not always stop to document everything. She then had to go back and retrace some of her work.

Using data to tell the story of a debt problem

When I asked Jamie why the audit team did this specific analysis, she said that paying vendors who owe money to the state has been a long-standing problem. The Audits Division had first recommended vendor offset in 1997. However, in the past our office had only talked about it anecdotally.

Being able to show the extent of the problem through data analysis had a big impact. Actually going through the methodology also demonstrated that doing vendor offset was technically possible. During the course of the audit, in part due to testimony from the audit team showing this analysis, the Oregon Legislature passed Senate Bill 55. SB 55 requires the Oregon Department of Revenue to do a vendor offset.

Breaking down the methodology

Here is a step-by-step look at how Jamie analyzed vendor offset:

  1. She took a list of approved vendors from the Oregon Department of Administrative Services and a list of debtors from the Oregon Department of Revenue. She matched the lists based on tax id numbers. She found 9140 debtors who were approved as vendors to receive payment from the state. These vendors owed a total of $67 million in debt.
  2. Next, she pulled queries in the Statewide Financial Management Application (SFMA) to find and export records of all payments to these vendors for the time period of 2011 to 2014.
  3. She then summarized the debt by each year and summarized the payments each year.
  4. She took the debt for the first year (2011) and subtracted the payments for the following year (2012). If a balance of debt remained, it was rolled over to the next year (2012) to create a rolling balance of debt.
  5. For each year, the amount of debt that could have been collected through payments in the following year was also calculated and rolled forward, to create a rolling balance of what the state could have collected.
  6. She computed $3 million in debt that could have been collected, or an average of $750,000 a year.

 

Caroline Zavitkovski, OAD Senior Performance Auditor, MPA

Caroline Zavitkovski, OAD Senior Performance Auditor, MPA

Auditing and Methodology Data Wonk Featured

Audit Release: Oregon Needs Stronger Leadership, Sustained Focus to Improve Delinquent Debt Collection

Executive Summary


Liquidated and delinquent receivables owed to the state of Oregon have almost doubled since 2008, to nearly $3.2 billion, while collection rates on the debt have dropped. The state’s debt collection system needs more leadership, sustained focus and accountability to improve performance over time.

 

Read full report here.

Past due receivables are growing

Oregon’s liquidated and delinquent debt rose from $1.7 billion at the end of fiscal year 2008 to nearly $3.2 billion by 2014, while statewide collection rates on that debt dropped from 13.5% to 11.2%. Nearly $800 million of the debt is tied to the state’s general fund.

Liquidated and Delinquent Receivables

collections audit blog pic 1

Source: Legislative Fiscal Office. Adjusted for PERS errors. Excludes Department of Administrative Services interagency debt.

The recession contributed to the increased debt. Evidence indicates many of the debtors are low-income, and more than half the debt may be uncollectible.

However, bumping up Oregon’s collection rates could still make a substantial difference over time. At 2014 debt levels, every percentage point increase in the statewide collection rate would improve collections by about $38 million. If Oregon had collected delinquent debt at a 13.5% rate in 2014 – last achieved in 2008 – the state would have brought in nearly $90 million more in collections.

Our audit found four key improvements that could help Oregon increase collections:

  • Improved oversight of collections;
  • Enhanced performance measurement and reporting;
  • Increased expectations for private collection firms and the state’s central collection agency;
  • Better use of proven collection tools.

Oregon has not focused on improving collections

collections audit blog pic 2Our audit found Oregon’s highly decentralized approach to collections has contributed to a lack of sustained focus on improvement. This is our sixth collections-related audit since 1997. Significant improvements identified in those audits have not been implemented, some dating back 18 years.

Oregon has not implemented productive collection tools used by other states, has not resolved lingering legal issues that hinder collections, and has allowed inadequate performance measurement to persist.

Individual agencies have made some improvements. Statewide, however, no one has been tracking collection improvement efforts or encouraging them. Our discussions with leading states on debt collection highlighted the importance of having a system “expert” responsible for identifying potential improvements, looking outside the state system for new opportunities, and reporting to decision makers.

In Oregon, the statutory authority and history of the Department of Administrative Services indicate it is the best agency to serve as a statewide strategist on debt collection.

Performance reporting, measurement are flawed

collections audit blog pic 3State agencies routinely collect receivables, or bills for charges and services. Statewide performance reporting focuses on receivables that become “liquidated and delinquent” – past due debt that debtors have had a chance to contest.

The Legislative Fiscal Office prepares an annual report on liquidated and delinquent debt collection, designed 16 years ago by the Legislature to help drive collection improvements. However, the report includes few large-debtor agency details – not even their collection rates – contains noteworthy inaccuracies, and does little to hold agencies accountable for collections performance. It also does not identify potential collections improvements or detail the status of agency improvement efforts, key to encouraging advances.

In addition to reporting, we also focused on “assignment” of debt, accounts sent by agencies to private collection firms or the Other Agency Accounts unit at the Department of Revenue, the state’s collectors of last resort. Private collection firms carried nearly $1 billion of the state’s debt as of 2014 – more than double the 2008 balance – with a collection rate just over 1%. Other Agency Accounts, the state’s central collection agency, had a better rate, roughly 7%, according to Legislative Fiscal Office data. Assignment to OAA has stayed relatively flat, however, hitting $259 million in 2014.

We found the Department of Administrative Services is not evaluating the performance of OAA or private collection firms. We also found some large-debtor agencies are not using performance information to strategically assign debt.

Oregon is not using some proven collection tools

Our research, discussions with other states and interviews with Oregon officials suggested eight tools Oregon could pursue to increase collections, including some the state has considered for years but not implemented.

Among the most promising:

State vendor offset: Forty states are intercepting state payments to debtors who are also state vendors, including corporations and consultants. Our work indicates vendor offset in Oregon would collect at least $750,000 a year.

collections audit blog pic 4Bank levies: Other states have systems that allow for automated matching of a wide variety of debtors to bank account records, a process that yielded $30 million for Wisconsin in 2014.

Internet posting of debtors: Twenty-three states maintain public online lists of debtors, some focused only on large debtors, to increase collections. Many of the debtors pay after they receive a warning letter but before the information is posted.

2015 Legislative changes should help

The Institute for Modern Government at Willamette University drafted Senate Bill 55 in the 2015 legislative session to improve debt collection. We issued an interim report to the Legislature to suggest further legislative changes. Our recommendations were incorporated in the bill, which the Legislature passed and the governor signed in July.

At our recommendation, Senate Bill 55 charged the Department of Administrative Services with monitoring and improving debt collection. DAS’s duties, detailed in the bill, include improving performance reporting and assignment of debt for collection. DAS started a committee last year to address statewide collections, and contributed to Senate Bill 55.

Senate Bill 55, passed by the Legislature, included changes we recommended.

Even with stronger oversight, improving collection of Oregon’s rising debt will not happen overnight. During our audit, we found that improving collections requires meticulous work with agencies.

DAS officials – and policy makers – will also have to be persistent to ensure improvements are made.

Recommendations

Beyond the changes implemented in Senate Bill 55, we found improvements OAA could focus on. We also found other steps DAS could take, including:

  • Preparing meaningful annual reports on debt collection, relevant to the public and policy makers.
  • Helping agencies adopt successful collection tools.
  • Developing short- and long-term plans for a sustained focus on debt collection.

Agency Responses

Both the Department of Administrative Services and the Department of Revenue generally agreed with our recommendations, with DAS noting that it recognizes its oversight role.

DAS said it would focus efforts on current receivables as well as liquidated and delinquent debt. The response also included concerns about the difficulty of adopting a fully integrated vendor offset program.

The Department of Revenue said agency officials will continue to discuss many of the collection improvements noted in our audit with policymakers and stakeholders. A computer system upgrade now underway will help the agency make further improvements, the response said.

The full agency responses can be found at the end of the report.

 

Featured New Audit Release Performance Audit