Insights for auditors

I have been subscribing to a series of newsletters by Harvard professor Bob Behn for many years now. Behn teaches in the Kennedy School of Government. They cover many topics that touch on our audit world: accountability, performance, measures, management. The one-pagers are an interesting quick read on topics about government performance.

Behn has a good eye for the pragmatics that interfere with the best laid plans of policy makers and executives.

His narratives often bring in ideas and experiences we encounter in performance auditing, though he has raised a few questions about the profession. He wrote about one hearing he attended in Olympia in 2008 about a performance audit of the state’s transportation department, produced by contractors Ernst & Young. He learned about specific cost savings identified by the auditors but didn’t get a sense of the agency’s performance, and seemed to have an idea of what was missing:

“For a performance audit, however, what is the equivalent of the financial rules? Who has set what expectations for performance for which the auditors can check”?

It’s clear that Professor Behn comes from a measurement viewpoint, because he seems to want a target or expectation as a basis to gauge performance. The intent of performance auditors is to spot a solution or strategy that improves an organization’s performance, and results. We don’t grade agencies but we show agencies a path to save money, serve more clients, or produce better outcomes. And, as we say, there is always room for improvement…

It would be very difficult for us to produce a reliable grade because the performance of an organization is dependent upon too many variables, external and internal. Economic conditions can affect the costs of goods and services, or the workload in an employment department. Demographic changes, federal rule changes, weather conditions, and even availability of street drugs are just some of the factors that can make government agency outcomes better or worse. These agencies also face internal variables such as turnover in managers or staff, computer system efficacy, agility to change, and sufficiency of resources.

The professor writes about many of these challenges, so perhaps he is taking a second look at his first impressions of performance auditing. Some activities are subject to control, such as financial rules, while others can only be ‘managed’ within circumstances.

Here is a sampling of some of the more recent newsletters, and there are many others worth reading as well.

May 2015: Policy Design & Operational Capacity

November 2014: Avoiding the KPI Quagmire

October 2014: What Performance Mgmt Is and Is Not

August 2014: Designing a Test Worth Teaching To

July 2014: Be Both a Hedgehog and a Fox

June 2014: What Is Really Going On? [ II ]

November 2013: Measurement, Mgmt, & Leadership

October 2103: Punishing the Efficient

Gary Blackmer OAD Director

Gary Blackmer
OAD Director

 

Header Image: @Stephen Orsillo, Dreamstime Stock Photos

 

Accountability and Media Auditors at Work Featured

Audits and Oregon’s Integrity Ranking

The Center for Public Integrity recently issued a report that assessed the systems in place to deter corruption in state government. The report offers a different perspective on some of the issues in the sphere of an auditor’s work. Oregon didn’t fare well this year, 44th among the 50 states, and the allegations against former governor Kitzhaber were a key element in the scoring. Oregon’s F grade was not unique, with thirteen other states flunking, and only three earning better than a D+.

Oregon_rank

Considerable effort goes into the evaluation. Each state undergoes an in-depth analysis by a reporter or researcher familiar with that state. I was interviewed by Lee van der Voo about our auditing responsibilities and activities and she also obtained other viewpoints.

One gauge of integrity is the presence of laws related to the 13 evaluated categories. While I think that laws can be an important element of clean government, the absence of a law should not be equated with a lack of ‘integrity’.

As we all know, the law and a structure of enforcement are not always a sufficient deterrent to criminal behavior.  Government ethics laws are common, and commonly broken. Some ethicists argue that it is more effective to characterize good behaviors than to try to define all prohibited activities.

Ms. van der Voo seems to recognize Oregon’s values in her title “Land of ethics, manners hurt by rare scandal.” When I looked at our Washington neighbor’s rankings, the reporter stated that, “A 2012 report by the University of Illinois at Chicago, for instance, concluded that only Oregon had a lower per-capita rate for federal corruption convictions.”

I think public officials should examine the deficiencies identified in this review, but I would also caution that additional laws may not necessarily improve Oregon’s integrity if there exists a higher expectation among its citizens, government employees, and elected officials.

Auditing grade

I hated whiners in school who appealed a grade, so instead I’ll look at why we didn’t get an “A” and figure out what would improve the grade next time.

Foremost, it is gratifying to see that the C+ for the auditing section was the best grade, from these very tough graders, for Oregon. (Electoral Oversight and State Budgeting got Cs.)

I continue to cringe though when I see the category name ‘Internal audit’ for us. I actually had several conversations and emails with the director and reporter, trying to educate them about the categories of auditing. I explained that an internal auditor often answers to the executive of an agency, which can affect what topic is selected and whether the results will ever be released. I urged them to consider using the Government Auditing Standards term ‘independent’ for auditors who are elected, or answer to a legislature. It is obvious I was not persuasive. (Even ‘Audit Oversight’ would be an improvement.)

Below is a breakdown of the factors and scores that comprise the C+ grade. The factors have links on the website with some discussion and sources of information.

internal_auditing

“In law, the leadership of the audit entity is protected from political interference”: I understand the group’s intent and I don’t think there is much we could do to raise this score. We follow the national Government Audit Standards which assures objectivity and independence. The Secretary of State is the state auditor as set out in the Oregon Constitution and is protected from political interference. I wonder whether there might be some inconsistencies between scorers. The score for Washington’s elected state auditor is ‘yes’ and describes a structure very much like ours. I don’t know what protections are afforded Washington’s Audits Director but it may be different the position in Oregon, which is ‘at will.’

Washington’s overall internal auditor score is B+. While we did worse on the leadership issue above, we got 100s in the next two, better than Washington’s 75s, regarding the practical aspects of independence and sufficient funding.

Oregon got a 75 on government action of audit findings. We get about 80% implementation of our recommendations so this coincidentally aligns with the score. Incidentally, I think it is tricky to place a big value on this statistic because the content of recommendations can vary drastically. Some may be simple adjustments for an agency, others may require substantial resources and time to accomplish. We can get 100% action on trivial recommendations that are easily implemented. The most important dimension is difficult to measure: the benefit of the audit recommendations for Oregonians.

“In law, the audit agency is required to report on its investigations, activities, and advisory opinions.” I think I may have misunderstood the question and we could raise this score next time. In fact, we are required by law (ORS 197.180) to produce an annual Hotline report on our investigations, which is on our website. I’m not sure what ‘activities’ or ‘advisory opinions’ mean, but we if we do them, we would report on them. (If ‘activities’ and ‘advisory opinions’ refer to consulting-type services, we don’t do those, because that would violate our audit standards.)

“In law, citizens can access audit reports.” Again, perhaps more discussion of the reporting requirements imposed upon us by our audit standards could raise the score next time. We are required by state law to follow those standards and they require we widely distribute our audit reports. All our reports are on our website, accessible to everyone, and we make our workpapers public once the audit is complete.

Lastly, the ‘open data format’ is a category that puzzles me. Wikipedia identifies a number of formats that meet those standards and the pdf format of our audits is listed as acceptable. We ensure that our pdf reports are ADA compliant, and no one has ever complained about difficulties accessing our audits. As technologies advance we will continue to examine ways to publish our audit reports, and keep this criterion in mind.

So, next time we’re contacted, we’ll compare our structure to Washington’s to see if that will bring our score up. We’ll cite ORS 177.180 which requires that we report our Hotline investigations annually to the Legislature.  We’ll also point to our audit standards on audit reporting requirements to see if that might improve the score.

Gary Blackmer OAD Director

Gary Blackmer
OAD Director

Accountability and Media Featured Noteworthy

The Roots of Performance Auditing in the Northwest

Jewel Lansing

Jewel Lansing

Oregon has many good local auditors who don’t limit the scope of their work to financial matters. They also produce performance audits that trigger improvements in their government’s programs. The genealogy of performance auditing in Oregon traces back to Jewel Lansing, the elected Multnomah County Auditor from 1975 to 1982.

The story I heard from Jewel is that the Multnomah County elected chair, Don Clark, approached her, hoping to get a professional candidate to run for the elected auditor position against the incumbent. Jewel was a CPA, and former teacher, engaged in local issues and civic matters. She agreed to run and defeated her opponent.

Early in that campaign she agreed to pick up a guest speaker at the airport for a CPA training conference. The speaker was Lennis Knighton, a professor at Brigham Young University, who had written a paper defining a type of accountability that is now known as performance auditing. During the drive she decided to adopt it as part of her campaign platform.

This wasn’t the first performance auditing in the nation however. Up in King County, their legislative auditor, Rol Malan, had received support and federal funding to explore performance auditing at the local level. In this case the funding source was the GAO (now the Government Accountability Office, originally the General Accounting Office).

After two terms as Multnomah County Auditor, Jewel ran and was elected Portland City Auditor. There, against considerable resistance from the mayor and several city commissioners, she introduced performance auditing in 1984. I was fortunate to be one of her hires in 1985. Jewel chose not to run again and left office in 1986.

In 1990 Jewel came back through the City offices one day and suggested that I consider running for Multnomah County Auditor. The value of that office had declined because there had been no requirements that a candidate needed to be a professional auditor or accountant. I made the leap and she agreed to be my campaign chair. I was elected and served two terms, then ran for City Auditor and served for two terms and part of a third, following in Jewel’s footsteps.

Jewel had also planted seeds for two other local performance audit functions, in Washington County, and at Metro the regional government. Alan Percell, Jewel’s deputy auditor in Multnomah County, was approached and agreed to start a similar function in Washington County. In 1990 the Metro government was conducting a charter review and several local government auditors at Jewel’s encouragement testified that the regional government needed an independent elected auditor, which was added and approved by the voters.

The impact of Jewel’s efforts and the attention of her audits didn’t stop at the local level. Norma Paulus, a contemporary of Jewel’s and the Oregon Secretary of State from 1977 to 1985, first promoted performance auditing by her division. The next secretary, Barbara Roberts, expanded on the idea and the efforts, as did Phil Keisling, her successor.

After I left City Hall in 2009 Secretary of State Kate Brown asked me to lead her Audits Division. It was an honor and challenge to bring what I had learned in nearly 25 years of auditing to the state level. One of our principal auditors, Shanda Miller, was recently appointed the Lane County Auditor, and I like to think that some of Jewel’s legacy rubbed off during my nearly six years together here with Shanda.

And whenever prospective candidates for local elected offices contacted me, I made sure to send them to Jewel who still remains active, writing histories of Portland and Multnomah County. I know that auditors Suzanne Flynn, LaVonne Griffin-Valade, Steve March, Mary Hull Caballero, and Brian Evans have all met with her to talk about their aspirations.

Oregon’s important government accountability has been strongly influenced by the work and contributions of Jewel Lansing. I ran across an article she wrote in 1977 about her early experience in auditing, which you can see below. The observations are still very current.

All of this important government accountability has become a public expectation in Oregon, and in the Northwest for over 40 years. Much of it resulted from the contributions of Jewel Lansing.

Jewel1

Jewel2

Gary Blackmer Director, Oregon Audits Division

Gary Blackmer
Director, Oregon Audits Division

Auditors at Work Featured Noteworthy

The Evolution of New Auditor Training at OAD

How does training impact the work of new auditors?

We provide training to our newly hired auditors because government auditing is very different than business auditing. Unfortunately, no universities specialize in this field, and even a single class is rare. We try to update and improve our training each time, but what we’ve learned in the last year about our instructional design approach has given us new insights.

First, what we taught was not specifically designed to help new performance auditors conduct their day-to-day work. We explained about preparing a good workpaper, or using our software, Teammate. But there were gaps in the framework and perspective of auditing. To address those gaps, we began to develop trainings that complemented our practical nuts and bolts offerings with courses focusing on soft skills development and leadership.

Auditing has a maze-like feeling for the new hire but we sometimes forget that there are two perspectives of a maze – the bird’s eye view, and the view from inside. From above, our eye can trace the shortest route. But inside, there are no direction markers.  Success must come by assembling a clear memory of the layout of the maze through arduous trial and error.

Inside Hampton Court Maze

Inside Hampton Court Maze

Some people enjoy the puzzle of a maze and pay to wander through corn mazes. Some have no patience for mazes. The image to the right makes some folks feel like a lab rat. And not a very smart one, since I hear that lab rats are quite capable of negotiating mazes.

But even as we think about this frustration, we also need to ’embrace the maze’. Every new audit is an agency maze. We try to understand how all the program and financial pieces fit together, and build up the picture through interviews, data analysis, and observation.

On the financial side we have workpapers and team member continuity to guide us through the maze of risks each year. On the performance side, we are often entering new agency mazes and our initial, scoping phase produces a rough drawing of the maze to help us narrow down to an issue that we should audit.

Above Hampton Court

Above Hampton Court

The other maze is our audit process.  We want newly hired auditors to be able to negotiate their way through the agency maze without also getting lost in the auditing maze. The better we can prepare new auditors in our methods and procedures, how to audit, the more they can focus on what they are auditing.

Our instructional design efforts are laying out the tasks more clearly for new hires with 11 classes so far. For example, we have training on audit documentation, interviewing, elements of a finding, sufficiency of evidence, and report writing. This knowledge and skill set are spread out, presented when a new hire will encounter them, then reinforced as they do their work.

Nonetheless, we recognize that many aspects of auditing are affected by circumstances and context that can’t be set in procedure. Strategies, priorities, principles, and standards provide general guidance in these situations. Leadership is also important and we want to help our newly promoted team leads understand the new responsibilities and situations they must take on, and perform them with competence and confidence.

We are developing that team leader training in our next phase, and some intermediate auditor training, and some technical training on specific software or analytical tools.

This seems like a lot of work but we are asking our auditors to develop and offer the classes, which involves us all in the responsibility of defining expectations, developing the content, and presenting the instruction.

We want to shorten the learning curves of our auditors because that translates into highly professional work from everybody. Staff members that have had the opportunity to take part in thorough and holistic training are a key element of a learning organization.

Auditors at Work Featured Financial Audit Performance Audit

A new approach in financial accountability

Portland just released its biennial financial condition report, which is a great tool that auditors in our region use. Auditors for Portland, Metro, the State of Oregon, and Multnomah County all regularly issue financial condition reports. These reports follow the same method of looking at key financial indicators over a ten year period to spot trends. These trends may show the recovery from a problem, or could be a warning that a problem is creeping up, year by year.

They use graphics in creative ways to simply tell these stories. The reports are a great way to inform the public about the sources of a government’s money, and where that money is spent.

Featured Financial Audit

GED Audit wins national award

Audit Manager Sheronne Blasi

Audit Manager Sheronne Blasi

We are very pleased to learn that our

Featured Noteworthy Performance Audit

IT Audits help trigger budget action

In addition to a cyber attack, two of our audits helped prompt a state agency to request more resources for computer security and for better oversight of computer projects, as reported by the Portland Tribune.

Trib_IT_auditOur audits always include a security component and we prepare a confidential report on the weaknesses we find. State law allows confidential reporting to deny the information to hackers. Read our 2010 audit on security.

We recently issued an audit on IT Project Management, and concluded that inadequate resources would hamper the oversight by the Department of Administrative Services. Read our 2015 audit.

IT Audit