Insights for auditors

I have been subscribing to a series of newsletters by Harvard professor Bob Behn for many years now. Behn teaches in the Kennedy School of Government. They cover many topics that touch on our audit world: accountability, performance, measures, management. The one-pagers are an interesting quick read on topics about government performance.

Behn has a good eye for the pragmatics that interfere with the best laid plans of policy makers and executives.

His narratives often bring in ideas and experiences we encounter in performance auditing, though he has raised a few questions about the profession. He wrote about one hearing he attended in Olympia in 2008 about a performance audit of the state’s transportation department, produced by contractors Ernst & Young. He learned about specific cost savings identified by the auditors but didn’t get a sense of the agency’s performance, and seemed to have an idea of what was missing:

“For a performance audit, however, what is the equivalent of the financial rules? Who has set what expectations for performance for which the auditors can check”?

It’s clear that Professor Behn comes from a measurement viewpoint, because he seems to want a target or expectation as a basis to gauge performance. The intent of performance auditors is to spot a solution or strategy that improves an organization’s performance, and results. We don’t grade agencies but we show agencies a path to save money, serve more clients, or produce better outcomes. And, as we say, there is always room for improvement…

It would be very difficult for us to produce a reliable grade because the performance of an organization is dependent upon too many variables, external and internal. Economic conditions can affect the costs of goods and services, or the workload in an employment department. Demographic changes, federal rule changes, weather conditions, and even availability of street drugs are just some of the factors that can make government agency outcomes better or worse. These agencies also face internal variables such as turnover in managers or staff, computer system efficacy, agility to change, and sufficiency of resources.

The professor writes about many of these challenges, so perhaps he is taking a second look at his first impressions of performance auditing. Some activities are subject to control, such as financial rules, while others can only be ‘managed’ within circumstances.

Here is a sampling of some of the more recent newsletters, and there are many others worth reading as well.

May 2015: Policy Design & Operational Capacity

November 2014: Avoiding the KPI Quagmire

October 2014: What Performance Mgmt Is and Is Not

August 2014: Designing a Test Worth Teaching To

July 2014: Be Both a Hedgehog and a Fox

June 2014: What Is Really Going On? [ II ]

November 2013: Measurement, Mgmt, & Leadership

October 2103: Punishing the Efficient

Gary Blackmer OAD Director

Gary Blackmer
OAD Director

 

Header Image: @Stephen Orsillo, Dreamstime Stock Photos

 

Accountability and Media Auditors at Work Featured

How do we find great auditors?

The Oregon Audits Division is holding a ‘Meet the Agency’ event for people interested in a job as a financial or performance auditor.  Come join us Friday, October 30, 2015 from 11:30 – 2:30 at the Oregon State Capitol, Room 50 to speak with our staff and learn what we do. To see the event flier, click here. Watch the Secretary of State jobs page for the chance to apply.

Every fall we initiate our hiring efforts. Auditing is a profession that is often misunderstood, but has many features that, once discovered, can offer a very fulfilling career.

We start by announcing on our website that we will be holding a ‘Meet the Agency’ session, usually late October. Meet the Agency is a chance for us to show off our office, and to speak to potential applicants about what we do, and who we are. We describe our activities in financial, performance, and information technology auditing. We also provide an opportunity for applicants to sit down with our auditors and ask questions. We want them to have a clear understanding of the role, responsibilities, and satisfactions of becoming an auditor.

mtaOur application is posted on the Secretary of State’s website and is conducted online. We ask for education and employment history and usually ask a couple questions to get a more detailed sense of the applicant. We require applicants to submit their college transcripts as well.

Applicants who meet the minimum qualifications get their written application reviewed for a possible interview. We schedule an interview with the top-ranked applicants then make a decision on hiring.

From the close of the announcements to a hiring decision is generally about 6 to 8 weeks. New hires must also successfully pass a criminal background check.

We consider our hiring decisions the most important ones we make. New hires should also know that we conduct several evaluations during their trial service period to give them feedback on their learning and progress to be an auditor. We take this period very seriously because we sometimes have to make the difficult decision to dismiss those who struggle with the work.

We look for certain characteristics of success in all our applicants, and special kinds of skills and expertise for our financial, performance, and information technology auditors. All our auditors need good interpersonal skills because so much of our work depends upon engaging and interviewing agency personnel. We look for candidates with a nature to ask questions and get answers. We must also be quick to learn the wide range of the activities performed by more than 40,000 other state employees 150 agencies, boards, commissions, and departments. Lastly, our auditors need to communicate what they find, so clear and concise writing is a fundamental skill.

Fin wallFor our financial auditors we require a strong accounting background, of course. But auditing is different than accounting and we either hire or develop a ‘reviewer’ perspective in these auditors. We expect them to apply their judgment in determining whether agencies have carried out their financial work properly.

IT wallOur information technology auditors should have a good grounding in computer systems and some accounting skills can be a bonus, since we are often testing the reliability of financial data in major state applications. We are also looking for applicants with an interest in learning about computer systems and checking them for security vulnerabilities.

We want our performance auditors to be comfortable with numbers, and also have an ability to see the ‘big picture’ and connect it to the critical details. Applicants should bring an understanding of organizations and management principles which can reveal the sources of many of our audit findings.

We have high expectations of our auditor applicants but that also brings them a deep satisfaction when they earn a place in the Secretary of State’s office. Once here, they find a rich variety of experiences that offer long-term personal growth, teamwork with peers and managers that break big challenges down into answers and solutions. That means a fulfilling career serving Oregonians by improving government and holding it accountable.

Auditors at Work Featured Noteworthy

The Roots of Performance Auditing in the Northwest

Jewel Lansing

Jewel Lansing

Oregon has many good local auditors who don’t limit the scope of their work to financial matters. They also produce performance audits that trigger improvements in their government’s programs. The genealogy of performance auditing in Oregon traces back to Jewel Lansing, the elected Multnomah County Auditor from 1975 to 1982.

The story I heard from Jewel is that the Multnomah County elected chair, Don Clark, approached her, hoping to get a professional candidate to run for the elected auditor position against the incumbent. Jewel was a CPA, and former teacher, engaged in local issues and civic matters. She agreed to run and defeated her opponent.

Early in that campaign she agreed to pick up a guest speaker at the airport for a CPA training conference. The speaker was Lennis Knighton, a professor at Brigham Young University, who had written a paper defining a type of accountability that is now known as performance auditing. During the drive she decided to adopt it as part of her campaign platform.

This wasn’t the first performance auditing in the nation however. Up in King County, their legislative auditor, Rol Malan, had received support and federal funding to explore performance auditing at the local level. In this case the funding source was the GAO (now the Government Accountability Office, originally the General Accounting Office).

After two terms as Multnomah County Auditor, Jewel ran and was elected Portland City Auditor. There, against considerable resistance from the mayor and several city commissioners, she introduced performance auditing in 1984. I was fortunate to be one of her hires in 1985. Jewel chose not to run again and left office in 1986.

In 1990 Jewel came back through the City offices one day and suggested that I consider running for Multnomah County Auditor. The value of that office had declined because there had been no requirements that a candidate needed to be a professional auditor or accountant. I made the leap and she agreed to be my campaign chair. I was elected and served two terms, then ran for City Auditor and served for two terms and part of a third, following in Jewel’s footsteps.

Jewel had also planted seeds for two other local performance audit functions, in Washington County, and at Metro the regional government. Alan Percell, Jewel’s deputy auditor in Multnomah County, was approached and agreed to start a similar function in Washington County. In 1990 the Metro government was conducting a charter review and several local government auditors at Jewel’s encouragement testified that the regional government needed an independent elected auditor, which was added and approved by the voters.

The impact of Jewel’s efforts and the attention of her audits didn’t stop at the local level. Norma Paulus, a contemporary of Jewel’s and the Oregon Secretary of State from 1977 to 1985, first promoted performance auditing by her division. The next secretary, Barbara Roberts, expanded on the idea and the efforts, as did Phil Keisling, her successor.

After I left City Hall in 2009 Secretary of State Kate Brown asked me to lead her Audits Division. It was an honor and challenge to bring what I had learned in nearly 25 years of auditing to the state level. One of our principal auditors, Shanda Miller, was recently appointed the Lane County Auditor, and I like to think that some of Jewel’s legacy rubbed off during my nearly six years together here with Shanda.

And whenever prospective candidates for local elected offices contacted me, I made sure to send them to Jewel who still remains active, writing histories of Portland and Multnomah County. I know that auditors Suzanne Flynn, LaVonne Griffin-Valade, Steve March, Mary Hull Caballero, and Brian Evans have all met with her to talk about their aspirations.

Oregon’s important government accountability has been strongly influenced by the work and contributions of Jewel Lansing. I ran across an article she wrote in 1977 about her early experience in auditing, which you can see below. The observations are still very current.

All of this important government accountability has become a public expectation in Oregon, and in the Northwest for over 40 years. Much of it resulted from the contributions of Jewel Lansing.

Jewel1

Jewel2

Gary Blackmer Director, Oregon Audits Division

Gary Blackmer
Director, Oregon Audits Division

Auditors at Work Featured Noteworthy

The Evolution of New Auditor Training at OAD

How does training impact the work of new auditors?

We provide training to our newly hired auditors because government auditing is very different than business auditing. Unfortunately, no universities specialize in this field, and even a single class is rare. We try to update and improve our training each time, but what we’ve learned in the last year about our instructional design approach has given us new insights.

First, what we taught was not specifically designed to help new performance auditors conduct their day-to-day work. We explained about preparing a good workpaper, or using our software, Teammate. But there were gaps in the framework and perspective of auditing. To address those gaps, we began to develop trainings that complemented our practical nuts and bolts offerings with courses focusing on soft skills development and leadership.

Auditing has a maze-like feeling for the new hire but we sometimes forget that there are two perspectives of a maze – the bird’s eye view, and the view from inside. From above, our eye can trace the shortest route. But inside, there are no direction markers.  Success must come by assembling a clear memory of the layout of the maze through arduous trial and error.

Inside Hampton Court Maze

Inside Hampton Court Maze

Some people enjoy the puzzle of a maze and pay to wander through corn mazes. Some have no patience for mazes. The image to the right makes some folks feel like a lab rat. And not a very smart one, since I hear that lab rats are quite capable of negotiating mazes.

But even as we think about this frustration, we also need to ’embrace the maze’. Every new audit is an agency maze. We try to understand how all the program and financial pieces fit together, and build up the picture through interviews, data analysis, and observation.

On the financial side we have workpapers and team member continuity to guide us through the maze of risks each year. On the performance side, we are often entering new agency mazes and our initial, scoping phase produces a rough drawing of the maze to help us narrow down to an issue that we should audit.

Above Hampton Court

Above Hampton Court

The other maze is our audit process.  We want newly hired auditors to be able to negotiate their way through the agency maze without also getting lost in the auditing maze. The better we can prepare new auditors in our methods and procedures, how to audit, the more they can focus on what they are auditing.

Our instructional design efforts are laying out the tasks more clearly for new hires with 11 classes so far. For example, we have training on audit documentation, interviewing, elements of a finding, sufficiency of evidence, and report writing. This knowledge and skill set are spread out, presented when a new hire will encounter them, then reinforced as they do their work.

Nonetheless, we recognize that many aspects of auditing are affected by circumstances and context that can’t be set in procedure. Strategies, priorities, principles, and standards provide general guidance in these situations. Leadership is also important and we want to help our newly promoted team leads understand the new responsibilities and situations they must take on, and perform them with competence and confidence.

We are developing that team leader training in our next phase, and some intermediate auditor training, and some technical training on specific software or analytical tools.

This seems like a lot of work but we are asking our auditors to develop and offer the classes, which involves us all in the responsibility of defining expectations, developing the content, and presenting the instruction.

We want to shorten the learning curves of our auditors because that translates into highly professional work from everybody. Staff members that have had the opportunity to take part in thorough and holistic training are a key element of a learning organization.

Auditors at Work Featured Financial Audit Performance Audit

Auditors at Work: Spotlight on Sandy Hilton

Once a quarter we will be discussing the wonders of the world of auditing with (you guessed it) actual auditors! Our Summer Spotlight fell on Sandra Hilton, Performance Audit Manager with the Oregon Audits Division, and in her free time, newly adopted Cat Assistant.

What should we call you?
Call me Sandy.

How long have you been an auditor?
I’ve been an auditor for 26 years. I now work primarily on the performance side, but over the course of my career about half of that time was spent working on financial audits and hotline related work.

What led you to auditing?
I wanted to get my CPA certificate! Two years of relevant experience was required. That was the original reason I applied for the job as a staff auditor; however, once I got into it I found that I really, really liked it. I’ve been here ever since.

Many people, upon hearing the word ‘auditing,’ may assume that the job is just deadly dull. Is auditing boring?
I am never bored. Auditing is exciting! There are so many different things to look at, and it’s important work that you get to take on with a whole team. Teamwork, among other things, makes what we do interesting. Of course, after a few rounds of reading the same audit report, it does have its dull moments…

Why is your job important?
Because we can make a difference. We push things along- changing the course of a program or agency by incremental degrees and effecting some substantial long term changes. Similar to altering the trajectory of a rocket, tiny degree shifts make huge differences down the line. Audits build upon previous audit work and are often interconnected. We help the agencies and people we work with know which ‘tweaks’ can make a meaningful difference to achieving their goals.

Any personal victory stories, or tales of challenges overcome and differences made?
The TANF Audit we released a few years ago issued a number of recommendations that DHS used to  overhaul an underperforming program. In following, the most recent Governor’s Budget added $30 million to redesign and improve the program. Some of that money went to investments in support services and smoothing the transition for families off the ‘welfare cliff.’

Any auditing horror stories?
Not exactly a horror story, but it was definitely amusing. Years ago I did a site visit where I saw a safe with an ‘Open’ sign hanging on it. Apparently the safe was difficult to open, so the agency in question just left it open and put a double-sided ‘open-closed’ sign on the front. The safe was full of blank checks, and even had a signature stamp machine. Combined with the lack of segregation of duties and easy access to the safe, it was definitely a ‘one-stop’ shopping opportunity!

What do you do when you’re not auditing?
My number one hobby is gardening, and I enjoy feeding birds as well. I also adopted a cat recently- well, he adopted me. I’ve outfitted him with a break-away collar and a bell to warn the birds that a cat is around. I really enjoy being outside and seeing the new plant growth. When I’m inside I play the piano, and I am very interested in tracking genealogy.

Cats or dogs, and why?
I always thought I was a dog person. I love the affection. Cats are more independent. My husband had a cat before we were married, so I guess you could say I married the cat, too. I’ve learned to respect cats, and I respect them on their terms. I grew to understand and love our first cat. This new cat has also wiggled his way into my heart.

Any words of advice for new auditors?
That’s a tough one! But here goes… Be open to learning. Don’t be close-minded to ideas. Cultivate an attitude of willingness to hear different perspectives- it will help you realize the richness of what we do. As people we tend to focus a lot on ‘what we like.’ You may get a lot out of hearing stuff from people that you may not agree with. Be willing to learn from anybody, no matter who they are. Also, cut each other some slack. You’ll never know everything.

Final thoughts?
I’m really proud of our work here. I want to leave the world a better place, and here you have the opportunity to have a positive impact. That drives my work. When you get a whole group of smart people passionate about making a difference together, you can make change happen.

Auditors at Work Featured