Stephanie Evergreen Reblog: Qualitative chart chooser

The reality is, most people are never going to be excited to read your text heavy 50-page report with no visuals. This is where data visualization can come in handy. Visualization is a great tool to get people interested and engaged with your story. The problem is, many of the qualitative visualizations I see are reports with endless callout quotes or ugly charts that were spit out of data analysis software. We can do better than this.

Stephanie Evergreen and Jennifer Lyons explore the possibilities of charting tricky and complex qualitative data in this blog post. Read more here, and explore some of the chart options below.

qualitative-chooser-1_0-791x1024

Featured Noteworthy

Oregon Secretary of State Newsroom Reblog: User-friendly election night results website launched by Secretary of State

The Oregon Secretary of State in partnership with county elections officials has launched today a new centralized website that will make it easier to track and follow results on election night.

The website, which can be found at results.oregonvotes.gov, replaces the state election results website and includes individual election results pages for each county.

Read more at: User-friendly election night results website launched by Secretary of State — The Oregon Secretary of State Newsroom

Accountability and Media Featured Noteworthy

Department of Human Services: To Better Achieve its Mission, Vision, and Goals, DHS Must Increase Efforts to Address Employees’ Concerns

Executive Summary


The engagement level of employees can directly influence their ability to do their job and thrive professionally and personally. In April 2016, we conducted a survey of Department of Human Services (DHS) employees to help DHS management identify work environment factors positively or negatively affecting employee engagement.

Survey respondents generally reported they know the agency’s mission vision, and goals and are proud to work there. But their responses also highlighted areas within DHS that need improvement. These included tools and resources to accomplish the work, compensation, hiring practices, recognition, professional development, stress and workload distribution, and communication. Addressing these issues will help DHS improve employee engagement and better achieve the agency’s mission, vision and values.

Read the full report here.

Overview of DHS

The Department of Human Services’ (DHS) mission is to help Oregonians in their own communities achieve safety, well-being, and independence through services that protect, empower, respect choice, and preserve dignity. The agency’s biennial budget is about $10 billion with 7,897 full time equivalent staff.

The agency serves over a million Oregonians each year through two support services units and five program areas. The five programs provide services through numerous field and local offices throughout the state. Central Services, which includes the Director’s office, and Shared Services, provide support and leadership to the following programs: Aging and People with Disabilities, Child Welfare, Intellectual/Developmental Disabilities, Self-Sufficiency, and Vocational Rehabilitation.

Employee engagement is important

Engaged employees are passionate, energetic, and dedicated to their job and organization. One study indicates that a higher level of employee engagement correlates with higher rates of success in achieving strategic goals, higher employee retention, and fewer days of sick leave and lost time.

Work environment surveys can help an organization measure its level of employee engagement. DHS has been conducting an employee survey since 2012 that consists of seven questions designed to measure employee engagement.

Our survey was designed to measure the factors that influence employee engagement. DHS management could use the results of our survey to identify areas to improve, and set priorities for action.

Core knowledge and respectful work units given high ratings among respondents

Survey results indicate that DHS is doing well in four areas that influence engagement: mission, vision, goals; job suitability; respectful work units and reporting of harassing and discriminating behavior; and teamwork.

Nearly all respondents reported they knew the mission, vision, and goals of the agency; and how their work relates to these goals. Furthermore, over 85% of respondents reported they are proud to work at DHS. Almost all of the respondents reported they found their work to be meaningful.

Survey highlights concerns DHS management should address

DHS management should address perceived deficiencies that influence employee engagement. We surveyed 7,426 DHS employees and received 4,580 completed surveys, resulting in a 62% response rate. Employees rated their level of agreement with survey questions regarding factors that influence employee engagement. The response benchmarks we used were based on the existing DHS metrics, which are as follows: 85% and above means the respondent perceives DHS as doing well for that factor; between 66% – 84% means a factor that needs some improvement; and 65% and below means a factor that is in critical need of management attention.

Survey respondents identified seven factors in need of improvement – tools and resources, compensation, hiring practices, recognition, stress and workload, professional development, and communication.

Only 55% of respondents felt they had sufficient tools and resources to do their job. At least 50% of respondents across two units and five programs reported a high level of stress. Many respondents reported concerns about the fairness and competitiveness of hiring practices, and a lack of recognition for the work they do.

Another key factor related to employee engagement and organizational success is communication. For an agency as large as DHS, with offices all over the state, communication can be particularly challenging. However, according to a Newfoundland and Labrador Statistics Agency’s work environment survey, direct and timely communication from senior leaders can go a long way in making employees feel informed and connected.

Leaders also need honest feedback from employees who provide services to clients, in order to help them make the best decisions. Overall, less than half of the respondents felt that communication and information flows effectively between the central office and the field offices.

Recommendations

To better achieve its mission, vision, and goals, we recommend DHS management develop and implement a plan to address the seven areas needing improvement: tools and resources, compensation, hiring practices, recognition, professional development, stress and workload, and communication.

To gauge whether efforts are improving engagement, we recommend DHS management administer a work environment survey at least annually that includes the factors we identified that influence engagement.

Last, we recommend management use the future survey results to revise the plan, as needed.

Agency Response

The agency generally agreed with our recommendations. The full agency response can be found at the end of the report.

Featured New Audit Release Noteworthy

Oregon Department of Education: Clearer Communication, Consistent Use of Results and an Ongoing Commitment to Improvement Could Help Address Testing Concerns

Executive Summary


Our audit responds to House Bill 2713 (2015), developed with input from the State Auditor. It called for a Secretary of State audit to review the impacts of the statewide summative assessment on Oregon’s public schools, and make recommendations for improvement.

Through a series of surveys, site visits and interviews, we learned many schools faced challenges in the first year of administering the new Smarter Balanced test, including adjusting to the demands on staff and school resources. Some reported fewer challenges in the second year.

Some educators are concerned that certain student populations may experience more negative impacts than others. Some also told us that a more comprehensive assessment system would be useful.

Read full report here.

Oregon introduced Smarter Balanced in 2015

The Smarter Balanced assessment is a new test introduced by the Oregon Department of Education to all public schools in the spring of 2015. Smarter Balanced tests 3rd – 8th graders and 11th graders in math and English language arts near the end of the school year. The test assesses students’ progress toward meeting Oregon’s college- and career-ready standards, the Common Core State Standards. Smarter Balanced requires more time and depth of knowledge than the previous test.

There is not clear agreement on the test’s purpose

The Smarter Balanced test is intended to provide a measure for accountability, data to identify achievement gaps, and information about whether students meet standards overall, and many value these purposes. We also heard from educators who feel the test should be more useful in the classroom. However, other tools may be better suited for that purpose. The Oregon Department of Education could take a more active role in communicating about the test’s purpose.

The results of the test are not used consistently

Schools, school districts and the state use Smarter Balanced test results inconsistently, and sometimes not at all. Educators told us that it would be easier to use results if they received them sooner. Many reported that additional guidance on how to use results would be helpful. Some also reported that a more comprehensive assessment system would be useful.

Many reported test administration challenges

Educators described schoolwide challenges in the first year of administering Smarter Balanced. Testing did not just affect the classrooms that were actively testing, but could also place additional staffing and resource demands on the entire school. However, some said there were fewer challenges in the second year.

Testing took away from other duties of school and school district personnel. Some schools hired additional staff or substitutes specifically for testing. Testing also tied up computer labs for months at some schools. Time spent taking and preparing for the test took away from instruction time.

Some student populations may experience more negative impacts than others

Standardized testing may affect certain student groups more than others. Despite having accommodations, we heard concerns that the test’s greater use of technology and language may increase the risk that some students will not be able to demonstrate their abilities accurately. Students who take longer to complete the assessment may miss more instruction time.

Students in special education, English Language Learners, and students with less exposure to technology and typing may be particularly affected.

Recommendations

We recommend that the Oregon Department of Education improve communication, foster consistent use of results and continue its commitment to improve test administration. Our specific recommendations can be found on page 18 of the report.

Agency Response

The full agency response can be found at the end of the report.

Featured New Audit Release Noteworthy

Oregon Department of Forestry: Actions Needed to Address Strain on Workforce and Programs from Wildfires

Executive Summary


Three consecutive severe fire seasons have forced the Oregon Department of Forestry (ODF) to spend more time fighting fires and less time on its other programs. Recent fires have also strained ODF personnel, who often work long hours away from home.

ODF needs to take action to reduce these impacts on personnel and programs. Systematic, long-term workforce planning that takes into account resources needed for both fire and non-fire programs; development of a more effective business improvement process; better evaluation of wildfire prevention and detection measures; and increased mitigation efforts are steps ODF should take to help address current and future challenges.

Read full report here

ODF needs to analyze and clearly communicate full impacts of wildfires on the agency

Photo by Oregon Department of Forestry, CC BY

Since 2013, intense fire seasons have resulted in ODF staff spending more time on fire assignments. However, ODF does not currently collect, analyze and communicate to the Legislature and its stakeholders the full impacts of fires on its programs and personnel. This information is necessary for ODF to adequately plan and manage its workforce to meet existing and future demands.

Not only are more employees participating in fire related assignments, but these employees are working much longer hours. Overtime hours spent on fire protection by permanent employees have increased by 197% in recent years.

While the wildfire suppression workload has increased, staffing has not kept pace. ODF is fighting more severe fires with about the same full-time equivalent employees it had nearly 20 years ago. Fires have also created more administrative work, including preparing claims for cost reimbursement from the Federal Emergency Management Agency and other federal agencies, and making catastrophic wildfire insurance claims and emergency funding requests. To date several of these claims have not been completely processed, which resulted in ODF borrowing to finance these fire related costs. During the past three fire seasons, ODF paid $1.5 million in interest on this borrowing.

ODF staff in Salem and field offices are feeling overworked and are experiencing stress and fatigue as a result of fire related work. Despite the strain of consecutive severe fire seasons, ODF Management reports that employees remain committed to participating in the agency’s firefighting efforts. However, as staff devote more time to wildfire seasons, employees and agency leadership have expressed concerns about ODF’s ability to continue performing at current service levels.

Recent fires have caused delays in work for ODF’s non-fire programs, as employees in these programs are deployed to fire incidents. Examples include delays in developing annual operations plans for state forests, completing Forest Practices Act compliance reporting, and updating bald eagle protection rules. Fires have also created more work for employees in these programs after fires are controlled, such as salvage logging operations and developing and implementing reforestation plans.

Non-fire program contributions to fire response capacity are not fully known

ODF2

Photo by Oregon Department of Forestry, CC BY

Non-fire programs contribute to ODF’s firefighting and to maintaining fire readiness. But ODF is not tracking the contributions these programs make, which are absorbed into their respective budgets. While we identified some of these contributions, ODF needs a full accounting of the contributions and related costs to adequately plan for both fire and non-fire work.

Non-fire programs contribute staff hours to fight fires and to Incident Management command and support teams. For example during the last three fire seasons, the average number of hours State Forests Program staff billed to fire protection doubled to 19,038 hours.
These programs also pay other fire-related expenses such as the cost of specialized fire qualification training, and certain fire equipment and supplies. ODF needs better information on these costs and how changes to staffing, funding and workload in these non-fire programs affect fire operation capacity.

Agency wide workforce planning needed

ODF needs a systematic workforce planning process to effectively address current and emerging challenges to its programs and workforce. Workforce analysis is needed to identify gaps and to monitor, evaluate, and revise resources in order to meet the agency’s strategic goals now and in the future. ODF has completed some analysis, but more is needed, and it should include all necessary firefighting resources.

At ODF, workforce planning is complicated by staff who have program duties and firefighting responsibilities; long training times for fire duties; and the need to meet multiple program missions, including responding to wildfires. But complete workforce analysis and planning, that takes these factors into account, can help ODF ensure it sustainably meets both its fire and non-fire responsibilities.

Systematic process for business improvements needed

As fires have increased in recent years, so have the complexity and number of financial transactions associated with suppressing those fires. Today, ODF has systems in place to collect and assess process improvement suggestions, but some are fragmented and incomplete. We found that sometimes suggestions were not fully reviewed and/or implemented, and decisions were not made or communicated.

A better system that fully reviews, implements and communicates decisions made could help ODF address its increased workload by reducing unnecessary costs and inefficiencies. It could also help to improve the alignment between existing resources and program objectives and priorities.

Evaluation of prevention and detection efforts can be improved

ODF takes some proactive steps to prevent and detect wildfires, but the agency does not systematically evaluate the costs and relative effectiveness of different strategies. Evaluating these strategies could help ODF focus its resources on the most cost-effective strategies to keep suppression costs and wildfire damages low. Better information about the money and staff time spent on different prevention and detection activities and fire causes could aid this evaluation.

More work needed to mitigate wildfire risks and target strategies

ODF3

Fuel Reduction. Photo by Oregon State University, CC BY-SA

ODF, private landowners, and federal agencies work to reduce wildfire risks posed by the accumulation of small trees and vegetation in forests resulting from decades of fire suppression, land use changes, past land management practices, and other factors. Despite these efforts, there are millions of acres of land in Oregon in high wildfire risk areas across all ownerships, including federal, state and private lands. Some of this land is highly important to our water supply. The resources currently dedicated to mitigation work are unlikely to meet this challenge. To reduce wildfire risks, some of these areas may need treatment through methods such as prescribed burning, thinning, or removal of forest underbrush in forests and around homes.

Recommendations

This audit recommends ways ODF can build on its current efforts and accomplishments, and make improvements to address current and future challenges. Our detailed recommendations for ODF management are included on Page 31. They include recommendations for collecting and analyzing better information on fire impacts and costs, developing a systematic, future oriented workforce planning process, and enhancing the agency’s business improvement process. We also recommend actions for ODF to improve wildfire prevention, detection, and mitigation efforts.

Agency Response

The agency agrees with the report findings and recommendations. The full agency response can be found at the end of this report.

Featured New Audit Release Noteworthy Performance Audit

Oregon SoS Newsroom Reblog: Oregon Motor Voter Registers an Average of 735 Voters a Day

The August data report for the Oregon Motor Voter program shows over 220,000 new voter records added to the rolls since the program took effect on January 1, 2016.

Read more at: Oregon Motor Voter Registers an Average of 735 Voters a Day — The Oregon Secretary of State Newsroom

Accountability and Media Featured Noteworthy