Measuring and Justifying the Government Experience

Private sector organizations use revenue as the primary measurement to justify improving experiences. Many government services don’t have revenue as a lever, so how can we justify work to improve experience? We have leveraged a few metrics on our team at the Office of Natural Resources Revenue (ONRR). They fall into three categories: laws and regulations, improving the experience for users, and saving time for employees.

Laws and regulations

Laws and regulations are one of the most effective means to justify our work. Some people have an easy time getting behind laws because they are mandates. That said, there is often a lack of clarity when it comes to the details of implementing laws. Below, we’ve outlined a few of the regulations we’ve used to justify our work. You can also view Digital.gov’s full list of customer experience laws dating back to 1993.

21st Century Integrated Digital Experience Act (IDEA)

The 21st Century IDEA aims to improve the digital experience for government customers. It reinforces existing requirements for federal websites and requires agencies to:

• “Modernize their websites,
• Digitize services and forms,
• Accelerate use of e-signatures,
• Improve customer experience, and
• Standardize and transition to centralized shared services”.

We’ve used this Act to justify our work. We’ve primarily used the requirements related to modernizing websites, accessibility, and customer experience. We also participated on a Department of the Interior (DOI) task force to define how the Act will be implemented throughout the department. Here is the DOI IDEA Implementation Guidance.

Open, Public, Electronic, and Necessary (OPEN) Government Data Act

Then OPEN Government Data Act requires data to:

• Be published in machine-readable format
• Use open formats under open licenses
• Be user-centered and tracked using analytics and metrics

We’ve used this Act to justify the existence of the Natural Resources Revenue Data (NRRD) website. That site provides open data to the public. On both of our websites, we work in the open and use open-source software whenever possible. We also practice user-centered design and track our work using analytics and metrics.

President’s Management Agenda (PMA) Vision

Each administration produces a President’s Management Agenda (PMA). The past few administrations have had agendas supporting customer experience. We cite the current administration’s agenda when it supports our work. The current administration’s Priority 2: Delivering Excellent, Equitable, and Secure Federal Services and Customer Experience supports customer experience.

“Every interaction between the Government and the public is an opportunity to deliver the value and competency Americans expect and deserve.

People are at the center of everything the Government does. In their daily lives as well as in critical moments of need, people rely on Federal services to help support them through disasters, advance their businesses, provide opportunities for their families, safeguard their rights, and aid them in rebuilding their communities. That is why the Federal Government must center its services around those who use them–delivering simple, secure, effective, equitable, and responsive solutions for all who the Government serves. When individuals and organizations interact with any part of the Federal Government, they want that interaction to work seamlessly.”

“This focus on customer experience will not only improve the delivery, efficiency, security, and effectiveness of our Government programs, it will advance equity and enhance everyday interactions with public services and uplift the lives of those who need it the most.”

Executive Order 14058: Customer Experience

Out of the PMA vision, the current administration enacted a customer experience executive order. This primarily applies to 35 High-Impact Service Providers, but we still cite it as direction from the administration. The White House put out a fact sheet that sums up the purpose of the order well:

“The Government’s primary mission is to serve. By placing people at the center of everything we do, the Government will be able to deliver timely, modern, and secure services to you – the people. We will rebuild trust in our Government, ensure no one is left behind, and inspire others to join us in serving future generations of Americans.”

“As part of this Executive Order, agencies commit to putting their customers at the center of everything they do. These actions include modernizing programs, reducing administrative burdens, and piloting new online tools and technologies that can provide a simple, seamless, and secure customer experience.”

Section 508 accessibility

We cite Section 508 compliance whenever we do accessibility work. It’s an old law, but many agencies still aren’t in compliance. There have been several updates made to it over the years. It applies to all public-facing federal websites and documents. It also applies to agency official internal-facing content.

“Section 508 of the Rehabilitation Act of 1973 requires Federal agencies to make their electronic and information technology (EIT) accessible to people with disabilities. The law (29 U.S.C § 794 (d)) applies to all Federal agencies when they develop, procure, maintain, or use electronic and information technology. Under Section 508, agencies must give disabled employees and members of the public access to information comparable to the access available to others.”

Extractive Industries Transparency Initiative Withdrawal Letter

ONRR created the Natural Resources Revenue Data website as part of the Extractives Industries Transparency Initiative (EITI). The United States withdrew from the EITI in November of 2017. At that point, ONRR issued a withdrawal letter committing to continue data publication in the spirit of the EITI. We cite that letter to justify the existence of the NRRD website.

“The Office of Natural Resources Revenue (ONRR), which maintains the primary role in the U.S. Government for the collection and disbursement of revenue related to energy and non-energy mineral resources, remains fully committed to institutionalizing the EITI principles of transparency and accountability consistent with U.S. law. ONRR intends to mainstream government reporting of energy production and the associated revenue collection and disbursement. ONRR is also committed to continue its efforts to promote public awareness and engage stakeholders in a public conversation of the potential impacts of proposed policies and regulations related to revenue collection from such development. We will continue to unilaterally disclose revenue payments received for extractive operations on federal land through our open data portal, and we will continue to improve our reporting through the inclusion of additional states and tribes.”

Improving the experience for users

We try to improve the experience for users, as much as we can. We know any improvements for users will save them time. Improved experience also increases the goodwill users have for ONRR and the federal government.

Analytics

Analytics tell you what users are doing out in the wild. They’re good at quantifying the effect of changes and helping to prioritize work. We track analytics for NRRD, onrr.gov, and our blog. We look at page views and document downloads to tell us whether a change has increased or decreased traffic to a given page. Sometimes it’s better to increase views, like if we want more people accessing open data. Sometimes it’s better to reduce views. An example of this is when we made the handbooks on onrr.gov easier to search. The change reduced the number of chapter documents the user opens before finding the desired content.

We also use page views and document download numbers to prioritize work. When we wanted to make all the documents on onrr.gov accessible, we looked at the number of downloads for each document. We looked at the numbers over the course of a year and prioritized those with the most downloads. We were able to cover 70% of the downloads by updating just 100 of the 5,000 documents on the site. This knowledge made the task seem manageable.

We could also look at search terms to see if there’s information that people are looking for and can’t find. We don’t have a lot of search volume on our websites, but I know a lot of other agencies use this metric.

We use the blog analytics to see what content is popular and help guide our blog topics. We also look at referring pages to see how people discover blog posts.

User research

User research helps understand why problems are occurring. You can take high-volume areas identified in analytics and understand qualitative details through user research. Any time you can site a problem you’ve seen by observing users, it comes off as a powerful story. And if you can attach a number of times you’ve seen it happen, even better.

Saving time for employees

Saving time for employees equates to dollars and improves employee satisfaction. Below are some ways we help ONRR employees save time.

Decreasing the number of Freedom of Information Act (FOIA) requests and other data requests

Reducing Freedom of Information Act (FOIA) requests or the amount of time it takes to respond is an effective measurement. It saves time for both employees and the public. One of our team’s goals is to reduce the number of FOIA requests for data that could be public. To help measure progress towards this goal, our Data Optimization, Retrieval, and Coordination team is adding a checkbox to their data request tracking tool. It will allow them to indicate which data requests they field could be made publicly available. This will help us identify high-volume requests we can reduce by adding the data to the Natural Resources Revenue Data website.

Reducing the frequency that ONRR employees are contacted for help

Reducing contact volume is an effective way to show that improving the experience saves money. However, contacts aren’t often well tracked. We just designed a troubleshooting guide that we hope reduces contact volume. But none of the teams that get contacted by industry are currently tracking the numbers of contacts they receive. Those teams have said that it’s on their wish list for a new Customer Relationship Management system in development.

Reducing time to update the website

All of our data uploads used to be a manual process that involved several analysts manually creating our downloaded spreadsheets from raw data and repeatedly manually checking our data throughout the publication process. We’ve created a database that automatically formats and generates the download files from the raw files. We still thoroughly spend time checking the automated generated files for data accuracy against the raw files but reduced time spent to manually create download files by around 75%. What used to take a full day and lots of back and forth between analysts now only takes a few hours.

Executive champions

If you’re lucky, you’ll have executive support for some initiatives. Here are some examples where we’ve had support.

Section 508 compliance

ONRR’s Director took a course about Section 508 compliance and mandated compliance for all documents on onrr.gov. This mandate got all document owners on board with getting trained to make documents accessible. Our team still had to come up with a plan for tackling the 5,000 documents, but it definitely helped to have support coming from the top of the agency.

Continuing the spirit of the EITI

After we withdrew from the EITI we had the support of ONRR’s Director and Program Manager at that period of time. They allowed us to continue the Natural Resources Revenue Data website because they believed in continuing in the spirit of EITI. They were passionate about continuing to make the data public and designing the site to meet user needs.

Query tool

We also had executive support for another project in a more roundabout way. We’d heard from a number of internal stakeholders that they wanted the NRRD site to just do what an old “stats” site did. That site allowed users to filter and download the data. We also heard from users that they wanted to slice and dice the data. We had it on our roadmap, but didn’t have resources to get to making that change.

Then a political appointee came in and said they wanted that functionality. We used this request to increase our development capacity. The appointee adjusted our program’s budget to be able to hire contract developers to make the request happen. It was nice that that demand aligned with what we were hearing from users, so we could use it to improve the site. We were then able to use the success of that implementation to keep the developers longer and make more improvements to the site.

Conclusion

You have to get creative to justify improving experiences without monetization as a lever, but there are ways to do it. Our first line of defense is our legal obligation. After that, we rely on saving time and improving efficiency for both users and employees as selling points. It also helps when we get support from our executives who can dictate the direction in favor of users. At the end of the day, customer feedback is the most compelling measurement of success. Showing stakeholders how customer struggle is a great motivator for change. On the flip side, sharing positive stories from customers after making improvements demonstrates the value of the work and creates buy-in for more change.