ParliMeter’s newest feature reveals: About two-thirds of all parliamentary budget recommendations have gone unresolved across seven years

Have you ever wondered what happens to parliamentary recommendations on departmental budgets, or what happens when they are not actioned?

Each year, every parliamentary portfolio committee in South Africa releases a Budgetary Review and Recommendations Report (BRRR) for the department and entities it oversees, producing one report for each department across roughly 30 portfolio committees and one standing committee.

BRRRs are one of the most concrete instruments for accountability that Parliament has. They aren't speeches or press releases. They are written, public, on-the-record “asks” made by Members of Parliament to specific departments  –  with deadlines attached. If a department ignores these requests, that is itself a fact that can be tracked. Tracking BRRRs across years, committees or departments can produce critical insights on departmental activity and progress.

If a department ignores these requests, that is itself a fact that can be tracked.

But here’s the catch: BRRRs are released as long, dense PDFs that often use different formatting conventions. This leaves citizens, journalists or even other parliamentary committees with no easy way to read or analyse a full set of BRRRs.

Introducing ParliMeter’s new BRRR dashboard

This is the gap that ParliMeter’s new BRRR dashboard aims to fill. ParliMeter is a parliamentary oversight dashboard, aimed at improving public access to governance processes and strengthening oversight mechanisms, and developed by the Organisation Undoing Tax Abuse (OUTA), OpenUp and the Parliamentary Monitoring Group (PMG).

ParliMeter’s existing attendance and bill-tracking dashboards already give South Africans insight into parliamentary activity, but the team wanted to expand the project’s scope to provide insight into actual follow-through on decisions and recommendations made in Parliament.

“OUTA’s Parliamentary Oversight Reports include the observation of the BRRR recommendations each year, but the process is time-consuming because the information is spread across many lengthy reports,” says Naailah Parbhoo from OUTA. “Hence, the dashboard was developed to make these recommendations easier to search, compare and use for research, oversight and public awareness.”

...the dashboard was developed to make these recommendations easier to search, compare and use for research, oversight and public awareness.

The BRRR dashboard draws on the full set of publicly available BRRRs tabled by South Africa's National Assembly portfolio committees from 2019 to 2025, starting with the Sixth Parliament. It breaks down data from parliamentary PDFs, including over 5,800 recommendations made across roughly 30 portfolio committees and one standing committee, into easily understandable visualisations and statistics.

“Parliament makes thousands of asks of the executive every year,” says Carl Teffo, OpenUp Research and Data Lead. “Almost none of those asks have been systematically tracked over time, even though tracking them is exactly the kind of accountability work civil society needs.”

....tracking [BRRR recommendations] is exactly the kind of accountability work civil society needs.

Additionally, the dashboard provides two layers of analysis that does not exist in the source PDFs, namely:

• Entities: The departments, agencies, pieces of legislation, programmes, and named officials mentioned in each recommendation, which makes filtering possible.
• Threads: Groupings of recommendations that are substantively the same “ask” repeated across years. A thread might span a single year or run unbroken across all seven years, indicating an “ask” that has never been resolved.

How we tackled the data

With such a vast tranche of recommendations scattered across many differently structured PDFs, cleaning and sorting the data to prepare it for the dashboard is no small feat.

At first, Naailah Parbhoo from OUTA spent months reading through many of the BRRR PDFs and extracting the data manually. She worked to identify recommendations and mentioned entities, and organised the data in Excel spreadsheets. Her work was integral in defining how the process would work going forward, specifically with regards to what we would define as a “recommendation” in the first place.

“One major challenge was tracking recommendations across portfolio committees that were renamed, split or removed during the Sixth Parliament,” says Parbhoo. “Each recommendation had to be checked carefully to ensure it was assigned to the correct committee and matched accurately. Carl from OpenUp also double-checked the quality of the metadata to ensure correctness.”

Each recommendation had to be checked carefully to ensure it was assigned to the correct committee and matched accurately.

From there, we were able to scale the extraction process using a large language model (LLM), which allowed us to complete the process within a more expedient timeframe.

“Two years ago, extracting structured recommendations out of inconsistent PDFs and matching them across years would have required a small team of human coders working for months,” says Teffo. “Today, [...] it's a project a small civic-tech team can take on. We wanted to put that capability to work in the public interest.”

Of course, it isn’t as simple as feeding the PDFs into Claude and copy-pasting the results. Teffo implemented careful prompt design, informed by the foundational work done by Parbhoo, as well as specific validation scaffolding and continuous human oversight to ensure the accuracy and reliability of the LLM’s output.

LLMs allow us to do public interest work at a scale that wasn’t possible before, but in no way does it eliminate the necessity of real, human input, preparation and design behind the scenes. For more insight into this process, check out Teffo’s article on the methodology behind the new BRRR dashboard.

...in no way [do large language models] eliminate the necessity of real, human input, preparation and design behind the scenes.

What can we learn from the BRRR dashboard?

At a glance, the BRRR dashboard reveals a key finding: Roughly 65% or two-thirds of all recommendations are part of an ongoing thread, having gone unresolved for years on end. 

The dashboard’s capabilities to filter and track recommendations across years or departments are crucial in gaining a big-picture view of departmental progress and follow-through on specific budgetary recommendations. Curious citizens, journalists on a deadline, civil society organisations, political researchers or even Members of Parliament themselves can all benefit from the easy-to-use BRRR dashboard.

Curious citizens, journalists on a deadline, civil society organisations, political researchers or even Members of Parliament themselves can all benefit from the easy-to-use BRRR dashboard.

Concretely, the dashboard is built to answer questions like:

• Which recommendations have been repeated for the most years without resolution?
• Which committees have the highest share of recurring “asks”?
• What recommendations does committee X keep making to department Y?
• Are recommendations carried across when committees get restructured (for example, split, merged or renamed)?
• What was new in the most recent BRRR cycle and what is worth following up on?

Explore the BRRR dashboard on ParliMeter’s website today for unique insights into the work that your parliamentary representatives are doing. You can also access the original, full PDF reports on PMG’s website.

‍The BRRR dashboard’s data is continuously being refined and updated. We welcome all user feedback.