Disclaimer: Just like @zachxbt, I'm one of the projects making impact for OP full time.
Risking my vote here :)
We all recognize that retroPGF is an ongoing experiment designed to improve over time. Nevertheless, we can see room to improve here. (I hope this article may useful for next retroPGF)
We currently have 643 projects and 146 badgeholders selected through a variety of methods.
Similar to most members, I hold a deep belief in and respect for badgeholders. Reviewing projects is undeniably challenging.
The primary issue I aim to enhance lies within the PROCESS, as these improvements could prove invaluable for retroPGF4 and subsequent iterations.
Unlike the previous screening period where all projects were assigned 5 badgeholders, Optimism might fail to reward some impactful projects that passed the screening round, leading to builders/contributors leaving the OP ecosystem.
This time we have a new system in place. It's a step in the right direction, but there's still room for improvement in the following new idea:
Lists
Ballots
List
Like all inventions, list has pros and cons.
While we have list, there is no strategy nor incentive to get all projects checked. Badgeholders might spend intensive time but some undiscovered projects might not get any view.
While it sounds good, if all 146 badgeholders make list, we likely have more than 146 lists. But in reality there aren’t many lists. And if majority of badgeholders rely on other badgeholder lists. It means more projects get less attention.
Ballots - The major problem
In order to be eligible, projects need 17 ballots from badgeholders. This is likely the major flaw here.
Origin of the number 17 has no explaination. Many badgeholders and contributors have raise concern in Round3 design forum since before start.
Many projects trying to get their voice heard. Here are some notable ones.
Numbers aren't looking good.
While 11% of badgeholders doesn't sound big. Reality is not all of them interested in your category.
If your category, has 40 reviewers. You need 17/40 = 42.5% threshold.
If your category, has 30 reviewers. You need 17/40 = 56.66% threshold.
This becomes a problem when projects have confidence in our work but aren't certain whether they will receive reviews from 20 people.
At the end of the day, all badgeholders are working really hard and should earn respect.
But some undiscovered projects likely not get enough review. And it will ruin OP.
At this rate, not only will projects be disappointed.
But badgeholders, who invest their time and effort in reviewing undiscovered projects, might struggle to reach their goal of supporting small impactful contributors/builders.
Some improvement options?
Some screenshots above are also good options.
And here another one. Learned from retroPGF3.
We can't be so sure how many badgeholders are tech orient. How many are education orient.
We can set proper criteria (e.g. 25% threshold) and let badgeholders select their expertise.
For example
50 badgeholders wanna review 165 OP Stacks.
40 badgeholders wanna review 472 End user UX.
We get this.
In this example, non-tech badgeholders work is much harder. As we can see that tech projects, despite most OP end users doesn't know them, get ballots fast
Badgeholders in each category might divide their work and cross-check with other badgeholders in the same categories.
Considering the reviewer-to-project ratio, we can take prompt action and plan ahead before any issues arise.
This approach ensures that all projects receive reviews.
We can adjust the number of badgeholders in proportion to the number of project applications.
Projects eligible for multiple categories should submit work for each part separately.
RetroPGF should allocate OP for each category before commencement.
Possibly allowing badgeholders to vote for percentage allocations.
Summary
As an engaged project contributor within OP, I believe in its continual improvement. Currently, with 643 projects and 146 badgeholders, reviewing projects presents notable challenges.
While no plan to iterate all projects, participants need to get 17 ballots.
To enhance the process, I propose refining the current screening system. To improve, establishing a clear strategy for list creation, ensuring all projects receive attention, and revisiting the 17-ballot requirement by considering category-specific variations could lead to fairer and more comprehensive project assessments.
Moreover, exploring category-specific expertise percentages, adjusting reviewer numbers accordingly, and establishing preset OP allocations per category might significantly enhance the evaluation process.
And, as always, Stay Optimistic! 🔴
I also wrote analysis about badgeholders distribution. Check it out :)
Enjoy this? follow for more at @gadgeteerth @0x_nanobro
