PR Maturity Metric

Definition

The PR Maturity metric is a measure of how refined and well-prepared a pull request (PR) is when it is submitted for review. It assesses the extent of changes made by the developer after the initial submission to determine how much work was required post-submission to address feedback, fix issues, or improve the PR.

Why Is This Metric Useful?

The PR Maturity Metric highlights how well-prepared a PR is before being submitted.

High Maturity: Indicates that the PR was well-prepared before submission, with minimal post-submission changes required. This often reflects good development practices, such as thorough testing, documentation, and adherence to coding standards. It’s worth noting however, that a high PR maturity may sometimes lead to a shallow review process. In such cases, reviewers might simply auto-approve the PR without suggesting any changes, which can potentially undermine the review's effectiveness.

Low Maturity: Suggests that significant revisions were needed after submission, possibly due to unclear requirements, rushed development, or inadequate self-review.

• Quality Indicator: It highlights the readiness of a PR, helping teams understand the quality of work at the time of submission.
• Efficient Reviews: Well-prepared PRs (high maturity) often lead to faster and smoother review cycles.
• Development Insights: Low maturity scores can signal the need for improved development workflows, better requirement gathering, or enhanced pre-review practices.

The PR Maturity metric provides actionable insights for improving both individual developer performance and team collaboration.

Calculation

To calculate the PR Maturity metric, we compare the state of the branch at two points in time:

1. At PR creation: The state of the branch when the PR was initially issued.
2. At PR closure: The state of the branch after additional changes have been made.

We use the diff command to measure the changes (additions and deletions) introduced after the PR was issued (Changes After PR Issued) and divide this value by the total size of the PR (Current PR Size).

Formula:

Definitions:

• Changes After PR Issued: The sum of additions and deletions made after the PR was published.
• Current PR Size: The total size of the PR, defined as the sum of all additions and deletions in the PR

Important Notes

• This metric is only calculated for PRs larger than 10 lines of code.
• The result is rounded up to two decimal places and constrained to a minimum value of 0.1 to avoid unrealistic results.

Example for Context

Consider a PR with the following characteristics:

• Total size (Current PR Size): 100 lines (additions + deletions).
• Changes made after PR was issued (Changes After PR Issued): 20 lines.

The PR Maturity Metric is calculated as: round(max(0.1, 1 - (20 / 100)), 2) = 0.8

Result: The PR Maturity Metric for this example is 80% (0.8).