Review Time Metric

Definition

The Review Time Metric measures the time spent reviewing a pull request (PR) until approval or merging. This calculation only considers reviews that occur after the PR is moved out of draft status and is ready for review. For PRs merged without a review, no review time is calculated.

Why is This Metric Useful?

• This metric highlights the efficiency and thoroughness of the review process, helping teams balance speed and code quality. It provides actionable insights to streamline reviews and avoid bottlenecks.

How to Use it?

• Monitor review timeliness while ensuring evaluations remain thorough.
• Identify opportunities to improve review workflows without compromising the quality of code assessments.

Examples for Context

• Teams that reduced Review Time improved delivery velocity while maintaining high code quality.
• Faster reviews help avoid bottlenecks and accelerate the development pipeline.

Data Sources

• PR creation and draft transitions.
• Reviewer activity, including first comments or approvals.

Calculation

• For PRs Merged With a Review:

The metric is calculated as:

Review Time = PR merged at - PR first reviewed at

This measures the duration from the first reviewer comment or approval to the merging of the PR.

• For PRs Merged Without a Review:

No review time is calculated, as no review activity occurred.

This ensures the metric reflects only the time spent on actual reviews.

Tunable Configurations

• Option to include iterative reviews, such as additional comments or requested changes, for a more detailed analysis of review times.

Benchmarking Guidance

• Review Time should ideally be under 2 days for standard PRs.

Error Margins and Limitations

• Lengthy or complex PRs can skew the average review times. To address this, break down large PRs into smaller, more manageable chunks.

Stakeholder Use Cases

• Developers: Optimize PR size to ensure faster and more efficient reviews.
• Reviewers: Monitor personal response times to maintain a balance between speed and thoroughness.