Microsoft is using AI to accelerate pull requests — the feature that lets developers tell others about changes they’ve made to code — toward completion by reminding authors to engage with their overdue requests. That’s according to a new whitepaper published this week detailing Nudge, an end-to-end service that leverages models based on effort estimation to predict the completion time for a given pull request. Microsoft says that Nudge has been deployed on 147 of its internal repositories since 2019 and that it’s “significantly” reduced completion time for 60% for the 8,500 pull requests for which it sent notifications.
With the adoption of platforms like GitHub and GitLab, pull requests have become the standard mechanism for distributed code reviews. They enable changes to be reviewed by one or more developers and even bots; once the reviewers have signed off, the changes can be merged with the main branch and deployed. But while pull requests streamline the review process, they can also slow down the software development process. For instance, if reviewers become overloaded and lose track of the request, it might not be reviewed in a timely manner.
In search of a solution, researchers at Microsoft and Dell developed Nudge, which uses effort estimation and activity detection algorithms to predict the completion time for a given pull request. Nudge also goes one step further, analyzing the request state to determine the pull request author or reviewers holding up the request from completion and notifying them via comments in the pull request environment.
To build Nudge, the researchers performed a correlation analysis to analyze various factors related to pull requests, pull request authors, pull request reviewers, and even temporal factors that impact request completion time. They then built the effort estimation model to predict the pull request completion time at the time of request creation, drawing on metrics like code churn, reviewer information, and ownership information. Next, they implemented activity detection to monitor any updates on the pull request (for example, new commits and review comments) and adjust the notifications accordingly. Lastly, to make the Nudge notifications actionable, the researchers deployed what they call dependency determination, which infers the pull request author or a specific reviewer.
After deploying Nudge for 18 months on repositories in Microsoft, the researchers report that 73% of the pull request notifications were “positively acknowledged” by the users. “Pull request are a key part of the collaborative software development process,” they wrote. “Only 1,570 pull requests out of 8,500 pull requests (18.47%) have taken more than a week to close. 81.53% of the pull requests are closed with in a week. An important observation to make is 2,300 pull requests (27.05%) of the pull requests on which Nudge sent the notification were completed within a day. This distribution indicates that majority of the pull requests on which Nudge sends notifications are completed relatively quickly.”
Nudge builds on Microsoft’s earlier efforts to inject the DevOps process with AI and machine learning. For instance, in April, the company detailed a system that it claims correctly distinguishes between security and non-security software bugs 99% of the time, and that accurately identifies critical, high-priority security bugs on average 97% of the time. Microsoft says it’s using AI to help triage cloud service outages.
