Ok, ballers, picture this: Imagine you played lights out in your dream-come-true match and beat top-rated Ben Johns 11-0, 11-0 in singles. How much would your Dynamic Universal Player Rating (DUPR) increase?

Unless you are a top pro, the answer is that your DUPR would not change at all.

The reason: DUPR ignores any singles match in which one player’s DUPR is at least 0.625 points higher than another. In doubles, DUPR ignores matches in which the average DUPR score of one team is at least 0.625 points higher than the average DUPR score of the other team.

Since Johns’ DUPR in singles is 7.19, a match between Johns and any player whose DUPR is less than 6.57 would be excluded by the DUPR algorithm.

The net effect is (a) to make it harder for underdogs to boost their DUPR ratings and (b) to protect the DUPR scores of higher-rated players when they play lower-rated players.

Dinkheads first learned of this loophole during an online Webinar hosted by DUPR on August 2. The Webinar — attended by DUPR CEO Jill Braverman and Data Scientist Mike Forzley — was open to the general public.

Earlier this month, Minnesota upstarts Aanik Lohani (DUPR 5.41) and Amrik Donkena (DUPR 5.66) defeated Thomas Wilson (DUPR 6.58) and DJ Young (DUPR 6.60) 11-6, 11-0 at the APP Chicago Open. Due to the large gap in ratings between the two teams, the upset has not been incorporated into any of these players’ DUPRs.

At the same tournament, Lohani and Donkena defeated William Sobek (DUPR 6.22) and Adam Stone (DUPR 6.29). This match has also been ignored by DUPR’s algorithm.

Lohani and Donkena narrowly lost to John Cincola (DUPR 6.47) and Ryler deHeart (DUPR 6.34). This is another match that DUPR has ignored. Update (9/19/2022): Lohani and Donkena played Cincola and deHeart twice. In the main draw, Lohani and Donkena won 11-3, 11-9. Both of these matches are currently excluded by DUPR’s algorithm.

DUPR addresses this issue on its website as follows:

Why don’t my matches count towards my ratings if my rating differs from my opponents by 0.625 or more?

This rule allows the algorithm to look at more impactful matches by ignoring expected blowouts. Even if an upset win is outside 0.625 at the time the match is recorded, it will be counted once the rating difference between players shrinks to less than 0.625.

The matches will be counted if and only if Lohani and Donkena’s average DUPR score rises to within 0.625 points of the average scores of their opponents. It’s a classic Catch-22: Lohani and Donkena can’t get these matches incorporated into their DUPR until their DUPR scores rise, but their DUPR scores won’t rise if their matches aren’t being counted.

We reached out to Jillian Braverman for comment. We asked her to reconcile the exclusion of “expected blowouts” with her website’s claim that “DUPR is the only global rating system in Pickleball that encompasses all of a player’s results.” The DUPR definition of “all” strikes us as a bit, well, Clintonian:

But we digress.

Dinkheads.com also asked Braverman to reconcile the decision to exclude “expected blowouts” with the claim that DUPR discourages sandbagging.

“Doesn’t this loophole do the opposite, since sandbaggers who lose against low-rated players will suffer no ramifications with respect to their DUPR scores?,” we asked.

Braverman explained to Dinkheads.com that DUPR will be modifying the exclusion rule for pros and referred us to DUPR’s director of communications, Kristin Walla, for further comment. Braverman also stated that our article would benefit if we were able to include information about the “percent of players affected, average time (number of matches) for players to experience the retroactivity, the reason behind the policy, and our plans for modifying the policy.” We replied that we would would welcome such information; we will update this post as needed.

Dinkheads emailed Walla earlier today (9/19/2022). We asked her why DUPR will change its algorithm to take into account “expected blowouts” in pro games but not in amateur games. In response, Walla asked us to hold off on publishing this article until DUPR changes its algorithm (“most likely later this week”). We decided not to wait. We will, however, update this post if and when Walla or another DUPR employee provides a substantive response.

Here’s the thing: Although we were initially skeptical of DUPR, we have come to like it a lot. We like that DUPR takes into account points won rather than simply looking at games won. We love that the DUPR database is searchable, making it possible to find any player’s DUPR. We like that DUPR is more transparent and open to questions than other rating systems. However, we strongly oppose the dismissal of “expected blowouts”–even if only a small number of players are affected. We also believe that DUPR has thus far done a poor job of explaining the rationale for this exclusion. Whether in professional games or amateur ones, it seems to us that DUPR’s anti-underdog loophole should be closed.

Update (9/20/22): Dinkheads just got off the phone with Kristin Walla, DUPR’s director of communications. She shares Dinkheads.com’s concerns about players like Lohani and Donkena whose upset victories against much higher-rated opponents are being excluded by the DUPR algorithm: “These guys are definitely test cases that obviously math can’t always perfectly portray. Between them and a couple other pros whom I’m aware of, this is one of the things I care a lot about. We want to make sure we are always perceived as accurate and fair. It is definitely not our intention to prevent someone from increasing their rating.”

Walla confirmed that DUPR is going to be tightening up its exclusion criteria and will likely eliminate it for professional players altogether.

She explained that if there is no loophole, the DUPR algorithm can produce results that unfairly punish highly-rated players at “super extreme” skills differentials. As an example, she said if a low-rated player went up against Ben Johns in singles and lost 1-11, Johns’ rating likely would decline. Even if Johns won 11-0, his rating may not improve. “We would be unfairly punishing that higher rated person before they even set foot on the court,” she said.

Dinkheads asked Walla why DUPR may close the anti-underdog loophole for pros while keeping it for amateurs. She said that pros range in skill level from 5.0 all the way up to nearly 8.0. As a result, it is common for pros with large gaps in skills to play each other, as Lohani and Donkena demonstrated in Chicago. By contrast, most people in the amateur brackets compete against players of roughly similar level. Walla, whose DUPR is 4.09, noted that she usually plays at 3.5 or 4.0 and “it’s very inconceivable that I would be playing someone a full point ahead of me.”

She also acknowledged, however, that this argument cuts both ways. If the anti-underdog loophole rarely applies in amateur competition, that could be an argument for eliminating it rather than keeping it.

We appreciate DUPR’s responsiveness and will continue to monitor the evolution of the algorithm.

Update (9/21/22): Walla followed up with some statistics from DUPR’s data analysts.

“Censored matches make up a very small proportion of our results data (2 percent), and on the player level they do not have the ability to influence a players rating to the degree that people think. For example, if we rerun ratings without the exclusionary rule, the average increase in rating is 0.01, and the average decrease in rating is 0.03. So, while most players would see a change in their rating, that change would be pretty negligible…” 

“Also, players tend to focus on excluded matches where they are the underdog, however most players have equal number of favorite-underdog matches that were excluded. The median number of excluded underdog matches for players is 5.0, and the median number of favorite excluded matches is 5.0. Players are just as likely to be the higher rated player in a censored match as they are the lower rated player.”

“Lastly, matches are not censored if/when the ratings of the players involved move within the threshold. The reality is, if you are good enough to consistently beat players that are 0.625 higher than you, your rating will likely come up organically. If you are not, then that result is an outlier by definition, and we remove outliers just as in any other statistical analysis. You do not need these excluded results, and you can increase or decrease your rating in any match. You can play well and exceed expectations in a match against a lower rated opponent just as easily as you can in a match against a higher rated opponent.”