Unpacking the Numbers: Is There a Better Way to Rank Volleyball Players?

Volleyball England’s Team of the Month

Volleyball England uses Hudl’s stats to decide the Player of the Month and Team of the Month (e.g. see below). You will always get people debating why certain players are in and why certain players didn’t make it. I wanted to look deeper into how it’s done and see if there was a better way.

Combined Weighed Performance Indicators

I had access to the formula used by Volleyball England to calculate the players for Team of the Month. It was essentially the addition of specific performance indicators plus a weight. The weight was added to value one skill over another. For example, outside hitters were measured on a formula similar to below (it’s not the exact formula):

Outside Hitter Total = (Hitting % * 0.9) + (Passing Average /3)

There are two challenges to this. The first is the Hitting % and passing average are measured on two different scales – one is a percentage (0 to 1) and the other is on a scale from 0 to 3. Since the passing average is the bigger number, it will disproportionately affect the final score. The second challenge is the weight assigned. The purpose of the weights is to value one skill more than the other. However, how can we accurately assign a correct weight? For example, the weight for the Hitting % above is 0.9 – is that an appropriate weight? Should it be higher or lower? If so, how much higher and lower?

These challenges are compounded when deciding the Player of the Month. How can you combine the other performance indicators – Assist %, block kills per set, digs per set, and serve average – into a single metric? They are measured on different scales. For example, a libero can achieve 5.0 digs per set and this will massively dwarf any hitting % (e.g. 0.250) or assist % (e.g. 0.380). A weight can balance things out but what weights do you use?

So, is there a better way to compare players so that each player has a fair shot of being in the Team or Player of the Month? Broadly speaking, is there one metric that considers all skills performed to see how well a player has performed overall? Word of warning, this only works if you use DataVolley.

Weighted Average

I previously did come up with a way to do this (see link). It was based on a weighted average, where all actions are rated out of 10. It makes sense that if we have a weighted average for serve and pass, why not do it for the other skills too. All technical skills are graded on 6 levels (Perfect, Positive, Neutral, Negative, Worst, and Error) in DataVolley. We can give each grade the same weight, regardless of the technical skill performed. We can make the weights range from 0 to 10, similar to how pundits rate football players out of 10. The equation can be seen below:

Overall Rating = [(Perfect Action * 10) + (Positive Action * 8) + (Neutral Action * 6) + (Negative Pass * 4) + (Worst Action * 2)]/ Total Attempts

This is a nice idea but it has one major flaw: the assigned weights should represent the importance of the action but they are arbitrarily assigned. I just assigned weights of 10, 8, 6, 4, 2, and 0 because it’s convenient. There’s no rule they have to be equidistant to each other (i.e. spaced by 2 points) or range from 0 to 10. I could have easily assigned the weights to 64, 48, 40.6, 24, 9, and -10.

The core issue is we know one action is better than the next (known as an ordinal scale), but we don’t know how much better. In other words, we know relative ranking but not the magnitude of the difference between them. For example, is a positive pass (8 points) worth two times as much as a negative pass (4 points)? Or is passing ten negative passes the same as passing five positive passes and five passing errors? Both have a 4.0 passing average but I know I would always choose the player with ten negative passes if there’s still a chance to win the rally in all ten passes. The other issue is when we consider the value of the different technical skills. For example, is a service ace (10 points) worth the same as a perfect dig (10 points)? Or is a positive pass (8 points) worth two times as much as a negative set (4 points)? The answer to these questions is we don’t know.

So we need a way to quantify the importance of each action so we can assign an appropriate weight. This also means we need to somehow quantify every action using the same measurement. It’s tricky but I have an idea to achieve this.

Using Expected Rally Win %

Taking inspiration from the Expected SideOut % to measure passing performance, we can quantify the importance of each action by its associated Rally Win %. In other words, what is the Rally Win % after you perform a specific action? Once we determine that, we can use those values as weights to calculate an Expected Rally Win % Contribution, as seen below.

Expected Rally Win % Contribution = [(Perfect Action * Expected Rally Win %) + (Positive Action * Expected Rally Win %) + (Neutral Action * Expected Rally Win %) + (Negative Pass * Expected Rally Win %) + (Worst Action * Expected Rally Win %)]/ Total Attempts

By determining the Expected Rally Win % of each action, we can quantify the importance of each action because you need to win rallies to win matches. It also allows us to measure each technical skill on the same scale, i.e. Expected Rally Win %.

This metric measures the player’s expected contribution to winning the rally. It’s a great idea in theory but how does it look like in practice?

Team SideOut Expected Rally Win %

I exported data from Team SideOut Polonia London in the 2023-24 season. The Rally Win % for each graded action can be found below.

It is genuinely interesting to see these numbers. So it makes sense that all errors (=) have a 0% rally win rate since they are by definition actions that result in losing the rally. It’s the same for worst attacks and blocks since they are attacks stuffed (A/) and block-outs (B/), respectively. It also makes sense that perfect attacks, serves, blocks and free-balls have a 100% rally win rate since they are kills (A#), aces (S#), block-kills (B#), and free-ball kills (F#). Values with “—” mean I didn’t use those grades in that season so there’s no Rally Win % associated with it. A worst serve (S/) is actually an opposition overpass so it should have the second-highest rally win % (67.8%) behind an ace (100%). Interestingly, there is pretty much no difference in rally win % between a perfect pass (68.9%) and a positive pass (68.1%). It was probably because we had an absolute monster middle attacker and we used her as much as we could. The positive dig (D+) is lower than the neutral dig (D!) (61.5% vs 63.0%). This is probably due to the fact I tend not to grade spike cover digs as they happen so fast, so they stay on the default positive dig (D+). This means the positive dig (D+) will contain worst, negative, and neutral spike cover digs, which should lower the overall Expected Rally Win % for positive digs. The reason why negative blocks (B-) have such a low Expected Rally Win % is because I normally code it after a block touch that results in a dig after block touch error (D=).

So while it’s interesting to see the Expected Rally Win % for each action, how does this look like when we calculate the Expected Rally Win % for individual players.

Player Ranks

Below is the Expected Rally Win % Contribution for the Team SideOut Polonia players, ranked from highest to lowest.

You can see how ranking is largely grouped and ranked by position. The ranked groups are 1) setters, 2) outside hitters, 3) middles/ opposites, and 4) liberos. I believe the reason why setters are highly ranked is because around 60% of their actions are positive sets. It means their overall Expected Rally Win % is anchored on its Expected Rally Win % of 67.2%.

It looks like outside hitters have a higher Expected Rally Win % compared to middles and opposites. I think it’s because outside hitters can pass and positive (68.1%) and perfect passes (68.1%) can bump up their Expected Rally Win %. The outlier to this ranked grouping is middle #5 Sviridova. She is ranked above all outside hitters. However, #5 Sviridova had the highest Hitting % in the league and in Team SideOut’s history (since my time with them anyway). She was also voted MVP in the league in that season!

It makes sense that the reason why liberos are the lowest-ranked group is because they don’t score points. It means if they had all perfect passes (68.9%) and perfect digs (71.2%), their upper limit is only around 70% Expected Rally Win % Contribution. For all outfield players who can score points, their upper limit is 100% (all kills, aces, block kills).

Discussion

It is interesting to see the players ranked in this way. It’s not great that players are ranked largely by position. So if we were to use this metric, the Player of the Month will always be a setter. It also doesn’t look great that our third setter (#15 Skandali) is ranked higher than the league’s MVP (#5 Sviridova). However, it does generally show the starting players towards the top and the bench players towards the bottom. Also, the first-ranked players in each position were the starting players (although this was always fluid depending on who was available and the time of the season). So I would say that it’s not bad at ranking players, especially by position.

I would be interested to see if rankings change if I fully graded the setting actions (see link). This means expanding the negative sets into sets too far off (E+), too far in (E!), too wide (E-), and too tight (E/) with good sets moved to the Perfect code (E#). If were to be more stringent in coding those poor sets, it should knock down the setter’s Expected Rally Win % so they may not all pool towards the top of the rankings. It would be the same if I fully graded the block (like here) and free balls, this could increase the accuracy of ranking players.

I do have to say that these results are specific to Team SideOut Polonia London players only. This is a biased sample so we can’t apply any findings to the general population such as the Women’s Super League. Results may also differ depending on the level of competition and by gender, e.g. would I find similar results in my American School of London Boys Varsity team?

Final Thoughts

The Expected Rally Win % Contribution is a metric that measures the player’s expected contribution to winning rallies for the team. It’s probably the best way to measure all players on the same scale so we can compare players in different positions. I do think this metric can do a good job picking the Team of the Month. This is because Team SideOut’s starting team was players ranked first and second by position. Of course, we don’t know if these results will appear in other teams or leagues.

However, the Expected Rally Win % Contribution is probably not the best for picking the Player of the Month. This is because the metric heavily favours setters. This means a setter will likely always be the Player of the Month. If we were to fully grade the setting, block, and free ball, this may knock a few setters down the rankings and give other players a chance.

Unfortunately, Volleyball England couldn’t use the Expected Rally Win % Contribution if they wanted to. It’s because Hudl doesn’t keep a full set of stats (e.g. dig errors are not coded) and it doesn’t calculate an Expected Rally Win % for each action.

I do think the Expected Rally Win % is worthy of future investigation. I am already thinking about what the expected values will look like in other competition levels and genders. Do setters still top the ranking? If they do then does that mean setters are the most important players on the team? I could also include the opposition data to double the dataset to see if that changes things. I also want to see what happens if I use actual values rather than expected values. This means I could measure a player’s actual contribution to winning rallies. Also, could an Expected Rally Win % be a better performance indicator for individual skills? It is already accepted in passing and serving. We will see what I will investigate next.