.png)
How Different Are thevolleyballanalyst’s Stats to Hudl’s Stats
tl:dr
I wanted to see how different my stats are to Hudl’s stats
There were very few differences in matches played/ sets played, serve and serve receive stats, more differences in the attack and setting stats, and most differences in the dig and free ball stats
Differences are largely explained by differences in how we define the stats (e.g. Hudl tags downballs as attack attempts whereas I would code them as free balls sent) and by human error (e.g. coding/ tagging the wrong player)
Introduction
With Volleyball England partnering with Hudl to allow Super League teams to collect stats (read my first thoughts on this link), I wanted to see how different Hudl’s stats are from what I get. How does my 13-year volleyball analysis experience compare to a professional sports performance analysis company? Let’s find out.
Method
The match where we will compare data is the Leeds Gorse vs Team SideOut Polonia London Women’s Super League match played on 15th October 2023. The full stats from Hudl and my stats can be found at the end of the blog post.
I coded the match in real time and cleaned the scout file post-match using my data-accuracy procedure (see link for more info). Hudl tagged the match from an uploaded match video with some information supplied by me, e.g. starting rotations, player numbers etc.
I exported and combined the data from Hudl and my data from DataVolley. I transformed the data so only similar data could be compared. For example, Hudl and I should use similar definitions for serve and serve receive attempts. This meant I removed data that can’t be compared, e.g. I removed Hudl’s Ball Handling Error data as I don’t collect that data in the same way (the error is recorded but it would fall under a technical skill such as a setting error).
We can calculate a disagreement % by total disagreements (or differences) divided by the total actions coded by Hudl and thevolleyballanalyst divided by two (to find an average of total actions coded). The calculation can be found below:
Disagreement % = Total Disagreements/ [(Total Actions Tagged by Hudl + Total Actions Coded by thevolleyballanlyst)/2]
Results
The table below shows my data (in grey) and how it is different to Hudl (in white and red). White highlight means there is no difference while red means there is a difference and the deeper the red colour, the further it is from no/ zero difference. So Team SideOut’s #7 Kasia Plucinska has serve attempt stats of 5 +2, which means I coded 5 serve attempts but Hudl tagged 2 more, i.e. 7 total. A Disagreement % is listed and anything below 5% is considered a satisfactory level of disagreement.
Matches Played (MP)/ Sets Played (SP) | 4.3% DisagreementThere were four disagreements that stemmed from two errors. I coded that #2 Millerd played one more set than what Hudl coded. I looked into it and Hudl incorrectly tagged #7 Plucinska as coming in as a substitute when it should have been #2 Millerd.
#17 Greenidge has played an extra set and match. Hudl Analyst thought that #8 Tszching subbed in when it should have been #17 Greenidge. This was probably my fault as Hudl did not list player #17 so I probably didn’t list #17 for Hudl in the original information.
Serve Attempts (S Att) | 2.0% DisagreementThere were four disagreements but it stemmed from the #2 Millerd/ #7 Plucinska substitution. #2 Millerd subbed in as a serving sub so she should have been given the two extra serves that Hudl gave to #7 Plucinska.
Serve Receive | 3.3% DisagreementThere were six disagreements that stemmed from four errors. Two errors were due to the #2 Millerd/ #7 Plucinska substitution error by Hudl as #2 Millerd stayed on to pass so she should have been given a pass continuation instead of #7 Plucinska.
The last two disagreements were due to Hudl tagging a serve receive error but not assigning it to a player. One serve hit the net and trickled over and I assigned the blame on setter #8 Canevari as she was closest. It was unplayable but I needed to record the passing error and by charging it to the setter, it wouldn’t affect the main passers’ stats. There was another serve receive error where the serve went in between two passers. I put the blame on #9 Garnero as the serve was moving towards her.
Hitting | 8.6% DisagreementWe have more differences here. I did not check all of them but I found some of the differences are due to Hudl tagging downballs as attack attempts when I code downballs as free ball attempts. This is because I believe downballs are not a good reflection of a player’s ability to spike the ball.
Hudl coded 3 extra attack errors for #19 Busa and 2 of them downball errors. I did think this was harsh as the sets went beyond the antenna so I put the fault on the setter (setting error). The third extra attack error was due to human error as #9 Parmova got the attack error, thus why #9 Parmova has 1 fewer attack error as coded by Hudl.
#8 Tszching’s kill was a mistake by Hudl as #6 Borrowdale got the kill. I had Team SideOut’s #6 Scott having one fewer kill and this was my mistake as I coded it as Leed’s #6 Borrowdale having got the kill (I have since changed this in DataVolley).
I did not check on the other differences.
Setting Attempts | 8.0% DisagreementThere are more differences here too. I did not check on all of them but I found a lot of the differences are due Hudl’s operational definition of an assist (or setting) attempt. When the setter attacks the ball on the second ball contact, the preceding pass or dig is additionally credited with an assist attempt. This was the reason why #19 Busa was credited with 4 assist attempts by Hudl whereas I coded 0 for her.
Digs | 19.6% DisagreementThere are a lot of disagreements. Again, I did not check on all of them. However, I found that since downballs are tagged as attack attempts and the resulting digs are tagged as digs. This explains why #4 Tsoumi was credited with 5 extra dig attempts. I would also code them as digs but I add an extended code so they are classed as free ball digs.
Free Ball Sent | 68.4% DisagreementThere are massive disagreements. I found that the biggest reason for this was that Hudl tags overpasses, over-digs and over-sets (set attempts that go over the net) as free balls sent. I use separate codes for overpasses (R/), over-digs (D/) and over-sets (E/). This explains why Hudl coded 5 extra free balls sent for #10 Kalinova. Four of them were overpasses/ over-digs and 1 one of them was due to an over-set.
Total Actions | 9.5% DisagreementAs seen in the table below, the total disagreement % for all comparable actions is 9.5%. A satisfactory level of disagreement is commonly set at 5% and this is double the disagreement rate. However, using a Disagreement % is an unfair measurement as Hudl does use slightly different operational definitions.
Discussion
There were the fewest disagreements in the matches played/ sets played, serve and serve receive stats (<5.0%). There are more disagreements in the attack (8.6%) and setting stats (8.0%). There are the most differences in the dig (19.6%) and free ball sent stats (68.4%).
Two main reasons emerge to explain the majority of the differences. The first reason for differences is due to differences in operational definitions, i.e. how we define the stats. For example, Hudl would tag downballs as attack attempts when I would code them as free balls sent. The second reason for differences is due to human tagging/ coding errors, e.g. when Hudl incorrectly substituted the wrong player and the subsequent stats were tagged to the incorrect player or when I tagged a kill for Leed’s #6 when it should have been for Team SideOut’s #6.
Limitation
One big limitation is totals may conceal differences that cancel each other out (see here and here). For example, I could have coded one attack kill for #9 Parmova but Hudl tagged that same attack kill to #19 Busa. I could have then coded a second attack kill for #19 Busa but Hudl tagged that same second attack kill to #9 Parmova. There are two errors but the totals cancel each other out as both datasets will have 1 kill each for both players. It is best to export the two timelines of events and match them for agreements (or differences) like I did in this study. However, I don’t have access to export Hudl’s timeline of events.
Final Thoughts
It was an interesting exercise to see how Hudl’s stats are different to mine. The serve and serve-receive stats are pretty much the same as mine. There are more differences in the attack and dig stats but I reckon if Hudl didn’t code downballs as attack attempts and the resulting digs as digs, the stats would be very similar to mine. I don’t mind the bigger disagreements in the setting and free balls sent stats. Setters are generally picked by coach’s eye rather than stats and it really doesn’t matter if the free ball sent stats are vastly different.
This knowledge would make it easier for me if I wanted to go back and clean up Hudl’s timeline. However, I have no plans on doing that because I want to spend more time on sidestepping careers to become a Data Analyst (see link for more info). Having said that, I did spend a lot of time undertaking this study and writing up my first thoughts on Hudl.
Appendix: Full Stats by Hudl and by thevolleyballanalyst
Comments