Intra-Operator Reliability Study Part 2: Cardiff Celts versus Cardiff Ladies (2011)

• Facebook

• Instagram

• Twitter

• Mail

• YouTube

• WordPress

Introduction

A discussion point that accounted for the lack of intra-operator reliability in my first study was due to operator’s inexperience.  The first data collection was taken with 8 match data collection experience.  The second data collection for the purposes of the study was taken with 26 match data collection experience.

This second study investigated the intra-operator reliability (the operator being the author) of one volleyball match when data collection experience was similar.

Method

Point ending actions plus attack attempts were taken from one volleyball match (Cardiff Celts versus Cardiff Ladies played on 25th September 2011).  I used NCAA’s (2011) Volleyball Statisticians’ Manual for operational definitions.  V1 values were taken in real-time using StatEasy software.  V2 values were taken in lapse-time using LongoMatch software.  As recommended by Hughes (2008), the following formula is used to calculate error percentage:

∑ (mod (V₁-V₂)) / V_mean) × 100%

Where V₁ and V₂ are variables, V_mean their mean, mod = modulus and ∑ means ‘sum of’. 

Results

Conclusion

A satisfactory degree of agreement is set below 5%.  It can be seen that the % Error Overall value is 0.73% and thus intra-operator reliability is highly satisfactory.

References

Hughes, M. (2008) “Analysis of Notational Data: Reliability” in Hughes, M. and Franks, I.M. (eds) “The Essentials of Performance Analysis: An Introduction”, Routledge, Oxon

NCAA (2010) “Volleyball Statisticians’ Manual” [online]. Available from: http://fs.ncaa.org/Docs/stats/Stats_Manuals/VB/2011easy.pdf [Accessed 28 September 2011]

Personal Note

I am quite happy that there was only one data difference between taking data live and post-match and between using two different software packages (>1%!!!).  That single different data entry could be explained too (it will take too long to explain here).  It should mean the data and further analyses for the upcoming volleyball season can be trusted.

*** Update 2 February 2021 | I recently did an intra-operator reliability study which you can read on this link.  This is where I statted the same match twice, months apart, and compared the two timeline of events.  This is a better way to test reliability compared to a simple error percentage because two errors can cancel each other out ***