The Process of Developing a New Setting Evaluation System

New Setting Evaluation System

In my new simplified VAS, I have changed the way I evaluate setting.  I did not like the Assist system because assists heavily relies on the proceeding attack. A setter cannot do much with bad passers and/or bad attackers.  The new system is now based on the number of options the attacker and thus making it largely independent from the quality of the attackers.  The new system is very similar to the blog post I wrote on this subject (see link).  The basic operational definitions of the new system is as follows:

– this set allows all hitting options, i.e. power, line, angle, roll/tip. – this set allows limited hitting options. – this set allows no hitting options, i.e. attacker can only put a free ball over.  – this set results in a point lost, i.e. ball handling error.

Developing Operational Definitions in Practice

As we know, providing precise operational definitions is important because it provides objectivity. That is, two different statisticians using the same VAS should be able to stat the same match and come up with the same data.  Precise operational definitions are developed via using them in real matches.  This is because unusual situations will occur that cannot be foreseen beforehand.  For example:

Revised Operational Definitions

Because of these unusual situations, the revised operational definitions has been developed into this:

A is the second ball touch that intends to set up an attack attempt.  Second ball touches simply to keep the ball in play are not considered an assist attempt but are still measured as part of managing the events list. 

(S) is a set attempt that results in an attacker having all attack options.  An attacker should have a normal attack approach and normal spike action, i.e. they are able to spike with power to anywhere on the court.

(AG) is a set attempt that results in an attacker having limited attack options.  Situations include:1.Set is short so attacker does not have power line option.2.   Set is too long beyond the antenna so attacker can only hit cross court. This is usually indicated by the attacker jumping and landing from outside the court.3.   Set is too tight on the net but the attacker is still able to attack the ball. This is usually indicated by an early outstretched arm for the tip.4.   Set is too far off the net. This is usually indicated by a shortened attack approach and the jump location of the attacker, e.g. if jumping and landing from the 3 metre line.

(AP) is a set attempt that results in an attacker having no attack options, i.e. a free ball or overpass, i.e. the ball was set-able and it should have resulted in an attack attempt but it did not.

(AE) is an assist attempt that directly results in a point lost. An assist error is charged to a player whenever an assist attempt:1.  Is called for a ball handling error, i.e. lift or double touch violation.2.  A set is poor AND the attacker cannot keep the ball in play.     a)  The set goes far beyond the antennae.     b)  The set is too low or tight on the net.     c)  The set is too high or low for the quick middle attack.

(AF) is a second ball touch simply to keep the ball in play, i.e. no set attempt. 

System Usefulness?

The operational definitions is not perfect but it will continually be refined as the season progresses.  However, the bigger question is whether this evaluation system is actually useful, e.g. can we use this system to differentiate the quality of setters?  I guess you will have to wait for a blog post at the end of the season for a review.

Although it probably makes the data a little more unreliable, I’ve decided to add location in addition to the other criteria.  This was because I had one Polonia player that could turn balls that were set short, i.e. a “good” set, into “excellent” sets because she was able to change her attack approach so she was still able to hit power line from sets that were short.  I couldn’t have the same set be either excellent or good depending on the player and thus why I’ve added location.  Below is a general guide to the locations of an excellent, good and poor sets.  A player’s attack approach and spike action are also a major factor of the rating of the sets.  For example, a set may go to the excellent zone for a middle player but if the middle player has to stretch and reach for the set ball then it is a good set.

*** Update 30 January 2021***

Since I moved to DataVolley in September 2015, I have stopped coding setting stats.  From what I remember, I didn’t have such a good system to measure setting.  Having read back the operational definitions, it does seem like a reasonable system but at City University London level only.  At Polonia SideOut London and IBB Polonia London, most sets would be excellent sets since the level is high so this system would be useless.  I haven’t really seen a good way to measure setting for higher level though.