Time blocks or natural phrases?
Some players expressed dissatisfaction with the current system for Difficulty judging that uses 15 second time blocks and would like to return to our previous process of judging Difficulty according to combinations of trick moves sequenced together (“combos” in the form of natural phrases). Since both approaches have significant advantages and disadvantages (see summary in annex below), the committee decided to issue a survey and ask membership for their preference.
In the end, almost 60 people participated and the survey results showed no clear preference for either of the systems (exact results available if needed). The committee agreed that such a result wouldn’t justify a total change of the system towards natural phrases. Instead the committee followed the idea of a hybrid approach that aims to combine the best of both phrase and time-block methods.
Hybrid Approach
The idea of the hybrid approach is a modification of the currently used time-block method of judging difficulty. The basic premise is for difficulty judges to not be mandated to give a mark exactly when the speaker says ‘mark,’ but instead record a score within a certain timeframe around each ‘mark’. To provide a time-frame, a metronome beat already recorded onto the difficulty tape shall start 4 seconds before and end 4 seconds after each ‘mark’ to define an 8 second opportunity within which time the judges shall rate the difficulty of play.
With the hybrid approach, judges remain attentive toward watching the freestyle players’ moves, consecutive combos are not artificially split up and fluff is penalized. Also, it still makes a difference if a combo is long or short, since Diff is still judged according to time and not according to combos.
A new audio file was created with the help of a sound technician, with a great deal of attention aimed at creating a file that is intuitive to use for the judges and with sound cues which will not annoy the audience. Pilot tests of the new audio file have occurred unofficially at tournaments and while viewing freestyle routines on videos; these “tests” showed that the new audio file appears to work well, and accomplish the goals of both phrase and time-block judging methods. The 3 minute version of the file is available for download.
Annex – Advantages and disadvantages of both systems:
Judging combos (natural phrases):
A difficulty mark is given for each “combo”(combination of trick moves sequenced together by a player). A combo starts with a throw and ends with a catch or a drop.
Advantages:
- Phrase-based judging is more naturally adapted to freestyle play in that judges don’t have to score during the middle of a player’s combination of moves and consecutive trick moves (“consecutivity”) can be better considered.
- A player’s combos are therefore judged fully as freestyle tricks combined together vs. judging part of a combo prior to the 15-second “mark” and the rest of the sequence after the mark.
Disadvantages:
- All combos, regardless of the length of time, are given the same scoring weight. A player’s very short combo has the same scoring weight as a long one. For example, Xavier does two short combos, each of which last 7.5 seconds. Annabelle, his partner, does a long combo of 15 seconds. At the end of the routine, when summing up single Diff scores, Xavier’s two short combos together have twice the weight as Anabelle’s long one.
In contrast, when using the time blocks system, Xavier’s two short combos together have the same weight as Anabelle’s long one (since they both fill a 15 second time block), which takes into account the fact that a short combo tends to be easier to complete flawlessly than a long one. - Situations where the disc is not in play (fluff) are not evaluated/scored in terms of difficulty. An extreme example would be if a player could simply lay down the disc for 10, 20 or even 60 seconds with no effect on difficulty scores (while not risking a drop during this time), because there is no ‘combo’ of freestyle moves occurring.
Judging time blocks:
The system as we currently use it: Difficulty judges hear a sound every 15 seconds and have to give a mark for the difficulty of moves that were performed within the last 15 second time section.
Advantages:
- The whole routine is judged and not just when the disc is in play.
- Short combos have less weight than long ones, which takes into account the fact that a short combo tends to be easier to complete flawlessly than a long one (see above).
Disadvantages:
- Judging using time blocks is not natural. Judges are obligated to score when they hear the “mark” sound, which is often in the middle of a combo. This is distracting them from the rest of the combo, and combos have to be split up in the middle, where the first half has to be judged as part of the first time block and the second half as part of the second time block. This is hardly possible in real time, and doesn’t take into account that a combo of moves are actually presented in competition often as one multi-faceted entity and not just a string of separate moves, thereby not suitably valuing consecutive play. In reality, the time blocks system has already been informally modified by many judges (as our survey showed) while in the midst of judging to not interrupt their observations and mark a score until after a player has completed their combo (with a catch or missed catch). Therefore, the time blocks system is adapted to phrase-based judging to some extent anyway.
- Judges have to calculate an average of the moves that happened within the last 15 seconds. That is quite demanding, especially since they have to decide quickly while having to keep track of the next moves already.
In my personal opinion, the “hybrid” system, leaving the judge a range of few seconds to vote the combo, is probably the most effective. I have used once this system and seemed to be “more naturally adapted to freestyle play”, as you said.
But still the problem about the very long combos, or the big scoring weight for short combos.
An idea can be implement a new categorie related only to the lenght of combos.
The judge would keep scoring every 15sec with the little range, and he would vote even at the end of the single combo, for example 3 for a 5-6 seconds combo and 8 for a 18-20sec combo.
The 2 totals (added up and devided by 2) would determine a more accurate score.
With phrase, there is no greater weight on shorter combos, because you divide by the total number of combos at the end (total number is the divisor). Shorter combos=more combos in same period of time=bigger divisor=lower score. Longer combos=less combos in same time period=lower divisor=higher score. Please remove this from the disadvantages of phrase…
In this case, compromise is wrong. The hybrid system should not exist. Give players an up or down vote, phrase or time block.
The new Audio file is very helpful and makes Judging much easier!
Because the Judge have an frame in witch he can decide when he will do his mark. This should be when the judge has made his decision and not if he will be forced to do it.
I also prefer the hybrid approach. It’s the best compromise.
The audio file makes it very easy for the judges.
Although, on this issue I’m goin old school and agreeing with the “leave the phrase alone” mentality, I believe there’s always been another option of scoring much more frequently, such as when the difficulty level varies. This is like grouping 2 or 3 moves that are linked as a sub-phrase, or each element of quick play. Only a completely consecutive combo, of all the same diff level of the sub-phrases, would get a single score using this method. I make this suggestion, because the reality is that there is a great deal of fluctuation in diff level within the average combo, or co-op, from the in-take, through the combo-o-moves, through to the seal. And the problem with long periods of time between notations, like either the phrase or the robot system, is recency error. We tend to reward what we most recently seen. That’s why average moves to big seals can sometimes tend to get higher scores than big pulls to average seals. And I’ve noticed, some have adapted their competitive game, either consciously or un-consciously, to peak every 15 seconds, playing to the robot system. A “no look” style of writing would have to be implemented, to go to a sub-phrase system, and that might require judging training, or a bigger space per team on the judging sheet, and some sort of a hand guide to help move along the page for some judges. I’m talking about scoring as many scores as it takes. Then average the scores. That kinda frees the diff judges up a lot, and as I’m always getting asked to judge diff, this is my view on how I’ve always wanted to see it judged.
But puhleeeeze, in any case……………………….. LOSE THE DIFF TAPE!!!!! MARK ONE
I would like to see a top tournament judge the best with out the Difficulty category. To judge difficulty successfully would mean that the judge is familiar with all the moves ever made, impossible. New moves that show up at the top tournaments would be judged how? The judge has never seen the move how are they going to judge its difficulty? Moves I do may seem difficult to some because they have never seen or done them. The same would apply to me judging or anyone else. I agree that by dropping Variety we have done a disservice to the sport. Now what do you do with aspects that really are outside the judging system, like a routine of just piddling, or quick catch or twirling or juggling, etc.? Having a robo clock chops up the overall impression of how difficult the routine was.
In some horse events the judges have scribes assigned to them so they don’t have to write things down, so the judges don’t take their attention away from the routine. The “difficult ” aspect of judging has
driven players from the sport , “Variety” would bring in new players.
For the beginning I think the hybrid approach is a good compromise…and is also how i have handled it so far. I cannot give a mark on diff in midst of a consecutive combo…
Furthermore i agree with dave that there is a tendency to judge only the last move and not the whole 15 sec period… we have to emphasize on this part when sticking to the tape. it is the whole period you judge, not a single move! which can also lead to the result that a 15 sec period ending with a double spinning barrel gitis can only get 4 points, if the rest of the elements were low diff stuff, lots of “the” receptions, etc…. it’s the average of what happened within the time block. Judges should be more aware of that when judging diff… which might help to take another step towards fairer results.
I don’t have significant with this change. The choice to include the completion of a consecutive phrase in a block has always been in the rules. This change seems to be a refinement to make the old flexibility more clear to judges.
This subject has been discussed so many times and many times too emotional. It’s not that important – both systems can work, the current works pretty good. One argument is incorrect: shorter combos (phrased based judging) don’t have more weight than long once because at the end we are going to divide by the number of marks (combos). One argument is incorrect. Most of us never judged the way that we put the number down quickly after the timemark. Most of us wait for a good timepoint to put the number down. This has allways been encouraged and it works good. So the hybrid approach is therefore not new –we just need to educate the judged to use the system like that. The strongest argument for or against one or the other system, is the one “situations where the disc is not in play (fluff) are not evaluated/scored in terms of difficulty”. This disadvantage of the system with the natural phrases is much stronger than any other (dis)advantage of the two systems that we should stay with the current one (blocks).
I’m totally agree with Arthur and Reto, I’ve learn to judge with flexibility and I think everyone can learn it. The only problem is using a full scale but I’ll speak about that in the next one.