1. Severe Error (-.5) Reserved for mistakes that disturb the routine in an extreme way, such as a wild throwaway, a long, embarrassing break in the routine, or an incident that clearly endangers the audience. Judges are cautioned to make a Severe Error deduction only when the audience is clearly endangered, not just when players perform near the audience. Catches near the audience may add to a routine’s excitement without endangering spectators.
I noticed something at Paganello while helping Lui tabulate the results of the Co-op finals. A judge gave three teams in my pool a Severe Error – including one in my routine with Gery Nemeth and Balu Major. As implied in the definition above, Severe Errors are most often wild throaways, discs thrown out of the reach of the teammate. I was surprised because I didn’t remember any throwaways during our performance. I’ll have to wait until the videos hit YouTube to remind myself of what happened during our five minutes.
I bring this up not as a sob story. We would have placed the same with or without that deduction. I bring this up because Severe Errors are rare, especially indoors, and yet three teams were given the Severe Error deduction during the finals, so it’s a good window into how judges perceive the judging system and the routine they watch.
Judging competency is essential. As so many players emerge into the competitive scene, it’s super important that judges are trained and that players can expect to be evaluated consistently. One of the best ways to improve our judging competency is through discussion. So let’s talk about the Severe Error. When do you give them, what’s your threshhold for the difference between Major and Severe Error? What do we need to do to communicate the difference to new judges?
I give Severe Errors rarely because that’s the intention of the judging system. They are Severe. Most teams don’t stray into Severe territory. They may make mistakes, but most don’t make gigantic, severe mistakes that match the definition of this deduction.
For throwaways, I adhere very strictly to the definition. It’s got to be a wild throwaway. The teammate doesn’t even have a chance to touch the disc. Someone used to use the phrase “throw to the ghost,” like the disc was thrown to someone not even on the team.
Regular bad throws usually get a 0.2 or 0.3 from me. If the disc is dropped or doesn’t reach the teammate, but it’s not a huge, embarrassing error, I give a 0.3. Occasionally, I’ll downgrade for 0.2 if someone is able to seamlessly pick or kick the disc back into play.
The other scenarios for Severe Error rarely happen. I personally have not seen a player endanger the audience. I’ve seen many flow interruptions, but the play hardly ever stops long enough for it to be a Severe Error.
What kind of disruptions would merit a Severe Error if I were a judge? Let’s say the team not only forgot where they were in the routine but started a long conversation about what to do. After about 10 seconds of absolutely nothing going on, I’d probably give the Severe Error. Usually, teams make some sort of throw before that, so this situation rarely comes up.
How about teams that endanger the audience? For me, the line that distinguishes between aggressive play, going after a disc off the field, and a Severe Error is body control. If the player is tracking the disc AND the audience, and if they make good decisions about where to run/jump, it’s not a severe error for this judge. But, if the player recklessly runs into the audience with a great chance of collision, or if they jump with little body control so there’s a very real possibility of them landing on an audience member, they get the 0.5 deduction.
I want to make an important distinction. There are players who can play into an audience without endangering them. It’s probably not the best situation, but I’ll let the Artistic Impression judges deal with that. The point is: the player isn’t endangering the audience by doing it, and that is the essential difference between a Severe Error and a questionable decision that’s not really an error.
—
i like to add some thoughts to this ..
when i started competing a few years ago .. i was asked to judge and did not had a clue what i have to do ..
i asked some other judges and got help but in the first years of competing my judging style changed a lot .. this is not only because of experience .. it is also because of knowledge about judging ..
when we had the first German Open in 2005 Hartmut Wahrmann was giving a briefing on judging and this helped a lot to learn and understand what judging is about.
since in Europe a lot of new Jammers appear i like to suggest that we do some judging teaching at the very start of a tournament..
this is not because I was unhappy with some judging decisions .. I think our sport lives from the different point of view that everybody has on the play of some one else .. It is because for me this would have been a great help and even today I think I can learn a lot about judging..
Big ZZZs
Jan Schreck
maybe 1.49
http://it.youtube.com/watch?v=rTgZLtMoRKU
ciao
π
Jan, I agree. Especially with all the new judges, a little judging "class" helps everyone – new judges and experienced judges. Probably the biggest danger is when an experienced judge thinks they know everything.
Thanks Paolo. Interesting possibility and a good example. Two ideas:
1. It was a kickbrush, and according to the rules, Severe errors are for "throwaways." A brush is not a throw; Gery’s kick should not qualify as a "throwaway." So, for a Severe Error, the judge would have determined that this was an "endangering the audience" situation.
2. In a stage this small, the criteria for Severe Error should be more narrow. There is not much that players can do to keep EVERY pass on the stage. Gery’s pass went off the stage, but it didn’t endanger the audience. Remember the "disturb the routine in an EXTREME way" definition. This was a mistake, a drop, but not extreme. Balu didn’t leap into the crowd and land on top of Ultimate players. And, there was no long interruption in flow.
If I were a judge this would definitely be a 0.3 because of the drop into the audience but definitely not a 0.5.
thanks Arthur for your explanations
π
ciao
I agree with Arthur, as a judge, I would give 0.3 to Gery’s kick.
—
GadDozZ
Here is one of the other two routines that got a Severe Error deduction during the Paga co-op finals. I haven’t screened this video yet, so I don’t know when the error happened:
http://www.youtube.com/watch?v=o8EbfAs-2K0
I heard a lot of times the statement "Every player should read the Judging manual before competing at a tournament".
But it seems to me that the most players doesn’t care about the judging method, except when they feel that they were judged in the wrong way.
I heard a lot of times the statement "Every Judge should talk with the other judges".
But it seems to me that the most judges doesn’t care about the other judges, and every judge has his own way to judge.
I heard a lot of times the statement "We should change the judging method".
But it seems to me that we should still learn how to judge…
I think that we need to find a way to give more responsability to the judges, because now judging is felt as boring work.
—
GadDozZ
i agree with jan too. paganello 08 was my first tournament ever, and i didn’t knew anything about judging. in the semis i was thrown in the judging pool, but cleverly Lui placed me in execution π
i think execution is not difficult for a newcomer (despite of arthur’s clear reporting of some errors in the finals!), but how to manage if i was in artistic impression or difficulty? definitely a brief explanation of judging system before every tournament is what we (newbies) need!
—
nOOb
As a judge, I don’t see any 0.5 in this routine.
The judge maybe give 0.5 at 0.38 or 2.25 or 3.27… but that’s totally crazy IMHO.
—
GadDozZ
maybe at the first throw…..the judge didn’t intend that this was an ironic scene about ultimate players
—
nOOb
Michele, welcoming new judges is a serious issue, and we have not been successful in supporting inexperienced judges at the table.
One solution has been to assign one very experienced judge per category, to assist the newer judges with questions. Also, we printed the judging instructions on the back of each judging sheet – with so many languages now, maybe this is impossible, but it was helpful for the English speakers at past tournaments.
Planning some training sessions is very important. Even if only a few judges attend, it helps to see examples without the pressure of competition and to learn about strategies for being a good judge. For now, we can talk with each other using video, but sessions at tournaments are also important.
I agree with both Jan and Gaddoz. As far as Italy is concerned, the problem of what needed to be defined as Severe Error emerged several times. Some Italian judges are giving -0.5 even in cases where I personally wouldn’t give it. Since the problem was so ticklish, last year I asked Larry to explain me what the judging system said. For my understanding, 0.5 is rarely given. At the beginning I thought 0.5 was used only in dangerous situation for the audience because in my mind "Severe" meant something dangerous while it didn’t concern the flow. However reading the judging system I would give it in the cases Arthur indicated: wild throaway (a throw that breaks the flow because the teammate cannot reach the disc) and when endangering audience/judges. Probably, I would consider giving it also when there is a long break as Arthur stated. It’s a delicate issue, but I think that it needs to be carefully defined and that we need an official and definitive definition of Severe Error. Execution is the only 99% mathematical category that we have in our judging system…
I’ve watched both videos now. It was clear to me in each video when a "throwaway" error occurred. I think I’m generally a pretty easy judge and I don’t want to seem like a hard ass, but I would have given both of these errors the 0.5 deduction too. A more than usual loss of control occurred to me. Was it extreme enough?
The rules reads "… , such as, …" and examples are given. A kick brush is not exempt just because it’s not listed. Any pass that is wildly or completely out of reach of the intended recipient is a candidate for the 0.5 deduction, in my opinion. I can imagine a chest roll pass gone so bad that the receiver can’t get to it, as well as other possible passes gone awry. Too often I see the ladies misfire on one of those chicken wing crank throws.
I agree there usually aren’t always (or many) severe errors in a round, but occasionally there are. Both of these passes in question went into the crowd untouched. While a crowd consisting mostly of disc players is not likely to be endangered and because of the small venue, the break in action was short, in my opinion, I think these were severe errors. I know I would have been embarrassed if I had authored either of these "into the crowd" maneuvers, more so than I’m embarrassed by my drops.
In a close venue like this, one may need to dial back on the force applied to achieve the accuracy needed on a small playing site. Our sport demands good throwing and passing skills.
Slightly off topic, I’m reminded of a freestyle ice skating performance where the skater was too close to the boards and actually went off the ice through a gap in the boards where a camera was while performing a multiple spin move. Funny as it looked, that was a severe error! I suppose a skater’s fall would be like our 0.3 deduction, a two footed landing a 0.2 deduction and a balance or wobble error a 0.1. I’m sure, everything works differently in ice skating, but I’m trying to draw an analogy.
There’s also a point of consistency. Since all three deductions came from the same judge, this was his/her interpretation and was applied evenly to all teams they judged. I’ll bet the third 0.5 deduction was a similar into-the-crowd, untouched pass of some kind (not a drop that rolled off the stage). I was always taught, when judging, to apply the same standards to all teams, even if it did not match that of fellow judges, who were being consistent within their own interpretation.
There used to be some judging standard of all category judges being within a certain range of each other, I guess in an attempt to homogenize the scores. This is particularly hard in the AI and Diff categories. To me, if one judge has lower scores across the board for all teams than the other two judges, that’s OK and it won’t have a bad effect on the overall category when totaled up.
Going a little beyond the execution category, if we had set scores for certain "book moves", and all judges knew the book scores for a move, then all judges scores would/should be close. Krae Van Sickle was a proponent of this type of scoring. This is similar to what is used in ice skating and olympic diving, where moves have an established difficulty. For example, a scare crow catch might be a difficulty 4.0, a spinning scarecrow (actually 1 1/2 spins) might be a 5.5, a double spinning scarecrow (2 1/2 spins) a 7.5, etc. If the disc is UD in any case, add another 0.3 in difficulty. If conditions are very windy, perhaps some moves would have another difficulty factor added. This is probably for the future, when we have retired players as judges and the scorebook for moves has been written. π
My 2
Thanks Doug. I want to just clarify: if I implied that the judge’s decision was "wrong," then that was not my intent. My goal for the conversation was to share our perspectives and calibrate our judging.
I think we can do a better job of defining each deduction for judges, but as you say consistency is essential. Equally essential is that players know what the standards are, which is why these conversations are nice.
Side note on the diff ratings: this is probably a completely separate (and much much longer) thread, but assigning marks to moves is trickier in our sport than things like figure skating because freestyle is all about transitions and combinations. A single spinning crow out of a difficult set might be twice as hard (or more) than a single spinning crow from a delay. We haven’t cracked a way of tracking the infinite complexity of combining our moves. But like with execution, the more we talk about it, the better we get.
I think Arthur’s write-up on severe error is right on – especially the section describing a throwaway resulting in a 0.2 – 0.5. Someone last summer in Italy asked me if it was common to give the 0.5 severe error, and I said it’s very rare. The italian player said it’s very common in Europe, so this discussion makes sense.
I feel these issues are rarely black and white and the judge should use discretion, keeping the essense of the rule in mind. Let’s not discourage a player to go into the crowd to save a disc, as long as it’s not reckless. That’s excitement and getting the crowd involved (use of elements ?). But don’t hesitate to give it when deserved – but make sure it is severe. Also other categories are probably docking the player for that move, so it’s already being severely penalized even without the 0.5.
Arthur’s standard for a "wild" throwaway would mean a higher threshold than what I’ve applied in the past. Unreachable throws always struck me as wild by definition. The introduction of "severe error" seemed intended to help sharpen that distinction. Throwaways are certainly more embarrasing than drops, and we would expect them to be far more rare. Also, in line with Doug, I would generally consider any kind of an intentional pass that flies farther than a couple of meters to be the functional equivalent of a throw.
Larry makes the point that this kind of mistake could end up docking points in other categories as well. Yes, but not always. Say the kick brush or some other complex pass had occurred toward the end of a 15 second segment, with the disc in the air by the start of the next one. The difficulty score in the first throwing/passing segment would be reasonably secure, and members of the competing team would still have a fair amount of time to build up their Diff score in the following segment (presuming the disc could be retrieved quickly enough).
For the same reason, when we’re just jamming, I think folks tend to show extra tolerance for the riskier and more interesting kinds of moves that end up as unreachable passes. The play may be busted either way, but a wild scarecrow brush is generally treated with more respect than a grounded backhand… (Unless the sender is repetitively and inconsiderately trying things far beyond his or her skill level.)
But Larry is right… in FPA-compliant jugding, the longer the disc is out of play, the more severe the impact on the other categories. So, my argument would be that equating an unreachable pass with "severe error" is not overpenalizing. The penalty just marks the starting point for some potentially even worse news.
I have a question about Judging a throwaway in Difficulty.
In Germany we are often using the alternative Difficulty Judging System, where you judge each combo (from Throw till Catch) without the 15 sec Segments.
In this case we have a big conflict between the categories Difficulty and Execution.
–> How should we judge a Throwaway in Difficulty?
Before 2 years Mark Regalbuti told me that he’ll give a ‘1’ in Difficulty for a simple Backhand throwaway.
And if you’ll judge consistent with that Judging System he is right I think.
But what happens if you handle that Throwaway as a Severe Error? You get a 0.5 in Execution and a 1 in Difficulty?! Than you will be punished two times and that could affect the result of your routine in a dramatic way!
Even if you give 0.3 in Ex the weighting is very hard!
What do you think about that problematic? Should Thowaways be judged in Difficulty?
maybe for first throw?
anyway I agree with Arthur
Flo,
Mistakes AND great moves get reflected in all the categories. If you make a drop, your difficulty was lower than what you wanted, you made an error, and there is a chance that the AI is affected. If you make a great combo, your difficulty improves, your execution stays high, and your AI probably improves.
I don’t think it’s a problem, and I’m not sure there is a way to separate the judging factors if it was a problem. The only solution is: shred. π
After a round, it’s sometimes interesting to look at the scores and imagine if teams had played a little different. What would have happened if the second place team had made two fewer drops? Their execution would go up, and their difficulty would improve. The excitement of their routine would also be higher, so probably they would improve on AI. Would it have given them 1st place? For the first place team, they might conclude "wow, we won by several points but we were very lucky," or the second place team might conclude "wow, I didn’t think we had a chance to win, but mathematically we could have won."
I’ve never given a .5 severe error, I think that giving this score for throwaways especially impacts the variety of throwing in a negative way. If we give that score for a bad throw then we really limit the types of throws jammers will attempt in their routines. I support Flo’s interpretation of judging by combos , than the 15 sec diff segments.
If it hits the ground it is a .3 deduction.
I don’t know where the .5 was given , but by way of explanation for another judge, I can see where a judge would be looking for more deduction after seeing the disc hit the ground a few times in succession, then a throwaway into the crowd.
After that Everyone picked up their game, Arthur shreded and all miscues were probably forgotten.