Eleonora Imazio has shared the Italian translation of the new FPA judging system. Download it as a 200kb PDF document.
There was a judging experiment this year at Beach Stylers, and I loved it.
The Beach Stylers judging system was a simplified version of the FPA system. Two panels of judges scored routines. The first panel handled Execution and Artistic Impression. The second panel handled Difficulty. Key changes across the board created a competition that served our sport by rewarding ambitious play.
Penalties for mistakes were reduced and collapsed into fewer deduction categories. With 0.2 as the worst penalty for any mistake, taking risks got very attractive.
Difficulty was scored by phrase. Normally this doesn’t affect risk incentive because the easier, transitional phrases mute the effect of the peak moments. At Beach Stylers, only the top 10 phrases counted, creating an incentive to go bigger and bigger. Every time a team replaced a weaker move with a stronger one, their mark went up noticeably. Combined with the reduced penalties from execution deductions, the top 10 approach encouraged players to pushing their limits.
AI was simplified but touched enough elements to measure the performance while not being a burden. With the added responsibility of judging Execution, it was helpful for AI judges to track fewer subcategories.
Linking Execution/Artistic Impression
In the Beach Stylers system, the AI score and the Ex scores are multiplied together. This is a cool approach to reducing the skewed impact AI and Ex traditionally have on the final score and preserving the importance of Difficulty. Here’s how it works. AI/Ex can contribute a maximum of 50 points to the score. Let’s say a team maxes out in AI for 50 points (10 x 5 subcategories). But they have 3 drops. That would result in an Ex score of 9.4 and an Ex multiplier of 0.94 (Ex score divided by 10). The AI/Ex score is 50 x 0.94 or 47 points.
This is a reversal of the scoring dynamic from the FPA system where AI adds points to the score and Difficulty is locked in a narrow averaged range. At Beach Stylers, Difficulty was unleashed by the top 10 approach, allowing teams to add to their score in a tangible way every time they replaced a weaker top 10 combination. Meanwhile, AI/Ex stayed in a solid range, generating modest distinctions between teams. Teams that sacrificed difficulty for AI were likely to be hurt more than teams that sacrificed a AI for difficulty. That said, I saw a team or two lose points by not addressing AI.
The Judging Experience
I judged only AI/Ex, and it wasn’t taxing. Cooperation among judges helped to minimize Execution tracking errors. It’s possible to judge AI without taking many notes, so focusing on Execution marks while taking in the whole performance felt relatively effortless.
Let’s Do This More Often
This judging approach is a breath of fresh air. Like the turboshred approach, it incentivizes state-of-the-art freestyle play. It unleashes us. It’s an engraved invitation to step up. Turboshred has a presumption of mistakes that the general public understands. That’s not usually the case in team play. Beach Stylers addresses this by including enough incentive for cleaner and cooperative play to be fun for the general public to see. Let’s try this approach to competition more often!
After reflecting on Worlds in Bologna I extracted one question about judging, which I may ask to the freestyle community.
Judging Difficulty needs some concentration, knowledge and experience.
Judging AI needs even more concentration, knowledge and experience.
BUT judging Execution needs by far the lowest concentration, knowledge and experience.
Why is the Execution kept so simple?
Why do we not put more intelligence in Execution?
I think counting drops and non restricted catches is by far not enough to reflect the execution of a routine!
What do you Think?
To help your preparations for FPAW08 and other competitions, here is a German translation of the judging system.
1. Severe Error (-.5) Reserved for mistakes that disturb the routine in an extreme way, such as a wild throwaway, a long, embarrassing break in the routine, or an incident that clearly endangers the audience. Judges are cautioned to make a Severe Error deduction only when the audience is clearly endangered, not just when players perform near the audience. Catches near the audience may add to a routine’s excitement without endangering spectators.
I noticed something at Paganello while helping Lui tabulate the results of the Co-op finals. A judge gave three teams in my pool a Severe Error – including one in my routine with Gery Nemeth and Balu Major. As implied in the definition above, Severe Errors are most often wild throaways, discs thrown out of the reach of the teammate. I was surprised because I didn’t remember any throwaways during our performance. I’ll have to wait until the videos hit YouTube to remind myself of what happened during our five minutes.
I bring this up not as a sob story. We would have placed the same with or without that deduction. I bring this up because Severe Errors are rare, especially indoors, and yet three teams were given the Severe Error deduction during the finals, so it’s a good window into how judges perceive the judging system and the routine they watch.
Judging competency is essential. As so many players emerge into the competitive scene, it’s super important that judges are trained and that players can expect to be evaluated consistently. One of the best ways to improve our judging competency is through discussion. So let’s talk about the Severe Error. When do you give them, what’s your threshhold for the difference between Major and Severe Error? What do we need to do to communicate the difference to new judges?
I give Severe Errors rarely because that’s the intention of the judging system. They are Severe. Most teams don’t stray into Severe territory. They may make mistakes, but most don’t make gigantic, severe mistakes that match the definition of this deduction.
For throwaways, I adhere very strictly to the definition. It’s got to be a wild throwaway. The teammate doesn’t even have a chance to touch the disc. Someone used to use the phrase “throw to the ghost,” like the disc was thrown to someone not even on the team.
Regular bad throws usually get a 0.2 or 0.3 from me. If the disc is dropped or doesn’t reach the teammate, but it’s not a huge, embarrassing error, I give a 0.3. Occasionally, I’ll downgrade for 0.2 if someone is able to seamlessly pick or kick the disc back into play.
The other scenarios for Severe Error rarely happen. I personally have not seen a player endanger the audience. I’ve seen many flow interruptions, but the play hardly ever stops long enough for it to be a Severe Error.
What kind of disruptions would merit a Severe Error if I were a judge? Let’s say the team not only forgot where they were in the routine but started a long conversation about what to do. After about 10 seconds of absolutely nothing going on, I’d probably give the Severe Error. Usually, teams make some sort of throw before that, so this situation rarely comes up.
How about teams that endanger the audience? For me, the line that distinguishes between aggressive play, going after a disc off the field, and a Severe Error is body control. If the player is tracking the disc AND the audience, and if they make good decisions about where to run/jump, it’s not a severe error for this judge. But, if the player recklessly runs into the audience with a great chance of collision, or if they jump with little body control so there’s a very real possibility of them landing on an audience member, they get the 0.5 deduction.
I want to make an important distinction. There are players who can play into an audience without endangering them. It’s probably not the best situation, but I’ll let the Artistic Impression judges deal with that. The point is: the player isn’t endangering the audience by doing it, and that is the essential difference between a Severe Error and a questionable decision that’s not really an error.