Increasing the weight of difficulty
Many players express frustration about the high influence of execution scores and the comparatively small influence of difficulty scores on tournament results leading to a risk-averse type of play. Systematic analyses of tournament results have demonstrated that this is not just a perception by the competitors, but a fact: the committee calculated the standard deviations of all 3 judging categories for 72 pools from Paganello and FPAW 2009 and 2010.
Summing up, the data shows that Difficulty has a lower weight than Execution. AI, however, is the dominating category (see table below; more detailed results available if needed). If AI is 100% in terms of weight, than Execution would be 74% and Difficulty 61% in this sample.
Since nobody wants to increase the scoring weight of execution, the committee thinks that the best idea would be to figure in a way to increase the weight of Difficulty to the same level as that of AI. All committee members agreed that the best way to do this is to educate judges in order to use the whole difficulty rating scale, instead of mainly using scores from 3-7, to increase variance for Difficulty. However, since judging education is a long process, as well as requiring a lot of effort to fully accomplish, committee members agreed to implement a multiplier as a transitional measure to solve this issue mathematically until judges are better educated and subsequent analyses of categories indicate a wider variation of difficulty scores on routine score sheets. The multiplier shall increase the weight of Diff simply by multiplying the overall Difficulty scores of all teams by 1.5 when calculating the final results of a pool. This would bring the weight of Difficulty to 92% in the analysed sample, while AI would stay at 100% and Execution at 74%.