The Performance Method

An Improved Archer Ranking System for Determining a “Shooter of the Year”

Tim Wilson https://timwilson.info/ (Minnesota Archers Alliance)https://themnaa.org/
April 26, 2019

Many archery organizations including the National Field Archery Association (NFAA), Archery Shooter’s Association (ASA), and numerous state-level archery associations have developed “Shooter of the Year” awards. Though their methods differ, the goal is to determine which archer (or archers in the case of multiple categories) is the highest performing competitor for a given year. This article describes a method that attempts to identify the best archer across a variety of archery disciplines while compensating for the difficulty of each discipline and isolating each individual performance from his or her peers.

Introduction

In many cases, the simplest approach is the best. In an archery league at a local club, for example, where all archers are shooting the same round, there’s little reason to do anything besides add up the participants’ scores to determine the winner. If, however, the league alternates rounds and shoots an NFAA indoor 360 round one week and a NFAA 450 Vegas round the next, some problems with the simple additive approach start to become apparent. It’s not uncommon for a given archer to be more proficient at one type of archery round than another. In the alternating league scenario where the final winner is determined by adding up all the scores, an archer who is better at the Vegas round would have an advantage over an archer who was better at the NFAA indoor round because he or she would accumulate points faster on a round with a possible score of 450 points versus a round with a possible score of 360 points.

In practice, the difference between the NFAA 360 and Vegas 450 rounds is relatively small. If one wanted to compare the performance of archers across a wider variety of disciplines including indoor and outdoor rounds, the problems with this approach are more apparent. Let’s examine some of the ways to mitigate those differences along with the advantages and disadvantages of each approach. We’ll start with the simple “additive” method as a baseline and then consider “ranked finish” and “weighted percentage” methods as improved, yet flawed, alternatives.

For the purpose of this discussion, let’s assume that a state archery association holds a variety of events throughout the year and wants to determine a “Shooter of the Year” (SotY) winner based on the archers’ performance in the following events: NFAA indoor round, USA Archery indoor round, NFAA field round, and USA Archery outdoor round. For the NFAA indoor round, we will add the score and X-count together to get the final score. The total points possible for each round are shown in the following table.

Table 1: Summary of archery rounds and total points possible.
Round Format Points Possible
NFAA Indoor 60 arrows (12 ends of 5 arrows) 360
USA Archery Indoor 120 arrows (40 ends of 3 arrows over two days) 1200
NFAA Field 112 arrows (28 targets with 4 arrows/target) 560
USA Archery Outdoor 144 arrows (24 ends of 6 arrows over two days) 1440

For the purpose of comparing the methods used to rank archers, we’ll use a group of four sample archers with scores from the four different events.

Table 2: Sample archers and their scores.
Archer NFAA Indoor USA Archery Indoor NFAA Field USA Archery Outdoor
Archer A 356 1,151 534 1,361
Archer B 346 1,130 539 1,318
Archer C 358 1,163 530 1,341
Archer D 351 1,141 544 1,313

Simple, But Faulty Approaches

The Additive Method

The most obvious (and most flawed) way to determine an archer’s overall ranking when events of different types are involved is to simply add all of the scores together. The NFAA uses an additive method, but adds some additional points for podium finishes in the professional divisions. While this method is easy to understand, the primary problem is that it unfairly weights events with a larger number of possible points. In our example scenario, the additive method would have the effect of weighting a USA Archery outdoor round four times more than an NFAA indoor round. It also means that an archer who missed the field event would be penalized four times more in the final rankings than an archer that missed the NFAA indoor round.

For the purpose of comparison, however, let’s see how the final rankings would work out using this approach.

Table 3: Results of using the additive method to determine archer rankings.
Rank Archer NFAA Indoor USA Archery Indoor NFAA Field USA Archery Outdoor Total
1 Archer A 356 1,151 534 1,361 3,402
2 Archer C 358 1,163 530 1,341 3,392
3 Archer D 351 1,141 544 1,313 3,349
4 Archer B 346 1,130 539 1,318 3,333

The Ranked Finish Method

An alternative method is to award points to individual archers based on the order of their finish in each of the competitions. The Formula One racing series uses this approach, awarding points for the top 10 finishers of each race. According to the regulations, the race winner is awarded 25 points, 2nd place 18 points, 3rd place 15 points, and continuing through the top ten with 12, 10, 8, 6, 4, 2, and 1 points. At the end of the racing season, the Formula One champion is the driver who has accumulated the most points during the season.

This approach has one great benefit over the additive method: each event has exactly the same weight in the final determination of the champion. There are, however, several significant drawbacks of the ranked finish method when applied to archery competitions. Since not every archer participates in every event during the year, an individual’s order of finish would be affected by the quality of the competing archers at any given event. This method would also make it impossible to compare archers across divisions. For example, an archer competing in the adult male freestyle division at an NFAA event couldn’t compare his scores to someone in the senior male freestyle division because they weren’t competing against the same archers.

The Percentage Method

If the additive method’s primary fault is in the different number of points possible at each event, one approach to solve that problem is to award points for the SotY competition based on the percentage of points possible earned by each archer. The percentage could be multiplied by 100 making a perfect score equal to 100 SotY points. For the USA Archery outdoor round where Archer A from our example shot 1,334 points out of 1,440 points possible, the SotY points would be given by the following formula:

\[ \begin{eqnarray} \text{Shooter of the Year Points} &=& \frac{\text{Score}}{\text{Points possible}} \times 100 \\ &=& \frac{1334}{1440} \times 100 \\ &=& 92.6 \end{eqnarray} \]

This is a great improvement on the additive method. It’s relatively easy to understand, and each event, no matter how many points are possible, would carry equal weight in the final shooter of the year calculation. Based on this approach, the points earned by each of our sample archers during the year and the final rankings are shown in Table 4.

Table 4: Results of using the percentage method to determine final archer rankings.
Rank Archer NFAA Indoor USA Archery Indoor NFAA Field USA Archery Outdoor Total
1 Archer A 98.9 95.9 95.4 94.5 384.7
2 Archer C 99.4 96.9 94.6 93.1 384.1
3 Archer D 97.5 95.1 97.1 91.2 380.9
4 Archer B 96.1 94.2 96.2 91.5 378.1

There are still a couple issues that make the percentage method less than ideal. First, this method fails to consider the relative difficulty of a given event. Scoring 90% in an NFAA Indoor round (e.g., 324/360 or 295 29X1) is very different than shooting 1,294/1,440 at an outdoor USA Archery event, yet both would earn the sample credit toward the shooter of the year rankings.

Another problem is that the percentage method has no way to take into account the nonlinear nature of archery performance as you approach a perfect score. For example, in practical terms, an improvement from 503 to 531 in a full 28-target NFAA field round is not nearly as significant as an improvement from 531 to 559 even though both increases equal 5% of the total 560 points.

Exploring a Better Approach

An improved method of ranking archers for a Shooter of the Year award would have the following qualities:

  1. Results in different events would be equally weighted in a manner that takes into the account the relative difficulty of shooting a perfect score in that event.
  2. The points earned by an archer at a particular event should be independent of the number and skill of the other competitors.
  3. The ranking method should be applicable across age divisions and genders as long as the different competitors are shooting the same rounds at the same distances. Different equipment classes would be considered separately. This would allow an organization to award a Shooter of the Year across a group of many different competitors in different age divisions or an award for each age division if they choose.

Decathlon scoring

A good model for this kind of approach has already been implemented for track and field’s decathlon and heptathlon. Like archery, the ten events of the decathlon vary in length and type making it necessary to have a method for awarding points in each event that doesn’t favor one event or style over the others. While no method is perfect, the decathlon scoring approach has been in place for many years, and its longevity speaks to its inherent fairness.

The decathlon consists of running events where the goal is to have the fastest time and throwing events where the goal is to have the longest distance. This requires two different formulas to calculate points for the different types of events. Points for running events are given by the following formula:

\[ \text{Track event points} = A \times (B - T)^C \] where \(A\), \(B\), and \(C\) are arbitrary factors derived from past events and \(T\) is the time recorded by the athlete. (\(B\) is the minimum time needed to score any points. A time slower than \(B\) earns 0 points toward the athlete’s point total.)

Field events use a similar formula:

\[ \text{Field event points} = A \times (D - B)^C \]

where \(D\) is the distance recorded and \(B\) is the minimum distance needed to score any points.

The values of \(A\), \(B\), and \(C\) are different for each event of the decathlon and are based on years of data from past events. While this approach seems complicated at first, it has many advantages:

  1. An athlete can look up the points earned from any result in any event by consulting a table (or a web site with a formula lookup function).
  2. It doesn’t matter who’s competing on any given day. A male running 100m in 10.0 seconds will always earn 1,096 points toward his total. A female who runs 200m in 20.0 seconds would earn 1,398 points toward her total in the heptathlon. (See ten7events.com for complete scoring calculators for the decathlon and heptathlon.)
  3. Any organization can stage an event using the standard decathlon scoring formulas, and the results will be consistent from event to event. That makes it possible to compare performances across the country or world.

An Archery Equivalent

Modifying the formula-based decathlon scoring method to work with an annual archery Shooter of the Year competition requires the following steps:

  1. Collect data from past competitions to serve as the baseline for the equations.
  2. Analyze the past data to derive an equation for each different archery round.
  3. Build the lookup tables to assign Shooter of the Year points for any given score.
  4. Crown your “Shooter of the Year”!

The Performance Method for Archery

If the goal, then, is to compare archers’ performances across disciplines and age divisions, we need a way to compare individual performances.

Archery is a performance, not a competition —George (GRIV) Ryals

In this section, we’ll work through the details of how the Performance Method was applied to the four archery rounds in our sample. Other types of rounds, or one-day versions of the two-day events could be calculated in the same way. The final equations will differ in form from the decathlon equations, but the basic approach is similar.

Collecting data

This method relies on a large sample of representative data on which to base the calculations. For this purpose, the last three years of results (2016–2018) were collected from NFAA Indoor Nationals, USA Archery Indoor Nationals, NFAA Outdoor Field Nationals, and USA Archery Outdoor Nationals. The dataset contains compound and recurve results for all adults (amateur and professional). Youth and senior (USA Archery master) results are not included. Compound and recurve results will be considered separately, and different equations will be derived for each. In other words, compound and recurve archers won’t be able to compare their results directly.

The total number of scores in the dataset are shown in the table below.

Table 5: Total number of scores in the dataset for each round and equipment class.
Total scores by class
Event Compound Recurve
NFAA Field Nationals 277 19
NFAA Indoor Nationals 1431 172
USA Archery Indoor Nationals 885 1161
USA Archery Outdoor Nationals 214 319

Starting the analysis

To develop an equation for each round and equipment class, we need to begin by understanding the underlying data. A standard histogram is a good place to start.

Two things are immediately apparent when you examine the distributions of scores for the four events. First, the difference in the number of participants in the compound and recurve classes at NFAA events is dramatic. Second, the shapes of the compound and recurve distributions are different too.

The recurve histograms are more like the typical “normal” distribution we’re familiar with from statistics class. The most common score lies somewhere in the middle with fewer scores as you move toward the tails of the histogram. The compound histograms are much different. They are weighted heavily toward the higher scores. This makes sense because the average compound score—especially for indoor events—is much closer to a perfect score than it is for recurve. It’s not unusual to see skewed distributions like this when there’s a fixed upper limit to the possible values (i.e. a maximum score).

We’ll see how the different shapes of the distributions affect the equations that are derived to generate the SotY points. In short, the more skewed the distribution toward the right, the more “progressive” the equation will be. In other words, an improvement in score near the top of the scale will produce a larger increase in points than an equivalent score improvement toward the bottom or middle of the scale. The more progressive curve will slope upward to the right in an “exponential” way. Distributions that are more “normal” will have a less progressive, even linear equation in some cases.

Setting the parameters

In order to ensure that the different events are weighted evenly, the method for deriving the points equation for each event and equipment class should be as consistent as possible. We’ll start with two rules that form the basis of this approach.

  1. A score at the 90th percentile of all adult scores should earn approximately 100 points toward the shooter of the year rankings.
  2. A score at the 10th percentile of all adult scores is the minimum score that will earn shooter of the year points.

Here’s a summary of the percentile data of the four events including the 50th percentile (median) value for comparison.

Table 6: Summary of percentiles for events by equipment class.
Compound
Recurve
Event 10th 50th 90th 10th 50th 90th
NFAA Field Nationals 504 538 555 398 463 496
NFAA Indoor Nationals 328 352 359 232 277 313
USA Archery Indoor Nationals 1,026 1,116 1,166 706 959 1,105
USA Archery Outdoor Nationals 1,148 1,328 1,398 860 1,096 1,252

We will use the 90th and 10th percentiles in the process of deriving equations that will produce the SotY points for a given score.

Setting the curves

Another method to examine the distribution of values in a sample of data is the cumulative distribution function or CDF. For each value in the dataset, the CDF calculates the probability that another random value would have a value less than or equal to the given value. Once determined, the CDF can be used to create a useful plot which describes the distribution of the sample data. Here are the same data from the histograms above, plotted as cumulative distribution functions.

Like in the histogram example, the difference between compound and recurve scores is obvious. The advantage of the CDF approach is that the resulting data can be fit to a line. First, however, we need to trim the data by removing the top and bottom 10% of scores based on our criteria above. The bottom 10% are removed in keeping with the requirement that archers need to score at least the 10th percentile to earn any shooter of the year points. By removing the top 10% of scores and recalculating the CDF, we can ensure that a score at the 90th percentile will earn approximately 100 points.

The new CDF plots based on the trimmed data are shown below.

Most of those curves will be fit to an equation of the form

\[ \text{Points} = 100 \times e^{a (\text{Score}-b)} \] where \(b\) is the highest score in the dataset for that specific event and equipment class and \(a\) is a parameter determined by the computer to fit to the data. Others will be fit to a simple linear model when the data fall mostly on a straight line.

Let’s consider a couple examples and see how the model is derived. The data from the compound and recurve classes at NFAA Indoor Nationals are quite different and will make a good example. In the graphs above you can see that the compound data curve sharply upward while the recurve data form a nearly a straight line. This is consistent with the fact that a relatively large number of participants in the compound class shoot perfect or nearly perfect scores at the event. The recurve results, as we’ve seen elsewhere, show a more typical distribution with an average somewhere in the middle of the range.

The calculated model fits well with the event scores, and it’s easy to see how the upward sloping graph rewards archers more as their scores approach a perfect 360 (i.e. 300 60X).

Once the model is determined, it’s easy to print a table that maps a certain score to the associated Shooter of the Year points.

Table 7: Shooter of the Year points table for the NFAA Indoor round (Compound).
Score Points Score Points Score Points Score Points
328 4.3 337 10.8 346 26.8 355 66.7
329 4.8 338 11.9 347 29.7 356 73.8
330 5.3 339 13.2 348 32.8 357 81.7
331 5.9 340 14.6 349 36.3 358 90.4
332 6.5 341 16.1 350 40.2 359 100.0
333 7.2 342 17.9 351 44.5 360 110.7
334 7.9 343 19.8 352 49.2
335 8.8 344 21.9 353 54.5
336 9.7 345 24.2 354 60.3

The approach for the NFAA Indoor recurve results is similar except that we’ll use a linear fit to build a model of the data