Who are the CFL's most successful runners?

Anyone who's watched a football game can tell you that not all yards are created equally.  QBs pile up yardage in failed comeback attempts, and running backs rack up the carries while teams protect a lead.  They might look the same on the score sheet at the end of the day, but there is a significant difference between an 8 yard gain on 2nd and 5, and an 8 yard gain on 2nd and 15.

Success Rate

Success rate is a simple metric that attempts to put a number on the difference between those plays - one of those plays is a successful one (it gained a first down), the other is not.

Your definition of success may differ from mine, but I've opted to define a successful running play as follows:

1) On first down, it gained at least 50% of the needed yards.

2) On second or third down, it gained 100% of the needed yards.

3) The runner did not fumble on the play.

In other words, a 5 yards on 1st and 10 is successful, but 5 yards on 2nd and 10 is not, and neither is a 15 yard run on 1st and 10 where the runner fumbled after gaining the yardage.  Possession of the fumble is not relevant, any fumble, recovered by the offense or not, is considered to be an unsuccessful play.  (Ask any coach and I think you'd find they agree.)

Success rate is shown as a percentage (successes / total attempts).  A rusher with a high yardage total and low success rate probably tends to have long runs mixed with frequent stops for short or no yardage.  A rusher with a low yardage total and high success rate is getting just enough to be successful, and not much more (perhaps indicative of a goal line QB or full back).

2014 Success Rates through Week 5

Through week 5 I've limited this list to running backs with at least 15 carries, and quarterbacks to those with at least 5 carries.  I will increase these value as the season goes on.

Success Rate - Running Backs (min 15 attempts)

Success Rate - Quarterbacks (min 5 attempts)

By itself, Success Rate doesn't tell the whole story about a runner (would anyone rather have Pat White's 100% success rate and 1.7 YPC than Tanner Marsh's 88% and 6.4 YPC?), but it does provide an interesting metric to add to the conversation.

2013 Success Rates

I intend to put up a page on cflstats.ca to display success rates for all seasons, but in the meantime, here are the values for last year.

Success Rate - Running Backs (minimum 50 attempts)

Success Rate - Quarterbacks (minimum 15 attempts)

Missing data status

At launch, data from 2009, 2010 and 2013 was available for the majority of games. All game boxscores for these years are in the system, but a small number are missing play data. Data for games in 2011 and 2012 is available, but has not yet been processed. 2014 data is being entered into the system on a weekly basis and will be kept up to date throughout the season. Data for 2011 is currently being processed as time allows, and 2012 data will follow. As games are completed they will become immediately available on the website. There is no time frame for completion, but I hope very much to have them ready before the start of the 2015 season. There is currently no work being done on games with missing play data, these games will remain flagged in the system (a note appears at the top of these games), and individual play data will remain unavailable until the back log of 2011 and 2012 games has been completed. This page will be updated as the status changes.

A deeper look at the Edmonton fake punt (July 24, 2014)

Some background: Expected Points and Expected Points Added

Expected points (EP), and more specifically, Expected Points Added (EPA), are metrics I use on http://www.cflstats.ca that has been used by NFL analyst Brian Burke for quite a few years now.

Using historical scoring data, we can assign a point value for every down/distance/line of scrimmage combination in a game.  By looking at every play and then the next score for either team, then grouping it by down/distance/LOS, we can come up with an average expectation for that play.  EP can be positive or negative, indicating whether we expect the offense (positive) or defense (negative) to score next.  In calculating the values, some game situations are filtered out in order to keep the values more accurate; plays which occur in the last 4 minutes of a half, or when the score margin is 14 points or more are not included in the calculations, in order to decrease the effect of garbage time or 2 minute drill type possessions.

Once we have a value for each game situation, we can then calculate EPA, which is simply the difference between EP on the next play and EP on the current play (EP After - EP Before). Positive EPA means the play moved the offence into a better position, negative means they are worse off than before.

Looking at EPA and comparing some possible outcomes can give us clues to whether in-game decisions were good or bad, or if risks were worth it.

The Play

On 3rd and 10 from their own 6 yard line with 26 seconds to go, Edmonton opted for a fake punt and gained just shy of the 10 yards necessary for the first down.  Calgary scored a touchdown on the next play, and Edmonton was left looking like they'd made a bad decision.

But was it really a bad decision?

Outcomes and potential EPA

3rd and 10 from your own 6 yard line is a bad place to be, and the EP value reflects that.  EP in this position is -1.3 points for the offense, meaning most of the time, the offense will give up the ball (or a safety) and the defense will be the next team to score.

Going into the play, they had three options: 

1) Punt - Edmonton averaged 38.7 net punt yards on the night, so punting from the 6 yard line would expect to give Calgary the ball back somewhere around the 44 yard line.  1st and 10 from their own 44 yard line carries an EP of -2.4 for the Edmonton defense.  Over the past 5 years, kickers have averaged 81% on field goal attempts from this range, which was the mostly likely scenario given the time remaining in the quarter.  EPA for this outcome would be -2.4 - -1.3 = -1.1

2) Go for it (and succeed) - Lets assume they got those extra needed inches, and kept the ball on their own 16 yard line.  That gives Edmonton 1st and 10, which carries an EP of 0.3.  In this situation though, Edmonton would certainly have opted to kneel out the quarter, so in actuality their EP for this case would be a flat 0 EP.  EPA for this outcome would be 0 - -1.3 = 1.3

3) Go for it (and fail) - Or exactly what happened, in other words. On average, teams on 3rd and 10 that go for it are successful 23% of the time, for an average gain of 6.3 yards.  Plugging in the average yardage gives Calgary back the ball on the Edmonton 12 yard line, for an EP of -4.0.  EPA for this outcome would be -4.0 - -1.3 = -2.7

The success rate for 2 and 3 are linked, meaning the true value of going for it must be calculated as a fraction of both, however.  Historically, teams have converted on 3rd and 10 just under 23% of the time, which includes fake punts.  That means the average EPA is a combination of the two:

EP_Success * SuccessRate + EP_Failure * FailureRate

For this situation, we get an average EPA of -1.78

To recap, that leaves us two outcomes - punt for an EPA of -1.1, or go for it, for an EPA of -1.78.  Those are surprisingly close.

Conclusion

In a vacuum, or as a standard third down gamble, going for it here is the wrong decision.  Both options are bad, as both indicate that Calgary is more likely to score next, but going for it is a little more than a half a point worse over all.

But, this didn't happen in a vacuum, time was a major factor here.  Ordinarily, gaining your team a first down on your own 16 yard line isn't worth all that much, especially when it comes at the risk of a -4.0 EPA swing.  But in this case, gaining the yardage would have allowed Edmonton to kneel out the quarter, giving up zero points, opposed to giving Calgary the ball back in position to kick a field goal from a spot on the field where kickers are fairly successful (81%).

And it wasn't a gamble, it was a fake punt.  Fake punts are a bit harder to quantify the success rate on, as they rely heavily on the element of surprise, and whether or not the team has found a weakness they hope to exploit.  I don't know the league average for success on fake punts, but I would wager that they are slightly more likely to succeed than a 3rd and long gamble, especially if the punting team has spotted something they think they can exploit.  Edmonton only needed to convert at a 40% rate in order to break even vs the punt.

It's a very close call on this one.  In a tie game, giving up the ball and a likely field goal could have been the difference in the game.  Conversely, giving up the ball inside your own 20 is a huge risk, but with less than 20 seconds left, there's a very good chance that most of the time, Calgary only walks away with a field goal here anyway.  

So should Edmonton have punted? I actually like the decision here: if the Edmonton coaches felt that the Calgary defense was unprepared or likely to cheat back to block, they may have felt their chances of succeeding were much higher than 40%, and after a good defensive half, they had to feel that they could hold Calgary to a short field goal in the case of a turnover.  Unfortunately, Edmonton's defense didn't hold, and the gamble resulted in the worst possible outcome, but I give credit to Chris Jones and his staff for making the aggressive choice at a reasonable time.

July 24th Update

CflStats.ca has been updated with a few minor enhancements:

1) Player and team pages now include "pass targets" (times a player made a catch or was targeted with a pass).  This data was always tracked, but for some reason not available on any pages.

2) Added a list of recent games to the front page for quicker access.

In addition, I've greatly improved the way that game day rosters are handled.  Going forward, the official game day roster will be used to determine which players were actually in the game, so the per game stats and player game logs should be much more accurate.  (Previously I used the transaction list to try and guess which players were available, but the transaction list is incomplete and results in some players appearing to still be on a team, when in fact they were inactive or even no longer on the roster.)  It will take some time to implement this into the games in the archive, but it will be done eventually.

/r/CFL Mathematical Rankings explained (And the Week 2 Math Rankings)

Many of the readers here were originally introduced to this blog via http://www.reddit.com/r/cfl, a great CFL-based community of which I am a regular participant.

One of the ways I participate is as one of the 10 voters (technically 9 currently, as we have no Edmonton voter) for the official /r/CFL Power Rankings.

The /r/CFL Power Rankings work in the usual way, we have a group of voters, and each voter ranks the teams at the end of each week.  The average of the votes determines a team's position on the list.  Most of the rankers vote from a team standpoint; they are designated as the official voter for their team, and they will contribute a short note regarding their team for the rankings.

My contribution is different, however.  The folks organizing the rankings decided that they would like to include my voice as well, and I'm happy to have the opportunity to contribute.  My rankings, as I'm not designated as a team ranker, are intended to be an unbiased vote, so as per my nature, my votes are math based.

For my vote, I've opted to use the Simple Ranking System, a new system I haven't talked about on this blog before (you can see each team's SRS rank on the standings page of www.cflstats.ca).

Simple Ranking System, or SRS, follows the same concept as Pythagorean Expectation and is based on the theory that points for and against are a better indicator of team strength than a team's actual record.  However, what Pythagorean Expectation and points for/against lack are adjustment's based on matchup.  The best team in the league beating the worst team in the league in a close game is both less impressive for the best team, and more impressive for the worst team.  SRS attempts to adjust results based on opponent rankings.

In basic terms, the formula for SRS is a team's average point margin, plus the average of their opponent's ratings.

I'll quote www.pro-football-reference/blog?p=37 for this part:

So every team's rating is their average point margin, adjusted up or down depending on the strength of their opponents. Thus an average team would have a rating of zero. Suppose a team plays a schedule that is, overall, exactly average. Then the sum of the terms in parentheses would be zero and the team's rating would be its average point margin. If a team played a tougher-than-average schedule, the sum of the terms in parentheses would be positive and so a team's rating would be bigger than its average point margin.

You can figure out any team's rating if you know their opponent ratings. Which sounds easy, except you can't know an opponent rating until you've figured out their own opponent rating.  Which brings you back to the first opponent, and leaves you in an infinite loop.

Fortunately, the loop will stabilize after a number of iterations.  On cflstats.ca, SRS is calculated first with an opponent adjustment of 0, then again once we've calculated them all once.  And then again and again, until the ratings stop changing.  Once the ratings stop changing, you have your ratings for the week.

How does this apply to the Power Rankings?

Simple.  My votes are simply the order of the teams ranked by SRS on www.cflstats.ca.  It's bias-free because I have no direct input on the process, and it provides a good way to contextualize a team's perfomance, especially in the early parts of the season when there aren't too many common opponents.

There's one caveat though: with a small sample size, the usefulness of a stat like this is reduced, as a single game makes up a significant portion of the rating and may be an outlier in the teams actual season.  Over time, those even out, but early on, they count too heavily.  So I have introduced an element of human intervention for the early part of the season.  It's not based on any tested math, it's just a means to avoid wild swings to a certain extent.  

Going into the season, I ranked the teams based on their expected win total change (from historical Pythagorean Expectation data).  After 1 game, a team's movement was capped at +/- 3 spots on the list (ie: the 9th place team on the list was limited to no higher than 6th position).  After 2 games, the cap was raised to +/- 6 positions.  After 3 games, the limit will be removed and SRS will be used directly.

The Week 2 Math Rankings

1) Winnipeg (SRS rank 1)

2) Toronto (SRS rank 3)

3) Calgary (SRS rank 4)

4) Saskatchewan (SRS rank 5)

5) Montreal (SRS rank 6)

6) Ottawa (SRS rank 2)

7) Edmonton (SRS rank 7)

8) Hamilton (SRS rank 8)

9) Montreal (SRS rank 9)

With the movement cap up to 6 for most teams this week (Calgary and Ottawa were restricted to +/- 3), the cap was mostly a non-issue, and only Ottawa was affected.  They started the season in 9th and weren't moved in the bye week, so despite a strong performance against what SRS thinks is the best team in the league, Winnipeg, they were moved down to 6th by the cap.

Some might find the rankings of 1-1 Toronto and 2-0 Edmonton to be rather odd, but they can be explained by opponent adjustments.  Toronto dominated a Saskatchewan team which had a very strong week 1 ranking, and that makes Winnipeg look that much better in week 1, and subsequently Toronto's loss to a strong Winnipeg team no longer hurts as much.  Likewise, while Edmonton is sitting pretty at 2-0, their two wins have come against the 8th and 9th ranked teams, both 0-2 with an average point differential of -23.5 between them.