Start or sit accuracy, explained: what the number means and how to check anyone's

The definition

Start or sit accuracy is the share of two player calls where the player you were told to start outscored the player you were told to sit. That is the whole idea. Everything else lives in the counting rules, and the counting rules are where most published numbers go soft.

Ours are written on the record page and were fixed before any game was graded: "A call is right when the player we said to start outscored the other one in your scoring format. A tie counts as neither." Ties drop out because a tie tells you nothing about which player to start. There were 3 in Week 1 and 6 in Week 2.

A player who does not play is treated differently. He scores zero, and the call is graded against that zero whichever side he was on. We keep those in, because your lineup took the zero too. The record page says 78 of the 595 graded calls were settled that way, and it says plainly that those results are real but do not test the comparison we made.

The baseline is 50 percent, not zero

A coin gets half of these right. So does a friend who always starts the more famous name. Any accuracy number has to be read against 50, and a number near 50 is not a skill yet.

The second thing to read is the sample. A rate built on a few dozen calls can land almost anywhere by luck. The standard check is a 95 percent margin of error. By the normal approximation it is 1.96 times the square root of p times (1 minus p), divided by n, where p is the hit rate and n is the number of decided calls.

Here it is on our Week 1, step by step.

1. p = 83 / 177 = 0.469, so 1 minus p = 0.531.
2. 0.469 times 0.531 = 0.249.
3. 0.249 divided by 177 = 0.00141.
4. The square root of 0.00141 is 0.0375.
5. 0.0375 times 1.96 = 0.0735, or about 7.4 points.

So Week 1 was 46.9 percent, plus or minus 7.4: anything from 39.5 to 54.2 percent fits what happened. A perfect coin over 177 calls wanders by the same 7.4 points either way. The margin shrinks only with the square root of the sample, so four times as many calls buys half the margin. That is why one week proves little and a season proves more.

Our record, the bad week first

Week 1: 83 right of 177, 46.9 percent. Worse than a coin. The margin says it could have been bad luck on a break even process, but it gave no sign of skill, and we are not going to argue it up.

The clearest weakness is on the record page itself: 140 of the 180 graded Week 1 calls were made when only one of the two players had a market-based projection, "comparing a real projection against a blank." Receivers took the worst of it, 28 right of 82 (34.1 percent, plus or minus 10.3). From Week 2 on, a pair is only published when both players have a market-based projection. We cannot prove that rule change explains the whole gap between the two weeks. It is the change we made.

Week 1 was also first graded from an incomplete box score feed. Every call was checked again against Sleeper's official stats, and five published results changed: two from right to wrong, three from wrong to right. All five are listed under Corrections on the record page, and the counts above include them.

Week 2: 262 right of 412, 63.6 percent. Same arithmetic: plus or minus 4.7, a range of 58.9 to 68.2 percent. The bottom of that range sits clear of 50.

The record today: 348 right of 595 graded, 58.5 percent. That is the headline on the record page. It includes 6 Week 3 calls graded after Thursday night's game, 3 of them right. Weeks 1 and 2 alone are 345 of 589, 58.6 percent.

By position

Published calls, Weeks 1 and 2 together, ties excluded.

Running backs are the only position whose whole range sits above 60 percent. Our working explanation is that a back's carries are the steadiest weekly role in the game, so the market-based projections read them well. These numbers are consistent with that. They do not prove it.

Receivers are the largest group and the hardest. At 54.5 percent on 266 calls, the range still dips below 50. The split tells the story: 34.1 percent in Week 1 on 82 calls, 63.6 percent in Week 2 on 184.

Quarterbacks and tight ends have margins of 11 and 15 points. Their rates are a guess with a sample attached. Kickers went 4 of 13, all in Week 1, and a range from 5.7 to 55.9 percent means 13 calls cannot tell you anything useful in either direction.

How to check anyone's number

Four questions sort a record from a highlight reel.

1. Does it publish the misses? Every graded call on our week pages says "We were wrong" when it was wrong, beside the ones we got right, and changed grades are listed as corrections.
2. Does it count every call, or only the featured ones? Ours counts all 1,539 calls we published. The page also shows the pairs our builder scored but never published: 655 right of 1,137 (57.6 percent). That number is lower than the headline, and it is on the page anyway.
3. Is the call locked before kickoff? Ours locks when the earlier of the two games kicks off and is never edited afterwards.
4. Was the grading rule stated in advance? If the rule appears only after the results, the number is a column, not a record.

Then run the arithmetic above on whatever figure survives. A source quoting 70 percent on 40 calls has a margin of about 14 points and has shown you less than one quoting 58 percent on 600.

What it means for your Sunday

A source right 60 percent of the time is right three times in five and wrong twice. Ours sits at 58.5 percent, with a range of 54.5 to 62.4. That is worth something on the calls where you are genuinely torn, and worth nothing on the calls where one player is obviously better. Start the obvious player without asking anyone.

The Week 3 calls are up now: 941 published pairs, each locked at kickoff and graded against Sleeper's official stats once Monday night is over.

Use a start or sit call only to break a genuine tie on your roster, and check its grade on Tuesday.