Abstract
Based on the stats of 3P shots in the early season of the NBA, I used Bayesian inference to predict the possible range of the final 3P% stats.
The results of the calculation were not so bad qualitatively, but there were some issues.
Is the width of the credible interval practical?
Is it possible to make predictions in the case of extreme values?
Is the actual value in the confidence interval in the same proportion as the proportion in the calculated confidence interval?
In short, there is no evaluation metric to determine which prediction is the best.
Intro
The other day, I saw the following article.
It is a very interesting approach to predicting the final stats by padding the noisy early season results with league-wide numbers.
However, there is a point of concern, which the predicted value is always a single value(point estimation), and the certainty of that value is not known.
For example, if a simulation predicts that this player’s 3P% at the end of the season will be 35%, it doesn’t mean that there is a 100% chance that it will be 35%. It may be 34% or 36%.
The question then becomes, what is the probability that it will be 35%, or 34% or 36%, but the above method cannot answer that question.
So I’m going to try to predict the final 3P% based on the early season results while using a Bayesian inference, which can simulate the interval of 3% with possibility, to eliminate the above shortcomings.
Analysis approach
In order to use Bayesian estimation, we need to set up a prior distribution.
Let’s say p is the 3P% we want to predict, and p follows a normal distribution , and p is between 0.2 and 0.5. Then, depending on the p, we set the Bernoulli distribution to generate the success or failure of the 3P shot.
The stan code to express above settings as below(I use R and stan to caluculate).
data {
int N;
int Y[N];
real prior_mean;
real prior_sd;
}parameters {
real<lower=0.2,upper=0.5> p;
}model {
p ~ normal(prior_mean, prior_sd);
for (n in 1:N)
Y[n] ~ bernoulli(p);
}
Then we need to set normal distribution parameters, mean and standard deviation(sd).
I make three parameter patterns as below.
Pattern1
mean : past two seasons’ 3P% average, which results from top 100 player who shot more 3P than others.
sd : past two seasons’ 3P% standard deviation, which results from top 100 player who shot more 3P than others.
* I check the distribution of 3P% in graph below and think this looks like normal distribution,
Pattern2
mean : each player’s last seasons’ 3P%.
sd : calculate square root of “last season 3P% * (1-last season 3P%) * 3P attempt num in last season” and divide by 3P attempt num in last season.
* It is calculated by approximating the normal distribution from the binomial distribution using the central limit theorem and dividing it by the number of trials n.
Pattern3
mean : each player’s last seasons’ 3P%.
sd : past two seasons’ 3P% standard deviation, which results from top 100 player who shot more 3P than others.
In short, pattern1 uses entire players’ stats, pattern2 uses each player’s own stats, and pattern3 is mixture of patten1 and pattern2.
In addition to these information, I use 2017–18 and 2018–19 regular season and 2019–20 early time data(by 2019–11–17) to predict 2019–20 final stats.
Result
The accuracy of predict
By running the code, I calculate 3P% interval like this.
The 3P% is between 35% and 40% by 80% possibility
(35% is 10% tile num and 40% is 90% tile num.)
So I want to check if each player’s 3P% is within the predicted range. If the ratio of within players is similar to the possibility(80% in this example), then the simulation is good.
(Calculating 10 players’ 3P% interval by 80% possibility, and 8players’ final 3P% being within the interval is good. If 10 players are within the interval, it’s no information, and if 3 players are within, it’s too much filtered.)
Here are each patterns’ within ratio by 80% and 50% possibility.
Pattern1 (40 players to predict)
80% possibility : 30 players are within (75.0%)
50% possibility : 20 players are within (50.0%)Pattern2 (38 players to predict)
80% possibility : 26 players are within (68.4%)
50% possibility : 17 players are within (44.7%)
* 2 players have no 3PA last season, so I exclude in pattern2 and 3.Pattern3 (38 players to predict)
80% possibility : 28 players are within (73.7%)
50% possibility : 20 players are within (52.6%)
It seems that pattern1 is more accurate than other patterns.
It is natural, however, that each player’s own 3P skills, in other words last season 3P%, is useful information. For example, a player whose 3P% is 40% last year, much better than league 3P%, and if his 3P% in early time is 42%, we think this stats is his own skill rather than this is extreme value.
So it is needed to check if pattern2 and 3 predict those who made extreme 3P% in early time.(It will be listed in the last.)
The range of credible interval
Next, I check the range of credible interval by the num of 3PA and each pattern.
Here are the graph to show above data.
Pattern1 and 3 have same trend, around 7% range by 80% possibility and 4% range by 50%.
Pattern2 is much more scattered than other patterns, because the standard deviation is different by each players(the fewer last season’s 3PA, the bigger the standard deviation and range of credible interval.)
I don’t know the range is narrow enough to use in practical situation, but I think it is not too wide.
Who is predictable?
In the last, I check who is predicted correctly by each patterns.
Here are tables which include each players’ 3PA, 3PM and 3P% by early time and final time, and flag which mean final 3P% is within 80% tile.
By comparing each tables,within_80per_flag in pattern1 and 3 looks biased in the middle(a bit more in pattern1, I think).Those of pattern2 looks uniformly distributed.
It seems that pattern2 is good at predicting 3P% which the player’s final stats ,who have extreme 3P% in early time.
However, each topic I write about has the same problem, how to evaluate the patterns.
The interpretations I said above is qualitative, and in order to decide which pattern we should select, it needs quantitative metrics.
To devise or come up with quantitative metrics is future prospects, so if someone devise or come up with metrics to evaluate predicted method with range, I’d like to discuss it!
Here is my Twitter account. If you are interested, please follow me.
R and stan code is below.
