Module 11: Intuition: p-values vs critical values


Module 11: Intuition: p-values vs critical values

Author
Message
NEAS
Supreme Being
Supreme Being (5.7K reputation)Supreme Being (5.7K reputation)Supreme Being (5.7K reputation)Supreme Being (5.7K reputation)Supreme Being (5.7K reputation)Supreme Being (5.7K reputation)Supreme Being (5.7K reputation)Supreme Being (5.7K reputation)Supreme Being (5.7K reputation)

Group: Administrators
Posts: 4.2K, Visits: 1.2K

Module 11: Statistical inference for simple linear regression

 

(The attached PDF file has better formatting.)

 

Intuition: p-values vs critical values

 

Jacob: We often speak of rejecting a null hypothesis at a 95% or a 90% confidence interval.  We can also phrase hypothesis testing with p-values.  Which is better?

 

Rachel: Statisticians prefer p-values.  If we reject a null hypothesis at a 5% significance level, we don’t know if the p-value is 5.1%, and the null hypothesis probably ought to be rejected (or viewed with suspicion) or the p-value is 50%, and the null hypothesis should not be rejected.  A 5% significance level is an arbitrary choice; it has no greater justification than a 6% level or a 4% level or any other level.  Yet social scientists sometimes speak of regression results as absolutes; they say a certain result is significant or is not significant.  This is misleading.

 

Jacob: If the p-value is better, why do we use arbitrary confidence intervals?

 

Rachel: A lay person may have trouble interpreting a p-value.  Suppose we want to know if women or more likely than men to vote for one of two candidates in an election.  If we say “the p-value is 8.2%,” the listener says: “What does that mean?”  Explaining the statistical meaning to a lay person may not help. So we choose a significance level and say: “yes” or “no.”  The listener may not realize that we could change “yes” to “no” by changing the significance level.

 

Jacob: For actuaries, is the p-value a good measure?

 

Rachel: It is a better measure than a significant test, but it suffers from the same problems.  Listeners thinks we are testing the observed relation between the X and Y variables, but we are only testing the null hypothesis.  In many regression analyses, we are confident that â is not zero, but we don’t know its true value.

 

Illustration: Suppose we are determining the inflation rate, the interest rate, or a loss cost trend.  We know that the trend is not 0%, but we don’t know its true value, such as 8%, 9%, or 10%.  A p-value is no help.  If the observed trend is 8.7%, the p-value may be 0.01%.  This doesn’t tell us that the trend is 8.7%; it says that the trend is not 0%, which we know.

 

Jacob: Is a confidence interval better?

 

Rachel: It is better to say that we are P% confident that the true trend is between 8.7% – z and 8.7% + z.

 

Jacob: This seems like a good statement; it answers our concerns about the true trend.

 

Rachel: Not necessarily.  We want to know the current trend.  But the statistical statement says the following: “If the trend has been stable over the experience period, and any observed differences over the years stem solely from sampling error, then the true trend is between 8.7% – z and 8.7% + z.”  Our listeners respond: “We do not assume the trend is the same every year. It may change from year to year. We want to know the best estimate of the current trend.”

 

Jacob: Isn’t the ordinary least squares estimator the best estimate of the current trend?

 

Rachel: Suppose we have 11 years with trends of 8.0%, 8.2%, 8.4%, …, 9.8%, and 10.0%.  The standard trend analysis gives an ordinary least squares estimator of 9.0%.  Our listeners are likely to reject this in favor of a 10.0% current trend.

 

Jacob: For trend analyses, should we should use the most recent value?

 

Rachel: Suppose we examine 11 years, and we find trends of

 

9.0%, 8.2%, 8.8%, 9.8%, 8.4%, 9.6%, 8.6%, 9.4%, 8.0%, and 10.0%.

 

We ascribe the differences to sampling error, and we choose a trend of 9%, not 10%.

 

Jacob: How do we choose between these two scenarios?

 

Rachel: The time series course deals with this choice.  The first scenario is a random walk, and the second scenario is white noise.  The time series question is “How much of the observed annual differences is the drift of a random walk and how much is sampling error of white noise?”

 

 


Attachments
GO
Merge Selected
Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...





Reading This Topic


Login
Existing Account
Email Address:


Password:


Social Logins

  • Login with twitter
  • Login with twitter
Select a Forum....











































































































































































































































Neas-Seminars

Search