In the framework of a univariate linear regression model, one can
be interested in testing two different groups of hypotheses about
,
and
. In the first group, the user has
some prior knowledge about the value of
, for example he
believes
, then he is interested in knowing whether
this value,
, is compatible with the sample data. In this
case the null hypothesis will be
, and
the alternative
. This is what is
called a two sided test. In the other group, the prior
knowledge about the parameter
can be more diffuse. For
example we may have some knowledge about the sign of the
parameter, and we want to know whether this sign agrees with our
data. Then, two possible tests are available,
against
, (for
this would be a test of positive sign); and
against
, (for
this would be a test of negative sign). These are the
so called on sided tests. Equivalent tests for
are
available.
The tool we are going to use to test for the previous hypotheses is the sampling distribution for the different estimators. The key to design a testing procedure lies in being able to analyze the potential variability of the estimated value, that is, one must be able to say whether a large divergence between it and the hypothetical value is better ascribed to sampling variability alone or whether it is better ascribed to the hypothetical value being incorrect. In order to do so, we need to know the sampling distribution of the parameters.
In section 1.2.6, equations (1.52) to (1.58)
show that the joint finite sample distribution of the OLS
estimators of and
is a normal density. Then, by
standard properties of the multivariate gaussian distribution (see
Greene (1993), p. 76), and under assumptions (A.1) to (A.7)
from Section (1.2.6)it is possible to show that
then, by a standard transformation
is standard normal. is unknown and therefore the
previous expression is unfeasible. Replacing the unknown value of
with
(the unbiased estimator of
) the result
is the ratio of a standard normal variable (see (1.63)) and
the square root of a chi-squared variable divided by its degrees
of freedom (see (1.61)). It is not difficult to show that
both random variables are independent, and therefore in
(1.65) follows a student-t distribution with
degrees
of freedom (see Johnston and Dinardo (1997), p. 489 for a proof). i.
e.
To test the hypotheses, we have the following alternative procedures:
Null Hypothesis | Alternative Hypothesis | |
a) Two-sided test |
H![]() |
H![]() |
b) one-sided test | ||
Right-sided test |
H![]() |
H![]() |
Left-sided test |
H![]() |
H![]() |
According to this set of hypotheses, next, we present the steps for a one-sided test, after this, we present the procedure for a two-sided test.
One-sided Test
The steps for a one-sided test are as follows:
If the calculated -statistic
falls in the critical
region, then
. In that case the null
hypothesis is rejected and we conclude that
is
significantly greater than
The -value Approach to Hypothesis Testing
The -statistic can also be carried out in an equivalent way.
First, calculate the probability that the random variable
(
-distribution with
degrees of freedom) is greater than
the observed
, that is, calculate
This probability is the area to the right of in the
-distribution. A high value for this probability implies that
the consequences of erroneously rejecting a true
is severe.
A low
-value implies that the consequences of rejecting a true
erroneously are not very severe, and hence we are "safe" in
rejecting
. The decision rule is therefore to "accept"
(that is, not reject it) if the
-value is too high. In
other words, if the
-value is higher than the specified level
of significance (say
), we conclude that the regression
coefficient
is not significantly greater than
at
the level
. If the
-value is less than
we
reject
and conclude that
is significantly greater
than
. The modified steps for the
-value approach are
as follows:
If we want to establish a more constrained null hypothesis, that
is, the set of possible values that can take under the
null hypothesis is only one value, we must use a two-sided test.
Two-sided Test
The procedure for a two-sided alternative is quite similar. The steps are as follows:
because of the symmetry of the -distribution around the origin.
The different sets of hypotheses and their decision regions for
testing at a significance level of can be summarized in
the following table:
Test | Rejection region for
H![]() |
Non-rejection region
for
H![]() |
Two-sided |
![]() |
![]() |
right-sided |
![]() |
![]() |
left-sided |
![]() |
![]() |
We implement the following Monte Carlo experiment. We generate one
sample of size n = 20 of the model
.
has a uniform distribution generated as
follows
, and the error term
. We
estimate
,
,
. The program gives the
three possible test for
when
, showing the
critical values and the rejection regions.
The previous hypothesis-testing procedure is confined to the slope
coefficient, . In the next section we present the process
based on the fit of the regression
In this section we present an alternative view to the two sided
test on that we have developed in the previous section.
Recall that the null hypothesis is
against
the alternative hypothesis that
.
In order to implement the test statistic remind that the OLS
estimators, and
, are such that they
minimize the residual sum of squares (RSS). Since
,
equivalently
and
maximize the
, and
therefore any other value of
, leads to a relevant loss
of fit. Consider, now, the value under the null,
rather
than
(the OLS estimator). We can investigate the
changes in the regression fit when using
instead of
. To this end, consider the following residual sum of
squares where
has been replaced by
.
Then, the value of ,
, that minimizes
(1.67) is
Substituting (1.68) into (1.67) we obtain
Doing some standard algebra we can show that this last expression is equal to
and since
and defining
then (1.70) is equal to
which is positive, because must be smaller than
,
that is, the alternative regression will not fit as well as the
OLS regression line. Finally,
where
is an F-Snedecor distribution with
and
degrees of freedom. The last statement is easily proved
since under the assumptions established in Section 1.2.6 then
and
The proof of (1.73) is closed by remarking that (1.74) and (1.75) are independent.
The procedure in the two-sided test
With the same data of the previous example, the program computes
the hypothesis test for
by using the regression
fit. The output is the critical value and the rejection regions.
As in Section 1.3.1, by standard properties of the multivariate gaussian distribution (see Greene (1993), p. 76), and under assumptions (A.1) to (A.7) from Section (1.2.6) it is possible to show that
The construction of the test are made similar to the test of
, a two- or one-sided test will be carried out:
1)Two-sided test
2) Right-sided test
3) Left-sided test
If we assume a two-sided test, the steps for this test are as follows
With the same data of the previous example, the program gives the
three possible tests for
when
, showing
the critical values and the rejection regions.
Although a test for the variance of the error term is
not as common as one for the parameters of the regression line,
for the sake of completeness we present it here. The test on
can be obtained from the large sample distribution of
,
Using this result, one may write:
which states that percent of the values of a
variable will lie between the values that cut off
percent in each tail of the distribution. The critical values are
taken from the
distribution with
degrees of
freedom. Remember that the
is an asymmetric distribution.
The
percent confidence interval for
will
be:
Now, similar to test the coefficients of the regression, we can
consider a test for the significance of the error variance
. The steps are as follows: