Math330 Additional Exercises 2

This sheet gives additional exercises covering all of the course.

1.

Let $X_{1},\ldots,X_{n}$ be independent with $X_{i}\sim N(\alpha+\beta z_{i},1)$ for known constants $\{z_{i}\}$ .

Calculate the log-likelihood $\ell(\alpha,\beta)$ .

Hence calculate the MLE, $(\hat{\alpha},\hat{\beta})$ .

Calculate also the Fisher’s information matrix $I_{E}(\alpha,\beta)$ .

Give a condition on $\{z_{i}\}$ which leads to the orthogonality of $\alpha$ and $\beta$ . (2006 A2)
2.

Suppose $X_{1},\dots,X_{n}$ are IID Normal $(\mu,\sigma^{2})$ random variables with ${\bm{\theta}}$ $=(\mu,\sigma)$ unknown. Find
- (a)
  
  the maximum likelihood estimator, $\hat{\bm{\theta}}$ , of ${\bm{\theta}}$ ;
- (b)
  
  the maximum likelihood estimator of $\Pr(\displaystyle\sum^{n}_{i=1}X_{i}<10)$ ;
- (c)
  
  the asymptotic variance matrix for $\hat{\bm{\theta}}$ by using the observed information at $\hat{\bm{\theta}}$ ;
- (d)
  
  an approximate 95% confidence interval for $\mu$ .
(1997 A3)
3.

Consider the normal model

$f(x|\mu,\sigma)=(2\pi\sigma^{2})^{-\frac{1}{2}}\exp\left\{-\frac{(x-\mu)^{2}}{% 2\sigma^{2}}\right\}$

where $-\infty<x<\infty$ . Find an expression for $(\hat{\mu},\hat{\sigma})$ and calculate its asymptotic distribution.

Particular interest lies in the parameter $\phi=\frac{\mu}{\sigma^{2}}$ . Calculate $\hat{\phi}$ and its asymptotic distribution. (2006 B3 part)
4.

Assume that $X_{1},\ldots,X_{n}$ are independent random variables with a Normal( $\mu$ , $\sigma^{2}$ ) distribution. It is well known that, in this case, the maximum likelihood estimator of ${\bm{\theta}}=(\mu,\sigma)$ is given by

$\hat{\mu}=\overline{X}=\frac{\sum_{i=1}^{n}X_{i}}{n}$

and

$\hat{\sigma}=\left(\frac{\sum_{i=1}^{n}(X_{i}-\overline{X})^{2}}{n}\right)^{1/2}$

Furthermore the expected information matrix is

$I_{E}(\mu,\sigma)=\left(\begin{array}[]{cc}n/\sigma^{2}&0\\ 0&2n/\sigma^{2}\end{array}\right).$
- (a)
  
  Find the asymptotic distribution of the maximum likelihood estimator of ${\bm{\theta}}=(\mu,\sigma)$ .
- (b)
  
  Briefly comment on the implications that the diagonality of the information matrix $I_{E}(\mu,\sigma)$ has for parameter estimation.
- (c)
  
  Give the maximum likelihood estimator of $\phi=g({\bm{\theta}})=\sigma/\mu$ .
- (d)
  
  Find the asymptotic distribution of the maximum likelihood estimator of $\phi=g({\bm{\theta}})=\sigma/\mu$ , stating the general result used.
(2005 A3)
5.
Suppose $x_{1},\ldots,x_{n}$ are observations independently drawn from

$X_{i}\sim{\rm Poisson}\Bigl{(}\mu_{i}\Bigr{)}$

where $i=1,\ldots,n$ , and $\mu_{i}=\exp\Bigl{(}\alpha w_{i}+\beta z_{i}\Bigr{)}$ , and $w_{i}$ and $z_{i}$ are two known covariates associated with observation $i$ .
- (a)
  
  Explain why $\exp(\alpha w_{i}+\beta z_{i})$ is a more suitable specification for $\mu_{i}$ than $\alpha w_{i}+\beta z_{i}$ .
- (b)
  
  Write down the log-likelihood function, $l(\alpha,\beta)$ .
- (c)
  
  Show that the inverse of the expected information matrix is given by
  
  $I_{E}(\alpha,\beta)^{-1}=\frac{1}{\Delta(\alpha,\beta)}\left(\begin{array}[]{% cc}\sum_{i}z_{i}^{2}\exp(\alpha w_{i}+\beta z_{i})&-\sum_{i}w_{i}z_{i}\exp(% \alpha w_{i}+\beta z_{i})\\ -\sum_{i}w_{i}z_{i}\exp(\alpha w_{i}+\beta z_{i})&\sum_{i}w_{i}^{2}\exp(\alpha w% _{i}+\beta z_{i})\end{array}\right),$
  
  where
  
  $\Delta(\alpha,\beta)=\Bigl{(}\sum_{i}w_{i}^{2}\exp(\alpha w_{i}+\beta z_{i})% \Bigr{)}\Bigl{(}\sum_{i}z_{i}^{2}\exp(\alpha w_{i}+\beta z_{i})\Bigr{)}-\Bigl{% (}\sum_{i}w_{i}z_{i}\exp(\alpha w_{i}+\beta z_{i})\Bigr{)}^{2}.$
  
  Note: You may use the following result
  
  $\left(\begin{array}[]{ll}a&b\\ c&d\end{array}\right)^{-1}=\frac{1}{ad-bc}\left(\begin{array}[]{cc}d&-b\\ -c&a\end{array}\right).$
- (d)
  
  Find the asymptotic distribution of the maximum likelihood estimator of $\mu_{1}$ , where $\mu_{1}$ is the expected value of $X_{1}$ . Quote all results used.
(2004 B1)