We consider the problem of making asymptotically valid inference on structural parameters in instrumental variables regression with weak instruments. Using the local-to-zero asymptotics of Staiger and Stock (1997), we derive the asymptotic distributions of LR and LM type statistics for testing simple hypotheses on structural parameters based on maximum likelihood and generalized method of moments estimation methods. In contrast to the nonstandard limiting behavior of Wald statistics, the limiting distributions of certain LM and LR statistics are bounded by a chi-square distribution with degrees of freedom given by the number of instruments. Further, we show how to construct asymptotically valid confidence sets for structural parameters by inverting these statistics.