We examine the second order properties of various quantities of interest in the partially linear regression model. We obtain a stochastic expansion with remainder $O_P(n^{-2\mu})$, where $\mu < 1/2$, for the standardized semiparametric least squares estimator, a standard error estimator, and a studentized statistic. We use the second order expansions to correct the standard error estimates for second order effects, and to define a method of bandwidth choice. A Monte Carlo experiment provides favorable evidence on our method of bandwidth choice.