This paper provides deterministic approximation results for stochastic processes that arise when finite populations recurrently play finite games. The processes are Markov chains, and the approximation is defined in continuous time as a system of ordinary differential equations of the type studied in evolutionary game theory. We establish precise connections between the long‐run behavior of the discrete stochastic process, for large populations, and its deterministic flow approximation. In particular, we provide probabilistic bounds on exit times from and visitation rates to neighborhoods of attractors to the deterministic flow. We sharpen these results in the special case of ergodic processes.