There have been many recent advances in the simulation of biologically realistic systems, but controlling these systems remains a challenge. In this thesis, we focus on methods for learning to control these systems without prior knowledge of the dynamics of the system or its environment. We present two algorithms. The first, designed for quasistatic systems, combines Gaussian process regression and stochastic gradient descent. By testing on a model of the human mid-face, we show that this combined method gives better control accuracy than either regression or gradient descent alone, and improves the efficiency of the optimization routine. The second addresses the trajectory-tracking problem for dynamical systems. Our method automatically learns the relationship between muscle activations and resulting movements. We also incorporate passive dynamics compensation and propose a novel gain-scheduling algorithm. Experiments performed on a model of the human index finger demonstrate that each component we add to the control formulation improves performance of fingertip precision tasks.