Quasi-free photoproduction of $\eta '$ mesons off nucleons bound in the deuteron has been measured with the combined Crystal Barrel - TAPS detector. The experiment was done at a tagged photon beam of the ELSA electron accelerator in Bonn for incident photon energies from the production threshold up to 2.5 GeV. The $\eta '$-mesons have been detected in coincidence with recoil protons and recoil neutrons. The quasi-free proton data are in good agreement with the results for free protons, indicating that nuclear effects have no significant impact. The coincidence with recoil neutrons provides the first data for the $\gamma n \rightarrow n\eta '$ reaction. In addition, also first estimates for coherent $\eta '$-production off the deuteron have been obtained. In agreement with model predictions, the total cross section for this channel is found to be very small, at most at the level of a few nb. The data are compared to model calculations taking into account contributions from nucleon resonances and $t$-channel exchanges.