We prove that any action of a finite dimensional Hopf algebra H on a Weyl algebra A over an algebraically closed field of characteristic zero factors through a group action. In other words, Weyl algebras do not admit genuine finite quantum symmetries. This improves a previous result by the authors, where the statement was established for semisimple H. The proof relies on a refinement of the method previously used: namely, considering reductions of the action of H on A modulo prime powers rather than primes. We also show that the result holds, more generally, for algebras of differential operators. This gives an affirmative answer to a question posed by the last two authors.