We provide a significant extension of the twisted connected sum construction of G_2-manifolds, i.e. Riemannian 7-manifolds with holonomy group G_2, first developed by Kovalev; along the way we address some foundational questions at the heart of the twisted connected sum construction. Some of the main contributions of the paper are: (i) We correct, clarify and extend several aspects of the K3 "matching problem" that occurs as a key step in the twisted connected sum construction. (ii) We show that the large class of asymptotically cylindrical Calabi-Yau 3-folds built from semi-Fano 3-folds (a subclass of weak Fano 3-folds) can be used as components in the twisted connected sum construction. (iii) We construct many new topological types of compact G_2-manifolds by applying the twisted connected sum to asymptotically Calabi-Yau 3-folds of semi-Fano type studied in arXiv:1206.2277. (iv) We obtain much more precise topological information about twisted connected sum G_2-manifolds; one application is the determination for the first time of the diffeomorphism type of many compact G_2-manifolds. (v) We describe "geometric transitions" between G_2-metrics on different 7-manifolds mimicking "flopping" behaviour among semi-Fano 3-folds and "conifold transitions" between Fano and semi-Fano 3-folds. (vi) We construct many G_2-manifolds that contain rigid compact associative 3-folds. (vii) We prove that many smooth 2-connected 7-manifolds can be realised as twisted connected sums in numerous ways; by varying the building blocks matched we can vary the number of rigid associative 3-folds constructed therein. This leads to speculation that the moduli space of G_2-metrics on a given 7-manifold may consist of many different connected components.