Branching processes in varying environment (BPVEs) are a natural extension of Galton-Watson processes where the offpsring distribution depends on time. In this work, we define a notion of near criticality for BPVEs. We show that the genealogy of a nearly critical BPVE, viewed as a random metric space, converges to a limiting tree in the Gromov-Hausdorff-Prohorov topology. This limit is expressed as a time-change of the corresponding limit for Galton-Watson processes: the Brownian coalescent point process. This work also illustrates and extends a general approach to tackle convergence of genealogies for branching processes which I will discuss. It relies on computing the so-called moments of the genealogy using a many-to-few formula.