A method based on separated integration to estimate anharmonic corrections to energy and vibration of molecules in a second-order diagrammatic vibrational many-body Green’s function formalism has already been presented. A severe bottleneck in extending this approach to bigger molecules is that the storage of the Green’s function scales exponentially with the number of atoms in the molecule. In this article, we present a method that overcomes this limitation by approximating the Green’s function in the Hierarchical Tucker tensor format. We illustrate that the storage cost is linear in dimension and hence one can obtain accurate representations of the Green’s function for a molecule of any size. Application of this method to estimate the second-order correction to energy of molecules illustrates the advantage of this approach.