Model error estimation remains one of the key challenges in uncertainty quantification and predictive science. For computational models of complex physical systems, model error, also known as structural error or model inadequacy, is often the largest contributor to the overall predictive uncertainty. This work builds on a recently developed frame-work of embedded, internal model correction, in order to represent and quantify structural errors, together with model parameters, within a Bayesian inference context. We focus specifically on a polynomial chaos representation with additive modification of existing model parameters, enabling a nonintrusive procedure for efficient approximate likelihood construction, model error estimation, and disambiguation of model and data errors' contributions to predictive uncertainty. The framework is demonstrated on several synthetic examples, as well as on a chemical ignition problem.