Conditional deep gaussian processes: Multi-fidelity kernel learning

Chi Ken Lu, Patrick Shafto

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


Deep Gaussian Processes (DGPs) were proposed as an expressive Bayesian model capable of a mathematically grounded estimation of uncertainty. The expressivity of DPGs results from not only the compositional character but the distribution propagation within the hierarchy. Recently, it was pointed out that the hierarchical structure of DGP well suited modeling the multi-fidelity regression, in which one is provided sparse observations with high precision and plenty of low fidelity observations. We propose the conditional DGP model in which the latent GPs are directly supported by the fixed lower fidelity data. Then the moment matching method is applied to approximate the marginal prior of conditional DGP with a GP. The obtained effective kernels are implicit functions of the lower-fidelity data, manifesting the expressivity contributed by distribution propagation within the hierarchy. The hyperparameters are learned via optimizing the approximate marginal likelihood. Experiments with synthetic and high dimensional data show comparable performance against other multi-fidelity regression methods, variational inference, and multi-output GP. We conclude that, with the low fidelity data and the hierarchical DGP structure, the effective kernel encodes the inductive bias for true function allowing the compositional freedom.

Original languageEnglish (US)
Article number1545
Issue number11
StatePublished - Nov 2021

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Mathematical Physics
  • Physics and Astronomy (miscellaneous)
  • Electrical and Electronic Engineering


  • Approximate inference
  • Deep Gaussian Process
  • Kernel composition
  • Moment matching
  • Multi-fidelity regression
  • Neural network


Dive into the research topics of 'Conditional deep gaussian processes: Multi-fidelity kernel learning'. Together they form a unique fingerprint.

Cite this