Scalable Inference for Bayesian Proportion Models: Variational Approaches under Contaminated Likelihoods
DOI:
https://doi.org/10.31185/bsj.Vol21.Iss35.1241Keywords:
Bayesian Proportion Models, Variational Inference, Contaminated Likelihoods, Robust Bayesian Methods, Scalable InferenceAbstract
Bayesian proportion models, such as Binomial and Beta regression, are widely applied in health sciences, social research, and digital analytics. While these models provide coherent uncertainty quantification, they are highly sensitive to contamination and outliers. Furthermore, traditional inference methods, particularly Markov chain Monte Carlo (MCMC), are computationally prohibitive for large datasets. This study develops scalable variational inference approaches for Bayesian proportion models under contaminated likelihoods. Two complementary strategies are considered: (i) mixture-based inference that explicitly models contamination via an ε-contamination likelihood, and (ii) divergence-based inference that employs β-divergence to reduce the influence of outliers. Both methods are implemented in a stochastic variational framework suitable for large-scale applications. Simulation studies demonstrate that the proposed approaches substantially improve estimation accuracy, posterior calibration, and predictive performance under contamination, while maintaining scalability. The results highlight that robustness and efficiency can be achieved simultaneously, providing practical tools for modern Bayesian analysis of proportion data.
References
1. Arbel, J., & van Erven, T. (2024). Adaptive divergence-based variational inference for large-scale Bayesian models. Statistics and Computing, 34(3), 67. https://doi.org/10.1007/s11222-024-10367-2
2. Basu, A., Harris, I. R., Hjort, N. L., & Jones, M. (1998). Robust and efficient estimation by minimizing a density power divergence. Biometrika, 85(3), 549–559.
3. Bishop, C. M. (1994). Mixture density networks [Technical report]. Aston University.
4. Bishop, C. M. (2006). Pattern recognition and machine learning. Springer.
5. Blei, D. M., Kucukelbir, A., & McAuliffe, J. (2017). Variational inference: A review for statisticians. Journal of the American Statistical Association, 112(518), 859–877.
6. Brooks, A., Gelman, A., Jones, G., & Meng, X. (2011). Handbook of Markov chain Monte Carlo. Chapman & Hall/CRC.
7. Celeux, G., Forbes, F., Robert, C. P., & Titterington, D. M. (2006). Deviance information criteria for missing data models. Bayesian Analysis, 1(4), 651–673.
8. Chen, L. M., & Aravkin, M. S. (2020). Generalized divergences in robust inference. Statistics and Computing, 30(5), 1183–1202.
9. Chen, M., Liu, Q., & Carin, L. (2015). On the convergence of stochastic gradient MCMC algorithms with high-order integrators. Advances in Neural Information Processing Systems.
10. Cichocki, A., & Amari, S. (2010). Families of α–β–γ divergences: Flexible and robust measures of similarities. Entropy, 12(6), 1532–1568.
11. Congdon, P. (2014). Bayesian statistical modelling. Wiley.
12. Cribari-Neto, F., & Zeileis, A. (2010). Beta regression in R. Journal of Statistical Software, 34(2), 1–24.
13. Demir, C. G. B. (2013). Robust Bayesian analysis via contaminated likelihoods. Bayesian Analysis, 8(3), 591–622.
14. Ferrari, S., & Cribari-Neto, F. (2004). Beta regression for modelling rates and proportions. Journal of Applied Statistics, 31(7), 799–815.
15. Frühwirth-Schnatter, S. (2006). Finite mixture and Markov switching models. Springer.
16. Fujisawa, S., & Eguchi, H. (2008). Robust parameter estimation with a small bias against heavy contamination. Journal of Multivariate Analysis, 99(9), 2053–2081.
17. Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian data analysis (3rd ed.). CRC Press.
18. Ghosh, S., & Basu, A. (2023). Advances in robust Bayesian inference using generalized divergence measures. Computational Statistics & Data Analysis, 182, 107727. https://doi.org/10.1016/j.csda.2023.107727
19. Hoffman, M., Blei, D., Wang, C., & Paisley, J. (2013). Stochastic variational inference. Journal of Machine Learning Research, 14, 1303–1347.
20. Huber, P. J., & Ronchetti, E. (2009). Robust statistics. Wiley.
21. Kingma, K., & Welling, M. (2014). Auto-encoding variational Bayes. In Proceedings of the International Conference on Learning Representations (ICLR).
22. Lesaffre, J., & Lawson, A. (2012). Bayesian biostatistics. Wiley.
23. Li, M., Turner, D., & Gretton, A. (2016). Rényi divergence variational inference. In Advances in Neural Information Processing Systems.
24. López-Serrano, J., & Morales, D. (2025). Bayesian beta regression under contamination: A variational approach with applications in epidemiological data. Bayesian Analysis, 20(1), 45–70. https://doi.org/10.1214/24-BA1456
25. McCullagh, P., & Nelder, J. A. (1989). Generalized linear models (2nd ed.). Chapman & Hall.
26. Minka, T. (2005). Divergence measures and message passing [Technical report]. Microsoft Research.
27. Nakagawa, Y., & Omori, J. (2019). Bayesian analysis of robust beta regression models. Statistics and Computing, 29(1), 29–46.
28. Owen, A. (2001). Empirical likelihood. Chapman & Hall/CRC.
29. Robbins, H., & Monro, S. (1951). A stochastic approximation method. Annals of Mathematical Statistics, 22(3), 400–407.
30. Rue, H., Martino, S., & Chopin, N. (2009). Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(2), 319–392.
31. Salimans, R., & Knowles, D. (2013). Fixed-form variational posterior approximation through stochastic linear regression. Bayesian Analysis, 8(4), 837–882.
32. Tiwari, R., & Gupta, M. (2024). Robust β-divergence variational inference for contaminated data in probabilistic modeling. Entropy, 26(2), 150. https://doi.org/10.3390/e26020150
33. Wainwright, M. J., & Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 1(1–2), 1–305.
34. Walker, T. O., & Hjort, D. (2011). Robust Bayesian inference. Scandinavian Journal of Statistics, 38(3), 514–531.
35. Wang, Y., & Blei, J. (2019). The blessings of multiple causes: Robust inference with latent confounders. Journal of Machine Learning Research, 20(150), 1–50.
36. Zhang, C., Butepage, J., Kjellstrom, H., & Mandt, S. (2013). Stochastic variational inference. Journal of Machine Learning Research, 14, 1303–1347.
37. Zhou, L., & Zhang, Y. (2023). Scalable variational Bayes for generalized linear mixed models with outlier-robust likelihoods. Journal of Machine Learning Research, 24(112), 1–28.
38. Feng, X., Sun, S., & Shapiro, A. (2024). Robust likelihood tempering for scalable Bayesian inference. Journal of Machine Learning Research, 25(88), 1–32.
39. Martín-Yebra, A., & Walker, S. (2024). Generalized robust divergence priors for Bayesian modeling under contamination. Bayesian Analysis, 19(2), 405–430. https://doi.org/10.1214/23-BA1405
Downloads
Published
Issue
Section
License
Copyright (c) 2026 علي طلال محمد جامعة محقق أردبيلي، إيران/ كلية العلوم/ قسم الرياضيات
This work is licensed under a Creative Commons Attribution 4.0 International License.
