Bayesian network parameter learning algorithm based on improved QMAP

基于改进QMAP的贝叶斯网络参数学习算法

¹ School of Electronic and Information Engineering, Xi'an Technological University, Xi'an 710021, China
² School of Electronic and Information, Northwestern Polytechnical University, Xi'an 710072, China

Received: 14 April 2021

Abstract

Small data sets make the statistical information in Bayesian network parameter learning inaccurate, which makes it difficult to get accurate Bayesian network parameters based on data. Qualitative maximum a posteriori estimation (QMAP) is the most accurate algorithm for Bayesian network parameter learning under the condition of small data sets. However, when the number of parameter constraints is large or the parameter feasible region is small, the rejection-acceptance sampling process in QMAP algorithm will become extremely time-consuming. In order to improve the learning efficiency of QMAP algorithm and not affect its learning accuracy as much as possible, a new analytical calculation method of the center point of constrained region is designed to replace the original rejection-acceptance sampling calculation method. Firstly, a new objective function is designed, and a constrained objective optimization problem for solving the boundary points of the constrained region is constructed. Secondly, a new optimization engine is used to solve the objective optimization problem, and the boundary points and center points of the constrained region are obtained. Finally, the existing QMAP algorithm is improved by the obtained center points. The simulation results show that the CMAP algorithm proposed in this paper has a slightly worse parameter learning accuracy than the QMAP algorithm, but its computational efficiency is 2-5 times higher than that of the QMAP algorithm.

摘要

小数据集使得贝叶斯网络参数学习中的统计信息不准确，导致只依靠数据难以得到准确的贝叶斯网络参数。定性最大后验估计（QMAP）方法是目前小数据集条件下贝叶斯网络参数学习精度最高的算法。然而，当参数约束数量较多或参数可行域较小时，QMAP算法中的拒绝-接受采样过程会变得极为耗时甚至难以完成。为了提高QMAP算法的学习效率同时又尽量不影响其学习精度，设计了一种约束区域中心点的解析计算方法来替代原有的拒绝-接受采样计算方法。结合参数约束构建一个求解约束区域边界点的目标优化模型；利用凸优化引擎来求解该目标优化模型，获得约束区域的边界点和中心点；通过获得的约束区域中心点改进现有的QMAP算法。仿真实验证明，所提出的CMAP算法的参数学习精度稍差于QMAP算法，但计算效率比QMAP算法提高了2~5倍。

1. Wittig F, Jameson A. Exploiting qualitative knowledge in the learning of conditional probabilities of Bayesian networks[C]//Proceedings of the Sixteenth International Conference on Uncertainty in Artificial Intelligence, California, 2000 [Google Scholar]
2. Feelders A, Gaag L. Learning Bayesian network parameters under order constraints[J]. International Journal of Approximate Reasoning, 2006, 42(1/2) : 37–53 [Article] [CrossRef] [Google Scholar]
3. Masegosa A R, Feelders A J, Gaag L C V D. Learning from incomplete data in Bayesian networks with qualitative influences[J]. International Journal of Approximate Reasoning, 2016, 69 : 18–34. 10.1016/j.ijar.2015.11.004 [Google Scholar]
4. Martin Plajner, Jirí Vomlel. Learning bipartite Bayesian networks under monotonicity restrictions[J]. International Journal of General Systems, 2020, 49(1) : 88–111 10.1080/03081079.2019.1692004 [NASA ADS] [CrossRef] [Google Scholar]
5. Zhou Y, Fenton N, Zhu C. An empirical study of Bayesian network parameter learning with monotonic influence constraints[J]. Decision Support Systems, 2016, 87 : 69–79 10.1016/j.dss.2016.05.001 [Google Scholar]
6. Di Ruohai, Gao Xiaoguang, Guo Zhigao. Parameter learning of discrete Bayesian networks based on monotonicity constraints[J]. Systems Engineering and Electronics, 2014, 18(5) : 272–277 [Article] (in Chinese) [Google Scholar]
7. Di Ruohai, Gao Xiaoguang, Guo Zhigao. Learning Bayesian networks parameters under new monotonic constraints[J]. Journal of Systems Engineering and Electronics, 2017, 28(6) : 1248–1255 10.21629/JSEE.2017.06.22 [CrossRef] [Google Scholar]
8. Ren Jia, Gao Xiaoguang, Bai Yong. Discrete DBN parameter learning under the condition of incomplete information with small samples[J]. Systems Engineering and Electronics, 2012, 34(8) : 1723–1728 [Article] (in Chinese) [Google Scholar]
9. Zeng Qiang, Huang Zheng, Wei Shuhuan. Bayesian network parameter learning method based on expert prior knowledge and monotonicity constraint[J]. Systems Engineering and Electronics, 2020, 42(3) : 642–656 [Article] (in Chinese) [Google Scholar]
10. Niculescu R, Mitchell T, Rao B. Bayesian network learning with parameter constraints[J]. Journal of Machine Learning Research, 2006, 7(1) : 1357–1383 [Google Scholar]
11. Campos C, Ji Q. Improving Bayesian network parameter learning using constraints[C]//Proceedings of the Nineteenth International Conference on Pattern Recognition in Florida, 2008 [Google Scholar]
12. Liao Wenhui, Ji Qiang. Learning Bayesian network parameters under incomplete data with domain knowledge[J]. Pattern Recognition, 2009, 42(11) : 3046–3056 10.1016/j.patcog.2009.04.006 [CrossRef] [Google Scholar]
13. Zhou Y, Fenton F, Neil M. Bayesian network approach to multinomial parameter learning using data and expert judgments[J]. International Journal of Approximate Reasoning, 2014, 55(5) : 1252–1268 10.1016/j.ijar.2014.02.008 [CrossRef] [Google Scholar]
14. Zhou Y, Fenton N, Hospadales T, et al. Probabilistic graphical models parameter learning with transferred prior and constraints[C]//Proceedings of the Thirty First International Conference on Uncertainty in Artificial Intelligence, 2015 [Google Scholar]
15. Guo Zhigao, Gao Xiaoguang, Di Ruohai. BN parameter learning based on double constraints under the condition of small dataset[J]. Acta Automatica Sinica, 2014, 40(7) : 1509–1516 [Article] (in Chinese) [Google Scholar]
16. Rui Chang, Shoemaker R, Wei Wang. A novel knowledge-driven systems biology approach for phenotype prediction upon genetic intervention[J]. IEEE/ACM Transactions on Computational Bioinformatics, 2011, 3(8) : 683–697 [Article] [Google Scholar]
17. Guo Z G, Gao X G, Ren H, et al. Learning Bayesian network parameters from small data sets: a further constrained qualitatively maximum a posteriori method[J]. International Journal of Approximate Reasoning, 2017, 91(12)22–35 [Google Scholar]
18. Heckerman D, Wellman M P. Bayesian networks[J]. Communications of ACM, 1995, 38(3): 27–30 10.1145/203330.203336 [CrossRef] [Google Scholar]