Volume 40, Number 4, August 2022
|Page(s)||804 - 811|
|Published online||30 September 2022|
Visual judgment approach of isomorphic GUI for automated mobile app testing
School of Computer Science, Northwestern Polytechnical University, Xi’an 710072, China
2 School of Software, Northwestern Polytechnical University, Xi’an 710072, China
3 School of Computer Science and Engineering, Xi’an Technological University, Xi’an 710021, China
Currently, the rapid growth of mobile apps requires automated testing technology to ensure their quality. Automated testing of mobile apps is usually closely related to the recognition and judgment of their graphical user interface (GUI), but there usually are numerous isomorphic GUIs with different styles and contents, and similar structure and function in mobile apps. In automatic testing, isomorphic GUI is easy to cause the issue of state space explosion, which leads to low efficiency or failure of testing. In view of the limitations of traditional automatic recognition of isomorphic GUI, this paper presents a GUI similarity judgment approach based on visual feature information. Firstly, the GUI component elements are identified by object detection, and then the GUI skeleton is abstracted. Secondly, the visual features of the GUI skeleton are extracted by a convolutional autoencoder. Finally, the isomorphic GUI judgment is completed by comparing the similarity of GUI visual features. The experimental results show that the proposed approach can effectively shield the influence of GUI style and content, complete the isomorphic GUI recognition more accurately and optimize the efficiency of automated mobile app testing.
当前快速增长的移动应用程序迫切需求自动化测试技术以保证其质量。移动应用的自动化测试与其图形用户界面(GUI)的识别与判断紧密相关, 但移动应用却通常存在大量的样式与内容有差异而结构与功能相类似的同构GUI。在自动化测试中, 同构GUI容易引发应用状态空间的爆炸问题, 进而导致测试的低效或失败。针对传统自动化识别同构GUI的局限性, 提出一种基于视觉特征信息的GUI相似度判断方法。通过目标检测技术识别GUI组件元素进而抽象出GUI结构框架; 采用卷积自编码器提取出GUI结构视觉特征; 对比GUI视觉特征的相似度完成同构GUI判断。经过实验验证, 所提方法能够屏蔽GUI的样式、内容等影响, 从而更精确地完成同构GUI识别, 优化自动化测试效率。
Key words: mobile app testing / GUI visual features / isomorphic GUI / GUI similarity judgment
关键字 : 移动应用测试 / GUI视觉特征 / 同构GUI / GUI相似判断
© 2022 Journal of Northwestern Polytechnical University. All rights reserved.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.