Learning ultrasound scanning skills from human demonstrations
Deng X T, Lei Z W, Wang Y, et al
Sci China Inf Sci, 2022, 65(8): 184201
Recently, the robotic ultrasound system has become an emerging topic owing to the widespread use of medical ultrasound. According to the level of the system autonomy, robotic ultrasound systems can be categorized into three levels: teleoperated, semiautonomous and full-autonomous. Related work to fulfill ultrasound scanning guidance could be divided into three stages in autonomous ultrasound systems according to the dimensions of the input information: (1) manually planning trajectory (human guidance), (2) ultrasound image evaluation model with proper servo controller (ultrasound image), (3) learning part of ultrasound scanning skills (ultrasound image, probe pose). Although studies have achieved certain results, the modeling method of ultrasound scanning skills needs to be improved. In this study, focusing on extracorporeal ultrasound, we not only consider ultrasound images and probe pose, but also encode the contact force between the probe and humans into ultrasound scanning guidance, which is regarded as a critical factor during ultrasound scanning. The main contribution of this study is twofold. (1) A multimodal model of ultrasound scanning skills is proposed and trained from human demonstrations, which considers the ultrasound images, probe pose and contact force. (2) A sampling-based strategy is proposed with the learned model to adjust the extracorporeal ultrasound scanning process to obtain a highquality ultrasound image. Note that the primary goal of this article is to offer a learning-based framework to understand and acquire extracorporeal ultrasound scanning skills from human demonstrations. However, the learned model can apparently be applied to a robot system, which is our future work.