COMPUTER-AIDED MEDICAL DIAGNOSTIC SYSTEM ON VISUAL QUESTION-ANSWER USING DEEP LEARNING

Abstract

Medical-VQA can enhance the capabilities of computer-aided diagnosis that can not only bridge the imbalance in the current state of medical resources in the healthcare sector but also reduce the cost for both the hospitals as well as for the patients. Medical images are rapidly being used in medical diagnosis, however, VQA systems are lacking behind due to capabilities of the domain transfer of general VQA model to medical-VQA as there is a huge gap between the nature and complexity of general domain and medical domain datasets. Moreover, medical-VQA datasets are also not very big which also enhances the need for such a VQA system that can learn from the lesser data and can be generalized to diversify medical imaging. In this proposal, a multimodal Transformer based VQA system is suggested. The suggested system incorporates the interaction among input images, question and answer text.