Hybrid Vision-and-Language Fusion: A Threefold Learning Approach for elevating Image Captioning through Adaptive Strategies
1Masters in Artificial Intelligence & Machine Learning, Liverpool John Moores University, India
2Dept. of CSE, Chandigarh University, India
3Amity School of Engineering and Technology, Amity University Punjab, Mohali, India
4Chitkara University School of Engineering & Technology, Chitkara University, India
*Author to whom correspondence should be addressed:
E-mail: batta.priya1@gmail.com (PB)
E-mail: batta.priya1@gmail.com (PB)
Received: January 27, 2025 | Revised: July 08, 2025 | Accepted: August 02, 2025 | Published: December 2025
Abstract
Image captioning is a significant area of application for artificial intelligence techniques. When a machine can interpret an image similar to humans, it indicates a higher intelligence level and comprehension of the image. This research displays advancements in real-time image collection and labeling systems using a triad of computer vision, natural language processing, and classification. The approach employs three deep learning models to generate human-level natural language descriptors, resulting in a user-friendly system. The model comprises a multimodal pipeline of deep learning architectures, enabling the extraction of probabilistic features for each object category. Our model surpasses other image captioning models, achieving a CIDEr score of 37.93% on the common MS-COCO Captioning task test baseline, thereby exhibiting superior syntactical saliency when integrated with advanced object features. Additionally, we observed that incorporating an intermediate step of clustering objects before classification enhances the final model's performance. By implementing these methodologies, we have developed a more capable and accurate model, proficient in object classification and generating informative image descriptions. Such capabilities can significantly augment human comprehension and decision-making across various applications, particularly in advancing sustainable cities and communities, fostering quality education through improved accessibility of visual content, promoting industry, innovation, and infrastructure with cutting-edge AI technologies.
Keywords
Deep Learning; K-Means Clustering; KNN classification; MS COCO; Multimodality; POS tagging; YOLO
Available Repositories
Share Article
Article Metrics
--
Views
--
Downloads
--
Citations
Export Citation
Full Text









