Arabic Image Captioning Using GoogLeNet

Abstract

This thesisfallswithin the scope of Arabic image captioning, a researchfieldthat combines computer vision and naturallanguageprocessing (NLP). The primary objective of thisworkis to design and develop an intelligent system capable of comprehending and interpreting image content, and subsequentlygenerating an accurate and appropriatetextual description in Arabic. The first part of thisworkreviews the fundamental concepts of image captioningtechnology, itshistoricalevolution, and its operating principles, alongside key application areas— particularly in assisting the visuallyimpaired, as well as in medicine, e-commerce, social media platforms, and e-learning environments. This section also examines the mostcommon architectures, specifically the encoder-decodermodel;here, Convolutional Neural Networks (CNNs) are employed to extractvisualfeatures, while Long Short-Term Memory (LSTM) networks or Transformer modelshandle the decoding process to generate descriptive textincrementally. Special emphasisisplaced on attention mechanisms, whichenhance the quality of generatedtext by guiding the model to focus on the mostsignificantregions of the image. The second part isdedicated to the practicalimplementation of the proposed system, highlighting the developmentenvironment, the Flickr8k dataset, and the various stages of image and textpreprocessing. Visual featureswereextractedusingtransferlearning techniques— leveraging twopre-trainedmodels, VGG16 and GoogLeNet (InceptionV3) —and a comparative analysis of their performance wasconducted. Meanwhile, the Arabic descriptions underwentpreprocessingthatincludedcleaning, normalization, and tokensization, utilizingboth a classicalapproach and an advancedmethodbased on the AraBERT model. Finally, thisresearchpresents the complete architecture of the developed model, its training stages, the key hyperparameterstuned, and the metricsused for performance evaluation. Experimentalresultsdemonstrate the proposedapproach'sability to generatecoherent and semanticallymeaningfulArabic descriptions. Furthermore, the resultsclearly highlight the positive impact of leveraging transferlearning, as well as the added value of AraBERT- basedtext segmentation in enhancing the quality of generated descriptions and accounting for

the unique morphologicalcharacteristics of the Arabiclanguage.