Comprehensive Visual Features and Pseudo Labeling for Robust Natural Language-based Vehicle Retrieval

IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2023 – AICity Challenge Track 2(2023)
International conferences
19 Jun 2023
intelligent transportation systemsfeature fusionvehicle retrievallarge-scale transportation videosvehicle re-identificationvehicle trackingtraffic video feedscross-modal retrievalnatural languagevisual representationpseudo labelingpost-processingretrieval accuracymean reciprocal rankmrr scoretracked-vehicle retrievalaicity 2023contrastive learningcomputer visionpublic transportation

Abstract

Vehicle retrieval has become crucial for public transportation and intelligent transportation systems due to the exponential development of large-scale transportation videos. Vehicle re-identification and vehicle tracking are the main components of most vision-based vehicle recovery systems. Unfortunately, the limited amount of information provided by traffic video feeds limits the vision-based vehicle retrieval algorithm’s efficacy. Therefore, this article proposes a contrastive cross-modal vehicle retrieval approach to maximize the complementarity of natural language and visual representations. An efficient method to fuse multiple input image features is also proposed to extract comprehensive information from various vehicles along with pseudo labeling and efficient post-processing techniques to enhance retrieval accuracy. The proposed method achieved the 3rd ranking of Mean Reciprocal Rank (MRR) score of 0.4795 on the test set for the Challenge Track 2: Tracked-Vehicle Retrieval by Natural Language Descriptions 2023. Source code for the proposed approaches is openly accessible at https://github.com/anminhhung/AICity-2023-Track2.

Authors

Bach Hoang NgoOtherOrder: 1CorrespondingEqual Contribution
BachHoangNgo@gmail.com
Dat Thanh NguyenOtherOrder: 2Equal Contribution
DatThanhNguyen@gmail.com
Nhat-Tuong Do-TranOtherOrder: 3Equal Contribution
NhatTuongDoTran@gmail.com
Phuc Pham Huy ThienOtherOrder: 4Equal Contribution
PhucPhamHuyThien@gmail.com
Minh-Hung AnOtherOrder: 5Equal Contribution
MinhHungAn@gmail.com
Tuan-Ngoc NguyenOtherOrder: 6Equal Contribution
TuanNgocNguyen@gmail.com
Loi Nguyen HoangOtherOrder: 7Equal Contribution
LoiNguyenHoang@gmail.com
Nguyen Dinh Vinh
Nguyen Dinh VinhFacultyOrder: 8Equal Contribution
vinhnd29@fe.edu.vn, nguyendinhvinh2004@gmail.com

We use necessary cookies to run this site and optional analytics cookies to understand how it is used. Analytics only loads if you accept. Read our cookie policy