Transformer-Based Object Detection in Natural Images. State-of-the-Art Architectures and Recent Algorithms

Abstract

Object detection in natural images is a fundamental computer-vision task that underpins applications ranging from autonomous driving to industrial inspection. Since the introduction of the DEtection TRansformer (DETR), the field has shifted from anchor-based convolutional pipelines toward end-to-end, attention-driven set-prediction frameworks that remove hand-crafted components such as anchor generation and non-maximum suppression. This paper presents a structured review of transformer-based object detectors, tracing their evolution from the original DETR through deformable attention, query-design refinements (DAB-DETR, DN-DETR), contrastive denoising (DINO), collaborative hybrid assignment (Co-DETR), and the most recent real-time variants (RT-DETR, RT-DETRv2, RF-DETR). We organise these methods into a taxonomy according to their core innovations in attention mechanisms, query formulation, and label-assignment strategies, and we compare their reported accuracy and inference speed on the MS-COCO benchmark. The analysis shows that contemporary detection transformers now match or surpass convolutional and YOLO-family detectors in both accuracy and real-time efficiency, with leading models exceeding 60 mean Average Precision on standard benchmarks. We further discuss persistent challenges—small-object localisation, training convergence, and computational cost—and outline promising research directions, including open-vocabulary detection and lightweight deployment.