Examining the role of simulated imagery in the evaluation and training of computer vision systems

This thesis addresses the significant limitations of "Scenario Evaluation" methodologies in Visual Simultaneous Localization and Mapping (SLAM) systems by proposing a scalable and generalisable "Technology Evaluation" framework driven by high fidelity synthetic imagery. Traditionally, the evaluation of computer vision systems has relied upon expensive, environment-specific real-world data collection, making it inherently difficult to predict system performance accurately under fluctuating or previously unseen operational conditions. To overcome these considerable constraints, this research thoroughly investigates the viability of using synthetic imagery as a robust, repeatable, and highly predictive substitute for real-world datasets. The core methodological contribution of this work is the systematic construction of "Digital Twins" that meticulously replicate not only the geometric structures of real-world environments but also the crucial optical artifacts that fundamentally influence computer vision performance. Specifically, this research focuses on two critical imaging phenomena: Signal-to-Noise Ratio (SNR) degradation and optical lens blur. Both parameters are carefully modelled and precisely calibrated within the synthetic environment to faithfully mirror the characteristics observed in real captured imagery. Through a rigorous and systematic comparative analysis of fundamental computer vision operators, including corner detection algorithms and Oriented FAST and Rotated BRIEF (ORB) feature matching, the experimental results demonstrate a consistently strong and statistically meaningful correlation between real world and virtual environment performance outcomes. These findings provide compelling empirical evidence that accurately constructed and carefully matched synthetic datasets can reproduce the pattern of real-world performance degradation observed in the sensors tested, and can predict the failure boundaries of the operators examined. By establishing this correlation at the operator level, this work provides evidence that engineering simulation offers a viable and cost-effective complement to physical data collection for the evaluation of computer vision operators. Extension of the approach to complete Visual SLAM systems is identified as the principal direction for future work. This contribution opens significant opportunities for more efficient, flexible, and comprehensive evaluation of computer vision technologies across diverse and demanding operational scenarios.