Engineering a multi model fallback system for edge devices

Machine learning (ML) is an effective way to extract information from data and perform decision making on it. The growing deployment of machine learning applications in edge environments, such as IoT devices and embedded systems, has highlighted the need for efficient, resource-aware systems for edge environments. This paper presents a novel multi-model fallback system designed for deployment on resource-constrained edge devices, leveraging the advancements of TinyML. The proposed system utilizes a model pool of optimized models and a confidence-based switching mechanism to dynamically select the most reliable model for inference. Through the integration of optimization techniques such as quantization, pruning, and clustering, the system achieves significant reductions in model size and inference time without compromising accuracy. The system's efficacy is validated through extensive experiments using the benchmark MNIST and CIFAR10 datasets, demonstrating its ability to balance trade-offs among accuracy, speed, and resource usage. The experimental analysis shows that the proposed optimization techniques reduce the size of the model ranging from 57% to 90% while ensuring its effectiveness. The proposed approach highlights the selection of a threshold value to make models adaptive to the considered application. The results show the system's adaptability, positioning it as a compelling option for real-time, low-power tasks in different engineering areas such as IoT, autonomous technologies, and continuous monitoring.