AI-Powered Floor Plan Reading for Smart Home Automation Using YOLO and OpenCV

Introduction

Smart home automation has witnessed significant growth in recent years, with an increasing demand for intelligent solutions that can optimize various aspects of home living. One crucial element in achieving this is the ability to accurately interpret and understand floor plans. Advances in computer vision, combined with deep learning models such as YOLO (You Only Look Once), have enabled AI-powered floor plan reading. This case study explores the application of the You Only Look Once (YOLO) model, a state-of-the-art object detection algorithm, for floor plan reading in the context of smart home automation.

Problem Statement

The primary challenge in smart home automation is effectively utilizing floor plans to optimize device placement, energy efficiency, and user experience. The construction and real estate industries rely heavily on accurate interpretation of architectural floor plans. Manually extracting information like room layouts, dimensions, symbols, furniture like windows, doors, stairs, basin, commode, showcase, sofa, tea table, Dining table, chair, kitchen cabinet etc and other design elements can lead to inconsistencies, errors, delays and cost. Given the diversity and complexity of floor plans, the need to automate this process becomes essential. The key challenges in automating floor plan reading include:

• Variability in floor plan designs and formats.
• Complex symbol recognition for architectural features.
• Limited scalability of traditional methods.
• Time and cost constraints in manual interpretation.

The Solution: AI Powered Floor Plan Reading

To address these challenges, a computer vision-based solution using the YOLO model is proposed. YOLO is a state-of-the-art object detection algorithm that provides fast and accurate detection by framing object detection as a single regression problem. We used two YOLO models, one for room detection and another for furniture detection like door, window, dining-table, chair, showcase, wall-cabinet etc. The workflow of the solution is as follows:

Used key steps for floor plan reading:

• Data Collection and Processing:
• We gathered a number of floor plan image dataset of around 1000 images.
• Converted all of the image’s shape into 512 width and 512 height.
• Data Annotation:

• For annotating the dataset, use the Roboflow platform. That is too user friendly. Users can preview annotated data and also can divide the dataset.
• We annotated rooms, doors, windows, furniture like bed, dining table, reading table, showcase, wall cabinet, sofa, chair, tv cabinet, commode, basin, shower, wall mirror, water tap etc.
• From Roboflow we downloaded our desired model extension dataset where we chose the Yolov8 format for both the models.
• Dataset Divide:

• Divided room prediction dataset into 3 parts. Where the training dataset carried 80% images, the validation dataset carried 10% images and the testing dataset carried 10% images.
• For furniture prediction dataset is also divided into 3 parts training, validation, testing accordingly 80%, 10%, and 10%
• Model Architecture:

• We used a pre-trained YOLOv8 point detection model. That involves detecting rooms in an image frame and drawing bounding boxes around them. For furniture detection we used the YOLOv8 detection model.
• YOLOv8 Architecture utilizes a novel C2f module instead of the traditional Feature Pyramid Network (FPN). This module combines high-level semantic features with low-level spatial information, leading to improved detection accuracy, especially for small objects. Head: The head is responsible for making predictions.
• Model Training and Validation:

• YOLOv8 pose model is fine tuned with floor plan images for detecting rooms and corners accurately according to the walls of the rooms.
• YOLOv8 detection model is used for detecting furniture and exact center position of the detected furniture in the bounding box.
• Where epochs number was 400 selected for both the models. Trained with full 400 epochs each model.
• After training, for room detection model Training-Accuracy=0.83, Training-Loss=0.23, and Validation-Accuracy=76, Validation-Loss=0.25.
• For furniture detection model Training-Accuracy=0.93, Training-Loss=0.13, Validation-Accuracy=0.91, Validation-Loss=0.12.
• Prediction and Interpretation:

• Testing dataset is used to test the trained room detection pose model and got 0.74 testing accuracy. In the same way testing dataset is used for testing furniture prediction models and has 0.94 accuracy.

Benefits of AI-Powered Floor Plan Reading

AI-powered floor plan reading offers several significant benefits in a accuracy, time, cost and most importantly flexible for property user and for some architectural company:

• Accuracy: YOLO's object detection capabilities enable highly accurate identification and localization of objects in floor plans.
• Efficiency: The model can process floor plans in real-time, making it suitable for various smart home applications.
• Flexibility: YOLO can be adapted to detect a wide range of objects in floor plans, including custom-defined elements.
• Scalability: The model can handle large-scale datasets and complex floor plan layouts.
• Time Save: The system saves the huge time of customer/client by converting a 2D floor plan image to 3D object image.

The Challenges and Limitations

There were several challenges and limitations appeared during building this application, some of them for data, accuracy, complex floor plan, country-base-culture and furniture prediction. Still there are some challenges and limitations are below:   

• Variability in Floor Plan Formats: Floor plans come in various formats (raster, vector, hand-drawn, etc.), which poses a challenge for consistent interpretation across different styles.
• Limited Training Data: High-quality labeled data is crucial for training. The accuracy of the model depends on having a well-curated dataset that covers a wide range of floor plan designs.
• Complex Symbol Recognition: Architectural symbols can vary in design and size, making it difficult for the model to generalize to all possible symbols without extensive training.
• Occlusions and Noise: Floor plans often contain overlapping elements or annotations that may confuse the detection algorithm.
• Generalization Issues: The model may struggle with floor plans from different regions or companies that use unique conventions or symbology not present in the training data.

Technologies Use

There are several advanced technologies we used for solving this problem and some of them are paid like Google Colab GPU service. Below are some of the list of technology:

• YOLOv8: For object detection in floor plans, YOLOv5 was chosen for its balance between speed and accuracy.
• OpenCV: Used for image preprocessing tasks such as resizing, noise reduction, and contour detection.
• TensorFlow/PyTorch: Machine learning frameworks used to build and train the deep learning models. Here we used PyTorch
• GPU/TPU/CPU: T4 X2 GPU is used for training both the model, with 15 GB RAM, Intel COREi7 processor,
• Roboflow: An annotation tool RoboFlow is used to label the training dataset for supervised learning.
• Building Information Modeling (BIM) Integration: For post-processing and exporting the detected objects into compatible formats for architecture, engineering, and construction (AEC) workflows

Conclusion

The case study demonstrates the effectiveness of the YOLO model for floor plan reading in the context of smart home automation. By accurately detecting and classifying objects within floor plans, the model can provide valuable insights for optimizing device placement, energy efficiency, and user experience. As AI and computer vision technologies continue to advance, we can expect even more sophisticated and versatile solutions for floor plan analysis in the future.