DefectDetect Bot
Introduction
My role
Our Team
Duration
Project Management
Team Collaboration
Product Design
Interaction Design
Circuit Design
Mechanical System Design
CAD Modeling
Prototype Fabrication
9/20/2022 - 12/31/2022
We have designed a robot for Changhong Corporation that assists in the inspection of prefabricated buildings. This robot helps Changhong address issues such as time-consuming professional training for inspectors, limited human resources, and the time-intensive nature of manual inspections. This robot is user-friendly, allowing volunteers without professional backgrounds to operate it effortlessly. The robot is capable of detecting defects in prefabricated constructions and promptly delivers reports to headquarters engineers, creating an efficient system. Additionally, its aesthetically pleasing design is intended to attract more community members to explore the company's prefabricated buildings, potentially generating additional revenue.
Project Type
Group Project
Human-Computer Interaction
User Experience Design
Product Design
Demo link: https://youtu.be/hwtNYaVvG5c
Storyboard: Let us see how people use our robot
Background research
This is a collaboration project with Changhong Company in the year 2022.
Target users and stakeholders
The process of constructing a prefabricated community center
Users
Community service personnel without a professional background
Stakeholders
community residents
Changhong Group
BIM
Prefabrication
15-minute living circle
Smart technology
Changhong Community Prefabricated Leisure Center
Now
Future
Inconvenient storage and transmission of data records during the detection process.
Uses human resources for inspection, risking oversight of less noticeable corners or detection errors.
Inspector-maintenance communication lacks speed and comprehensiveness.
Before putting into use
Order
Maintenance and operations
Horizontal and vertical deviation
Connection tightness
Structural corrosion
Appearance integrity
Acceptance
Use
1
2
3
1
2
3
4
1
2
4
5
5
6
8
9
7
6
7
8
9
3
1
2
3
Changhong is China's pioneering BIM company and China's first company to collaborate with communities and implement on-site Community Modular Center
Sports
Shopping
Entertainment
INterview
yuetong Li
27
Community staff, without a professional background in construction engineering.
She worries about the safety of prefabricated houses; arranging inspection workers is time-consuming and labor-intensive.
Pains:
Goals：
Huigang
Changhong Operations Center Design Department Manager, Senior Engineer
40
Trying computer vision and angle measurement sensors to achieve automated detection.
To reduce manpower, improve efficiency, implement systematic management, and integrate detection processes
Manual inspections are time-consuming, feedback efficiency is low, and accuracy is often influenced by human factors.
“Manual inspections are time-consuming and lack precision. ”
Pains:
Goals:
Suggestions:
"I need to ensure the safety of community residents."
Safe and easy-to-operate machines with professional testing devices can assist in inspections.
Yihan Liu
14
"I want to focus on exercising in the modular house without any disturbances."
Middle school student, community resident
At times, he was disturbed by routine inspections of the house.
Pains:
Goals:
He hopes that future inspection work can be quieter, or during times of low human traffic.
Affinity diagram
Needs Analysis:
The company aims to reduce labor costs and training time while enhancing inspection efficiency and accuracy.
The community seeks a more efficient and accurate inspection process, faster inspection reports, and desires a safer and quieter environment for activities without disturbances from inspection work.
Observation
Check the panels
Inspect the equipment
Inspect the steel structure
Inspect screws, bolts, and nuts
Check the wall surface
Check the ground level
Inspect for signs of termite damage
painpoints
The current inspection of prefabricated construction mostly relies on manual visual inspection or traditional measurement tools.
INSPIRATION
visual inspiration
Reliable
Compact
Sleek
Approachable
Minimalistic
35.1
140.0
120.0
100.0
80.0
60.0
40.0
20.0
0.0
41.8
54.0
69.3
82.9
94.6
110.3
131.4
2014
2015
2016
2017
2018
2019
2020
2021
2.2
3.0
3.9
5.2
6.3
7.5
9.1
11.1
The market size of specialized robots in China reached 1.1 billion dollars in 2021.
The data from the International Federation of Robotics (IFR)
The global special robotics sales scale
China's special robotics sales scale
Sales scale of the special robotics
The specialized robots demonstrate outstanding performance in responding to tasks such as inspections, floods, extreme weather conditions, as well as emergencies like fires and public safety events.
I also want to design a specialized robot to assist in Modular buildings quality inspections
The Youai Zhihe Inspection Robot
Conceptual Inspiration
DEFINE
Accurate
Comprehensive Inspection Process
Accurate Detection Methods
Efficient
Reducing Operation Time
Faster Detection and Reporting
Integrated
Various inspection technologies
Compact
Easy to learn
Simple Usage Steps
User-friendly Instructions
concept 01
IDEATION
EVALUATION
concept 02
concept 03
concept 04
Evaluation and Discussion
A robot equipped with a laser radar scans the entire cabin, generating a point cloud file. The point cloud file is uploaded to the control center for data processing and report generation.
Utilizing a multi-axis robotic arm equipped with a camera, various defects can be detected using computer vision methods.
Using a lever with a two-axis gimbal, coupled with a camera, enables vertical shooting and scanning.
Using infrared ranging sensors for point-to-point distance measurements helps assess ground level and board warping, determining potential warping above or uneven ground.
“Using laser radar is a high-risk solution, given its high cost, substantial development challenges, and time-consuming nature. It's not strongly advised for your team to pursue this.”
Robotics Engineer
Yuan Zhou
“The tilt detection needs to consider the issue of horizontal plane calibration. Additionally, if data drift occurs, a filtering algorithm should be employed for processing.”
Civil Engineer
Junpeng Zhao
“To ensure no corner of the building is missed, it's essential to sequentially scan all walls with a camera.”
Architect
Yujia Lei
On October 14, 2022, a meeting was held at Changhong Shenzhen Operations Center, attended by the head of Changhong Shenzhen, designers and engineers from Changhong Headquarters.We presented our proposal to them, and they provided feedback and suggestions.
Multi-axis robotic arm + Camera
Inertial Measurement Unit
Two-axis gimbal
Pushing rod + Camera
Lidar + Camera
Infrared Distance Sensor + Camera
Detecting floor level
Detecting wall surface
To develop a Automated inspection robot for Prefabricated Community Center that is:
PERSONA
Service blueprint
Leila
About
Painpoint
Motivation
35
Staff of Shangxiawei community
More convenient and precise detection
Regular, efficient inspections, and maintenance
In charge of safety of the community
Concerned about the accuracy of community center inspections
Reaching out to professional inspection personnel and communicating about maintenance matters consumes significant time
Yue Wang
About
Painpoint
Motivation
64
CEO of Changhong
Hopes that the process can be more efficient and accurate
Aims for an effective communication system between the community and the company
Plans to build 50 community centers next year
Manual inspection is time-consuming, inefficient, and prone to human error
Training personnel for inspection also takes time
Evidence
Line of interaction
Customer actions
Backstage actions
Front actions
Supporting processes
Line of internal action
Line of visuability
App
App and Robot
App
App
Makes Inspection Schedule
Displays user inspection schedule and informs users of robot reception time
Displays the startup and status of the robot
Displays real-time location, status, inspection progress, and camera feed of the robot
Displays inspection result and uploads the report to engineers
Displays the return of the robot and indicates the completion of the inspection
Prepares for the charging and deployment of the vehicle
Pairs the vehicle with the user
Notifies the courier to deliver the robot
Receives real-time data, location, and images
Analyzes data, generates report, sends to engineer
Notifies the courier to retrieve the robot
Reviews report and schedules repairs if needed
Recharges and upkeeps the robot
Delivers the robot to the community
Deliver the robot to the robot charging center
Places the robot and turns it on
Waits and Monitors robot’s Status in the App
Views the Report
Robot
final concept
We created a visual mockup with copperplate paper to test dimensions and identify potential structural interferences. Discovered issues include the need for a smoother transition at the rear of the casing and adjustments to side dimensions to avoid interference with the tires.
1
Commence patrol from the starting point
Automatic Inspection Technology
2
Capture images of walls and identify defects
Computer vision and Deep learning
1
2
2
2
2
4
3
Testing step
Technology
How it works
3
Determine if the ground is level
Inertial Measurement Unit
4
Stationed and awaiting retrieval
Automatic Inspection Technology
EXPERIMENT: TEChNOLOGY ANALISIS
Test Purpose
Process
Result
Testing the accuracy of the IMU module and installing it
Test whether the car stops at designated points and finish the designed process
Ensure the LED matrix displays status
Compute fixed positions using construction data
Gimbal, push rod, and camera collaboration
More simulations using Simulink will be conducted to find the appropriate PID values and enhance the response speed during turns.
Assembling and testing the sensor
MATLAB simulation, PID tuning, and on-site testing
Installing and testing the obstacle avoidance function
The push rod height is insufficient, and the camera's shooting angle range is not enough to capture the bolt area.
Robots autonomously inspect prefabricated construction using SLAM, Path Planning, and Obstacle Avoidance tech, enhanced by algorithms like ORB-SLAM, A*, and ROS Navigation Stack for efficient and precise inspections.
Measuring the angle between device acceleration and gravity, often using an Inertial Measurement Unit (IMU) with accelerometers and gyroscopes, enables precise detection of the device tilt relative to the horizontal surface.
By capturing multiple photos, utilizing YOLO to construct a training database, and employing image recognition algorithms for precise analysis, issues like rust, warping, and loosening bolts can be identified.
Five-degree Grayscale Sensor
PID Control
Joint control of push rod, two servos, camera, and LED matrix
Ultrasonic sensor
IMU and filtering
Inertial Measurement Unit
Computer vision and Deep learning
Automatic Inspection Technology
Test whether the robot can accomplish automatic line following
Test whether the robot can maintain stable motion at an appropriate speed
The technology
Test Purpose
Test Purpose
Process
Process
Result
Result
The technology
The technology
Test whether the robot can accomplish obstacle avoidance
Test whether the IMU starts properly measures accurately
There is some drift in the IMU data. Kalman filtering is required to address this issue and achieve more accurate results.
3D renderings
Emergency stop upon encountering obstacles
On patrol
In detection
Schedule Inspection
Waiting for delivery
View inspection updates on the phone
Waiting for the maintenance personnel
Start the robot
Check the report
Using process
Camera
Servos
Camera
LED Dot Matrix Display
Push Rod
Light Strip
Ultrasonic Module
Grayscale Sensor
Gimbal
Light
Grayscale Sensor
Sensor output
Turning test
PID control
Camera capturing pictures
IMU
Ultrasonic sensor
LED light changing
APP DESIGN
SToryBoARD
Site map
User Interface
UI Specification
Color
Fonts
Montserrat
Montserrat
Semibold / Italic / regular
40pt
24pt
20pt
14pt
12pt
16pt
12pt
Icons
B3BFFF
007AFF
E3E7FD
Administrator Login
Robot status
Detection Report
Settings
Current detection
Current position
Progress
Battery power
Real-time video
Real-time data
Latest report
Past report
Detailed content
Detailed content
Schedule
Plan inspection time
Contact
Schedule the next inspection
Robot received and turn it on
The robot patrols autonomously
Check the status in the app
The inspection complete
View the report
Detection results
Detailed report
Previous detailed reports
the progress and battery level
Real-time video and data
What the robot is currently detecting
The position of the robot
