On Applying Machine Learning/Object Detection Models for Analysing
  Digitally Captured Physical Prototypes from Engineering Design Projects
  While computer vision has received increasing attention in computer science
over the last decade, there are few efforts in applying this to leverage
engineering design research. Existing datasets and technologies allow
researchers to capture and access more observations and video files, hence
analysis is becoming a limiting factor. Therefore, this paper is investigating
the application of machine learning, namely object detection methods to aid in
the analysis of physical porotypes. With access to a large dataset of digitally
captured physical prototypes from early-stage development projects (5950 images
from 850 prototypes), the authors investigate applications that can be used for
analysing this dataset. The authors retrained two pre-trained object detection
models from two known frameworks, the TensorFlow Object Detection API and
Darknet, using custom image sets of images of physical prototypes. As a result,
a proof-of-concept of four trained models are presented; two models for
detecting samples of wood-based sheet materials and two models for detecting
samples containing microcontrollers. All models are evaluated using standard
metrics for object detection model performance and the applicability of using
object detection models in engineering design research is discussed. Results
indicate that the models can successfully classify the type of material and
type of pre-made component, respectively. However, more work is needed to fully
integrate object detection models in the engineering design analysis workflow.
The authors also extrapolate that the use of object detection for analysing
images of physical prototypes will substantially reduce the effort required for
analysing large datasets in engineering design research.
