A robot arm digital twin utilising reinforcement learning

Matulis, Marius and Harvey, Carlo (2021) A robot arm digital twin utilising reinforcement learning. Computers and Graphics. ISSN 0097-8493

CAG_RL_accepted_version.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (3MB)


For many industry contexts, the implementation of Artificial Intelligence (AI) has contributed to what has become known as the fourth industrial revolution or “Industry 4.0” and creates an opportunity to deliver significant benefit to both businesses and their stakeholders. Robot arms are one of the most common devices utilised in manufacturing and industrial processes, used for a wide variety of automation tasks on, for example, a factory floor but the effective use of these devices requires AI to be appropriately trained. One approach to support AI training of these devices is the use of a “Digital Twin”. There are, however, a number of challenges that exist within this domain, in particular, success depends upon the ability to collect data of what are considered as observations within the environment and the application of one or many trained AI policies to the task that is to be completed. This project presents a case-study of creating and training a Robot Arm Digital Twin as an approach for AI training in a virtual space and applying this simulation learning within physical space. A virtual space, created using Unity (a contemporary Game Engine), incorporating a virtual robot arm was linked to a physical space, being a 3D printed replica of the virtual space and robot arm. These linked environments were applied to solve a task and provide training for an AI model. The contribution of this work is to provide guidance on training protocols for a digital twin together with details of the necessary architecture to support effective simulation in a virtual space through the use of Tensorflow and hyperparameter tuning. It provides an approach to addressing the mapping of learning in the virtual domain to the physical robot twin.

Item Type: Article
Identification Number: https://doi.org/10.1016/j.cag.2021.01.011
29 January 2021Accepted
4 February 2021Published Online
Uncontrolled Keywords: Robot arm Reinforcement learning Artificial intelligence Digital twin
Subjects: CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science
CAH11 - computing > CAH11-01 - computing > CAH11-01-05 - artificial intelligence
Divisions: Faculty of Computing, Engineering and the Built Environment > School of Computing and Digital Technology
Depositing User: Carlo Harvey
Date Deposited: 20 Feb 2021 10:40
Last Modified: 22 Mar 2023 12:00
URI: https://www.open-access.bcu.ac.uk/id/eprint/10987

Actions (login required)

View Item View Item


In this section...