Search Unity

Safely automating heavy vehicles using simulations

December 20, 2021 in Industry | 13 min. read
Real machine on the left, digital twin on the right
Real machine on the left, digital twin on the right
Share

Is this article helpful for you?

Thank you for your feedback!

Using Unity’s real-time 3D development platform for machine learning can help engineers safely design, develop and test complex autonomous machines. Read this guest post from Daniel Lindmark at Algoryx, a Unity Verified Solutions Partner, to learn more.

Industrial automation is everywhere. Factory floors are filled with safety cages containing robot arms that perform precise repetitive automated tasks. Large warehouses have robots that drive along predetermined tracks, fetching and delivering packages. In underground mines, a large wheel loader can follow a preprogrammed track without a human operator. 

However, many industries still rely on human operators and labor. These are usually highly unstructured and varying environments and, paradoxically, also the most dangerous environments for people to work in, such as construction, tree harvesting and mining.

 

computer simulated crane picking up dirt
Automated and non-automated tasks: a simple organized environment in a factory compared with a much more complex environment in an underground mine

The importance of accurate physics

Algoryx has more than a decade of experience with full-system physics simulation of heavy machinery. The core Algoryx product, AGX Dynamics, is an SDK for numerical simulation of multibody and multidomain dynamics.

With the goal of helping the industry solve real-world problems, Algoryx has moved beyond idealized models and developed numerical methods that both follow the fundamental principles of physics and also allow for complex real-world modeling. For example, the models allow for gaps or clearance in joints, elastoplastic beams, joint limits, breakable joints, piecewise linear models and much more.

These are all common phenomena we encounter in reality, and if not modeled to the right level, the simulation will not be able to solve any real-world problems. The variational time stepper in AGX Dynamics makes it possible to simulate discontinuous (or nonsmooth) physical events, such as impacts, of these non-idealized models in real-time or faster. This is achieved by resolving the conditions both geometrically and globally at each time step, resulting in stable physics over indefinitely long simulation times. Combined with Algoryx’s fast direct solver, which can solve large sparse systems with machine precision, this makes it possible to simulate “stiff” systems with large mass ratios without any special treatment.

The dynamics of machines depend on many machine components, including engine type, gearbox, differentials, hydraulics, tires, tracks, and deformable objects like cables or wires, and the machines must be able to interact with the environment. A ship must float in water and a wheel loader must drive on and load soil. All of these components are modeled using the same framework, leading to unified, tightly coupled simulations. This combination of models and numerical methods is based on the most recent scientific literature.

Industry partners trust the performance and accuracy of AGX Dynamics for their professional training simulators or engineering tools. In training simulators, human operators learn how to control the machines in a safe and efficient way. The operator’s skills, often measured in terms of eye-muscle coordination, have high transfer to reality thanks to the fidelity of the models.

Engineering tools, on the other hand, require in-depth analysis of a machine’s dynamics while it interacts with the environment – for example, to measure real forces in joints or energy consumption of an electric drivetrain. Simulations that can guide early design decisions with the confidence that the same behavior will be observed in the real world are of tremendous value to customers. Now, thanks to the integration of AGX Dynamics with Unity, it is possible to use the powerful Unity Editor to set up the machine and environment model and simulate it using AGX Dynamics.

Crane picking up a box
Wires
robotic arm fixing a car door
Cables and cable damage
Granular
Granular
Terrain
Terrain
Drivetrain, tires and tracks
Drivetrain, tires and tracks
Hydrodynamics and wind
Hydrodynamics and wind

Successfully transfer simulations to the real world

Automating the work of these vehicles is much harder than automating a robot repeating the same pick-and-place motion in a factory. The varying environment makes the problem more complex, as do the dynamics of the task. Highly skilled operators can exploit the dynamics of the vehicle and the objects they interact with to greatly improve their performance, compared to less experienced operators.

What to measure – how to relate it to the control of a vehicle or make an automated decision – has proven difficult to solve. Human operators often make decisions according to intuition, based on many different sensory inputs such as vision, sound and reaction forces of the vehicle. That intuition is built up over thousands of hours of training in which operators experience different, yet similar, situations, first in training simulators and then on the real machine. A training simulator requires accurate physics for the experience gained to be successfully transferred when driving the real machine.

Physical training simulator, images courtesy of Oryx Simulations
Physical training simulator, images courtesy of Oryx Simulations

In recent years, reinforcement learning has shown promise in figuring out these advanced environments where classical solutions have failed. Using reinforcement learning, automation engineers do not need to solve every part of the automation problem separately and then connect the pieces.

It is possible to define the task to solve, and which actions and observations are available to solve it, and then let an agent explore the environment and "figure out" how to solve it. This sounds easy in theory, but in reality, the amount of exploration and training required to solve a single task can be daunting, and doing it on a real machine can be both dangerous and costly.

The solution? Do it in a simulation instead. It is not possible to damage the machine in a simulation, the simulation can run faster than real time and you can run several simulations in parallel without significant extra cost. However, just as for human operators, accurate physics is necessary for learned experience to usefully transfer from the simulation to the real machine. If your model is wrong, the solution the agent learns might not work in the real world.

A sample of elements that constitute reinforcement learning
A sample of elements that constitute reinforcement learning

Using the Unity engine and Unity Machine Learning Agents (ML-Agents), it is easy to model the environment with one or multiple agents in it. You can use Unity SystemGraph to simulate different visual sensors that the agent uses to observe its surroundings.

In the beginning, the agent will not know the series of actions to perform to solve the task, but by giving the agent a reward that depends on the previous state-action transition, the reinforcement learning algorithms in ML-Agents aim to find a policy that maximizes the return, ultimately allowing the agent to solve the task. Combining this with the fast and accurate physics from AGX Dynamics for Unity, it is possible to model and train heavy machinery to intelligently and autonomously solve very complex problems using only simulations.

 

Autonomous forestry: Log-grasping control

Jennifer Andersson of Umeå University worked with Algoryx to investigate how to use reinforcement learning to automate the log-grasping motion of a forestry crane manipulator. Experienced human operators can control the underactuated crane on a forestry machine to successfully grasp multiple logs while driving over uneven ground and avoiding collisions with trees. This is no small feat, requiring counterintuitive coordination of several actuators, resulting in work that is both mentally and physically exhausting for the operators. 

By using reinforcement learning in an environment created with Unity and ML-Agents and simulated with AGX Dynamics for Unity, an agent was trained to individually control the six joints of the crane on a forest forwarder to perform single log grasping. The best control policy succeeded in grasping the logs 97% of the time, and the strategy and cycle time is comparable to that of experienced human operators. For example, the agent is able to use the swing of the underactuated grasping claw while grasping, a technique that improves cycle times but is hard to learn. 

The study also showed that agents trained with an energy optimization goal in the reward function significantly decreased the energy consumed compared to agents trained without this goal. If you want to learn more, please refer to the scientific paper presented at International Conference on Intelligent Robots and Systems (IROS) 2021 (video and PDF).

A side-by-side comparison of real and virtual forestry log grasping, from IROS 2021

Autonomous consecutive loading trajectories in underground mines

In another example, Algoryx partnered with Epiroc, a world-leading manufacturer of tunneling and mining equipment, to automate the loading of one of its larger underground mine wheel loaders, the ST-18. The vehicle is capable of loading 18 tonnes of blasted rock in its bucket and runs nonstop in many underground mines around the world. When driving an underground drift forward, the end of the drift is blasted, resulting in a large pile of fragmented rock (muck) that must be removed before the next blasting.

In simulations, an agent has been trained both to plan loading positions, using depth images, and to control the vehicle to efficiently fill the bucket over many subsequent loading cycles. Planning, filling the bucket, and avoiding collisions and wheel slippage were all achieved by having the agent control throttle, steering and lift/tilt of the bucket. The best results, on average 75% of the max capacity filled, were obtained by including a penalty for energy consumption in the reward function. For more information, see the scientific paper published in Machines.

AGX Dynamics for Unity enables engineers to safely design, develop and test complex autonomous machines. The above examples are two of many applications where Algoryx used the Unity platform for machine learning. Together, our technology can solve other types of difficult-to-automate tasks, making the world better by simulating it first.

This content is hosted by a third party provider that does not allow video views without acceptance of Targeting Cookies. Please set your cookie preferences for Targeting Cookies to yes if you wish to view videos from these providers.

The vehicle solving the task after training

AGX Dynamics for Unity enables engineers to safely design, develop and test complex autonomous machines. The above examples are two of many applications where Algoryx used the Unity platform for machine learning. Together, our technology can solve other types of difficult-to-automate tasks, making the world better by simulating it first.

Learn more

Interested in exploring the potential of simulation for your industrial scenarios? Find out how Algoryx can enable virtual prototyping, training simulators, and digital twins.

Or contact Algoryx directly to discuss your machine autonomy plans.

December 20, 2021 in Industry | 13 min. read

Is this article helpful for you?

Thank you for your feedback!