In this paper, reinforcement learning is applied to coordinate, in a decentralized fashion, the motions of a pair of hydraulic actuators whose task is to firmly hold and move an object along a specified trajectory under conventional position control. The learning goal is to reduce the interaction forces acting on the object that arise due to inevitable positioning errors resulting from the imperfect closed-loop actuator dynamics. Each actuator is therefore outfitted with a reinforcement learning neural network that modifies a centrally planned formation constrained position trajectory in response to the locally measured interaction force. It is shown that the actuators, which form a multiagent learning system, can learn decentralized control strategies that reduce the object interaction forces and thus greatly improve their coordination on the manipulation task. However, the problem of credit assignment, a common difficulty in multiagent learning systems, prevents the actuators from learning control strategies where each actuator contributes equally to reducing the interaction force. This problem is resolved in this paper via the periodic communication of limited local state information between the reinforcement learning actuators. Using both simulations and experiments, this paper examines some of the issues pertaining to learning in dynamic multiagent environments and establishes reinforcement learning as a potential technique for coordinating several nonlinear hydraulic manipulators performing a common task.
Skip Nav Destination
e-mail: nariman@cc.umanitoba.ca
Article navigation
September 2007
Technical Papers
Decentralized Coordinated Motion Control of Two Hydraulic Actuators Handling a Common Object
Mark Karpenko,
Mark Karpenko
Department of Mechanical and Manufacturing Engineering,
University of Manitoba
, Winnipeg, Manitoba, R3T 5V6, Canada
Search for other works by this author on:
Nariman Sepehri,
Nariman Sepehri
Department of Mechanical and Manufacturing Engineering,
e-mail: nariman@cc.umanitoba.ca
University of Manitoba
, Winnipeg, Manitoba, R3T 5V6, Canada
Search for other works by this author on:
John Anderson
John Anderson
Department of Computer Science,
University of Manitoba
, Winnipeg, Manitoba, R3T 5V6, Canada
Search for other works by this author on:
Mark Karpenko
Department of Mechanical and Manufacturing Engineering,
University of Manitoba
, Winnipeg, Manitoba, R3T 5V6, Canada
Nariman Sepehri
Department of Mechanical and Manufacturing Engineering,
University of Manitoba
, Winnipeg, Manitoba, R3T 5V6, Canadae-mail: nariman@cc.umanitoba.ca
John Anderson
Department of Computer Science,
University of Manitoba
, Winnipeg, Manitoba, R3T 5V6, CanadaJ. Dyn. Sys., Meas., Control. Sep 2007, 129(5): 729-741 (13 pages)
Published Online: January 25, 2007
Article history
Received:
June 23, 2006
Revised:
January 25, 2007
Citation
Karpenko, M., Sepehri, N., and Anderson, J. (January 25, 2007). "Decentralized Coordinated Motion Control of Two Hydraulic Actuators Handling a Common Object." ASME. J. Dyn. Sys., Meas., Control. September 2007; 129(5): 729–741. https://doi.org/10.1115/1.2764516
Download citation file:
Get Email Alerts
Offline and Online Exergy-Based Strategies for Hybrid Electric Vehicles
J. Dyn. Sys., Meas., Control (May 2025)
Multi Combustor Turbine Engine Acceleration Process Control Law Design
J. Dyn. Sys., Meas., Control
A Distributed Layered Planning and Control Algorithm for Teams of Quadrupedal Robots: An Obstacle-Aware Nonlinear Model Predictive Control Approach
J. Dyn. Sys., Meas., Control (May 2025)
Active Data-Enabled Robot Learning of Elastic Workpiece Interactions
J. Dyn. Sys., Meas., Control (May 2025)
Related Articles
Impact Control in Hydraulic Actuators
J. Dyn. Sys., Meas., Control (June,2005)
Flatness-Based High Frequency Control of a Hydraulic Actuator
J. Dyn. Sys., Meas., Control (March,2012)
Tip-Over Stability of Manipulator-Like Mobile Hydraulic Machines
J. Dyn. Sys., Meas., Control (June,2005)
Hydraulic Remotely Adjustable Pulmonary Artery Banding
J. Med. Devices (June,2010)
Related Proceedings Papers
Related Chapters
A Novel Approach for LFC and AVR of an Autonomous Power Generating System
International Conference on Mechanical Engineering and Technology (ICMET-London 2011)
Manipulability-Maximizing SMP Scheme
Robot Manipulator Redundancy Resolution
A New Exploratory Neural Network Training Method
Intelligent Engineering Systems through Artificial Neural Networks, Volume 16