Abstract

Hydraulic pressure ripple in a pump, as a result of converting rotational power to fluid power, continues to be a problem faced when developing hydraulic systems due to the resulting noise generated. In this paper, we present simulation results from leveraging an actor-critic reinforcement learning method as the control method for active noise control in a hydraulic system. The results demonstrate greater than 96%, 81%, and 61% pressure ripple reduction for the first, second, and third harmonics, respectively, in a single operating point test, along with the advantage of feed forward like control for high bandwidth response during dynamic changes in the operating point. It also demonstrates the disadvantage of long convergence times while the controller is effectively learning the optimal control policy. Additionally, this work demonstrates the ancillary benefit of the elimination of the injection of white noise for the purpose of system identification in the current state of the art.

References

1.
Guicking
,
D.
,
2002
,
An Overview of ASVC From Laboratory Curiosity to Commercial Products, Institute of Engineering and Technology
, London, UK, Chap. 1.
2.
Wang
,
L.
,
2008
, “
Active Control of Fluid-Borne Noise
,” Ph.D. thesis,
University of Bath
,
Bath, UK
.
3.
Eriksson
,
L. J.
, and
Allie
,
M. C.
,
1989
, “
Use of Random Noise for On-Line Transducer Modeling in an Adaptive Active Attenuation System
,”
J. Acoust. Soc. Am.
,
85
(
2
), pp.
797
802
.10.1121/1.397552
4.
Kuo
,
S. M.
, and
Vijayan
,
D.
,
1997
, “
A Secondary Path Modeling Technique for Active Noise Control Systems
,”
IEEE Trans. Speech Audio Process.
,
5
(
4
), pp.
374
377
.10.1109/89.593319
5.
Kojima
,
E.
, and
Shinada
,
M.
,
1991
, “
Development of an Active Attenuator for Pressure Pulsation in Liquid Piping Systems: A Real Time-Measuring Method of Progressive Wave in a Pipe
,”
JSME Int. J. Ser.
,
34
(
4
), pp.
466
473
.10.1299/jsmeb1988.34.4_466
6.
Jiao
,
Z.
,
Chen
,
P.
,
Hua
,
Q.
, and
Wang
,
S.
,
2003
, “
Adaptive Vibration Active Control of Fluid Pressure Pulsations
,”
Proc. Inst. Mech. Eng., Part
,
217
(
4
), pp.
311
318
.10.1177/095965180321700407
7.
Yokota
,
S.
,
Somada
,
H.
, and
Yamaguchi
,
H.
,
1996
, “
Study on an Active Accumulator. (Active Control of High-Frequency Pulsation of Flow Rate in Hydraulic Systems)
,”
JSME Int. J. Ser. B
,
39
(
1
), pp.
119
124
.10.1299/jsmeb.39.119
8.
Maillard
,
J.
,
Lagö
,
T. L.
,
Winberg
,
M.
, and
Fuller
,
C.
,
1999
, “
Fluid Wave Actuator for the Active Control of Hydraulic Pulsations in Piping Systems
,”
IMAC-XVII, International Modal Analysis Conference
, Bethel, CT, pp.
1806
1812
.
9.
Sutton
,
R. S.
, and
Barto
,
A. G.
,
2018
,
Reinforcement Learning: An Introduction
,
MIT Press
,
Cambridge, MA
.
10.
ISO
,
2015
, “
Hydraulic Fluid Power – Determination of Pressure Ripple Levels Generated in Systems and Components – Part 1: Method for Determining Source Flow Ripple and Source Impedance of Pumps
,”
Standard, International Organization for Standardization
,
ISO
,
Switzerland, Geneva
, Standard No. ISO 10767-1:2015(E).
11.
Manring
,
N. D.
,
2000
, “
The Discharge Flow Ripple of an Axial-Piston Swash-Plate Type Hydrostatic Pump
,”
ASME J. Dyn. Syst., Meas., Control
,
122
(
2
), pp.
263
268
.10.1115/1.482452
12.
MOOG
,
2019
, “
Servovalves with Integrated Electronics d765 Series ISO 10372 Size 04
,” Brochure.
13.
Trikha
,
A. K.
,
1975
, “
An Efficient Method for Simulating Frequency-Dependent Friction in Transient Liquid Flow
,”
ASME J. Fluids Eng.
,
97
(
1
), pp.
97
105
.10.1115/1.3447224
14.
Taylor
,
S. E. M.
,
Johnston
,
D. N.
, and
Longmore
,
D. K.
,
1997
, “
Modelling of Transient Flow in Hydraulic Pipelines
,”
Proc. Inst. Mech. Eng., Part I
,
211
(
6
), pp.
447
455
.10.1243/0959651981540035
15.
Johnston
,
N.
,
2012
, “
The Transmission Line Method for Modelling Laminar Flow of Liquid in Pipelines
,”
Proc. Inst. Mech. Eng., Part I
,
226
(
5
), pp.
586
597
.10.1177/0959651811430035
16.
Andrychowicz
,
M.
,
Crow
,
D.
,
Ray
,
A.
,
Schneider
,
J.
,
Fong
,
R.
,
Welinder
,
P.
,
McGrew
,
B.
,
Tobin
,
J.
,
Abbeel
,
O. P.
, and
Zaremba
,
W.
,
2017
, “
Hindsight Experience Replay
,” Advances in Neural Information Processing Systems, Long Beach, CA, pp.
5055
5065
.
17.
Wang
,
Z.
,
Bapst
,
V.
,
Heess
,
N.
,
Mnih
,
V.
,
Munos
,
R.
,
Kavukcuoglu
,
K.
, and
de Freitas
,
N.
,
2016
, “
Sample Efficient Actor-Critic With Experience Replay
,” eprint arXiv:1611.01224 [cs].
You do not currently have access to this content.