Abstract

Model order determination of linear dynamic systems is important for system analysis and control system design. Determining the order of a linear dynamic system usually involves computing the rank of a Hankel matrix formulated using the system's Markov parameters. In a singular value plot of this Hankel matrix, a distinct drop is evident between the nonzero and zero singular values. This drop is most notable at the singular value whose index aligns with the system's order. However, this clear separation can be substantially diminished by even slight noise in the Markov parameters, thus posing difficulties in the accurate estimation of the system's order. Heuristic order estimation methods such as methods based on nuclear norm minimization (NNM) can suffer from scalability issues. As the order of the dynamic system increases, the computational time and memory requirement for these methods increase significantly. In this paper, we introduce a nonheuristic and noniterative algorithm to estimate the order of a linear dynamic system from noisy Markov parameters. Input–output data obtained from running a single experiment of the system can be split into two halves to obtain two vectors of estimated Markov parameters. We show that the largest singular value of the Hankel matrix constructed from half the difference between the two vectors of estimated Markov parameters serves as a threshold, separating true and spurious singular values. The proposed algorithm is demonstrated on single-input, single-output, multi-input, multi-output, stable, and unstable systems, and verified numerically and experimentally, providing a more scalable alternative to heuristic methods.

References

1.
Brunton
,
S. L.
, and
Kutz
,
J. N.
,
2019
,
Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control
,
Cambridge University Press
, Cambridge, UK.
2.
Jolliffe
,
I.
,
2002
, “
Mathematical and Statistical Properties of Population Principal Components
,”
Principal Component Analysis
, Springer, New York, pp.
10
28
.
3.
Child
,
D.
,
1990
,
The Essentials of Factor Analysis
,
Cassell Educational
, New York.
4.
Al-Tawaha
,
A.
, and
Jin
,
M.
,
2024
, “
Does Online Gradient Descent (and Variants) Still Work With Biased Gradient and Variance?
,”
2024
American Control Conference (
ACC
), Toronto, ON, Canada, July 10–12, pp.
3570
3575
.10.23919/ACC60939.2024.10644262
5.
Sel
,
B.
,
Tawaha
,
A.
,
Ding
,
Y.
,
Jia
,
R.
,
Ji
,
B.
,
Lavaei
,
J.
, and
Jin
,
M.
,
2023
, “
Learning-to-Learn to Guide Random Search: Derivative-Free Meta Blackbox Optimization on Manifold
,”
Learning for Dynamics and Control Conference
, Philadelphia, PA, June 15–16, pp.
38
50
.https://proceedings.mlr.press/v211/sel23a/sel23a.pdf
6.
Khattar
,
V.
,
Ding
,
Y.
,
Sel
,
B.
,
Lavaei
,
J.
, and
Jin
,
M.
,
2024
, “
A CMDP-Within-Online Framework for Meta-Safe Reinforcement Learning
,”
ICLR 2023 Conference Program Chairs
, Kigali, Rwanda, May 1, pp. 1–49.https://openreview.net/pdf?id=mbxz9Cjehr
7.
Al-Tawaha
,
A.
,
Kaushik
,
H.
,
Sel
,
B.
,
Jia
,
R.
, and
Jin
,
M.
,
2023
, “
Decision-Focused Learning for Inverse Noncooperative Games: Generalization Bounds and Convergence Analysis
,”
IFAC-PapersOnLine
,
56
(
2
), pp.
9336
9341
.10.1016/j.ifacol.2023.10.221
8.
Akaike
,
H.
,
1974
, “
A New Look at the Statistical Model Identification
,”
IEEE Trans. Autom. Control
, 19(6), pp. 716–723.10.1109/TAC.1974.1100705
9.
Ghosh
,
J. K.
,
Delampady
,
M.
, and
Samanta
,
T.
,
2007
,
An Introduction to Bayesian Analysis: Theory and Methods
,
Springer Science & Business Media
, New York.
10.
Burnham
,
K. P.
, and
Anderson
,
D. R.
,
2004
, “
Multimodel Inference: Understanding AIC and BIC in Model Selection
,”
Sociol. Methods Res.
,
33
(
2
), pp.
261
304
.10.1177/0049124104268644
11.
Sontag
,
E. D.
,
2013
,
Mathematical Control Theory: Deterministic Finite Dimensional Systems
, Vol.
6
,
Springer Science & Business Media
, New York.
12.
Fazel
,
M.
,
Pong
,
T. K.
,
Sun
,
D.
, and
Tseng
,
P.
,
2013
, “
Hankel Matrix Rank Minimization With Applications to System Identification and Realization
,”
SIAM J. Matrix Anal. Appl.
,
34
(
3
), pp.
946
977
.10.1137/110853996
13.
Gavish
,
M.
, and
Donoho
,
D. L.
,
2014
, “
The Optimal Hard Threshold for Singular Values Is 4/3
,”
IEEE Trans. Inf. Theory
,
60
(
8
), pp.
5040
5053
.10.1109/TIT.2014.2323359
14.
Bai
,
Z.
, and
Silverstein
,
J. W.
,
2010
,
Spectral Analysis of Large Dimensional Random Matrices
, Vol.
20
,
Springer
, New York.
15.
Chatterjee
,
S.
,
2015
, “
Matrix Estimation by Universal Singular Value Thresholding
,”
Ann. Stat.
,
43
(
1
), pp.
177
214
.10.1214/14-AOS1272
16.
Yin
,
H.
,
Zhu
,
Z.
, and
Ding
,
F.
,
2011
, “
Model Order Determination Using the Hankel Matrix of Impulse Responses
,”
Appl. Math. Lett.
,
24
(
5
), pp.
797
802
.10.1016/j.aml.2010.12.046
17.
Smith
,
R. S.
,
2014
, “
Frequency Domain Subspace Identification Using Nuclear Norm Minimization and Hankel Matrix Realizations
,”
IEEE Trans. Autom. Control
,
59
(
11
), pp.
2886
2896
.10.1109/TAC.2014.2351731
18.
Wang
,
L.
,
Ma
,
S.
, and
Han
,
Q.
,
2022
, “
Enhanced Sparse Low-Rank Representation Via Nonconvex Regularization for Rotating Machinery Early Fault Feature Extraction
,”
IEEE/ASME Trans. Mechatron.
,
27
(
5
), pp.
3570
3578
.10.1109/TMECH.2021.3135284
19.
Liu
,
H.
,
Zhang
,
X.
,
Liao
,
C.
,
Dong
,
H.
,
Liu
,
Z.
, and
Hu
,
X.
,
2023
, “
Synergistic Hankel Structured Low-Rank Approximation With Total Variation Regularization for Complex Magnetic Anomaly Detection
,”
IEEE Trans. Instrum. Meas.
,
72
, pp.
1
10
.10.1109/TIM.2023.3264036
20.
Fazel
,
M.
,
Hindi
,
H.
, and
Boyd
,
S. P.
,
2001
, “
A Rank Minimization Heuristic With Application to Minimum Order System Approximation
,”
Proceedings of the American Control Conference
, Arlington, VA, June 25–27, pp.
4734
4739
.10.1109/ACC.2001.945730
21.
Yin
,
M.
, and
Smith
,
R. S.
,
2021
, “
On Low-Rank Hankel Matrix Denoising
,”
IFAC-PapersOnLine
,
54
(
7
), pp.
198
203
.10.1016/j.ifacol.2021.08.358
22.
Sadigh
,
D.
,
Ohlsson
,
H.
,
Sastry
,
S. S.
, and
Seshia
,
S. A.
,
2014
, “
Robust Subspace System Identification Via Weighted Nuclear Norm Optimization
,”
IFAC Proc. Vol.
,
47
(
3
), pp.
9510
9515
.10.3182/20140824-6-ZA-1003.02698
23.
Liu
,
Z.
,
Hansson
,
A.
, and
Vandenberghe
,
L.
,
2013
, “
Nuclear Norm System Identification With Missing Inputs and Outputs
,”
Syst. Control Lett.
,
62
(
8
), pp.
605
612
.10.1016/j.sysconle.2013.04.005
24.
Hansson
,
A.
,
Liu
,
Z.
, and
Vandenberghe
,
L.
,
2012
, “
Subspace System Identification Via Weighted Nuclear Norm Optimization
,”
Proceedings of the IEEE Conference on Decision and Control
, Maui, HI, Dec. 10–13, pp.
3439
3444
.10.1109/CDC.2012.6426980
25.
Sun
,
Y.
,
Oymak
,
S.
, and
Fazel
,
M.
,
2022
, “
System Identification Via Nuclear Norm Regularization
,” preprint arXiv:2203.16673.
26.
Chiuso
,
A.
, and
Pillonetto
,
G.
,
2019
, “
System Identification: A Machine Learning Perspective
,”
Annu. Rev. Control, Rob., Auton. Syst.
,
2
(
1
), pp.
281
304
.10.1146/annurev-control-053018-023744
27.
Wang
,
X.
,
Wang
,
L.
,
Li
,
X.
, and
Bi
,
G.
,
2017
, “
Nuclear Norm Minimization Framework for DOA Estimation in MIMO Radar
,”
Signal Process.
,
135
, pp.
147
152
.10.1016/j.sigpro.2016.12.031
28.
Xu
,
J.
,
Fu
,
Y.
, and
Xiang
,
Y.
,
2023
, “
An Edge Map-Guided Acceleration Strategy for Multi-Scale Weighted Nuclear Norm Minimization-Based Image Denoising
,”
Digital Signal Process.
,
134
, p.
103932
.10.1016/j.dsp.2023.103932
29.
Fawzi
,
H.
, and
Goulbourne
,
H.
,
2021
, “
Faster Proximal Algorithms for Matrix Optimization Using Jacobi-Based Eigenvalue Methods
,”
35th International Conference on Neural Information Processing Systems
, Dec. 6–14, pp.
11397
11408
.https://proceedings.neurips.cc/paper_files/paper/2021/file/5ef78f63ba22e7dfb2fa44613311b932-Paper.pdf
30.
Al-Tawaha
,
A. S.
,
Aljanaideh
,
K.
, and
Alshorman
,
A.
,
2021
, “
A Singular Value Thresholding Algorithm for Order Estimation
,” American Control Conference (
ACC
), New Orleans, LA, May 25–28, pp.
4478
4483
.10.23919/ACC50511.2021.9482743
31.
Aljanaideh
,
K. F.
, and
Bernstein
,
D. S.
,
2017
, “
Closed-Loop Identification of Unstable Systems Using Noncausal FIR Models
,”
Int. J. Control
,
90
(
2
), pp.
168
185
.10.1080/00207179.2016.1172733
32.
Ljung
,
L.
,
1999
, “
System Identification
,” Prentice Hall PTR, Upper Saddle River, NJ.
33.
Paul
,
D.
, and
Aue
,
A.
,
2014
, “
Random Matrix Theory in Statistics: A Review
,”
J. Stat. Plann. Inference
,
150
, pp.
1
29
.10.1016/j.jspi.2013.09.005
34.
Weyl
,
H.
,
1912
, “
Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung)
,”
Math. Ann.
,
71
(
4
), pp.
441
479
.10.1007/BF01456804
35.
Stewart
,
G. W.
,
1993
, “
On the Early History of the Singular Value Decomposition
,”
SIAM Rev.
,
35
(
4
), pp.
551
566
.10.1137/1035134
36.
Knowles
,
A.
, and
Yin
,
J.
,
2013
, “
Eigenvector Distribution of Wigner Matrices
,”
Probab. Theory Relat. Fields
,
155
(
3–4
), pp.
543
582
.10.1007/s00440-011-0407-y
37.
Benaych-Georges
,
F.
, and
Nadakuditi
,
R. R.
,
2012
, “
The Singular Values and Vectors of Low Rank Perturbations of Large Rectangular Random Matrices
,”
J. Multivar. Anal.
,
111
, pp.
120
135
.10.1016/j.jmva.2012.04.019
38.
Fazel
,
M.
,
Hindi
,
H.
, and
Boyd
,
S. P.
,
2003
, “
Log-Det Heuristic for Matrix Rank Minimization With Applications to Hankel and Euclidean Distance Matrices
,”
Proceedings of the American Control Conference,
Denver, CO, June 4–6, pp.
2156
2162
.10.1109/ACC.2003.1243393
39.
Gavish
,
M.
, and
Donoho
,
D. L.
,
2017
, “
Optimal Shrinkage of Singular Values
,”
IEEE Trans. Inf. Theory
,
63
(
4
), pp.
2137
2152
.10.1109/TIT.2017.2653801
40.
De Moor
,
B.
,
De Gersem
,
P.
,
De Schutter
,
B.
, and
Favoreel
,
W.
,
1997
, “
DAISY: A Database for Identification of Systems
,” ESAT/SISTA, KU Leuven, Belgium, accessed Mar. 19, 2025, https://homes.esat.kuleuven.be/~smc/daisy/
41.
Ho
,
B.
, and
Kalman
,
R.
,
1966
, “
Effective Construction of Linear State Variable Models From Input/Output Functions
,”
Regelungstechnik
,
14
, pp.
545
548
.10.1524/auto.1966.14.112.545
You do not currently have access to this content.