Combined with the free-running model tests of KVLCC ship, the system identification (SI) based on support vector machines (SVM) is proposed for the prediction of ship maneuvering motion. The hydrodynamic derivatives in an Abkowitz model are determined by the Lagrangian factors and the support vectors in the SVM regression model. To obtain the optimized structural factors in SVM, particle swarm optimization (PSO) is incorporated into SVM. To diminish the drift of hydrodynamic derivatives after regression, a difference method is adopted to reconstruct the training samples before identification. The validity of the difference method is verified by correlation analysis. Based on the Abkowitz mathematical model, the simulation of ship maneuvering motion is conducted. Comparison between the predicted results and the test results demonstrates the validity of the proposed methods in this paper.