Echo state network (ESN) is an important method for time series prediction. However, the overfitting problem is likely to occur when the training data contain noise or outliers. To solve this problem, an ESN model based on smoothly clipped absolute deviation (SCAD) penalty function was proposed in this paper. Different from the traditional methods, such as ridge regression, L1 norm penalty, wavelet denoising and other methods added into the ESN model, the SCAD penalty function was used to select the variables of the ESN model. Specially, to meet the variable sparseness, the small coefficients are set to zero. And the large coefficients are taken as constants,which can well solve the over-fitting problem of ESN and satisfy approximate unbiased estimation. For the nonconvex optimization problem of SCAD penalty function, the local quadratic approximation (LQA) solution was presented in the paper, and the enormous computational complexity of the least angle regression (LQR) method for solving the SCAD penalty function was overcome.Then,the particle swarm optimization (PSO) is used to quickly determine the hyperparameters selection of smoothly clipped absolute deviation-echo state network (SCAD-ESN) model. The proposed method overcame the blindness of the conventional methods using the experience to select the hyperparameters, which is blind and difficult to determine the global optimum. Finally, the chaotic system simulation and network traffic simulation showed that,compared with the conventional models, the model can effectively reduce the test error and overcome overfitting problem.