In order to solve the difficult problems of caused by mutual occlusion of face and face orientation, a PNMS algorithm based on penalty factors was proposed to improve the accuracy of face detection and alignment. Firstly, according to the overlap degree between face candidate windows and the corresponding detection scores of candidate windows, a non-continuous linear function and a continuous function based on Gaussian distribution were proposed and used as penalty factors for non-maximum suppression. Then the traditional non-maximum suppression algorithm was improved and replaced, and the detection score of the candidate window was redistributed. On this basis, combining the characteristics of the first two kinds of penalty factors, the continuous nonlinear function was further proposed as the penalty factor of the non-maximum suppression algorithm. Consequently, the greater the overlap value between windows, the more severe the penalty is, and the function is continuous throughout the overlapping value range. The proposed algorithm performed detailed face detection experiment verification on two face detection data sets of FDDB and WIDER FACE. The face alignment experiments were verified on the AFLW data set. The results showed that the proposed PNMS algorithms compared with other algorithms not only effectively improves the accuracy and reliability of face detection and alignment, but also solves a certain degree of face occlusion, and reduces the rate of detection failure of occluded faces.