The paper addresses the problem of improving the spectral efficiency of radio communication systems operating in channels with Doppler frequency shift by employing optimal signals with amplitude-phase modulation. The relevance of the study is driven by the need for high data rate transmission in low-Earth orbit satellite communication systems, where conventional signals based on RRC pulses exhibit significant energy losses when the symbol rate is increased. A synthesis technique for optimal shaping pulses is proposed, based on the criterion of out-of-band emission suppression under given constraints on the cross-correlation coefficient that determines the BER performance. A computer simulation model of the transmission channel is developed, taking into account carrier frequency instability, timing synchronization offset, and Doppler shift modeled with time-scale transformation. Numerical simulations for 64APSK constellation and coding rate 4/5 demonstrate that the use of optimal signals with doubled symbol rate provides a spectral efficiency gain of up to 12.7% compared to RRC-based signals when the occupied bandwidth is estimated by the 99% energy concentration criterion. The associated energy penalties range from 3 to 4 dB depending on the satellite elevation angle, which is attributed to the need for wider receiver front-end bandwidth under Doppler-induced spectral distortions. It is shown that the choice of the cross-correlation coefficient enables a trade-off between BER performance and spectral efficiency. The obtained results confirm the feasibility of optimal signals in satellite communication systems with high spectral efficiency requirements and limited link power budgets.