Such requirements will vary depending on the design of the synthesizer.
A more general statement has been given by James A. Crawford: 50 reference cycles as a rule of thumb.
By this rule, a reference frequency of 50 kHz has a settling time of 1 millisecond.
[1] Two other authors state (Hamid Rategh and Thomas H. Lee) that the switching time (i.e., settling time) is a function of the percentage change in the feedback division ratio.
[2] So according to them, the delta N over N itself determines the switching time, where N is the frequency synthesizer's feedback divisor.