The main control problem of visual servoing is to cope with the delay introduced by image acquisition and image processing. This delay is the main reason for limited tracking velocity and acceleration. Predictive algorithms are one solution to handle the delay. The drawback of prediction algorithms is the bad prediction behavior for the discontinuity in the target motion, for example, a velocity step. In this paper, a switching Kalman filter (SKF) is proposed to overcome this problem. The SKF introduces three cooperating components. A prediction monitor supervises the prediction quality of an adaptive Kalman filter (AKF). If a discontinuity is detected, a transition filter switches to an appropriate steady-state Kalman filter (alpha beta or alpha beta gamma), which handles a discontinuity better than the AKF. During this transition, an auxiliary controller ensures that overall control is continuous. This new prediction algorithm is able to achieve a good prediction quality for smooth and for discontinuous motions. It is evaluated using a pan/tilt unit to track a colored object. The SKF and its components are compared to the classical AKF with four different target motions.