Device-free passive localization enables locating targets (e.g., intruders or victims) that do not carry any radio devices nor do they actively participate in the wireless localization process. This is because the wireless environments will get affected when people move into the area, which result in the changes of Received Signal Strength (RSS) of the wireless links. In this paper, we first show that the localization performance degrades significantly when people are moving in dynamic speeds. This is because existing studies in device-free passive localization system have an implicit assumption that the target is moving at a constant speed, which is not always true in practical scenarios. To cope with targets moving with dynamic speeds, we propose an adaptive speed change detection framework including three components: speed change detection, determination of time-window size and adaptive localization. Two speed change detection schemes have been developed to capture the changes of moving speed and adjust the time-window size adaptively to facilitate effective localization. We demonstrate that our framework is flexible to work with any device-free localization method using signal strength. Results from the real experiments confirm that our approach has over 30% improvement on both median and max localization error, under dynamically changing speed of the target.