Not a chance. The orbits are all much too stable for anything like that to happen, but asteroids CAN and DO get knocked out of the asteroid belt and impact the Earth from time to time. That's what we really have to worry about. The dinosaurs didn't, and look where it got them?