A meteor could take out a whole planet if it were big enough, not that it would destroy the planet but make it almost impossible to live there. This is what we think happened to the dinosaurs. So the answer is yes.
The mass and speed does definitely matter, particularly the mass, since more massive asteroids are less likely to get burnt up in the atmosphere. The range of speeds is at least partly set by the Earth’s gravity, which pulls the asteroid towards us, so doesn’t vary as much as the mass. Another thing that matters is the direction the asteroid comes from, since one that comes in directly from above has less atmosphere to travel through than one that comes in at an angle. So ones that come straight down are more likely to hit the Earth.