I'm gonna say that it matters somewhat. I think most people would agree that a female prefers to be shorter. But I've also seen tall women (5'8"+) with short men (5'3"). So it is definitely a depends situation.
I think social media just really makes this such a polarizing topic though.