Why would that be the case? Why would there be the same amount of girls as boys who are born 4 years earlier?
If they're arguing that 4 years isn't a significant amount of time to affect the numbers, then that should hold true whether there's a 4 year gap, 1 year gap, or no gap at all.
There are many possible reasons, and it's likely a combination of at least some of them. None of which are that it's too short to make a difference. Here are 2 I can think of offhand.
1) The average age gap is actually less than 4 years (anecdotally, in my family it's 1.5 years, 2.5 years, and 6 weeks. For my wife's family it's 2 years.)
2) There are more boys born every year
ETA: The whole premise of the age gap theory is based on a few percent difference each year (growth rate minus the greater number of boys born each year). It's ridiculous to think that G-d can't figure out how to make some minor adjustments of a few percentage points to make everything work out.