Calling a twenty-mill world "average" is a little misleading because it's the arithmetic mean; the population of a median world would be even lower.
Trying to sum up published worlds can also be misleading if there's booms or mass relocations in between one set of figures and the next. Are the TTSs set in 3067, or later?
Most of the first world have special pressures of both artificial (higher costs of raising children, social promotion of having few/no children, etc) and natural origins (biological feedback from overcrowding cities, etc). These pressures are unlikely to occur in colonies, where I would expect the families to have 6+ children.
I don't think the average colonist has the ability to triple their arable land every twenty years, but perhaps we are thinking of different spectrums of colonies and different phases of the colonization.
Not sure what this has to do with the OP to be honest, though.
It's a question of how plausible it is for world populations to plateau in the thousands or millions instead of proceeding unchecked to near-Earth levels.
Though of course, once we determine
that there's still the question of just how many unpublished worlds there are.