What you're missing here is that a lot of the time (as in this case) when people say "the Universe" what they really mean is "The OBSERVABLE Universe", which is a VERY large but definitely finite size (about 93 billion light years). It is defined by the distance that light has had time to reach us since the beginning of the universe. https://en.wikipedia.org/wiki/Observable_universe
So you may be right about how large the universe outside of that area could be (which is unknowable, since it can literally never be observed) but it's not really relevant in this context.