I think it should be about the relation between diameter(D) and square side (x).

(1.) Square Peg in a Round Hole

To fit in, Diameter (D) = x sqrt (2)

(2.) Round Peg in a Square Hole
To fit in, Diameter (D) = x

So, (2) shall be tighter than (1) since square peg required less diameter hole to fit in (which is D/sqrt(2)).