For round peg in square hole-
P = pi x 5 x 5 = 25pi
H = 5 x 5= 25
P : H = pi : 1
__________________________

For square peg in round hole-
P = 5 x 5 = 25
H = 5 x 5 x pi= 25pi
P : H = 1 : pi
________________________________
Clearly ratio of P : H is greater in 1st case, so the best fit would be the first one.

-Ajay Porwal