It's fascinating to hear how Mr. Wiles describes doing mathematics :)
For me, too, the part on "being stuck" was most insightful.

"Discovered" is obviously the right term when it comes to a general mathematical structure:
people working independently on the same problem will discover the same one,
e.g. AFAIK several people discovered complex numbers independently
(however there might be a smooth transition to "invented" when it comes to details of a proof).

Also the structure/landscape of math doesn't "exist" in a physical way - it's just everything that's (logically) consistent. There's no need for any entity to "create" it.
Indeed it's the other way round: any entity - whether mathematical or physical - is limited by the structure of mathematics.