Is the invisible hand infallible and does that assumption lead to the conclusion that non-moral actors like corporations should rightfully be free to do whatever they care to do?
It does seem as though the notion of a "free market" has been corrupted to reinforce this interpretation. The traditional definition of a free market was that of a market where there was freedom of entry - meaning there were no barriers to anyone competing with existing suppliers. Necessarily, a free market would have a large number of small suppliers with none so large as to be able, alone, to influence price.
Today, the notion of a "free market" has been corrupted to mean nearly the opposite of this traditional definition. Today, there seems a widespread acceptance of the idea that a market is free if and only if the government does not regulate the market. Presumably, this notion is motivated by the assumption that the invisible hand is truly infallible. But we know that a market that is not regulated by government, monopolies emerge to eliminate the true freedom in that market. Once there is a monopoly, new competitors will not be able to compete. We also know, if we examine the issue at all dispassionately, that business cannot function properly without any government regulation. Business depends on the structure and enforcement of law much more than private individuals do.
If the free hand is infallible and if it was so important in the thinking of Adam Smith you would think he would have mentioned it often. But the quote given at the start of this article seems to be his only mention of the concept.