Technically, the Bible and, thus, Mormon theology considers convoluted euphemisms like "gosh-darn", "heck", and "dangit" to be swearing, too. Matt 5:34-37 establishes the standard:
Matthew, in chapter 5, KJV, wrote:
34. But I say unto you, Swear not at all; neither by heaven; for it is God’s throne:
35. Nor by the earth; for it is his footstool: neither by Jerusalem; for it is the city of the great King.
36. Neither shalt thou swear by thy head, because thou canst not make one hair white or black.
37. But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil.
Invoking entities you have no power over to emphasize your claims ("Hell yes!") is considered a lie, since it is technically a claim to take additional consequence for your actions that, typically, you don't have the power to take. In the case of nonsense like "darn" or "golly gee", you can't take the extra consequence because it doesn't actually mean anything. Sure, people don't often mean to take an oath when they say such things, but if you don't mean it why say it?
Using such silly euphemisms is, thus, a concession to cultural pressures rather than theological reasoning.