Non-offensive substitute for a swear word

I would call this a minced oath:

an expression based on a profanity or a taboo term that has been altered to reduce the objectionable characteristics.

Lots of fun stuff like "strewth", "cheese and rice"...

The general term for these is euphemism.

The use of a word or phrase to replace another with one that is considered less offensive or less vulgar than the word or phrase it replaces.

Edit: just found another term: minced oath. I've never actually heard this term used, mind you, but still, it's a closer match than plain "euphemism".

The basic process is called euphemizing (replacing a "harsh" word with a "softer" word or phrase).

Another common term is "minced oath". This term literally refers to a substitute for the profane rather than the obscene or vulgar, however, I think it is sometimes used as a more general term for any such substitution.

So "gosh darn it", a minced oath for "God damn it" would be literally a minced oath. But "shut the front door", a substitute for "shut the f*** up" would not be an oath, and so not literally a minced oath. But I think the term is applied to both.