What do you call the thing on which something is attached?

For the tick example, the first word in my mind is host. I'm not sure that works as well for a rifle, but it doesn't sound unnatural to my ear, at least.

Well, what do you know? I found evidence it is used just that way for firearms:

Silencers generally increase the accuracy of a host firearm while reducing recoil and eliminating up to 90% of the muzzle signature.

Source found here.

The main part of the thing would be the base.

base (n.) - A main or important element or ingredient to which other things are added

It really depends on what type of thing the thing is and what type of something it is on.

Host is certainly appropriate for a parasitic relationship such as a tick or a leech on an animal. As indicated, it could also work for inanimate objects, but people might first reach for 'bearer', 'holder', or 'carrier' instead, depending on how they wish to characterize the relationship.

Depending upon context there are options, mostly engineering terms.

The principal, primary, or major (assembly, structure).

The chassis.

The mother (board, lode, ship). The master.

Also consider:

Equipped - The rifle was equipped with a scope. A scope equipped rifle.

"Host" has usage beyond the lowly parasite as well.

Might as well add carrier, as in any sort of shuttle that delivers, and also vector, as in mosquitos are a disease vector.