Why don't we pluralize "foot" in measurements?

There are many such cases of using singular forms for plural meaning, not only in English, but also in German, and possibly other Germanic languages, or even non-Germanic languages. For example, you order "drei Bier" instead of "drei Biere", and in a football match, there are "elf Mann" on the pitch, rather than "elf Männer". (The plural forms are not strictly incorrect, just less common.)

In general, the "why" question doesn't have a satisfactory answer for this kind of linguistic fact; don't search for logic and reason where there's only history and convention.

Ah, and of course you also say "drei Fuß" in German (on the rare occasion you'd be referring to pre-metric measures), not "drei Füße", which, in this case, would have to be qualified as wrong, just as in English, I guess.

While, to my ear, the distinction you mention sounds right, it is not quite supported by what we see in published literature.

For example, both of the following are attested as replies to How tall are you?, and I'm not sure one is significantly more frequent than the other:

[1] a. I'm about five f̲o̲o̲t̲ ten. (sources)
      b. I'm just five f̲e̲e̲t̲ two. (sources)

Similarly, both of the following are attested as well:

[2] a. I am five f̲o̲o̲t̲ two inches tall. (sources)
      b. I am five f̲e̲e̲t̲ three inches tall. (sources)

In CGEL, for example, we find both of the following:

[3] a. Our room is twenty f̲e̲e̲t̲ by thirty f̲e̲e̲t̲ (p. 655)
      b. My other table is six f̲o̲o̲t̲ by four. (p. 693)

Just in case one is tempted to think it significant that in [3a] the unit appears twice while in [3b] the second appearance it is ellipted, note that e.g. the following are attested:

[4] a. Heizer's Double Negative is a 50 f̲o̲o̲t̲ by 30 f̲o̲o̲t̲ by 1,500 f̲o̲o̲t̲ double cut in Virgin River                  Mesa, Nevada. (sources)
      b. Here is likewise a court-yard 40 f̲e̲e̲t̲ by 37. (sources)

I conclude that the distinction under discussion is regional or idiolectical---that it doesn't really exist in Standard English. It seems that in Standard English, both foot and feet are acceptable in the present context, though in other contexts it may not be so (e.g. it is definitely a ten-f̲o̲o̲t̲ pole, not a ^*a ten-f̲e̲e̲t̲ pole; see here).

I'm not sure the addition of inches matters much, as I've heard it both ways for straight foot/feet measurements as well. My best guess is that it has to do with an implied adjective vs. noun. For example:

When a person says "I'm 6 foot" my mind hears it like an adjective, similar to "I am a 6-foot man" where the person has colloquially dropped the "a" and the hyphen and the noun is implied by whoever is doing the speaking.

When a person says "I'm 6 feet" my mind hears it like a noun, similar to "I measure 6 feet" where the dimension of measurement is implied.

In writing, it helps to use hyphens and commas as necessary to clear up the meaning (e.g. "6-foot, 2-inches" adjective vs. "6 feet and 2 inches" noun, but in speech they can sound almost identical other than the singular/plural usage.

The ability to choose between the singular and plural form of foot/feet is common to several other measurement words, and may be a quality of some non-metric measurement words.

First, this usage seems superficially similar to the compound usage of other measurements, where they appear to be in a singular form because they describe how long / big / heavy something is. Compare:

• 100 meter dash (source)
• 12 ounce can (source)
• 20-pound turkey (source)

In all of these uses, the measurement and number are an attribute of the noun. So it's possible that this attributive usage has been generalized to frequently-used forms of measurement like "five foot three" when nouns are absent. As a result, in these idiomatic circumstances, adjusting the count of the measurement word is optional.

I can think of a few other examples of count-ambivalence among non-metric units of measure. These examples show this kind of usage is hundreds of years old, whatever its origin may be:

Stone - occurs in both singular and plural historically:

• Singular: "seed ten stone per acre" (Source from 1802)
• Plural: "ten stones of seed" (Source from 1804)

Pound - specific to money, and an example of idiomatic usage with currency:

• Singular: "the Sum of Ten Pound, over and above the said Four Shillings" (Source from 1801)
• Plural: "For the best four year old steer, fifteen pounds, Mr. J. A. Drought" (Source from 1808)

Mile - same pattern

• Singular: "Not to come near our person by ten mile" (Source from Shakespeare, 2 Henry IV)
• Plural: "So near our public court as twenty miles" (Source from Shakespeare, As You Like It)

Foot, too, follows a similar pattern. After running a search for "six foot three," I found the following examples:

• Singular: "The portrait had some points of resemblance, and six foot three was just his height" (Source from 1851)
• Plural: "Then the average would be Six Feet Three, but not Thirteen Feet" (Source from 1800)

In other words, English has had this idiomatic usage for a while. Why is something idiomatic? Usage over time is enough to answer that for foot.

feet is the unit of measurement. feet, pounds, miles
not
foot, pound, mile
*She is five feet 3. This does not work because feet is not a unit of measure yet it becomes okay to use if you specify a unit of measurement such as inches after 3.