When a sentence starts with "e.g.", should the e be capitalized?

Yes. Sentences start with capital letters; abbreviations are no exception.

A possible* exception is when a proper name starts with a lower case letter. E.g., if I changed my name to matthew then "matthew is awesome." would be correct. This is because the word is intended to be lower case. E.g., on the other hand, has no such association with it.

_{* Don't make an exception. This is just playing devil's advocate. See comments.}

Yes. For example is capitalized at the beginning of a sentence, so is its Latin equivalent exempli gratia, and so is its abbreviated form e.g.