Is "a total of 10 payments" singular or plural?

Since "payment" is countable, I would go with "were" to reinforce the notion that there was more than one payment.

This thread contains more examples:

Number, majority and total are singular if preceded by the, but plural if preceded by a.

• A number of people believe he is innocent.
• A majority of residents want the town to reduce the recreation fee.
• A total of 15 people were arrested for burglary last month in our town.

That being said, this is not a strict rule, and if the focus is specifically on the fact of something being a total, you would use "was".

• "A total of five cars is impossible: you must have miscounted."

The subject of the sentence were the payments, and not the total number of payments, therefore were is correct.

A total of 10 payments were made.

You could re-phrase the sentence to make the total the subject:

The total number of payments made was 10

VonC offers a useful rule-of-thumb - 'A total...' = plural, 'The total...' = singular - but it all boils down to the subject of the sentence.

You need to read things like a total of and a lot of and the majority of as premodifiers of the head noun, not as singulars. So they work like adjectives, which means they don't affect grammatical number for purposes of verb agreement. That also means that it's what follows it that determines the number.

• A total of ten payments ✅were made.
• A total of twelve people ✅were on the jury.
• A lot of people ✅were on the jury.
• A lot of trouble ✅is easy to avoid.
• Lots of trouble ✅is easy to avoid.
• A majority of voters ✅prefer the first candidate on the list.
• The majority of the field ✅is going to need to be replanted.

But these are all ungrammatical:

• A total of ten payments ❌was made.
• A total of twelve people ❌was on the jury.
• A lot of people ❌was on the jury.
• Lots of trouble ❌are easy to avoid.
• A majority of voters ❌prefers the first candidate on the list.
• The majority of the field ❌are going to need to be replanted.