cavil - to oppose by inconsequential, frivolous, or sham objections

There's also quibble, which is very similar. But I think that more strongly implies avoiding the main issue by focusing on minor details, so it doesn't work very well if the subject has already effectively ceded the main argument, and is simply stringing out nitpicky details (often it's little more than a way of showing that the person is unhappy about losing the argument).

This works nicely: -

Arguing for the sake of arguing

Arguing simply to be difficult or contrary

Free Dictionary

That would be a case of sticking to one's guns.

From dictionary.com:

17. stick to one's guns, to maintain one's position in the face of opposition; stand firm:

They stuck to their guns and refused to submit.

Also, stand by one's guns.

From macmillandictionary.com:

5. stick to your guns INFORMAL

to refuse to change what you are saying or doing despite the opposition or criticism of other people

They tried to persuade me, but I stuck to my guns.

Synonyms and related words: To not change, or to refuse to change your opinion:hold to, remain, have the courage of your (own) convictions...

"I don't mean starting out on pedantic points, but from a sincere position, which, through the course of the argument, became clearly untenable..."

If you're looking for idioms, both "not to concede defeat" and "not to throw in the towel" seem to fit.

• concede defeat - to admit one has been defeated

• throw in the towel - admit defeat, give up in the face of defeat of lacking hope.

Examples:

"She never concedes defeat, that's why I call her Mrs Wright."

"No one is going to think the less of you if you concede defeat now."

"Don't be such a mule. It's time you threw in the towel.

"He lost the election but refused to throw in the towel. He says he is running for mayor next time."