Is there an English idiom “in threes and fives” to describe arriving, gathering, or leaving of people in a pair, trio, or group in succession?

You're right in your suspicion that "in threes and fives" is a literal translation and not idiomatic. You mention "in ones and twos," which is what I would have suggested as an answer. If you're looking for a phrase that means people coming or going specifically in groups of three to five, I can't think of one, but you could try:

The people were coming (going) in dribs and drabs. ("A series of negligible amounts.")

This gives a sense of disorganization, of a few here, a few there.

But in ones and twos is idiomatic and I would suspect quite satisfactory for most purposes: in small numbers.

Finally, it would be perfectly acceptable (though not idiomatic) to say

The people were coming (going) in groups of three to five.

It's not really idiomatic, but it is a common utterance:

a few at a time

few : 1. An indefinitely small number of persons or things

The dictionary says indefinitely small and I think this covers the 3-5 range nicely, because two would be 'a couple', or we have 'in ones and twos' as mentioned in other answers for those specific cases.

When a group of people who are all meant to be in one place arrive in small groups over a period of time, the common description is "in twos and threes". I wonder if Japanese groups really are larger on average?

The term dribs and drabs is also sometimes applied to people moving in and/or out of a space.

An Australianism that is typically applied to the movement of a large group of children is like Brown's cows to convey the apparently pointless meandering of small groups that will eventually result in everyone arriving at the designated place.

It can also be said that "small knots (bunches, parties) of people trickled (straggled, meandered) in"