No, you’re right. It’s a counter word (助数詞/じょすうし), and the convenient thing is that it is always にん in that use, with two exceptions, so you never have to worry about a potential じん reading as you do in other compounds.
The confusing thing is that numers under 10 in this case can also form 大和言葉 (やまとことば)(Japanese-“original” words, as opposed to ones based on readings from China and other regions) compounds, though luckily for us learners, the only ones still in use are the readings for 一人 (ひとり) and 二人(ふたり). (Incidentally, numbers over 10 can also be counted using 大和言葉, but it’d be the equivalent of using Old English. Occassional 10+ 大和言葉-based readings do still rear their heads in a few in-use counters though, ex. 10日, 20日, and 二十歳.)
As for why the 大和言葉 readings are still in use for 一人 and 二人, I have no idea, except a guess that it’s just because “いちにん” and "ににん” don’t roll off the tongue as well. But those readings are used in longer compounds like 一人称 (いちにんしょう)and 二人称(ににんしょう), for first- and second-person perspective.
Welcome to Japanese. Languages are weird.