Etymology, I guess. は can also replace が or を, but always joins forces with に or で.
In some ways it’s the same as English. Even when you’re saying “he also went to the library”, you’re still saying he went to the library - the “also” doesn’t change that. The postposition に functions here the same as the prepostion “to”.
As @Belthazar said, if you don’t include に in にも it would be too confusing because に there is the location particle (図書館に).
It would be like “He went to the park. The library went to the park too”
は and も are non-logical topic particles - they say something about the topic of the sentence without actually influencing the core sentence.
CureDolly has some very good and clear explanation about it.
Older video on も More recent video - part of “Japanese from scratch” video series, so maybe she’ll be referring too much to things covered in past videos.
Standard disclaimer when linking to CureDolly: The voice filter will make you want to turn the video off, but consider giving it a go. Full subtitles are added to the videos.
Edit:
I admit I was rushing and didn’t read your post very thoroughly, so apologies if this wasn’t what you were asking about at all.
Doesn’t that mean that omitting を should also be confusing? As in: I ate an apple. I also ate yakisoba.
私はりんごを食べました。
やきそばも食べました。
With the same logic you could argue that the second sentence says “Yakisoba also ate an apple”. I don’t see how omitting に makes the phrase confusing if omitting を doesn’t.
Note that は (for topic or contrast) also does this with が and を, but not に or other particles.
Ame ga futta. → Ame wa futta
Eiga o miru. → Eiga wa miru.
Tokyo ni iku. → Tokyo ni wa iku.
Enpitsu de kaku. → Enpitsu de wa kaku.
In general markers of subject/direct object have a tendency to disappear in constructions.
Consider passive in English:
English example: I helped him. He was helped.
Note: “him” becomes “He”.
But: I gave it to him. To him it was given.
“To him” stays the way it is.
I helped him in Rome. In Rome he was helped.
“In Rome” stays the way it is.
(Passive in German is a much better example: Verbs that take accusative case objects have it transformed into nominative case. But verbs that take dative case objects keep it in dative case and you get a sentence with a dative subject!
Er schlug mich (He hit me) → Ich wurde geschlagen. (I was hit)
“Mich” (me) becomes “ich” (I).
Er half mir (He helped me) → Mir wurde geholfen. (I was helped)
“Mir” (me) moves into subject position but doesn’t change.)