That’s a lot of poem in one place! You’ve given some nice explanations of some of the vocab/grammar, so I’ll just add some complementary stuff I found while looking these up.
A lot of the translations into contemporary Japanese replace this with ぼんやりと, giving a sense of being lost in thought while staring at the rain. Of course as you mentioned it also hints at the ceaselessness of the rain.
Unfortunately, this kanji was only used in this way starting in the 江戸 period, so it wouldn’t have been used in the original poem. Furthermore, although I’ve seen it used for this poem once online, most websites use the kanji 寂 instead. Apparently, there is a difference in nuance here. 淋 is used when there is a psychological loneliness, so for phrases like 淋しい人間, while 寂 is used for physical or natural scenes. In this case, although there is a lonely atmosphere, it seems the 寂しき is describing ながめ, so it would probably fall under the natural scene category. All this being said, I like 淋 as a kanji too (I just try not to think about the disease that utilizes it 
).
This definition is a bit abstract, but the image I got in my head while reading was that the raindrops from the eaves ran along the leaves of the shinobu and then fell down. Shinobu is a type of fern, so the raindrop following the natural downward curve of the leaves is a pretty strong impression.
Most of the interpretations I saw rewrote this as 旧都, or ancient capital like you said. Although even looking at a 古文 dictionary, both definitions are listed so it may be up to interpretation.
I understand your explanation of the 絶えて being a kakekotoba, but it does feel a bit strange to call 跡 a kakekotoba here when the word 跡絶ゆ existed and meant 訪れる人がなくなる. I’m assuming that the etymology was indeed footprints disappearing, but does that mean that whenever you use this word, you are using a kakekotoba?
It’s very hard to encompass the meaning of kanji like this in English translations. Especially with shorter poems, it feels regrettable to not be able to subtly express things in the way kanji can.
Yeah this seems like a very difficult phrase to pick apart. The その here actually has the meaning of どの and と is more like ということも, so a more understandable phrase is どの色ということもなく.
Hermitage is definitely a word that’s hard for me to imagine in real life. But I think searching 草庵 gives you the right idea, and it’s basically just a thatched hut. The background here is that the poet used to live in the city, but quit as a government official to become a monk due to his illness two years before the writing of the poem. The contrast of the lively city with his current ascetic lifestyle intensifies the feeling of loneliness.
For some reason, some translators just can’t get birds and flowers right sometimes
This is definitely a cuckoo, but the important context is that during the summer most cuckoos leave the forest for the city, so the song of the remaining cuckoo must be lonely indeed. Fun fact: ホトトギス has the most kanji variations of any bird in Japanese. The one I saw the most for this poem was 郭公, which I like particularly because the other reading of it (かっこう) sounds like the sound of the bird.
Given that the cuckoo is singing, it can be implied that this isn’t just any rain, but 五月雨, which is particularly lengthy and can be quite depressing.