When a word is written with a normal sized つ, such as in いつつ. Its pronounced and written the way it looks. In romaji, this would be “itsutsu”. When a word is written with a small tsu っ, such as in むっつ, it’s actually an indicator that the consonant sound following it is doubled. So, this would be written “muttsu”. When pronouncing it, small っ is silent but still takes up one beat of time in the word. So where むつ is a 2 beat word, むっつ is a 3 beat word pronounced |む|Silence|つ|. I hope that helps!
Counters have a lot of irregularities, unfortunately. You aren’t missing a rule or anything like that - you just have to memorize that five things is pronounced いつつ and six things is pronounced むっつ.
It’s not quite silence. You need to form and hold the “ts-” consonant shape with your mouth, before releasing it on the next beat. To use the linguistics terms, it’s gemination rather than a glottal stop. (Small っ can function as a glottal stop in specific situations, but this is not it.)
The difference between the big and small つ is already explained above. As for the why, the small っ seems to serve as a marker that a sound was once there but is now omitted, similar to the ’ in English. Since 六日 is pronounced むいか, you would expect 六つ to be pronounced むいつ, but the い got dropped for one reason or the other and in these situations the dropped sound is often replaced with っ.
(This is a historical digression, not really important to understanding modern Japanese.)
I think that both むいか and むっつ are not the “original” form. In Old Japanese, it was almost never the case that two vowels were together in a word, as they are in むいか (u-i); the language was almost purely CVCVCV, with the vowel-only syllables あいうえお almost always at the start of words only. The sound changes (“onbin”) that arrived with the change from Old Japanese to Early Middle Japanese about a thousand years ago could reduce a CV syllable to a vowel, to an ん nasal, or to the “long consonant” now written with the small-tsu っ[*], and you don’t get word forms like either むいか or むっつ before then. 六日 and 六つ happened to end up taking different routes. Wikipedia thinks the original OJ stem for 6 was just “mu”, so maybe “muka” and “mutu” is where they started?
[*] You can see this in the way verbs inflect for their te forms: よむ : よんで, かく : かいて and わかる : わかって were all originally the more regular looking よみて かきて わかりて, but the reduction of their syllables went down different paths.