I’m a Chinese speaker, so I guess I can’t really not pay attention when a reading sounds like its cousin in Mandarin or other Chinese dialects. However, I don’t think that knowing which reading is which is useful per se beyond increased general knowledge: I’m pretty sure that before the modern Japanese education system, there were relatively uneducated but otherwise fluent Japanese people who went around without worrying too much about whether they were using an on’yomi or kun’yomi. However, if you use the classification as a means of helping you figure out how to read a kanji, then it has value. When you get a feel for it, you might say, ‘OK, this seems like more of a kun’yomi situation. I’ll use the kun’yomi,’ and reach for it in your memory. That’s the point of knowing which is which, in my opinion.
I guess I’ll just try to give you a few ideas about how to tell when you’re looking at an on’yomi. These are just general rules, and I’m pretty sure there are exceptions. Here goes:
- Most on’yomi are two syllables (the technical term is ‘morae’, since you could see that すう is one syllable, but two morae) long. A few are one syllable long, but those are a bit rarer. In any case, they’re never longer than two syllables, in my experience. (N.B.: using my terminology, ちょう = two syllables. Cho - u. In real life, of course, you just say ‘choo’ with an extended O sound.)
- Many on’yomi contain sounds that used to be non-existent in Japanese. Apparently the ちし + ゃゅょ system was invented for transcribing Chinese sounds.
- On’yomi often end with fairly hard consonant sounds. く and つ turn up quite a lot, as does ん. You still hear sounds like this in Chinese dialects today, like in Hokkien, Cantonese and Teochew. The one consonantal ending that turns up in Chinese dialects that doesn’t seem to pop up in Japanese is P: I still haven’t seen an on’yomi ending in ぷ.
That’s about all I can think of for now. I hope that helps.
Sorry to nitpick, but I’ve been seeing this quite a bit on the forums, and I just wanted to say: it’s rōmaji. Long O, no N. (The accent bar can be dropped if you want, since it’s kinda troublesome to type with all that.) I think the reason is that Rome is “Roma” in Latin. Your point about catering to one’s reader still stands though. 
Guess I’ll just break these down for OP: the first is read “watashi jishin”. The second is “tatemono”. The reason this seems unexpected is that if the on’yomi were used for all the kanji, the first would be “shijishin” and the second would be “kenbutsu”. I’m no expert, so I probably can’t explain everything, but I think readings are chosen depending on how words break down in the speaker’s head. It has to be “watashi jishin” in the first case because the three characters don’t form a single concept. It is, rather, ‘I’ (watashi) ‘myself’ (jishin). I’m not sure why the second word uses kun’yomi, but I think it has to do with where the word came from. I don’t think 建物 exists in Chinese. The equivalent in Mandarin is 建筑物 (the second character is the simplified version of 築, a character that appears in Tsukiji 築地, the name of that famous fish market in Tokyo), which means ‘constructed object’. The second character is necessary in Mandarin for clarification, I guess, since 建 means ‘to build’, but not necessarily in the literal sense. As support for my theory, well… you’ll notice that 建築物 also exists in Japanese, but it’s read ‘ken chiku butsu’ – the on’yomi is back. Tatemono probably comes from 建てる(tateru), meaning ‘to build’ and ‘mono’, which is the general word for ‘object’ and usually written as 物.
By the way, for future reference, 物 is probably one of the characters that switches between on’yomi (butsu) and kun’yomi (mono) the most in compounds, so it’s best to be aware of both readings.