Am I correct in my understanding of their corresponding explanation sections (and their kanji, now that I think about it) that 始めに refers more to space while 初めに refers more to time?
I was really confused that ‘beginning’ was not being accepted for 始めに until I rechecked the vocab lists and realized that the two shared the reading.
My approach, which is linked to what’s said in this article in Japanese and to how the kanji are used in Japanese and Chinese, is basically
初め = ‘the first’ (think of how 初(はつ)〜 I s used as a prefix meaning ‘the first ~’)
始め = ‘the start (of an action)’ (think of how the verb 始める is used if you know it)
The former is the first in a sequence; the latter is how something begins. The former is linked to order; the latter to the starting point of something. It’s not about space and time though. I wouldn’t say I’m an expert on these two because I get confused at times too, but they’re very clearly distinguished in Chinese, and I think the kanji compounds above should show you that as well. I’ll leave you to read the detailed explanations provided by everyone else: I just wanted to write something that was hopefully clear and concise for once. I hope this helps.