No it won’t screw you up, that’s how Japanese kids first learn to read - hiragana all the way.
You could also take a look through some of the study logs to see which combinations of other materials folks are using.
With reading a high priority, keep an eye on the Absolute Beginner Book Club thread, follow the links to previous books to have a look at how difficult things are, and join a group as soon as you feel able - it will be tough at first, but everyone is so lovely and supportive (it’s been the most rewarding experience for me since starting WK).
All the best!