I use a blend of 私, 僕, and 俺. 私 for first meetings or when speaking to important people (despite my friendly relations with the mayor, I wouldn’t dare try another pronoun!), 僕 for talking with anyone I view with respect (my significantly older co-workers, neighbors, etc), and 俺 with similarly aged coworkers, friends, and my students. Is this required or expected of me? Nope! The pronouns happen to align with my “image” of them.
I see each of the pronouns less as “masculine” or “feminine” as “indefinite” and “definite” rather than in abstract concepts like formality. From how I’ve noticed the pronouns used in real life (as opposed to media like anime and dramas), the “further away” a speaker gets from 私, the better the grasp they have on where they stand around people. This is the main reason I am not fond of 私 personally, as I feel it lacks any ability to connect with people (as a male, at least). This makes it the perfect choice for professional dealings, but bad for personal dealings.
僕 and 俺 I see as two sides of the same coin: they show an active desire to connect, but in two different ways. 僕 is the “soft” side, indicating that the user places equal/higher priority to those with whom their speaking than themselves. This is why it seems “boyish” and is used by younger males. Meanwhile, 俺 is the “hard” side, indicating that the user places higher priority to themselves than to those with whom they are speaking. This is not to say it make the speaker seem better than the other person, but simply at a position to take more responsibility for it. This is why 俺 should be used less often with people of higher status or if you are not an assertive person.
For the rarer わたくし, I think this also assume a more “definite” pronouns, but in a less natural (and, from my experience, a worse) way. Unlike 俺, I feel it actually does indicate that the speaker is more important at base than those with whom they are speaking. I view it about the same level as using 我; the problem is more the reasoning for using it than the word itself (unless, of course, your circumstances call for that much formality). I’ve heard very few natives using わたくし, and I think usually foreigners have no reason to use it.
Ultimately, your behavior will shape your image rather than your pronoun(s); if your pronoun(s) don’t match your image, someone will probably make that obvious, especially if you make close friends. Unless you are Asian by descent (where things can be much more complicated), more likely than not they’ll probably be impressed at first that you even know a pronoun other than 私. It’s media like anime that make it out that there is some code to crack due to the archetypal design of characters.