Like a lot of things, I think it depends on exactly where you'll be driving. People generally follow the rules of the road, so four way stops, red lights, passing, blah blah blah aren't all that difficult. Merging, pulling out, that kind of thing are slightly more trying (obviously, given we're talking tons of metal flying around at significant velocities), but more than manageable.
This is generally rurally or in small towns, though. In my experience the bigger and older the city, the more likely you are to run into ridiculous traffic nonsense. Especially if you aren't a local!
Personally, I don't think any anxiety I have enters into it much. It likely makes me more cautious, which may be frustrating to others, but **** them, I am in a giant metal deathtrap. Generally I don't have that much trouble thinking since a lot of driving is practically reflex, and the parts that aren't are very demanding of your attention.
