MG Siegler hits on a point of contention in my life. My wife doesn’t like it when I check my phone at dinner, or while we are out. I try to limit it to just checking in and reading @replies and the like. I don’t find it rude when someone does it — I do find it rude when I do it to my wife (e.g. it is just the two of us out to dinner).
I think the line is drawn somewhere between enhancing the conversation and tuning it out. Meaning go bonanza with your phone if you are adding information to the conversation, but put it away if you are just trying to escape the conversation. The real problem is that it is hard to tell the two apart.