I've been talking to a Nigerian guy for over 4 months should I give him my number?

[Ashes]

I have declined his request to talk to me on the phone for about a month because I'm not sure if there's some scam someone can do with my phone number. He's my friend on FB and he seems pretty normal, talking with friends and the like, but I know scammers can get really into the art of appearing normal, so I just wonder if anyone has heard of any type of scam art pertaining to my phone number. He has my picture and full name, since my picture and full name is on FB. He doesn't have my address or doesn't even know what city I live in.

I'm a very careful person and never thought I'd ever deal with a guy from another country, but they say never say never. I am far from naive, but I just wonder if I'm being paranoid. I'm pretty sure decent, moral people live in Nigeria. And he has pictures of him and his family on his Facebook.

However, I want to know if there's a chance of some scam with my phone number.

Any advice would be appreciated!
He's not royalty, and he actually gave me the name of the place where he works, which is a legit company I googled, but too ashamed to call. He's never mentioned anything about money. Decent people live in Nigeria, I know some Nigerians who came here and have opened businesses and some in the U.S. Military. They are actually more decent people than some Americans I know. Anyone else?
Yea, he said he has magic jack, but what I really want to know is, can he get my bank account information with my name and my phone number? Of course I wouldn't give him my bank information. Skype sounds like the way to go.