Should I break up with him?

[Veronica]

Physical attraction and being in love are two completely different things and it sounds like you weren't really in love with this guy to begin with. Sure the physical attraction was there but it didn't go any further than that. That happens and isn't a bad thing. What makes it hard in this case is that the guy is really in love with you and knowing you don't feel the same is uncomfortable.

I think it indeed would be better if you tell him you do care about him, but aren't in love with him. Don't know if he could be friends with you seen his feelings, but you could ask him. However don't count on it. Whatever you do (do or do not ask to stay friends) I do think you should let him know you're not in love and that it would be better if the two of you split up.

Hope this helps.