Should I care that he said my fiancee is attractive?

I'm in college about to wrap up the school year and graduate. I don't have a job lined up, but my fiancee and I are going to get married in the fall. My roommate is one of these guys who has a different girl in bed every week or sometimes more. He is alright to hang out with but sort of immature and also a bit of a caveman according to a female friend. But he means well and overall I like him. But recently he said if I wasn't careful he would steal my fiancee from me just for kicks. He said she is physically his type and he knows he could get her to sleep with him. I know he is joking, but now that I am going to marry her, I feel like I something to say in response to this kind of comment. Maybe something sharp but also still like a gentleman. Has anyone on here, especially married guys, dealt with this kind of comment? How did you shut the guy down? I don't think I could fight him physically but I am a smarter guy and would like to have something to say that would make him think twice about commenting again.