You have a right to say that you would like him to move out, without his making you feel like horrible parents. Short of physically trying to make him move out, you appear to be at an impasse. If he maintains the same attitudes and continues to show the same lack of initiative regarding moving out, he could stay with you indefinitely. His rages are his way of getting you to back off. You are both stuck in an arrangement that seems to serve neither of you well.
I would suggest that you seek some family therapy to provide all of you with an unbiased environment in which to seek a solution. A skilled family therapist can honor everyone's feelings and help you to reach a reasonable settlement. One of the possible outcomes of seeing a family therapist may be your son's forming a trusting bond with him and using him individually to break out of the rut he's in. It sounds as if he could use some life and career/vocational counseling and support. Short of seeking outside professional help, you need to give him a departure timetable and perhaps offer to help him search for an apartment he could afford on his own. Apartment agencies also connect people with other apartment seekers who need a roommate. I know this is a touchy situation but doing nothing is the surest route to perpetuating a bad situation.