My son likes this boy, but I am not enjoying this boys' presence in my house, or even in the schoolyard with my son. I have virtually cut off the playdates. Am I wrong to do so?
Keep in mind that you may be the only positive, structured adult in the life of your son's friend. Even though you are not responsible for him or his behavior, if you refuse to let him play with your son you will be denying him the chance to get positive feedback from you.
Consider allowing your son to play with him, but only in a structured setting (your house or yard) with your close supervision. Make sure that your son knows what you expect of his behavior, and make sure that his friend knows about those expectations as well.
In our society, parents have the right to raise their children any way that they choose so long as the child's health and safety is not endangered; that is why the teacher and school social worker were not able to be more helpful. If you ever feel that your son's friend is in danger, call the child protective services office in your area.