فَتَطْرُدَهُمْ فَتَكُونَ مِنَ الظَّالِمِينَ
“If you were to reject them, you would be from the wrongdoers.”
This is the asbab al-nuzul: Q6:52 rebukes exactly the demand described at this event -- Abu Jahal pressuring the Prophet ﷺ to expel poor believers like Ammar, Suhayb, and Bilal RA.