Maybe. It will modulate the speed of the belt/extruder carriage which will modulate the thickness of the plastic line being extruded. That will show up as a visible defect in the surface of the print, though it may be difficult to pick that particular defect out when mixed with other common surface defects. It probably won't hurt the strength or accuracy of the part much unless the pulley is really bad.
