There is some margin of discretion from the DM, but I would say that in most cases, yes. As mentioned in the question, Charm Person ends when
you or your companions something harmful to the target.
In D & D 5th, words mean what they mean in plain language. "Harmful" simply means "cause or likely to cause harm". Even if the attack ended up being totally blocked by the Arcane Ward (or was missing completely), it was "likely to harm" by the time you tried it.
Another clue about the use of the spell can be found later in its description.
The charmed creature regards you as a friendly acquaintance.
If you do something that prevents them from seeing you as a friendly acquaintance, the spell ends (probably – again, at the DM's discretion).