The spell shocking grasp has the clause
You have advantage on the attack roll if the target is wearing armour made of metal.
This is to simulate how fantasy-electricity $^1$ conducts through metal armour to the wearer, approximately negating the armour’s protection. You probably want to do something similar.
In past editions (e.g. 3.5e, or its relative Pathfinder), there existed situational variants of AC. The relevant variant here is touch AC, which ignores armour-like bonuses to AC. But 5e has done away with such details for the sake of simplicity. I have seen nothing in D&D 5e that has a different AC calculation in different circumstances. A creature’s AC is independent of its state or the type of attack.
What has replaced situational variants or modifiers to AC is advantage and disadvantage. When circumstances make a creature easier to hit than its normal AC would suggest, the attack roll has advantage. For example, attackers have advantage against unconscious creatures to approximate their inability to dodge. In a more detailed simulation you would remove their Dexterity bonus to AC (flat-footed AC in 3.5e and Pathfinder), but 5e approximates it with advantage. Likewise, shocking grasp has advantage against opponents in metal armour to approximate the ineffectiveness (or even detriment) of the metal armour.
In your case, you could grant the swarm advantage on attacks against targets wearing medium or heavy armour. Light armour, with +1 or +2 AC, is probably not worth granting advantage against, which normally works out to around +4 or +5 to hit (depending on AC and to-hit), but that’s your call. Also your call what to do with shields. But wherever you decide to draw the line, advantage is probably a good way to have an attack which has an easier time hitting armoured targets than their AC suggests.
Many spells and other features which only take effect if they touch the target with no regard to armour use a Dexterity saving throw.
For example, disintegrate calls for a Dexterity saving throw to avoid the thin green ray created by the spell, presumably to simulate dodging the ray.
Or see the Spawn of Kyuss (Volo’s Guide to Monsters p.192), which has a bug related ability.
Burrowing Worm. A worm launches from the spawn of Kyuss at one humanoid that the spawn can see within 10 feet of it. The worm latches onto the target’s skin unless the target succeeds on a DC 11 Dexterity saving throw.
Again, we a Dexterity saving throw used to simulate dodging something which ignores armour.
A different example is the infestation spell (Xanathar’s Guide to Everything p.158).
You cause a cloud of mites, fleas, and other parasites to appear momentarily on one creature you see within range. The target must succeed on a Constitution saving throw, or it takes 1d6 poison damage and moves 5 feet in a random direction…
For this spell it is assumed there is no opportunity to dodge. Rather, the saving throw represents the target resisting and enduring the biting creatures. Whether you think a Constitution saving throw or a Dexterity saving throw is more appropriate is up to you (and might also depend on whether you already use Constitution saving throws elsewhere in its stat block).
A few abilities, like the spell magic missile or a storm giant quintessent’s Wind Javelin (Volo p.151), always strike and damage their target, regardless of their AC. These are attack-like in that they use an action and have projectiles. These abilities simulate unerring accuracy, which might not be quite what you are looking for, but it ignores armour.
Other abilities deal damage automatically when creatures are in a certain area, like a balor’s Fire Aura, or a fire elemental’s Fire Form.
Fire Form. … The elemental can enter a hostile creature’s space and stop there. The first time it enters a creature’s space on a turn, that creature takes 5 (1d10) fire damage and catches fire.
This might be particularly appropriate for your swarm (which normally enter hostile creatures’ spaces to attack them anyway), dealing some automatic damage each time it enters another creature’s space in a turn. This would represent how it is impossible to escape unscathed when surrounded by the swarm.
$^1$ I say fantasy-electricity, because in real life metal armour would conduct electricity around the wearer, protecting them from the shock. This is the principle behind a Faraday cage.