The source of the advantage will tell you.
Generally, whatever rules text is granting advantage will explicitly tell you what attacks it applies to. If it only applies to the first attack roll, it’ll say that. For example, true strike says:
On your next turn, you gain advantage on your first attack roll against the target
So there’s no doubt that the second attack has no advantage. But then if the target is Blinded, then
Attack rolls against the creature have advantage
That’s all attack rolls, whether it’s your first or your fifth.
There are occasionally tricky examples, like if you’re attacking from hiding (PHB p.194-195, “Unseen Attackers and Targets”):
(…) When a creature can’t see you, you have advantage on
attack rolls against it.
If you are hidden when you make an attack, you give away your location when the attack hits or misses.
So in that case, generally being unseen means you have advantage on every attack, but there’s a specific rule that says you stop being hidden when you attack the first time. So in this particular case, your second attack won’t have advantage — not because there’s any standard rule that advantage applies only to the first attack, but because the first attack broke your stealth, and that means you are no longer unseen, which means you don’t qualify for advantage anymore.
The invisibility spell works the same way. “The spell ends for a target that attacks or casts a spell”, so you can get advantage on that first attack for being unseen, but you break the spell in the process, so your second attack no longer qualifies.