No, there is no way to know that Portent has been "used"
Wear is an unusual feature in that it exploits the distinction between the player and his character. Obviously, the choice of the player is to know when to replace a result by a Portent die, but in the fiction, it does not correspond to any action or even to any thought or intention on the part of the character. All character testifies to the fact that the glimpse into the future they received earlier in the day has actually come true.
This interpretation is supported by the first sentence of the feature, which is the only part that describes its operation rather than the mechanisms of its use:
From the 2nd level when you choose this school, a glimpse of the future begins to weigh more and more on your consciousness.
Also note that no action or reaction of any kind is required to use this feature. If Portent represented an active manipulation of the probabilities by the diviner, it would be more logical for this to require a reaction.
Immunity against the scary (or other condition) is not a lethal gift for your secret magic object
To answer your specific problem, fear immunity is not an extremely rare effect, so it does not necessarily warn players that a special magic item is at stake. Also, if your players make deduce that a magic object protects NPCs from fear, they will probably assume that the object does it and nothing more. According to your question, it seems that the item in question probably does a lot more than just protect from fear. But if you really fear that your players realize it, I think the best is to make sure that you can always tell an interesting story, even if it differs from the one you planned.
It seems that the effect of fear is only an example, but the same can be said of most other immunities. In the worst case, your players will probably suspect a magic item or spell that specifically protects it.