GSA SER – Duplicate target URLs become a real problem


Recently, I have seen a sharp increase in the number of duplicate URLs and the speed at which they grow. I have seen installations where, in a week, dup target URLs have increased by 20 GB

At the moment when GSA SER does not automatically check the dup target urls.

I understand that doing real-time checks would really slow things down, so please, @Sven can you consider adding a scheduler to remove duplicate domain \ urls at defined intervals.

Thank you