I have a Magento site and a WordPress site is used in a directory for the blog. My blog continues to receive hits as it is not mobile-friendly and the reason I think is that some resources are stuck in robots.txt. The part of my robots.txt this gives me problems is as follows:
# Do not index the session ID Not allowed: / *? SID = Not allowed: / *? Forbidden: /*.php$
This comes from the standard Magento model. One of the blocked resource URLs is:
I think this line
Not allowed: / *? blocks this example of URL. I would prefer not to delete this line. Is there a way to allow everything in the wp-content directory while blocking Magento URLs with this rule?