Google Updates Robots.txt Policy: Unsupported Fields Are Ignored

Google limits robots.txt support to four fields, clarifying its stance on unsupported directives.

• Google only supports four specific robots.txt fields.
• Unsupported directives in robots.txt will be ignored.
• Consider auditing your robots.txt files in light of this update.