| Copybara | 854996b | 2021-09-07 19:36:02 +0000 | [diff] [blame] | 1 | User-agent: * |
| 2 | # Start by disallowing everything. |
| 3 | Disallow: / |
| 4 | # Some specific things are okay, though. |
| 5 | Allow: /$ |
| 6 | Allow: /hosting |
| 7 | Allow: /p/*/adminIntro |
| 8 | # Allow files needed to render the new UI |
| 9 | Allow: /prpc/* |
| 10 | Allow: /static/* |
| 11 | # Query strings are hard. We only allow ?id=N, no other parameters. |
| 12 | Allow: /p/*/issues/detail?id=* |
| 13 | Allow: /p/*/issues/detail_ezt?id=* |
| 14 | Disallow: /p/*/issues/detail?id=*&* |
| 15 | Disallow: /p/*/issues/detail?*&id=* |
| 16 | # 10 second crawl delay for bots that honor it. |
| 17 | Crawl-delay: 10 |