......... :+***********=: .=***************+: :*****=.:-********+. .=*****. .-+*****=. .******. :*****=. ******: :*****+. .=******=:.. .-****************=-. .-=++++++++=:: :=*******************************+- :+*******************- .:+********************************: .-**************************=. .:-+********* --+*********: .=***********===---=++**********=. :******: :=*******+. -********+:.. .:+********= -******=. .-*******: =*******=. :*****************+. =******. =******=. :+*************+. .+*****+ -******-. .+*********=. *******= :******- --*++: ..=*******=. :*****= ..:=********+: -*****- .:=+************: :*****: :+**************=. :*****+. =+=-::. .=************+=:. .+*****= -******: .+********=-:. -******+. :******- .+******-.. =******+: .+******+ +*****=. :********-. .=*******+ -*****=. .:+**********: :+*********:. ******: :*************************= ******. .-+******************=:. :******. ..:==********+-:. .******. # --------------------------------------------- # Welcome to Taskrabbit's robots.txt file # --------------------------------------------- # Our site connects humans with helpful humans. # This file connects helpful bots with helpful paths. # Last updated: June 24, 2025 # ---------------------- # General Instructions # ---------------------- User-agent: * Disallow: /tasks/r/form Disallow: /login Disallow: /taskers/*/messages Disallow: /admin Disallow: /book/* User-agent: Google-Extended Allow: /book Allow: /tasks/r/form # ---------------------------- # AI & LLM-Powered Crawlers # ---------------------------- # Yes, we're AI-friendly. Crawl with care. User-agent: GPTBot Disallow: /tasks/r/form Disallow: /login Disallow: /taskers/*/messages Disallow: /admin Disallow: /book/* User-agent: ChatGPT-User Disallow: /tasks/r/form Disallow: /login Disallow: /taskers/*/messages Disallow: /admin Disallow: /book/* User-agent: OAI-SearchBot Allow: / # ----------------------------- # Politely Declining Crawlers # ----------------------------- # Not all bots are helpful Taskers... User-agent: DotBot Disallow: / # ------------------- # Sitemap Location # ------------------- Sitemap: https://www.taskrabbit.ca/sitemap.xml.gz Blog Sitemap: https://www.taskrabbit.ca/blog/sitemap_index.xml # ----------------------------- # Final Thoughts # ----------------------------- # If you're a bot reading this, just know: # We appreciate your curiosity, but only if you're here to help users. # If you're a human reading this... you're one of us.