# Spider Crawling Permission User-agent: * Allow: /wp-admin/admin-ajax.php Disallow: /cgi-bin/ Disallow: /c4p_log/ Disallow: /wp-admin/ Disallow: /cdn-cgi/ Disallow: /comments/feed/ Disallow: /trackback/ Disallow: /author/ Disallow: /index.php Disallow: /xmlrpc.php Disallow: /tag/ Disallow: /?s=* Disallow: /t=* Disallow: /*/?share=* Disallow: /*?relatedposts=* Disallow: /wp-content/plugins/ Disallow: /dgd_scrollbox/subscribe-test/ Disallow: /?_escaped_fragment_=* Disallow: /#!* Disallow: /go/ Disallow: /*/cyvb/ User-agent: NinjaBot Allow: / User-agent: Mediapartners-Google* Allow: / User-agent: Googlebot-Image Allow: /wp-content/uploads/ User-agent: Adsbot-Google Allow: / User-agent: Googlebot-Mobile Allow: / User-agent: MJ12bot Disallow: / User-agent: HTTrack Disallow: / User-agent: Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com) Disallow: / Sitemap: http://gennibbles.com/sitemap_index.xml