StudioPress

Forum navigation

Are You Using The WordPress Block Editor?

Genesis now offers plugins that help you build better sites faster with the WordPress block editor (Gutenberg). Try the feature-rich free versions of each plugin for yourself!

Genesis Blocks Genesis Custom Blocks

robots.txt Tester

Welcome!

These forums are for general discussion on WordPress and Genesis. Official support for StudioPress themes is offered exclusively at My StudioPress. Responses in this forum are not guaranteed. Please note that this forum will require a new username, separate from the one used for My.StudioPress.

Community Forums › Forums › Archived Forums › General Discussion › robots.txt Tester

This topic is: not resolved
Viewing 3 posts - 1 through 3 (of 3 total)
  • Author
    Posts
  • August 26, 2015 at 1:17 am #163532
    RavenManiac
    Participant

    I just tried to use Fetch as Google and I found out my website is being blocked. The file that's doing this is called robots.txt Tester and it contains the following:

    User-agent: *
    Disallow: /

    This is fine for my development server, but the live server should be accessible. Where can I find this file and how should I edit it to allow search engine access?

    Thanks!

    August 26, 2015 at 1:41 am #163534
    RavenManiac
    Participant

    BTW, I can't find the robots.txt file anywhere in the root directory of my live server. I've even turned on show invisible files, which unhides .htaccess, but still no robots.txt file.

    August 26, 2015 at 2:07 am #163536
    RavenManiac
    Participant

    Okay, I figured this out. It's a virtual file. To turn off disallow you basically have to go to Settings > Reading and uncheck "Search Engine Visibility", which is fine. But how do I add to the virtual robots.txt file and what should be in that file to protect my WP website?

  • Author
    Posts
Viewing 3 posts - 1 through 3 (of 3 total)
  • The forum ‘General Discussion’ is closed to new topics and replies.