[QUOTE=Gaystoryman;16884]Have you looked into this robots text plugin? Robots Meta
If trying it, might want to check the author’s page… seems there are issues with 2.6 and the options. But also Platinum SEO has a lot of the robots meta data to add to a post/page that might make it easier to use than this Robots Meta plugin…[/QUOTE]
I’ve got it working - I don’t need a plugin. I just noticed that when I didn’t have the settings quite right WordPress started serving a robots.txt file itself… It was very weird.
Still, if other people are going to set up robots.txt they’ll need a way to do it that’s comfortable for them. Editing htaccess is one way, as are plugins.
And other note - if you didn’t install WP in the root directory, then there’s no problem… You just put a robots.txt file in the root directory and your done. The issue is that WP likes to control all the files in the directory that it’s installed in and you have to write exceptions into htaccess (or VirtualHosts)…
Replace your current tag (found in header.php) with that and it will put the most important/relevant keywords/phrases first. The only time you’d want the name of your site first (which is the default) is when your brand is well known (for example “Gay Demon”).
To optimize that a bit you can hard code the name of the blog. This takes out a database call and makes the page load slightly faster, however if you change the name of your site in WordPress you’ll have to change it here too…
To do that change use the following and change “Name of Your Site” to the name of your site…
<title><?php wp_title('»',true,'right'); ?> Name of Your Site</title>
I would take out the the line that just has Disallow: in it. I’d be afraid the spiders might interpret it as Disalow: / which means everything is blocked.
Disallow: /wp-content/ should absolutely be taken out. Your stylesheet (style.css) is most likely in there and the spiders need to read it to make sure you’re not doing things like white text on a white background.
Disallow: /wp-admin/ is fine, of course
I’m not sure what to think of Disallow: /wp-includes/ - I think this is the default location of images that you upload. I’m not sure, since I changed it on my install to a /resources/ folder. But if there’s anything - images or Javascript files, etc. that’s in there that’s linked, not included, then you’ll want the spiders to be able to crawl it.
The one you’re missing that I’d add is /wp-login.php. It’s one of the first pages they’ll index (it was for me)…
One of the things that you may find confusing in WordPress is tags vs. categories… The best explanation I heard was based on a book analogy. Books have a chapter and sometimes subchapter outline at the front of the book, and they also have and index of terms in the back of the book.
Categories are like the outline at the front of the book - they should be the major themes of your blog. Tags are all the little things that are search terms, but not major themes. A good example of a category might be Latino, where a good thing for a tag might be the name of the model.
Where it gets confusing is the things that are sorta middle of the road. For me “thick dick” is something I don’t know where to put. I can see it being a theme of the site, but honestly, I’m not sure I want to elevate it to that point.
In those cases, just pick one and try to be consistent. Whatever you do, don’t have a tag and a category with the same name - you’ll be fighting yourself if you do. It’s not the end of the world if you want to promote a tag page like you might a category page - which is what I think I’ll end up doing with “thick dick”…
So just curious… Those of you who started blogs with this workshop… How’d it go? How many posts have you done? How many (real) comments have you gotten? How many pages do you have indexed in Google? How’s your traffic?
I started WilyWilly.com, but honestly it’s not my focus. I’ve done 238 posts, had 11 approved comments, 237 pages are indexed in Google, and I get a bit over 300 visits a day. Direct traffic is growing slowly. Search engine traffic is where I’ve done best, though the domain lacks authority and I feel the difference between it and rawtop.com - it’s so much easier to rank for terms with rawtop, but that will change in time…
I haven’t submitted it to any directories because while I’ve been dumping posts on the site, it really needs a lot more design work. Hopefully my boyfriend will have time to give it a face lift this summer. The homepage is the biggest problem - I’m envisioning a Flash/Flex-based dynamic page, but I need to figure out how that will work…
