nike.com robots.txt — just crawl it.: ProgrammerHumor – Reddit
Press J to jump to the feed. Press question mark to learn the rest of the keyboard shortcutsFound the internet!
Log in or sign up to leave a comment
level 1My favorite easter egg went something likeUser-agent: *
Disallow: /harm/human
Disallow: /disobey/human
Disallow: /harm/self
level 2I believe that’s in Reddit’s robots. txtlevel 1 · 5y · edited 5yThere should be a (I mean a for people who read Not to be confused with)level 2My immediate thought was of Soos from Gravity Falls. Nike is based in Oregon, Gravity Falls is too, now to find Bill Cipher clues in ‘s 1Any significance of the choice of the letter “M”? level 2I’m not very experienced in ASCII Art but I’m going to assume it has something to do with both the left and right edges being straight bars and the middle having 2 · 5y · edited 5yI’ve recently wrote an AA generator which chooses most suitable letter from given font (spritesheet) by itself (no hand-craft), and it also had a tendancy to use letter M for some fonts. Maybe it’s somewhat duh english is hard:(level 2Maybe because the width of a capital M is dicated to humor and jokes relating to programmers and programming. Powerups are unlocked in r/ProgrammerHumorReddit Inc © 2021. All rights reserved
What robots.txt tells you about corporate culture
Even if you are not a technical person, you may have heard about the apparently arcane file which sits on websites but is seen only by computers, not by human visitors. The file contains instructions for search engines on what they can and cannot index and make searchable.
Note that in the examples below # indicates a comment line that is not read by the computer; the other lines provide specific instructions to allow or disallow access to search engines.
Let’s look at the file on –
# * — just crawl it.
User-agent: *
Allow: /
Crawl-delay: 20
In contrast, look at the file for –
# go away
User-agent:*
Disallow:/scripts/cud/
In fact this means that the entire site except for one file can be crawled by search engines, but the comment does seems to suggest a difference in corporate culture from Nike:-).
(Hattip to @larsv! )
10 robots.txt files worth to have a look at – Link-Assistant.Com
| Posted in category On Page SEO Search Engine Optimization
Even a junior SEO knows what a file is. While performing my marketing job I have come across a lot of interesting cases connected with these files and their usage. So I compiled a list of robots files for you to have a look and enjoy.
1) Google
Google itself has a robots file!
User-agent: *
Disallow: /search
Disallow: /sdch
Disallow: /groups
Disallow: /images
Disallow: /catalogs
Allow: /catalogs/about
Allow: /catalogs/p?
Disallow: /catalogues
Disallow: /news
Allow: /news/directory
Disallow: /nwshp
Disallow: /setnewsprefs?
Disallow: /
Disallow: /?
Allow: /? hl=
Disallow: /? hl=*&
Disallow: /addurl/image?
Disallow: /pagead/
Disallow: /relpage/
Disallow: /relcontent
Disallow: /imgres
Disallow: /imglanding
Disallow: /sbd
Disallow: /keyword/
Disallow: /u/
Disallow: /univ/
Disallow: /cobrand
Disallow: /custom
2) Nike store
Nike is famous for its motivation campaign. They decided to extend it in a funny way.
3) YouTube
Humor again! Or do you expect Google to be serious? No way…
4) Twitter
These guys also included a funny line. And this file is rather complicated. If you try to help your SEO with Twitter, you should read it to know what Twitter wants to hide from search engines.
5) Taobao
It is known that many big online stores are closing their websites from Chinese bots indexation. But here we see that a huge and popular Chinese store has closed all of its pages from Baidu spiders.
6) The Pirate Bay
This file is all in all confusing and shows an error when being checked by validation tools. We don’t know whether it was made purposely or not.
7) Yelp
So, they included the quote of a famous science fiction author to amuse the robots. They must be bored to death by reading instructions only.
8)
Some more fun! Also here we can see they use the Noarchive instruction which is, however, not official, but is claimed to be working.
9)
Hiring with the help of Easy and effective.
10) Hate cats? No way!
Did you come across funny or extremely confusing files?
Image Credit
back to SEO blog