A few simple helpful hints from 'relative' definitions of a KB to help with Google Search.
Byte: |
8 Bits, or a single character, ~10 bytes in a single word |
Kilobyte: |
1,000 bytes, two kilobytes is one typed page. |
Megabyte: |
1 million bytes, a small novel |
Gigabyte: |
1 billion bytes, a pickup truck filled with paper documents |
Terabyte: |
1 trillion bytes, a research library is ~2 terabytes. |
Petabyte: |
1 quadrillion bytes, the web is ~10 petabytes. |
Exabyte: |
1 quintillion bytes, two exabytes is all data produced annually. |
Examples
User-agent: *
Disallow:
The above specifies that all robots can visit all files.
User-agent: *
Disallow: /cgi-bin/
Disallow: /img/
The above specific that all robots can visit all files accept those found in the cgi-bin and img directories.
User-agent: googlebot
Disallow: example.html
The above specifies that googlebot should not visit example.html.