What Is the Robots.txt File for My PhotoBiz Website

What Is the Robots.txt File for My PhotoBiz Website?

Overview

Every launched PhotoBiz Website automatically includes a robots.txt file. This file gives search engines and other automated crawlers instructions about which areas of your website they may access.

PhotoBiz creates, hosts, and manages this file for you. You do not need to configure it, and you cannot edit, replace, or remove it from your PhotoBiz account.

This guide explains how to view your robots.txt file, what the included instructions mean, and how robots.txt relates to search engine crawling and indexing.

What Is a Robots.txt File?

A robots.txt file is a plain-text file located at the top level of a website. Search engines and other automated services may review this file before crawling the website.

The file contains instructions for different crawlers, which may also be called bots or user agents. These instructions can allow access to website resources, restrict access to selected paths, and identify the location of the website’s XML sitemap.

Robots.txt primarily helps manage crawler traffic. It does not control your website navigation, change your page content, or provide security for private information.

How Robots.txt Works on PhotoBiz Websites

PhotoBiz uses a standardized robots.txt configuration for websites hosted on the platform. The crawler instructions are managed by PhotoBiz and are generally the same across all launched PhotoBiz Websites.

Domain-specific information, such as the Sitemap address, automatically reflects the domain connected to your website.

No action is required to activate or maintain the robots.txt file.

How to View Your Robots.txt File

Add /robots.txt to the end of your launched website’s domain name.

For example:

https://www.yourdomain.com/robots.txt

Replace yourdomain.com with your website’s domain. The file will appear as a plain-text page in your browser.

Your website must be launched on a domain before its domain-specific robots.txt file will be available.

What Is Included in the PhotoBiz Robots.txt File?

The PhotoBiz robots.txt file includes instructions that:

Allow Googlebot to access supported CSS and JavaScript resources
Allow supported Facebook and Twitter crawlers to access website content
Block selected automated bots
Restrict selected administrative and system-related paths
Restrict selected cart, checkout, billing, and account paths
Identify the location of your XML sitemap
Provide a crawl-delay instruction for crawlers that support it

These instructions help crawlers focus on public website content instead of internal or system-generated areas that are not intended to appear in search results.

Can I Edit My Robots.txt File?

No. PhotoBiz manages the robots.txt file at the platform level.

You cannot:

Edit the existing file
Upload a replacement robots.txt file
Remove the file
Add instructions for a specific crawler
Add custom rules for an individual page or path

PhotoBiz automatically maintains the file as part of your hosted website.

What Do the Robots.txt Instructions Mean?

User-agent

A User-agent identifies the crawler that should follow a group of instructions.

Some instructions apply to a specific crawler. The asterisk in User-agent: * represents crawlers that do not have a more specific group of instructions in the file.

Allow

An Allow instruction identifies a resource or website path that the specified crawler may access.

For example, the PhotoBiz robots.txt file includes Allow instructions for CSS and JavaScript resources used by your website.

Disallow

A Disallow instruction identifies a path that the specified crawler should not access.

PhotoBiz uses these instructions to restrict selected bots and system-related areas of the platform. A disallowed system path does not mean that your entire website is blocked from search engines.

A blank Disallow instruction means that no website paths are restricted for that crawler.

Sitemap

The Sitemap instruction identifies the location of your website’s XML sitemap.

You can view your sitemap by adding /sitemap.xml to the end of your domain name.

For example:

https://www.yourdomain.com/sitemap.xml

Your sitemap helps search engines discover the public pages included on your website. PhotoBiz automatically creates and maintains it for you.

Crawl-delay

The PhotoBiz robots.txt file includes a Crawl-delay: 60 instruction. This asks crawlers that support the directive to wait between requests to the website.

Google does not support the crawl-delay directive and ignores this setting. Google uses its own systems to determine how frequently its crawlers request content from a website.

Other crawlers may recognize and follow the 60-second crawl delay. This instruction does not prevent Google from crawling or indexing your public website pages, and it does not directly affect your Google search rankings.

Robots.txt and Search Engine Indexing

Crawling and indexing are separate processes.

Crawling occurs when a search engine visits and reviews a URL. Indexing occurs when the search engine evaluates that URL for possible inclusion in search results.

A robots.txt instruction can ask a compliant crawler not to access a URL. However, it does not guarantee that the URL will never appear in search results. A search engine may still discover a restricted URL through links, previously collected information, or other sources.

If you do not want an eligible public website page to appear in search results, add a noindex tag to the page instead of relying on robots.txt.

For content that should not be publicly accessible, use password protection, disable the page, delete the page, or remove the content from your public website. Robots.txt should not be used as a security or privacy tool.

Troubleshooting and FAQs

Does the PhotoBiz robots.txt file block Google?

No. The file does not contain an instruction that blocks Googlebot from your public website.

It includes specific instructions for supported website resources, while separate rules manage selected crawlers and PhotoBiz system paths.

Does the 60-second crawl delay hurt my SEO?

The crawl delay does not limit Googlebot or directly affect your Google rankings. Google does not support the crawl-delay instruction and ignores it.

Other crawlers may recognize and follow the instruction.

Why does an SEO audit report blocked URLs?

SEO auditing tools may identify administrative, cart, checkout, billing, or other PhotoBiz system paths that are intentionally restricted.

A blocked system URL does not necessarily indicate an SEO problem. These areas are generally not intended to appear in search results.

Review the exact URL listed in the report. If the audit identifies a normal public website page that should be available to search engines, contact PhotoBiz Support so the URL can be reviewed.

Can I add a custom rule for one page?

No. Custom robots.txt rules are not available for individual PhotoBiz websites.

Add a noindex tag to the page when you do not want it to appear in search results.

Can robots.txt protect private pages or images?

No. Robots.txt only provides instructions to automated crawlers. It does not prevent someone from visiting a publicly accessible URL, and some automated bots may not follow its instructions.

Use password protection for private content, Client Galleries, or pages that should only be available to selected visitors.

Do I need to submit my robots.txt file to Google?

No. Search engines automatically check for a robots.txt file at the standard location on your domain.

You do not need to upload, submit, or update it.

Does the robots.txt file include my sitemap?

Yes. The file contains a Sitemap instruction that identifies the XML sitemap associated with your domain.

PhotoBiz automatically creates and updates your sitemap as you make changes to your website.

Can removing a page from my sitemap prevent indexing?

No. Removing a page from the sitemap does not prevent search engines from discovering or indexing it through other sources.

Use a noindex tag when you want an eligible public page excluded from search results. Disable or delete the page when it should no longer be available online.

Related Articles

Need Help?

Support Hours: Monday to Friday, 9am to 6pm EST

Phone: 866.463.7620

Support Channels: Phone, Live Chat, Support Ticket

Previous Post Next Post

How do I add Google Analytics tracking to my website?

July 25, 2025

How do I submit my sitemap to Google?

January 27, 2026

How Do I Setup My SEO?

July 24, 2026

How To Customize SEO For Pages

June 16, 2026

What Is the Robots.txt File for My PhotoBiz Website