{"id":2412,"date":"2024-08-13T13:18:26","date_gmt":"2024-08-13T11:18:26","guid":{"rendered":"https:\/\/www.estugo.de\/howsting\/?p=2412"},"modified":"2024-09-30T12:37:20","modified_gmt":"2024-09-30T10:37:20","slug":"robots-txt","status":"publish","type":"post","link":"https:\/\/www.estugo.de\/howsting\/robots-txt\/","title":{"rendered":"robots.txt"},"content":{"rendered":"\n<h3 class=\"wp-block-heading\">1. <strong>Was ist die <code>robots.txt<\/code>-Datei?<\/strong><\/h3>\n\n\n\n<p>Die <code>robots.txt<\/code>-Datei ist eine einfache Textdatei, die im Stammverzeichnis Ihrer Website abgelegt wird. Sie wird von Webcrawlern (Bots) gelesen, um zu bestimmen, welche Bereiche der Website durchsucht oder ignoriert werden sollen. Durch entsprechende Anweisungen in dieser Datei k\u00f6nnen Sie verhindern, dass bestimmte Bots auf Ihre Website zugreifen.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">2. <strong>Erstellen der <code>robots.txt<\/code>-Datei<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u00d6ffnen Sie einen Texteditor (z. B. Notepad, Sublime Text oder ein anderes Textbearbeitungsprogramm).<\/li>\n\n\n\n<li>Erstellen Sie eine neue Datei und speichern Sie diese unter dem Namen <code>robots.txt<\/code>.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">3. <strong>Bots in der <code>robots.txt<\/code>-Datei sperren<\/strong><\/h3>\n\n\n\n<p>Um bestimmte Bots auszuschlie\u00dfen, verwenden Sie die Anweisung <code>User-agent<\/code> gefolgt von <code>Disallow<\/code>. Hier ist eine Beispielsdatei, die AhrefsBot, SemrushBot, BingBot und DotBot sperrt:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">User-agent: AhrefsBot<br>Disallow: \/<br><br>User-agent: SemrushBot<br>Disallow: \/<br><br>User-agent: MJ12Bot<br>Disallow: \/<br><br>User-agent: DotBot<br>Disallow: \/<br><br># Optional: Alle anderen Bots d\u00fcrfen auf die Website zugreifen<br>User-agent: *<br>Disallow:<br><\/pre>\n\n\n\n<p><strong>Erl\u00e4uterungen:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>User-agent: [Bot-Name]<\/code>: Gibt den Namen des Bots an, den Sie sperren m\u00f6chten.<\/li>\n\n\n\n<li><code>Disallow: \/<\/code>: Verhindert, dass der angegebene Bot auf irgendeinen Teil der Website zugreift.<\/li>\n\n\n\n<li><code>User-agent: *<\/code>: Diese Anweisung erlaubt allen anderen Bots den Zugriff auf die gesamte Website, sofern keine spezifischen Einschr\u00e4nkungen bestehen.<\/li>\n<\/ul>\n\n\n\n<p>Weitere Informationen zu Bots findest Du im folgenden Artikel <a href=\"https:\/\/www.estugo.de\/howsting\/bad-bots-und-spam-crawler-was-sind-sie-und-wie-kannst-du-sie-blockieren\/\">Bad Bots und Spam Crawler<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">4. <strong>Speichern und Hochladen der <code>robots.txt<\/code>-Datei<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Speichern Sie die Datei unter dem Namen <code>robots.txt<\/code>.<\/li>\n\n\n\n<li>Laden Sie die Datei in das Stammverzeichnis Ihrer Website hoch (das Verzeichnis, in dem sich auch die Startseite Ihrer Website befindet). Dies k\u00f6nnte der Ordner \/httpdocs\/shop sein.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">5. <strong>\u00dcberpr\u00fcfung der <code>robots.txt<\/code>-Datei<\/strong><\/h3>\n\n\n\n<p>Einige Suchmaschinen bieten auch Tools an, mit denen Sie die Syntax und Wirkung Ihrer <code>robots.txt<\/code>-Datei testen k\u00f6nnen (z. B. die Google Search Console).<\/p>\n\n\n\n<p>Sie k\u00f6nnen \u00fcberpr\u00fcfen, ob Ihre <code>robots.txt<\/code>-Datei korrekt funktioniert, indem Sie die Datei im Browser aufrufen: <code>https:\/\/www.ihre-website.de\/robots.txt<\/code>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>1. Was ist die robots.txt-Datei? Die robots.txt-Datei ist eine einfache Textdatei, die im Stammverzeichnis Ihrer Website abgelegt wird. Sie wird von Webcrawlern (Bots) gelesen, um zu bestimmen, welche Bereiche der Website durchsucht oder ignoriert werden sollen. Durch entsprechende Anweisungen in dieser Datei k\u00f6nnen Sie verhindern, dass bestimmte Bots auf Ihre Website zugreifen. 2. Erstellen der [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"set","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[16,1,52],"tags":[],"class_list":["post-2412","post","type-post","status-publish","format-standard","hentry","category-onlineshop-hilfe","category-allgemein","category-webhosting-hilfe"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>robots.txt - ho(w)sting.de<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.estugo.de\/howsting\/robots-txt\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"robots.txt - ho(w)sting.de\" \/>\n<meta property=\"og:description\" content=\"1. Was ist die robots.txt-Datei? Die robots.txt-Datei ist eine einfache Textdatei, die im Stammverzeichnis Ihrer Website abgelegt wird. Sie wird von Webcrawlern (Bots) gelesen, um zu bestimmen, welche Bereiche der Website durchsucht oder ignoriert werden sollen. Durch entsprechende Anweisungen in dieser Datei k\u00f6nnen Sie verhindern, dass bestimmte Bots auf Ihre Website zugreifen. 2. Erstellen der [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.estugo.de\/howsting\/robots-txt\/\" \/>\n<meta property=\"og:site_name\" content=\"ho(w)sting.de\" \/>\n<meta property=\"article:published_time\" content=\"2024-08-13T11:18:26+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-09-30T10:37:20+00:00\" \/>\n<meta name=\"author\" content=\"ESTUGO.net\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Verfasst von\" \/>\n\t<meta name=\"twitter:data1\" content=\"ESTUGO.net\" \/>\n\t<meta name=\"twitter:label2\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data2\" content=\"2\u00a0Minuten\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/robots-txt\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/robots-txt\\\/\"},\"author\":{\"name\":\"ESTUGO.net\",\"@id\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/#\\\/schema\\\/person\\\/32d59d1ace2e12c7ce24c6101a657b4f\"},\"headline\":\"robots.txt\",\"datePublished\":\"2024-08-13T11:18:26+00:00\",\"dateModified\":\"2024-09-30T10:37:20+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/robots-txt\\\/\"},\"wordCount\":251,\"articleSection\":[\"Onlineshop Hilfe\",\"Technisches\",\"Webhosting Hilfe\"],\"inLanguage\":\"de\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/robots-txt\\\/\",\"url\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/robots-txt\\\/\",\"name\":\"robots.txt - ho(w)sting.de\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/#website\"},\"datePublished\":\"2024-08-13T11:18:26+00:00\",\"dateModified\":\"2024-09-30T10:37:20+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/#\\\/schema\\\/person\\\/32d59d1ace2e12c7ce24c6101a657b4f\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/robots-txt\\\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/robots-txt\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/robots-txt\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Onlineshop Hilfe\",\"item\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/thema\\\/onlineshop-hilfe\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"robots.txt\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/#website\",\"url\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/\",\"name\":\"ho(w)sting.de\",\"description\":\"Wir liefern Ihnen das &quot;So wird es gemacht&quot;.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/#\\\/schema\\\/person\\\/32d59d1ace2e12c7ce24c6101a657b4f\",\"name\":\"ESTUGO.net\",\"sameAs\":[\"https:\\\/\\\/howsting.de\"],\"url\":\"https:\\\/\\\/www.estugo.de\\\/howsting\\\/author\\\/senor_wp_oberwal\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"robots.txt - ho(w)sting.de","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.estugo.de\/howsting\/robots-txt\/","og_locale":"de_DE","og_type":"article","og_title":"robots.txt - ho(w)sting.de","og_description":"1. Was ist die robots.txt-Datei? Die robots.txt-Datei ist eine einfache Textdatei, die im Stammverzeichnis Ihrer Website abgelegt wird. Sie wird von Webcrawlern (Bots) gelesen, um zu bestimmen, welche Bereiche der Website durchsucht oder ignoriert werden sollen. Durch entsprechende Anweisungen in dieser Datei k\u00f6nnen Sie verhindern, dass bestimmte Bots auf Ihre Website zugreifen. 2. Erstellen der [&hellip;]","og_url":"https:\/\/www.estugo.de\/howsting\/robots-txt\/","og_site_name":"ho(w)sting.de","article_published_time":"2024-08-13T11:18:26+00:00","article_modified_time":"2024-09-30T10:37:20+00:00","author":"ESTUGO.net","twitter_card":"summary_large_image","twitter_misc":{"Verfasst von":"ESTUGO.net","Gesch\u00e4tzte Lesezeit":"2\u00a0Minuten"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.estugo.de\/howsting\/robots-txt\/#article","isPartOf":{"@id":"https:\/\/www.estugo.de\/howsting\/robots-txt\/"},"author":{"name":"ESTUGO.net","@id":"https:\/\/www.estugo.de\/howsting\/#\/schema\/person\/32d59d1ace2e12c7ce24c6101a657b4f"},"headline":"robots.txt","datePublished":"2024-08-13T11:18:26+00:00","dateModified":"2024-09-30T10:37:20+00:00","mainEntityOfPage":{"@id":"https:\/\/www.estugo.de\/howsting\/robots-txt\/"},"wordCount":251,"articleSection":["Onlineshop Hilfe","Technisches","Webhosting Hilfe"],"inLanguage":"de"},{"@type":"WebPage","@id":"https:\/\/www.estugo.de\/howsting\/robots-txt\/","url":"https:\/\/www.estugo.de\/howsting\/robots-txt\/","name":"robots.txt - ho(w)sting.de","isPartOf":{"@id":"https:\/\/www.estugo.de\/howsting\/#website"},"datePublished":"2024-08-13T11:18:26+00:00","dateModified":"2024-09-30T10:37:20+00:00","author":{"@id":"https:\/\/www.estugo.de\/howsting\/#\/schema\/person\/32d59d1ace2e12c7ce24c6101a657b4f"},"breadcrumb":{"@id":"https:\/\/www.estugo.de\/howsting\/robots-txt\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.estugo.de\/howsting\/robots-txt\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.estugo.de\/howsting\/robots-txt\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/www.estugo.de\/howsting\/"},{"@type":"ListItem","position":2,"name":"Onlineshop Hilfe","item":"https:\/\/www.estugo.de\/howsting\/thema\/onlineshop-hilfe\/"},{"@type":"ListItem","position":3,"name":"robots.txt"}]},{"@type":"WebSite","@id":"https:\/\/www.estugo.de\/howsting\/#website","url":"https:\/\/www.estugo.de\/howsting\/","name":"ho(w)sting.de","description":"Wir liefern Ihnen das &quot;So wird es gemacht&quot;.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.estugo.de\/howsting\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Person","@id":"https:\/\/www.estugo.de\/howsting\/#\/schema\/person\/32d59d1ace2e12c7ce24c6101a657b4f","name":"ESTUGO.net","sameAs":["https:\/\/howsting.de"],"url":"https:\/\/www.estugo.de\/howsting\/author\/senor_wp_oberwal\/"}]}},"_links":{"self":[{"href":"https:\/\/www.estugo.de\/howsting\/wp-json\/wp\/v2\/posts\/2412","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.estugo.de\/howsting\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.estugo.de\/howsting\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.estugo.de\/howsting\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.estugo.de\/howsting\/wp-json\/wp\/v2\/comments?post=2412"}],"version-history":[{"count":3,"href":"https:\/\/www.estugo.de\/howsting\/wp-json\/wp\/v2\/posts\/2412\/revisions"}],"predecessor-version":[{"id":2621,"href":"https:\/\/www.estugo.de\/howsting\/wp-json\/wp\/v2\/posts\/2412\/revisions\/2621"}],"wp:attachment":[{"href":"https:\/\/www.estugo.de\/howsting\/wp-json\/wp\/v2\/media?parent=2412"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.estugo.de\/howsting\/wp-json\/wp\/v2\/categories?post=2412"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.estugo.de\/howsting\/wp-json\/wp\/v2\/tags?post=2412"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}