{"id":31157,"date":"2023-06-12T12:01:52","date_gmt":"2023-06-12T12:01:52","guid":{"rendered":"https:\/\/seo.london\/?p=31157"},"modified":"2023-06-29T16:47:23","modified_gmt":"2023-06-29T16:47:23","slug":"robots-txt-ile-botlar-nasil-engellenir","status":"publish","type":"post","link":"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/","title":{"rendered":"Robots.txt ile Botlar Nas\u0131l Engellenir?"},"content":{"rendered":"<p class=\"wp-block-paragraph\">Bilgisiz bir g\u00f6zlemci i\u00e7in web sitenizde dola\u015fan bir robot bilimkurgu filmlerinden f\u0131rlam\u0131\u015f gibi g\u00f6r\u00fcnebilir. \u0130ster inan\u0131n ister inanmay\u0131n, bu kurgudan \u00e7ok uzak ve ger\u00e7e\u011fe d\u00fc\u015f\u00fcnd\u00fc\u011f\u00fcn\u00fczden daha yak\u0131n! Web sitelerine sahip olma ve bak\u0131m\u0131n\u0131 yapma konusunda yol alan herkes i\u00e7in, botlar\u0131n \u00e7evrimi\u00e7i alanlar\u0131m\u0131zla nas\u0131l etkile\u015fime girdi\u011fini anlamak \u00e7ok \u00f6nemlidir. Bu etkile\u015fimi d\u00fczenleme kapasitesine sahip olmak da ayn\u0131 derecede \u00f6nemlidir. Bu ihtiya\u00e7 bizi kullan\u0131\u015fl\u0131 bir ara\u00e7la tan\u0131\u015ft\u0131r\u0131yor: <a href=\"https:\/\/seo.london\/tr\/seoda-robots-txt-dosyasi-nasil-kullanilir\/\">robotlar<\/a>.txt. Bu kapsaml\u0131 \u00e7al\u0131\u015fmada <a href=\"https:\/\/seo.london\/tr\/seo-ogrenmek\/\">rehber<\/a>, biz \u00e7\u00f6zece\u011fiz <a href=\"https:\/\/seo.london\/tr\/baglanti-kurma-nasil-yapilir-ne-yapilmaz\/\">ne<\/a> \"bots robots txt nas\u0131l engellenir\" ne anlama geliyor ve g\u00fcn\u00fcm\u00fcz dijital \u00e7a\u011f\u0131nda neden \u00f6nemli?<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-what-is-a-robots-txt-file\"><span class=\"ez-toc-section\" id=\"what-is-a-robotstxt-file\"><\/span>Robots.txt Dosyas\u0131 Nedir?<span class=\"ez-toc-section-end\"><\/span><\/h2><div id=\"ez-toc-container\" class=\"ez-toc-v2_0_69_1 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">\u0130\u00e7indekiler<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"\u0130\u00e7erik Tablosunu De\u011fi\u015ftir\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Ge\u00e7i\u015f<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #000000;color:#000000\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #000000;color:#000000\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 eztoc-toggle-hide-by-default' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#what-is-a-robotstxt-file\" title=\"Robots.txt Dosyas\u0131 Nedir?\">Robots.txt Dosyas\u0131 Nedir?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#technical-robotstxt-syntax\" title=\"Teknik Robots.txt S\u00f6zdizimi\">Teknik Robots.txt S\u00f6zdizimi<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#pattern-matching\" title=\"Desen E\u015fle\u015ftirme\">Desen E\u015fle\u015ftirme<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#checking-if-you-have-a-robotstxt-file\" title=\"Robots.txt dosyan\u0131z olup olmad\u0131\u011f\u0131n\u0131 kontrol etme\">Robots.txt dosyan\u0131z olup olmad\u0131\u011f\u0131n\u0131 kontrol etme<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#how-to-create-a-robotstxt-file\" title=\"Robots.txt Dosyas\u0131 Nas\u0131l Olu\u015fturulur\">Robots.txt Dosyas\u0131 Nas\u0131l Olu\u015fturulur<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#understanding-the-components-of-robotstxt\" title=\"Robots.txt Bile\u015fenlerini Anlama\">Robots.txt Bile\u015fenlerini Anlama<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#fresh-file-generation\" title=\"Taze Dosya \u00dcretimi\">Taze Dosya \u00dcretimi<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#uploading-your-file\" title=\"Dosyan\u0131z\u0131 Y\u00fckleme\">Dosyan\u0131z\u0131 Y\u00fckleme<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#blocking-specific-bots-and-filesfolders\" title=\"Belirli Botlar\u0131 ve Dosyalar\u0131\/Klas\u00f6rleri Engelleme\">Belirli Botlar\u0131 ve Dosyalar\u0131\/Klas\u00f6rleri Engelleme<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#robotstxt-vs-meta-robots-vs-x-robots\" title=\"Robots.txt vs Meta Robotlar vs X-Robotlar\">Robots.txt vs Meta Robotlar vs X-Robotlar<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#the-robotstxt-file\" title=\"Robots.txt Dosyas\u0131\">Robots.txt Dosyas\u0131<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#what-are-meta-robots-tags\" title=\"Meta Robots Etiketleri Nedir?\">Meta Robots Etiketleri Nedir?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/#how-do-x-robots-tags-work\" title=\"X-Robots Etiketleri Nas\u0131l \u00c7al\u0131\u015f\u0131r?\">X-Robots Etiketleri Nas\u0131l \u00c7al\u0131\u015f\u0131r?<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n\n\n\n\n<p class=\"wp-block-paragraph\">Robots.txt dosyas\u0131 esasen web sitenizin bek\u00e7isidir. Kontrol etmenizi sa\u011flar <a href=\"https:\/\/seo.london\/tr\/eskort-seo-hi%cc%87zmeti%cc%87\/\">hangi<\/a> Sitenizin baz\u0131 b\u00f6l\u00fcmleri botlar\u0131n ke\u015ffine a\u00e7\u0131kt\u0131r - Google'\u0131n <a href=\"https:\/\/seo.london\/tr\/googleda-gelismis-arama-nasil-yapilir\/\">arama<\/a> motor \u00f6r\u00fcmcekleri ve hangilerinin yasak olmas\u0131 gerekti\u011fi. Robot D\u0131\u015flama Standard\u0131n\u0131n (siteler taraf\u0131ndan kullan\u0131lan resmi olmayan bir standart) bir par\u00e7as\u0131 olarak \u00e7al\u0131\u015farak, web robotlar\u0131na web sitenizi ziyaret ettiklerinde izin verilen eylemler hakk\u0131nda talimat verir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bu m\u00fctevaz\u0131 metin dosyas\u0131, sayfa eri\u015filebilirlik tercihleriniz hakk\u0131nda \u00e7ok \u015fey anlat\u0131r. Merakl\u0131 robot g\u00f6zlerden uzak tutmak istedi\u011finiz belirli dizinler veya sayfalar m\u0131 var? Robots.txt dosyas\u0131 sizi korur! \u0130\u00e7eri\u011fi, site eri\u015fimini daha etkili bir \u015fekilde y\u00f6netmeye yard\u0131mc\u0131 olan direktifleri (web taray\u0131c\u0131lar\u0131na verilen \u00f6zel talimatlar) a\u00e7\u0131k\u00e7a belirtir. Bu beceriklilik, sitenin d\u00fczg\u00fcn bir \u015fekilde sunulmas\u0131n\u0131 <a href=\"https:\/\/seo.london\/tr\/i%cc%87ceri%cc%87k-pazarlama-seo-hi%cc%87zmetleri%cc%87\/\">\u0130\u00e7erik<\/a> hassas b\u00f6lgeleri kazara maruz kalmaya kar\u015f\u0131 korurken aramalar\u0131 da kolayla\u015ft\u0131r\u0131r.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Nihayetinde, siber d\u00fcnyam\u0131z\u0131n baz\u0131 b\u00f6l\u00fcmlerini nas\u0131l kordon alt\u0131na alaca\u011f\u0131m\u0131z\u0131 \u00f6\u011frenmek <a href=\"https:\/\/seo.london\/tr\/daha-fazla-yonlendiren-alan-adi-nasil-elde-edilir\/\">etki alanlar\u0131<\/a> Web y\u00f6neticileri olarak platformlar\u0131m\u0131z\u0131n de\u011ferli alanlar\u0131nda bot varl\u0131\u011f\u0131n\u0131 ve etkisini daha iyi y\u00f6nlendirmemizi sa\u011flar - dolay\u0131s\u0131yla bug\u00fcnk\u00fc odak noktam\u0131z.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-technical-robots-txt-syntax\"><span class=\"ez-toc-section\" id=\"technical-robotstxt-syntax\"><\/span>Teknik Robots.txt S\u00f6zdizimi<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Bir robots.txt dosyas\u0131n\u0131n s\u00f6zdizimi, esasen dil ve dilbilgisi <a href=\"https:\/\/seo.london\/tr\/makale-formati-ve-makale-yapisi\/\">yap\u0131<\/a> direktiflerini olu\u015fturmak i\u00e7in kullan\u0131l\u0131r. Bu s\u00f6zdiziminden do\u011fru \u015fekilde yararlanman\u0131n robots txt kullanarak botlar\u0131n nas\u0131l engellenece\u011fini \u00f6\u011frenmeye nas\u0131l yard\u0131mc\u0131 olabilece\u011fini anlamak \u00e7ok \u00f6nemlidir.<\/p>\n\n\n\n<ol class=\"wp-block-list\" type=\"1\" start=\"1\">\n<li><strong>Kullan\u0131c\u0131 arac\u0131s\u0131:<\/strong> User-agent y\u00f6nergesi, ileti\u015fim kurmak istedi\u011finiz bot t\u00fcr\u00fcn\u00fc belirtir; \u00f6rne\u011fin Google i\u00e7in Googlebot veya BingBot i\u00e7in <a href=\"https:\/\/seo.london\/tr\/web-sitesi-bings-aramasinda-dizine-eklenmiyor\/\">Bing<\/a>. Y\u00f6nerge setinize \"User-agent: *\" ile ba\u015flaman\u0131z, t\u00fcm web taray\u0131c\u0131lar\u0131n\u0131n bu talimatlar\u0131 dikkate almas\u0131 gerekti\u011fi anlam\u0131na gelir.<\/li>\n\n\n\n<li><strong>\u0130zin verme:<\/strong> Bu y\u00f6nerge basit bir mesaj g\u00f6nderir - hemen ard\u0131ndan tan\u0131mlanan yoldan ka\u00e7\u0131n\u0131n. Diyelim ki \"Disallow: \/images\/\" yazarsan\u0131z, bunu okuyan herhangi bir bota \u015funlar\u0131 yapmamas\u0131 talimat\u0131n\u0131 vermi\u015f olursunuz <a href=\"https:\/\/seo.london\/tr\/jetoctopus\/\">s\u00fcr\u00fcnmek<\/a> web sitenizin <a href=\"https:\/\/seo.london\/tr\/google-gorselleri-indekslemiyor\/\">g\u00f6r\u00fcnt\u00fcler<\/a> dizin.<\/li>\n\n\n\n<li><strong>\u0130zin ver:<\/strong> \u0130zin vermemenin tam tersi olarak, 'izin verilmeyen' dizinler i\u00e7inde, bir allow deyimi belirli alt dizinlere veya dosyalara eri\u015fim iznini geri verir.<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-pattern-matching\"><span class=\"ez-toc-section\" id=\"pattern-matching\"><\/span>Desen E\u015fle\u015ftirme<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Robots.txt dosyas\u0131 s\u00f6zdiziminin karma\u015f\u0131k ancak g\u00fc\u00e7l\u00fc bir unsuru da kal\u0131p e\u015fle\u015ftirmedir. Yollar\u0131 do\u011frudan belirtmenin yan\u0131 s\u0131ra, kal\u0131p e\u015fle\u015ftirme, basit semboller arac\u0131l\u0131\u011f\u0131yla bir robots txt dosyas\u0131nda botlar\u0131 nas\u0131l engelleyece\u011finize dair karma\u015f\u0131k talimatlar\u0131 ifade etmenizi sa\u011flar.<\/p>\n\n\n\n<ol class=\"wp-block-list\" type=\"1\" start=\"1\">\n<li>\u00d6r\u00fcnt\u00fc e\u015fle\u015ftirmeyi \u00f6\u011frenirken \u00f6ncelikle iki temel karaktere odaklan\u0131n: '*' (y\u0131ld\u0131z i\u015fareti) ve '$' (dolar i\u015fareti). Y\u0131ld\u0131z i\u015fareti joker karakter olarak i\u015flev g\u00f6r\u00fcrken dolar i\u015fareti URL'nin sonunu simgeler.<\/li>\n\n\n\n<li>Bir disallow deyiminin i\u00e7inde y\u0131ld\u0131z i\u015fareti kullanmak, orada bulunan herhangi bir dize dizisini belirtir. \u00d6rne\u011fin, 'Disallow: \/example' web taray\u0131c\u0131lar\u0131n\u0131n web sitenizde URL'sinde 'example' ge\u00e7en herhangi bir sayfaya eri\u015fmesini engelleyecektir.<\/li>\n\n\n\n<li>Aksine, farkl\u0131 terimlerinizin sonuna '$' eklenmesi, yaln\u0131zca <a href=\"https:\/\/seo.london\/tr\/seo-dostu-urller\/\">URL'ler<\/a> bu \u015fekilde sona ermesi yasaklanm\u0131\u015ft\u0131r <a href=\"https:\/\/seo.london\/tr\/seoda-crawling-ve-indexing-arasindaki-fark-nedir\/\">s\u00fcr\u00fcnme<\/a> botlar taraf\u0131ndan. 'Disallow: \/*example$' \u015feklindeki bir bildirim, yaln\u0131zca URL'si tam olarak 'example' ile biten sayfalara eri\u015fimi k\u0131s\u0131tlar.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Ancak unutmay\u0131n, t\u00fcm \u00f6r\u00fcmcekler bu kal\u0131plar\u0131 anlamaz veya takip etmez - \u00f6zellikle de spam odakl\u0131 olanlar - bu nedenle y\u00f6nergeleri olu\u015ftururken ve robots txt dosyalar\u0131n\u0131 kullanarak botlar\u0131 etkili bir \u015fekilde nas\u0131l engelleyece\u011finize dair etkili yollar\u0131 ay\u0131rt ederken bunu g\u00f6z \u00f6n\u00fcnde bulundurun.\"\"\"<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Robots.txt dosyan\u0131z\u0131n yerle\u015fiminde gezinmek g\u00f6z korkutucu g\u00f6r\u00fcnebilir, ancak emin olun, nispeten basit bir i\u015flemdir. Bu k\u00fc\u00e7\u00fck ama \u00f6nemli belge kesin bir yere aittir - web sitenizin k\u00f6k dizini.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Unutulmamas\u0131 gereken kritik nokta, bu basit metin dosyas\u0131n\u0131n taray\u0131c\u0131lar taraf\u0131ndan kolayca bulunabilmesi gerekti\u011fidir. \"K\u00f6k\" veya en \u00fcstteki dizin genellikle arama motoru botlar\u0131n\u0131n ilk olarak gitti\u011fi yerdir <a href=\"https:\/\/seo.london\/tr\/acilis-sayfalari-seo-icin-kotu-mu\/\">ini\u015f<\/a> senin <a href=\"https:\/\/seo.london\/tr\/tam-eslesmeli-alan-adinin-artilari-ve-eksileri\/\">etki alan\u0131<\/a>. Bu nedenle, robots.txt dosyas\u0131n\u0131 buraya yerle\u015ftirmek, sitenizin hangi b\u00f6l\u00fcmlerinin eri\u015filebilir olmas\u0131 gerekti\u011fi konusunda an\u0131nda ve net talimatlar sa\u011flar.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u015eimdi, web diline daha az a\u015fina olanlar i\u00e7in, 'k\u00f6k' dizinden bahsederken tam olarak ne demek istedi\u011fimizi merak ediyor olabilirsiniz. \u00d6z\u00fcnde, web sitenizin k\u00f6k dizini, di\u011fer t\u00fcm dizinlerin kendisinden \u00e7\u0131kt\u0131\u011f\u0131 bir a\u011fa\u00e7 g\u00f6vdesine benzer - \u00e7evrimi\u00e7i varl\u0131\u011f\u0131n\u0131z\u0131n omurgas\u0131n\u0131 olu\u015fturur. \u00d6rne\u011fin, web sitenizin URL'si www.example.com ise, k\u00f6k dizin \/ (.com'dan sonraki e\u011fik \u00e7izgi) olacakt\u0131r. B\u00f6ylece, www.example.com\/robots.txt k\u00f6k dizininizdeki yerini m\u00fckemmel bir \u015fekilde belirler.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Buna kar\u015f\u0131l\u0131k, ba\u015fka bir yerin alt\u0131na yerle\u015ftirmek <a href=\"https:\/\/seo.london\/tr\/alt-alan-adinda-veya-alt-dizinde-blog\/\">alt dizin<\/a> blog\/robots.txt gibi istenen etkiyi yaratmayacakt\u0131r \u00e7\u00fcnk\u00fc botlar talimatlar\u0131 almadan \u00f6nce sitenizde o kadar arama yapma zahmetine girmeyecektir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">En \u00f6nemlisi, yanl\u0131\u015f konumland\u0131rma, tarama ve indekslemenin verimsiz olmas\u0131na yol a\u00e7abilir. <a href=\"https:\/\/seo.london\/tr\/\">SEO<\/a> ba\u015far\u0131 - \u00e7\u00fcnk\u00fc arama motorlar\u0131 'kap\u0131n\u0131za' geldiklerinde nereleri ke\u015ffetmelerine izin verildi\u011fini veya yasakland\u0131\u011f\u0131n\u0131 bilmeyeceklerdir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bu nedenle, robots txt dosyalar\u0131n\u0131 kullanarak botlar\u0131 verimli bir \u015fekilde nas\u0131l engelleyece\u011finize bakarken yerle\u015fimi tam olarak anlad\u0131\u011f\u0131n\u0131zdan emin olun. Yerle\u015fim, bu teknik SEO temel ta\u015f\u0131 ayar\u0131nda ger\u00e7ekten ayr\u0131lmaz bir rol oynar.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Robots.txt dosyalar\u0131n\u0131n \u00f6nemini ve i\u015fleyi\u015fini anlad\u0131\u011f\u0131m\u0131zda, geriye bir soru kal\u0131yor: neden bir robots.txt dosyas\u0131na ihtiyac\u0131n\u0131z var?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u0130lk olarak, bir robots.txt dosyas\u0131na sahip olmak, web taray\u0131c\u0131lar\u0131na web sitenizle nas\u0131l etkile\u015fimde bulunmalar\u0131 gerekti\u011fi konusunda rehberlik sa\u011flar. Arama motorlar\u0131 sitenizi dizine eklemek i\u00e7in sitenize yakla\u015ft\u0131\u011f\u0131nda, robots.txt dosyan\u0131zdaki bu talimatlar devreye girer. Google'\u0131n Googlebot'u veya Bing'in Bingbot'u gibi arama botlar\u0131na alan\u0131n\u0131zdaki gezinme yollar\u0131nda rehberlik ederler.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u0130kinci olarak, bir robots.txt dosyas\u0131, sitenizin hassas veya geli\u015ftirilme a\u015famas\u0131nda olan \u00f6zel b\u00f6l\u00fcmlerine eri\u015fimi y\u00f6netmek i\u00e7in gereklidir. Botlara \u00f6zellikle \u015fu talimatlar\u0131 verebilirsiniz <a href=\"https:\/\/seo.london\/tr\/googlein-sitenizi-daha-hizli-indekslemesini-nasil-saglarsiniz\/\">indeksleme<\/a> bu t\u00fcr i\u00e7erikler. Bu, istenmeyen alanlar\u0131n indekslenmeden kalmas\u0131n\u0131 ve Arama Motoru Sonu\u00e7 Sayfalar\u0131 (SERP'ler) arac\u0131l\u0131\u011f\u0131yla genel g\u00f6r\u00fc\u015f\u00fcn d\u0131\u015f\u0131nda kalmas\u0131n\u0131 sa\u011flar.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Dahas\u0131, web \u00fczerinde hem iyi hem de k\u00f6t\u00fc huylu say\u0131s\u0131z tarama botu vard\u0131r. Robots.txt dosyan\u0131zdaki belirli 'User-agent' komutlar\u0131 arac\u0131l\u0131\u011f\u0131yla sitenizde kimin neyi tarayabilece\u011fini belirleyerek, masum tarama faaliyeti kisvesi alt\u0131nda ziyaret eden potansiyel tehditlere kar\u015f\u0131 koruma standartlar\u0131n\u0131 y\u00fcksek tutars\u0131n\u0131z.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Son olarak, bir Robots txt dosyas\u0131 taraf\u0131ndan sa\u011flanan k\u0131s\u0131tlamalar olmadan, baz\u0131 botlar sunuculara a\u015f\u0131r\u0131 istek y\u00fckleyerek kullan\u0131c\u0131lar\u0131n yava\u015flamas\u0131na neden olabilir <a href=\"https:\/\/seo.london\/tr\/sayfa-deneyimi-sinyalleri-nelerdir\/\">deneyim<\/a> veya DDoS (Da\u011f\u0131t\u0131lm\u0131\u015f Hizmet Engelleme) sald\u0131r\u0131lar\u0131. Bu nedenle, optimum hizmetin sa\u011flanmas\u0131 i\u00e7in \u00f6nemli bir ara\u00e7 g\u00f6revi g\u00f6r\u00fcr. <a href=\"https:\/\/seo.london\/tr\/sunucu-tarafi-vs-istemci-tarafi-isleme\/\">sunucu<\/a> <a href=\"https:\/\/seo.london\/tr\/bilgi-tabani\/nasil-izlenir-seo-performans\/\">performans<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bu makalenin ilerleyen b\u00f6l\u00fcmlerinde kendi Robots txt dosyan\u0131z\u0131 yap\u0131land\u0131rmaya ba\u015flarken bu anahtar kavram\u0131 hat\u0131rlay\u0131n: Taray\u0131c\u0131lar\u0131n web sitenizle etkile\u015fimleri \u00fczerindeki kontrol\u00fc \u00f6rneklendirmek, \u00f6zel bir Robots txt dosyas\u0131na sahip olman\u0131n herhangi bir alan ad\u0131n\u0131n \u00e7evrimi\u00e7i varl\u0131\u011f\u0131n\u0131 korumak ve optimize etmek i\u00e7in neden \u00e7ok \u00f6nemli oldu\u011funu tan\u0131mlar.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-checking-if-you-have-a-robots-txt-file\"><span class=\"ez-toc-section\" id=\"checking-if-you-have-a-robotstxt-file\"><\/span>Robots.txt dosyan\u0131z olup olmad\u0131\u011f\u0131n\u0131 kontrol etme<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">\u015eimdi web sitenizin bir 'robots.txt' dosyas\u0131na sahip olup olmad\u0131\u011f\u0131n\u0131 nas\u0131l tespit edebilece\u011finize ge\u00e7elim. Genellikle bu dosya sitenizin k\u00f6k dizininde bulunur.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Varl\u0131\u011f\u0131n\u0131 kontrol etmek i\u00e7in a\u015fa\u011f\u0131daki basit ad\u0131mlar\u0131 uygulaman\u0131z\u0131 tavsiye ederim:<\/p>\n\n\n\n<ol class=\"wp-block-list\" type=\"1\" start=\"1\">\n<li>Favori web taray\u0131c\u0131n\u0131z\u0131 a\u00e7\u0131n.<\/li>\n\n\n\n<li>\u0130\u00e7inde <a href=\"https:\/\/seo.london\/tr\/fiziksel-adres-olmadan-yerel-seo\/\">adres<\/a> barda <a href=\"https:\/\/seo.london\/tr\/seo-ajansi-londra\/\">\u00fcst<\/a>yoursitename.com\/robots.txt yaz\u0131n; \"yoursitename.com\" yerine ger\u00e7ek alan ad\u0131n\u0131z\u0131 yaz\u0131n.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Sitenizde mevcutsa, ekran\u0131n\u0131zda bu m\u00fctevazi ancak etkili 'robots.txt' dosyas\u0131n\u0131n i\u00e7eri\u011fi g\u00f6r\u00fcnt\u00fclenmelidir. Tersine, \"404 sayfa bulunamad\u0131\" veya \"dosya bulunamad\u0131\" benzeri bir hata mesaj\u0131, \u015fu anda robots.txt dosyas\u0131n\u0131n bulunmad\u0131\u011f\u0131n\u0131 g\u00f6sterir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Unutmay\u0131n ki 'botlar robotlar txt nas\u0131l engellenir'i do\u011fru bir \u015fekilde uygulamak <a href=\"https:\/\/seo.london\/tr\/seo-strateji%cc%87si%cc%87-londra\/\">strateji\u0307<\/a> Arama Motoru Optimizasyonunu (SEO) \u00f6nemli \u00f6l\u00e7\u00fcde etkiler. Bu nedenle, sahip olup olmad\u0131\u011f\u0131n\u0131z konusunda bilgi sahibi olman\u0131z \u00e7ok \u00f6nemlidir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u00d6zetle (zorunlu olmasa da), bir 'robots.txt' dosyas\u0131n\u0131 anlamak ve do\u011fru \u015fekilde kullanmak, g\u00fcn\u00fcm\u00fczde ba\u015far\u0131l\u0131 web sitelerini y\u00f6netmenin ayr\u0131lmaz bir par\u00e7as\u0131n\u0131 olu\u015fturmaktad\u0131r. Dosyan\u0131n varl\u0131\u011f\u0131n\u0131 kontrol etmek i\u00e7in bu ad\u0131mlar\u0131 uygulad\u0131ktan sonra hala emin de\u011filseniz, daha ileri d\u00fczey BT gerektirebilece\u011finden uzman tavsiyesi almay\u0131 d\u00fc\u015f\u00fcn\u00fcn <a href=\"https:\/\/seo.london\/tr\/google-bilgi-grafigini-nasil-kullanirim\/\">B\u0130LG\u0130<\/a> beklenenden daha fazla.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Ayr\u0131ca 'robots.txt' dosyas\u0131n\u0131n olmamas\u0131n\u0131n mutlaka zararl\u0131 olmad\u0131\u011f\u0131n\u0131 da unutmay\u0131n - bu yaln\u0131zca arama motoru botlar\u0131n\u0131n sitenizin t\u00fcm alanlar\u0131na s\u0131n\u0131rs\u0131z eri\u015fimi anlam\u0131na gelir. Sitelerimizde 'bots robots txt'yi etkili bir \u015fekilde nas\u0131l engelleyece\u011fimizi' anlad\u0131\u011f\u0131m\u0131zda, bu t\u00fcr eri\u015fim \u00fczerinde anlaml\u0131 bir kontrol son derece m\u00fcmk\u00fcn hale gelir!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-how-to-create-a-robots-txt-file\"><span class=\"ez-toc-section\" id=\"how-to-create-a-robotstxt-file\"><\/span>Robots.txt Dosyas\u0131 Nas\u0131l Olu\u015fturulur<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Bir robots.txt dosyas\u0131 olu\u015fturmak, arama motoru botlar\u0131n\u0131n web sitenizle nas\u0131l etkile\u015fime girece\u011fini y\u00f6netmek i\u00e7in \u00f6nemli bir ad\u0131md\u0131r. \u015eimdi bir tane olu\u015fturma s\u00fcrecine dalal\u0131m.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-understanding-the-components-of-robots-txt\"><span class=\"ez-toc-section\" id=\"understanding-the-components-of-robotstxt\"><\/span>Robots.txt Bile\u015fenlerini Anlama<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Tipik bir robots.txt dosyas\u0131 User-agent ve Disallow y\u00f6nergeleri olmak \u00fczere iki ana bile\u015fen i\u00e7erir. User-agent, Googlebot veya Bingbot gibi talimatlar\u0131n\u0131z\u0131 hedeflemesini istedi\u011finiz belirli web taray\u0131c\u0131s\u0131n\u0131 ifade eder. \u00d6te yandan, Disallow y\u00f6nergesi, belirli botlar\u0131n taramas\u0131n\u0131 istemedi\u011finiz sayfalar\u0131 veya dizinleri listeledi\u011finiz yerdir. \u00d6rne\u011fin:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Kullan\u0131c\u0131 arac\u0131s\u0131: * Disallow: \/\u00f6zel\/<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bu durumda, t\u00fcm botlar\u0131n ('*' t\u00fcm anlam\u0131na gelir) '\u00f6zel' dizin alt\u0131ndaki herhangi bir \u015feye eri\u015fimi engellenir.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-fresh-file-generation\"><span class=\"ez-toc-section\" id=\"fresh-file-generation\"><\/span>Taze Dosya \u00dcretimi<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u015eimdi bu \u015f\u0131k kod par\u00e7as\u0131n\u0131 olu\u015fturmaya ge\u00e7elim. Bir d\u00fcz metin edit\u00f6r\u00fcne ihtiyac\u0131n\u0131z olacak -Notepad i\u015finizi g\u00f6recektir. Microsoft Word gibi kelime i\u015flemciler, fazladan bi\u00e7imlendirme karakterleri ekleme e\u011filimleri nedeniyle bu g\u00f6rev i\u00e7in uygun de\u011fildir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Ba\u015flamak i\u00e7in yeni bir belge olu\u015fturun ve \"robots.txt\" olarak kaydedin. Burada b\u00fcy\u00fck harf kullan\u0131m\u0131n\u0131n \u00f6nemli oldu\u011funu unutmay\u0131n - her \u015feyin k\u00fc\u00e7\u00fck harfle yaz\u0131ld\u0131\u011f\u0131ndan emin olun. Ard\u0131ndan, hangi b\u00f6l\u00fcmleri engellemeyi ama\u00e7lad\u0131\u011f\u0131n\u0131za g\u00f6re s\u00f6zdizimini haz\u0131rlay\u0131n. Unutmay\u0131n, her kural kendi sat\u0131r\u0131nda olmal\u0131d\u0131r:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Kullan\u0131c\u0131 arac\u0131s\u0131: * Disallow: \/<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bu kural, t\u00fcm botlar\u0131n sitenizin herhangi bir b\u00f6l\u00fcm\u00fcne ('\/' ile g\u00f6sterilen) eri\u015fmesini engeller. Dikkatli kullan\u0131n!<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bu <a href=\"https:\/\/seo.london\/tr\/meta-anahtar-kelimeler-nedir\/\">anahtar kelime<\/a> Burada spesifiklik s\u00f6z konusudur; botlar\u0131n nas\u0131l engellenece\u011fini \u00f6\u011frenirken robot txt mod\u00fclleri bot eylemleri \u00fczerinde hassas kontrol sa\u011flayan \u00e7ok y\u00f6nl\u00fc ara\u00e7lard\u0131r.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-uploading-your-file\"><span class=\"ez-toc-section\" id=\"uploading-your-file\"><\/span>Dosyan\u0131z\u0131 Y\u00fckleme<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Olu\u015fturulduktan sonra robots.txt dosyan\u0131z\u0131 FTP (Dosya Aktar\u0131m Protokol\u00fc) kullanarak sitenizin k\u00f6k klas\u00f6r\u00fcne y\u00fckleyin. Genellikle wp-admin, wp-content ve wp-includes klas\u00f6r\u00fcn\u00fczle ayn\u0131 konumda bulunur.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bu ad\u0131mlar\u0131 ba\u015far\u0131yla tamamlad\u0131ktan sonra, kullan\u0131c\u0131lar Robots.txt dosyan\u0131z\u0131 birincil alan ad\u0131n\u0131zdan sonra \"\/robots.txt\" ekleyerek bulabilirler - \u00f6rne\u011fin, www.example.com\/robots.txt. Art\u0131k bir robots.txt dosyas\u0131n\u0131n nas\u0131l olu\u015fturulaca\u011f\u0131n\u0131 \u00f6\u011frendiniz!<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Yine de d\u00fcr\u00fcst taray\u0131c\u0131lar\u0131 y\u00f6nlendirmede etkili olsa da nezaketin yaln\u0131zca uyumu dikte etti\u011fini unutmay\u0131n; daha sinsi y\u0131k\u0131c\u0131 botlar bunlar\u0131 tamamen g\u00f6rmezden gelmeyi se\u00e7ebilir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bu bilgi art\u0131k kemerinizin alt\u0131nda g\u00fcvenli bir \u015fekilde s\u0131k\u0131\u015fm\u0131\u015fken, bak\u0131m\u0131n gerekli oldu\u011funu unutmay\u0131n - periyodik izleme, etkinli\u011fin devam etmesini sa\u011flar, bu nedenle d\u00fczenli denetimler i\u00e7in zaman ay\u0131r\u0131n. Mutlu kodlamalar!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-blocking-specific-bots-and-files-folders\"><span class=\"ez-toc-section\" id=\"blocking-specific-bots-and-filesfolders\"><\/span>Belirli Botlar\u0131 ve Dosyalar\u0131\/Klas\u00f6rleri Engelleme<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Robot txt botlar\u0131 nas\u0131l engellenir konusunu incelerken, bu g\u00f6revin her zaman t\u00fcm taray\u0131c\u0131lar\u0131 k\u0131s\u0131tlamakla ilgili olmad\u0131\u011f\u0131n\u0131 anlamak \u00f6nemlidir. \u00c7o\u011fu zaman, yaln\u0131zca belirli istenmeyen botlar\u0131 belirtmek veya yaln\u0131zca belirli dosya ve dizinlere eri\u015fimi k\u0131s\u0131tlamak isteyebilirsiniz. Bu incelikli senaryolarda, robots.txt dosyan\u0131z\u0131 kullanma konusundaki kavray\u0131\u015f\u0131n\u0131z\u0131 art\u0131rmak b\u00fcy\u00fck fark yaratabilir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u00c7e\u015fitlilik i\u00e7inde birlik, \u00e7e\u015fitli \u00e7evrimi\u00e7i hizmetler taraf\u0131ndan kullan\u0131lan yayg\u0131n bir taktiktir. Farkl\u0131 t\u00fcrde web taray\u0131c\u0131lar\u0131 etrafta dola\u015f\u0131yor <a href=\"https:\/\/seo.london\/tr\/bilisim-sirketleri-icin-seo\/\">internet<\/a> farkl\u0131 davran\u0131\u015flara ve yeteneklere sahiptir. Googlebot gibi baz\u0131 \u00f6r\u00fcmcekler i\u00e7eri\u011fi indekslemek i\u00e7in hayati \u00f6nem ta\u015f\u0131rken, spam botlar\u0131 gibi di\u011ferleri sitenizin performans\u0131na zarar verebilir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bu daha az yap\u0131c\u0131 botlar iki \u015fekilde engellenebilir: dar veya geni\u015f kapsaml\u0131. Dar yakla\u015f\u0131m belirli bir botun t\u00fcm web sitesinden engellenmesi anlam\u0131na gelirken, geni\u015f yakla\u015f\u0131m her botun belirli bir klas\u00f6r veya dosyadan engellenmesini i\u00e7erir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Devam etmeden \u00f6nce, robots.txt dosyan\u0131zda bir kullan\u0131c\u0131 arac\u0131s\u0131n\u0131 (yani bir botu) nas\u0131l belirleyebilece\u011finizi anlayal\u0131m. Bu belgedeki her kural 'Kullan\u0131c\u0131 arac\u0131s\u0131' belirtilerek ba\u015flamal\u0131, ard\u0131ndan iki nokta \u00fcst \u00fcste (:) i\u015fareti gelmeli ve ard\u0131ndan arac\u0131n\u0131n ad\u0131 belirtilmelidir. Bunu y\u0131ld\u0131z i\u015fareti (*) olarak b\u0131rakmak, sayfay\u0131 ziyaret eden herhangi bir bot anlam\u0131na gelir. Bunun yerine, belirli botlar i\u00e7in belirli isimler yaz\u0131lmas\u0131 tercih edilebilir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Ard\u0131ndan, web sitenizin belirli alanlar\u0131yla ilgili olarak tan\u0131mlanm\u0131\u015f kullan\u0131c\u0131-ajanlar i\u00e7in izin verilen eylemleri bildiren \"\u0130zin Verme\" veya \"\u0130zin Ver\" y\u00f6nergeleri gelir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Unutmay\u0131n, \u00f6nemli olan sadece txt robotlar\u0131n\u0131n nas\u0131l engellenece\u011fini bilmek de\u011fil, ayn\u0131 zamanda neden engellenece\u011fini bilmektir - hem kaynak israf\u0131n\u0131n \u00f6nlenmesine hem de tehlikeye at\u0131lm\u0131\u015f arac\u0131lar\u0131n k\u00f6t\u00fc niyetli faaliyetlerine kar\u015f\u0131 korunmaya odaklanmak.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Engelleme \u00f6zellikleriyle ilgili s\u00f6ylemimizi tamamlarken, bu kurallara uyma konusunda g\u00fcvenilirli\u011fin \u00f6nemli bir rol oynad\u0131\u011f\u0131n\u0131 unutmay\u0131n - ana ak\u0131m arama motorlar\u0131 genellikle s\u0131k\u0131 bir \u015fekilde uyarlar; ne yaz\u0131k ki daha az bilinen kaz\u0131y\u0131c\u0131 botlar nadiren d\u00fczg\u00fcn bir \u015fekilde takip ederler. Hassas verilerin g\u00fcvenli\u011fini sa\u011flamaya \u00e7al\u0131\u015f\u0131yorsan\u0131z yaln\u0131zca robots.txt'ye g\u00fcvenmeyin!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-robots-txt-vs-meta-robots-vs-x-robots\"><span class=\"ez-toc-section\" id=\"robotstxt-vs-meta-robots-vs-x-robots\"><\/span>Robots.txt vs Meta Robotlar vs X-Robotlar<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Botlar\u0131 robots txt ile nas\u0131l engelleyece\u011finizi bilmek \u00e7ok \u00f6nemlidir, ancak web sitenizdeki bot davran\u0131\u015f\u0131n\u0131 kontrol etmek i\u00e7in tek y\u00f6ntem bu de\u011fildir. Ayr\u0131ca \u015funlar da vard\u0131r <a href=\"https:\/\/seo.london\/tr\/iletisim-sayfasinda-meta-aciklama-baslik-etiketi-h1\/\">meta<\/a> robots ve x-robots etiketleri, \u00e7evrimi\u00e7i botlara siteniz hakk\u0131nda talimat vermenin di\u011fer iki etkili yoludur. Hangisini kullanaca\u011f\u0131n\u0131z\u0131 veya her birini di\u011ferlerinden neyin ay\u0131rd\u0131\u011f\u0131n\u0131 merak ediyorsan\u0131z, a\u00e7\u0131klamama izin verin.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-the-robots-txt-file\"><span class=\"ez-toc-section\" id=\"the-robotstxt-file\"><\/span>Robots.txt Dosyas\u0131<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Daha \u00f6nce de bahsetti\u011fimiz gibi, robots.txt dosyas\u0131, arama motorlar\u0131n\u0131 bir web sitesinin belirli b\u00f6l\u00fcmlerine y\u00f6nlendirmek veya bu b\u00f6l\u00fcmlerden uzakla\u015ft\u0131rmak i\u00e7in web y\u00f6neticisinin birincil k\u0131lavuzu olarak i\u015flev g\u00f6r\u00fcr. Bu k\u00fc\u00e7\u00fck metin dosyas\u0131 k\u00f6k dizin seviyesinde yer al\u0131r ve \u00f6zel bir durum belirtilmedi\u011fi s\u00fcrece genellikle t\u00fcm kullan\u0131c\u0131 arac\u0131 botlar\u0131 i\u00e7in genel direktifler sa\u011flar.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Esasen, robots.txt dosyas\u0131 botlara \u015f\u00f6yle der: \"Bu alanlar yasak b\u00f6lgedir.\" Ancak, t\u00fcm \u00f6r\u00fcmceklerin bu kurallara uymayaca\u011f\u0131n\u0131 unutmay\u0131n.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-what-are-meta-robots-tags\"><span class=\"ez-toc-section\" id=\"what-are-meta-robots-tags\"><\/span>Meta Robots Etiketleri Nedir?<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Meta Robots Etiketleri, a\u015fa\u011f\u0131dakilere k\u0131yasla daha ayr\u0131nt\u0131l\u0131 kontrol sunar <a href=\"https:\/\/seo.london\/tr\/kuresel-seo\/\">geni\u015f<\/a> robots.txt dosyas\u0131 taraf\u0131ndan sa\u011flanan y\u00f6nergeler. Bu HTML nitelikleri, arama motoru botlar\u0131na t\u00fcm dizinler veya siteler yerine tek tek sayfalar\u0131 dizine ekleme talimat\u0131 verir. Arama motorlar\u0131na bir sayfay\u0131 indeksleyip indekslemeyeceklerini (\"noindex\"), ba\u011flant\u0131lar\u0131n\u0131 takip edip etmeyeceklerini (\"nofollow\"), \"none\" (noindex ve <a href=\"https:\/\/seo.london\/tr\/ic-baglantilar-icin-nofollow-ozniteligini-kullanmali-miyim\/\">nofollow<\/a>) di\u011fer komutlar aras\u0131nda. Meta robot etiketleri, arama motoru taray\u0131c\u0131lar\u0131 ile sayfa baz\u0131nda do\u011frudan ileti\u015fim kurarak taray\u0131c\u0131 davran\u0131\u015f\u0131n\u0131 y\u00f6netmede ger\u00e7ek bir \u00e7ok y\u00f6nl\u00fcl\u00fck sunar.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-how-do-x-robots-tags-work\"><span class=\"ez-toc-section\" id=\"how-do-x-robots-tags-work\"><\/span>X-Robots Etiketleri Nas\u0131l \u00c7al\u0131\u015f\u0131r?<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">X-Robots etiketleri, sayfa d\u00fczeyinde ayr\u0131nt\u0131l\u0131 talimatlar sa\u011flad\u0131klar\u0131 i\u00e7in meta robots etiketleriyle baz\u0131 benzerlikler payla\u015f\u0131r. Ancak, HTML belgeleri i\u00e7inde g\u00f6r\u00fcnen benzerlerinden farkl\u0131 olarak, x-robots etiketleri <a href=\"https:\/\/seo.london\/tr\/karma-i%cc%87ceri%cc%87k-nedi%cc%87r\/\">HTTP<\/a> ba\u015fl\u0131klar.  \u00d6zellikle, bu yerle\u015ftirme PDF'ler veya resimler gibi HTML olmayan dosyalar i\u00e7in bile \u00e7al\u0131\u015fabilmelerini sa\u011flar. Meta robot etiketleri gibi, x-robot etiketi eylemleri de \"noindex\", \"nofollow\" ve hatta \"nosnippet\" aras\u0131nda de\u011fi\u015fmektedir.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bu nedenle, robots txt kullanarak botlar\u0131n nas\u0131l engellenece\u011fini \u00f6\u011frenmek her web y\u00f6neticisi i\u00e7in ger\u00e7ekten de\u011ferli bir bilgi olsa da, meta robotlar\u0131n ve x-robotlar\u0131n g\u00fc\u00e7l\u00fc y\u00f6nlerini ve uygulamalar\u0131n\u0131 anlamak, sitenizin web taray\u0131c\u0131lar\u0131yla ili\u015fkisini d\u00fczenlerken daha da geni\u015f bir ara\u00e7 seti sa\u011flar.<\/p>","protected":false},"excerpt":{"rendered":"<p>Bilgisiz bir g\u00f6zlemci i\u00e7in web sitenizde dola\u015fan bir robot bilimkurgu filmlerinden f\u0131rlam\u0131\u015f gibi g\u00f6r\u00fcnebilir. \u0130ster inan\u0131n ister inanmay\u0131n, bu kurgudan \u00e7ok uzak ve ger\u00e7e\u011fe d\u00fc\u015f\u00fcnd\u00fc\u011f\u00fcn\u00fczden daha yak\u0131n! Web sitelerine sahip olma ve bak\u0131m\u0131n\u0131 yapma konusunda yol alan herkes i\u00e7in botlar\u0131n \u00e7evrimi\u00e7i alanlar\u0131m\u0131zla nas\u0131l etkile\u015fime girdi\u011fini anlamak \u00e7ok \u00f6nemlidir. [...]<\/p>","protected":false},"author":6,"featured_media":31181,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[14],"tags":[],"class_list":["post-31157","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-seo"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v20.9 (Yoast SEO v26.4) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>How to Block Bots with Robots.txt? - SEO.London<\/title>\n<meta name=\"description\" content=\"A robots.txt file is essentially the gatekeeper of your website. It allows you to control which parts of your site are available for bot exploration\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/\" \/>\n<meta property=\"og:locale\" content=\"tr_TR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to Block Bots with Robots.txt?\" \/>\n<meta property=\"og:description\" content=\"For an uninformed observer, a robot wandering around your website might seem like something out of a sci-fi movie. Believe it or not, it&#039;s far from\" \/>\n<meta property=\"og:url\" content=\"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/\" \/>\n<meta property=\"og:site_name\" content=\"SEO.London\" \/>\n<meta property=\"article:published_time\" content=\"2023-06-12T12:01:52+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-29T16:47:23+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/seo.london\/wp-content\/uploads\/2023\/06\/How-to-Block-Bots-with-Robots.png\" \/>\n\t<meta property=\"og:image:width\" content=\"800\" \/>\n\t<meta property=\"og:image:height\" content=\"449\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Lukasz Zelezny\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Yazan:\" \/>\n\t<meta name=\"twitter:data1\" content=\"Lukasz Zelezny\" \/>\n\t<meta name=\"twitter:label2\" content=\"Tahmini okuma s\u00fcresi\" \/>\n\t<meta name=\"twitter:data2\" content=\"11 dakika\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Robots.txt ile Botlar Nas\u0131l Engellenir? - SEO.London","description":"Robots.txt dosyas\u0131 esasen web sitenizin bek\u00e7isidir. Sitenizin hangi b\u00f6l\u00fcmlerinin botlar\u0131n ke\u015ffine a\u00e7\u0131k oldu\u011funu kontrol etmenizi sa\u011flar","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/","og_locale":"tr_TR","og_type":"article","og_title":"How to Block Bots with Robots.txt?","og_description":"For an uninformed observer, a robot wandering around your website might seem like something out of a sci-fi movie. Believe it or not, it's far from","og_url":"https:\/\/seo.london\/tr\/robots-txt-ile-botlar-nasil-engellenir\/","og_site_name":"SEO.London","article_published_time":"2023-06-12T12:01:52+00:00","article_modified_time":"2023-06-29T16:47:23+00:00","og_image":[{"width":800,"height":449,"url":"https:\/\/seo.london\/wp-content\/uploads\/2023\/06\/How-to-Block-Bots-with-Robots.png","type":"image\/png"}],"author":"Lukasz Zelezny","twitter_card":"summary_large_image","twitter_misc":{"Yazan:":"Lukasz Zelezny","Tahmini okuma s\u00fcresi":"11 dakika"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/seo.london\/fi\/how-to-block-bots-with-robots-txt\/","url":"https:\/\/seo.london\/how-to-block-bots-with-robots-txt\/","name":"Robots.txt ile Botlar Nas\u0131l Engellenir? - SEO.London","isPartOf":{"@id":"https:\/\/seo.london\/fi\/#website"},"primaryImageOfPage":{"@id":"https:\/\/seo.london\/how-to-block-bots-with-robots-txt\/#primaryimage"},"image":{"@id":"https:\/\/seo.london\/how-to-block-bots-with-robots-txt\/#primaryimage"},"thumbnailUrl":"https:\/\/seo.london\/wp-content\/uploads\/2023\/06\/How-to-Block-Bots-with-Robots.png","datePublished":"2023-06-12T12:01:52+00:00","dateModified":"2023-06-29T16:47:23+00:00","author":{"@id":"https:\/\/seo.london\/fi\/#\/schema\/person\/e2143945a84522958a4a45d382439e90"},"description":"Robots.txt dosyas\u0131 esasen web sitenizin bek\u00e7isidir. Sitenizin hangi b\u00f6l\u00fcmlerinin botlar\u0131n ke\u015ffine a\u00e7\u0131k oldu\u011funu kontrol etmenizi sa\u011flar","breadcrumb":{"@id":"https:\/\/seo.london\/how-to-block-bots-with-robots-txt\/#breadcrumb"},"inLanguage":"tr","potentialAction":[{"@type":"ReadAction","target":["https:\/\/seo.london\/how-to-block-bots-with-robots-txt\/"]}]},{"@type":"ImageObject","inLanguage":"tr","@id":"https:\/\/seo.london\/how-to-block-bots-with-robots-txt\/#primaryimage","url":"https:\/\/seo.london\/wp-content\/uploads\/2023\/06\/How-to-Block-Bots-with-Robots.png","contentUrl":"https:\/\/seo.london\/wp-content\/uploads\/2023\/06\/How-to-Block-Bots-with-Robots.png","width":800,"height":449,"caption":"How to Block Bots with Robots.txt"},{"@type":"BreadcrumbList","@id":"https:\/\/seo.london\/how-to-block-bots-with-robots-txt\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/seo.london\/fr\/"},{"@type":"ListItem","position":2,"name":"How to Block Bots with Robots.txt?"}]},{"@type":"WebSite","@id":"https:\/\/seo.london\/fi\/#website","url":"https:\/\/seo.london\/fi\/","name":"SEO.London","description":"Londra, \u0130ngiltere'de SEO Dan\u0131\u015fman\u0131 ve Uzman \u015eirketi - Lukasz Zelezny","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/seo.london\/fi\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"tr"},{"@type":"Person","@id":"https:\/\/seo.london\/fi\/#\/schema\/person\/e2143945a84522958a4a45d382439e90","name":"Lukasz Zelezny","image":{"@type":"ImageObject","inLanguage":"tr","@id":"https:\/\/seo.london\/fi\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/55d18e132c3d0d63f7cd047da2d2a7f805a3392a84a615be5d6dca867281758e?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/55d18e132c3d0d63f7cd047da2d2a7f805a3392a84a615be5d6dca867281758e?s=96&d=mm&r=g","caption":"Lukasz Zelezny"},"description":"#1 Londra'da ya\u015fayan SEO Dan\u0131\u015fman\u0131, Zoopla, uSwitch, Mashable, Thomson Reuters ve di\u011ferleri gibi \u015firketlerle \u00e7al\u0131\u015f\u0131yordu. Lukasz Zelezny'i \u0130\u015fe Al\u0131n (MCIM, F IDM)","url":"https:\/\/seo.london\/tr\/author\/lukasz-zelezny\/"}]}},"_links":{"self":[{"href":"https:\/\/seo.london\/tr\/wp-json\/wp\/v2\/posts\/31157","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/seo.london\/tr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/seo.london\/tr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/seo.london\/tr\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/seo.london\/tr\/wp-json\/wp\/v2\/comments?post=31157"}],"version-history":[{"count":0,"href":"https:\/\/seo.london\/tr\/wp-json\/wp\/v2\/posts\/31157\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/seo.london\/tr\/wp-json\/wp\/v2\/media\/31181"}],"wp:attachment":[{"href":"https:\/\/seo.london\/tr\/wp-json\/wp\/v2\/media?parent=31157"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/seo.london\/tr\/wp-json\/wp\/v2\/categories?post=31157"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/seo.london\/tr\/wp-json\/wp\/v2\/tags?post=31157"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}