Hubi@feddit.de to Reddit@lemmy.worldEnglish · 6 months agoReddit shares plunge almost 25% in two days, finish the week below first day closewww.cnbc.comexternal-linkmessage-square135fedilinkarrow-up1740arrow-down113cross-posted to: modcoord@lemmit.online
arrow-up1727arrow-down1external-linkReddit shares plunge almost 25% in two days, finish the week below first day closewww.cnbc.comHubi@feddit.de to Reddit@lemmy.worldEnglish · 6 months agomessage-square135fedilinkcross-posted to: modcoord@lemmit.online
minus-squareLaterRedditor@lemmy.worldlinkfedilinkarrow-up3arrow-down2·6 months agoWasn’t there robots.txt at least?
minus-squareSacrificedBeans@lemmy.worldlinkfedilinkarrow-up9·6 months agoRobots.txt is a compliance standard afaik, more like an advisory guide. It cannot enforce bans on scrapers, it sets some polite boundaries for the website.
Wasn’t there robots.txt at least?
Robots.txt is a compliance standard afaik, more like an advisory guide. It cannot enforce bans on scrapers, it sets some polite boundaries for the website.
There was, and still is