Does the robots exclusion standard have any legal weight? There is a standard through which websites communicate to the web crawlers upon which search engines are based which pages should be indexed and included in the search results. This is called the Robots exclusion standard, is implemented in the robots.txt file that most web sites have, stack exchange's is here. If one was interested in the whole content of a site, for example one was researching the network structure of web sites, would it be legal to ignore the robots.txt and crawl the whole site (perhaps excluding /search[?/])?
The existence of a robots.txt file and the directory and file inclusions/exclusions in a robots.txt file do not constitute a legally binding contract for the use of the website by the visitor; if it exists, the Terms of Service would usually establish the contract for use of the site. Many TOSs prohibit bots, crawlers and any automated means from crawling or scraping a site, and you are contractually obligated to honor those stipulations simply by visiting the site. Violating a TOS is typically a civil matter, depending on jurisdiction. Simply crawling a site to &quot;map&quot; it, and crawling a site to &quot;scrape&quot; it and copy all content are two different things. Crawling could violate the TOS. Scraping content can violate the TOS and also be copyright infringement, depending on jurisdiction. See https://law.stackexchange.com/search?q=scraping I suppose a TOS could state that users are legally bound to the restrictions in a robots.txt, but I've never seen that in a TOS.
Patents are not that relevant in this case. Software patents are unenforceable in most parts of the world anyway. What matters here is copyright. Every work contract has a clause that everything an employee creates as part of their employment is copyrighted by the company. So using company-owned code to build an own project would be a copyright violation. There are also other legal tools in some jurisdiction which can be used against employees trying to misuse intellectual company property. But that's a topic for Law Stackexchange. Also, this isn't really related to a BYOD policy. Being able to bring your own device to work and then back home might make data theft more convenient, but isn't required. There are many other ways to steal sourcecode, like USB drives or uploading them to the internet. To prevent the first you would have to design your software development offices like a supermax prison facility with meter-high walls (so nobody can throw a device over it) and strip searches on everyone leaving the building. This is neither feasible nor reasonable for anything below matters of national security. To prevent the second, you would have to completely prevent internet access from developer workstations, which would greatly impede the productivity of any software developer. So most companies do not even try to physically prevent employees from stealing sourcecode. They rather rely on the legal safeguards and on maintaining a mutual trust relationship with their employees. It might seem counter-intuitive to some, but when you do not treat your employees like potential criminals they are in fact less likely to betray you.
First of all, Google's TOS says we reasonably believe that your conduct causes harm or liability to a user, third party, or Google — for example, by hacking, phishing, harassing, spamming, misleading others, or scraping content that doesn’t belong to you And you ask: But suppose a researcher, say based in UK, managed to work around them, get a big amount of data from Google searches and use it to publish some research. You're confusing methods with results. Someone uses methods to get a result, but if the results are not legal, the methods are usually illegal, too. Would the above depend on how they circumvented Google's checks (i.e., by using lots of proxies)? The words &quot;or scraping content that doesn’t belong to you&quot; means just that; it does not give any wiggle room for the actual method used to scrape. Could they get, theoretically, into trouble? Very much so. The researcher would have at least civil liability, and possibly criminal exposure. Would they, practically? Google can be very not kind to people who break their TOS. And Google has lots of money to spend on lawyers and court fees to enforce their TOS. Google would probably be able to easily prove the data came from their servers, and would probably have server logs to help prove it. And see user6726's answer for specific legal citations in the US and UK.
This seems to be a mix of question about law and a meta-question about this site, but I'll treat it as an on-topic question about law. The author of a question, or answer, owns the copyright to their contributions, and they can re-publish to their heart's content. Any user who posts here grants a license to SE and other users to use content posted here, so I don't have to ask you permission to quote you. As part of the permission granted by SE to use this website, you have agreed to &quot;follow the rules&quot; set by SE. There are many rules, some spelled out more clearly that others. For example, if you post a question, you indirectly agreed that your content can be upvoted or down-voted. Certain content can be &quot;closed&quot; and deleted, when the content is deemed to violate the rules in particular ways (is spam, porn, abuse, or judged to be poor-quality). Judgment (on different matters) can be rendered by community managers, moderators, or other users. The agreement is here, see especially here. If we take the post that you linked to, it is quite possible that it was deleted because it is not a general legal question, in violation of the acceptable use policy. If you want a historical analysis of your particular case, it should be asked on Law Meta.
Please see this related answer where this is discussed, although is a slightly different context. There have not been many cases on this subject, but it appears that, at least in the US and EU, a person is free to post web pages that link to others without permission and indeed against the wishes of the operator of the destination site. However, &quot;deep linking&quot; that bypasses a login or other access control page, or an advertising page that visitors who enter via a home page would be expected to load, particularly if such deep linking deprives the destination site of revenue, may constitute copyright infringement. &quot;framing&quot; a linked page so that it appears to be part of the linking site, may also be copyright infringement, and might also be actionable under trademark law. See Nolo's page on Linking, Framing, and Inlining And the Wikipedia article on Deep linking In Intellectual Reserve, Inc. v. Utah Lighthouse Ministry, Inc, 75 F. Supp. 2d 1290 (D. Utah 1999) deep linking was held to be contributory copyright infringement. See The Wikipedia article on the case In that case, the content being linked to had been posted without the authorization of the copyright holder, and no fair use issue was raised by the defense. In general, courts have found that publishing a page on the web invites others to visit it and link to it. In the Wikipedia article on &quot;Deep linking&quot; (linked above) it is said that: In a February 2006 ruling, the Danish Maritime and Commercial Court (Copenhagen) found systematic crawling, indexing and deep linking by portal site ofir.dk of real estate site Home.dk not to conflict with Danish law or the database directive of the European Union. The Court stated that search engines are desirable for the functioning of the Internet, and that, when publishing information on the Internet, one must assume—and accept—that search engines deep-link to individual pages of one's website. In Perfect 10, Inc. v. Amazon.com, Inc. 508 F.3d 1146 (9th Cir. 2007) a US court held that links to copyrighted images as part of an image search were not copyright infringement. The Ninth Circuit court of Appeals held that Google's display and caching of thumbnails was fair use, mainly because they were &quot;highly transformative.&quot; However, a ToS or ELUA may be a legally binding contract between a site operator and site users. If it forbids linking, that may be enforceable, depending on the laws of the country where suit is brought. Owners of publicly accessible databases are permitted, under EU law, to impose and enforce access restrictions. See the case of Ryanair vs PR Aviation brought in the European Court of Justice.
Yes, such a site can be created without infringing copyright Facts about the game are facts.They are not protected by copyright. Criticism of, and comment about the game, is an activity protected by the US First Amendment. Making such comments is very likely to be fair use under US copyright law. In general the author of a work, such as a book or a game, or the maker of a product, has no right to grant or withhold permission to discuss or comment on the work. This is true not only under US law, but also in the law of most countries (perhaps of all countries). The name of the game might well be protected as a trademark. But that does not allow the trademark owner to prevent discussion of the game, clearly identified by the name of of the game. As long as nothing is being sold or rented, or advertised for sale or rental under that name, and there is no attempt to claim that the site is sponsored or approved by the trademark owner, and there is no likelihood of confusion, there is no trademark infringement. This is true under US law, and under the trademark laws of most other countries. A wiki is a specific technology. It can be used for community discussion, or for a company's internal documentation, or for any of many other purposes. Wikipedia has popularized this technology. Not all community discussion sites are wikis, however, nor are all wikis for community discussion. Just as not all novels are books printed on paper, and not all books are novels. In any case, setting up a wiki about a topic such as a game, a movie, or a novel does not require permission from the owner or creator of the game or of any trademarks associated with the game or work. The same would be true for a discussion forum about such a game or work that is not a wiki. If a wiki uses excessive quotes from game dialog, or uses the game's logo without permission, or reproduces other game assets, such as character art, maps, and the like without permission, that might be copyright infringement.
"Public domain" refers to things in principle copyrightable but where protection has lapsed, been repudiated, or is a statutory exception (such as government works). A website is not "in the public domain". The idea that a website is "public property" is (*cough*) mistaken. There are basically two ways in which a web interaction could be illegal. The first regards whether accessing another person's computer is illegally accessing a computer, which is a crime. Authorization essentially comes down to "permission": if the owner permits me to access the computer, I am authorized. Putting stuff out there on a web server is an open-ended grant of permission to look at a web page. That simply means that if I create a web page (with a bunch of links or not), I am granting you permission to interact with my computer to that extent. It does not create permission to hack into a password-protected subdirectory. An ordinary web crawler automates what a clicking human does. Copyright law is also relevant, in that the stuff I put on my webpage is not to be copied without permission. Any webpage access necessarily involves automatic copying from machine to machine: in putting stuff out there for the world to see, I am saying that the world can do that level of automatic copying that arises from normal html-and-click interactions. It does not mean that you can download and do stuff with my copyrighted content (i.e., it is not an abandonment of copyright: I did not put that stuff in the public domain). Putting a web page out there in an unrestricted fashion means that you've given a certain level of permission to "copy" (at least in the automatic server-to-browser viewing sense). I may want to impose conditions on peoples' access to my stuff, so I can impose terms on such material. For instance, I may require users to agree to certain conditions before accessing the CoolStuff subdirectory. Users then have to jump through a minor hoop and agree to those terms. In that case, my permission is conditional, and if you violate the terms of that agreement, I may be able to sue you for copyright infringement. It could then be a violation of my terms of service (TOS) if I say "you may not crawl my website" (in less vague language). A TOS gets its legal power from copyright law, because every webpage interaction involves copying (I assume that technical point is obvious), and copying can only be done with permission. You may technologically overcome my weak click-through technology so that the bot just says "sure whatever" and proceeds to illegally use my web page: I can sue you now for copyright infringement. The robot-specific methods of meta-tags and robots.txt have no legal force. Although there is a way to say "no you may not," which is tailored to automated access, the meaning and enforcement of these devices has not yet reached the law. If my page uses NOFOLLOW and your program doesn't know or care, you (your program) do not (yet) have a duty to understand, detect and respect that tag. Prior registration is also not a legal requirement, and very many pages that are on the master crawl list get there from being linked to by someone else's web page. Again, there is at present no legal requirement of pre-registration (and there is no effective mechanism for verifying that the site owner has registered the site). Archiving and especially re-displaying someone's content is, on the other hand, not legal. It would be plainly copyright infringement if you were to scoop up someone else's webpage and host it. You can analyze their material and somehow associate it with some search terms, and display a link to that page, but you cannot copy and republish their material. You can put very short snippets out there taken from a web page, under the "fair use" doctrine, but you can't wholesale republish a webpage. (It should be noted that the archive.org is an internationally recognized library, and libraries have extra statutory powers to archive).
It is not different. But one company can decide to approach compliance differently from another. Here, TomTom has chosen a fairly safe/conservative interpretation, whereas Google and Apple decided that more data collection is appropriate. A “find my device” style functionality appears to be entirely unproblematic if the user books that particular service (regardless of whether the service is paid or gratis) and provides consent for the location data collection. The legal basis for such a service could then be Art 6(1)(a) consent or Art 6(1)(b) necessity for performance of a contract to which the data subject is party. Personally, I believe that Google is not sufficiently transparent about how Find My Device works, but that TomTom could provide a compliant service if they wanted to. That TomTom doesn't offer this service primarily shows that they don't think developing this service is worth it. Wiping a remote device is unrelated to this issue and doesn't seem to provide GDPR challenges. At least in a business context, remote wiping may be an appropriate security measure and may then even be mandated by the GDPR (e.g. see Art 24 or Art 32).