Do search engines need permission to start searching? I was talking to my friends a few days ago about search engines and we started discussing whether or not they need permission from websites to crawl them. I looked this one up and it said on Quora that you don't really need permission to crawl them since they are present in public domain and are public property. If the websites want privacy, they can change their settings appropriately. However, later on, I talked to a few other people who said that search engines like Google and Bing don't just crawl all websites. They only crawl websites which are registered on their SEO or are on their radar and ranking–I didn't really understand this part very much–but that doesn't make much sense either since they'd still need to crawl other websites to be updated. My question is: If I had a search engine that worked roughly like Google, Bing, etc., do could I just start crawling and displaying results, or would I need get special permission from the trillions of websites out there before I can actually run my search engine?
"Public domain" refers to things in principle copyrightable but where protection has lapsed, been repudiated, or is a statutory exception (such as government works). A website is not "in the public domain". The idea that a website is "public property" is (*cough*) mistaken. There are basically two ways in which a web interaction could be illegal. The first regards whether accessing another person's computer is illegally accessing a computer, which is a crime. Authorization essentially comes down to "permission": if the owner permits me to access the computer, I am authorized. Putting stuff out there on a web server is an open-ended grant of permission to look at a web page. That simply means that if I create a web page (with a bunch of links or not), I am granting you permission to interact with my computer to that extent. It does not create permission to hack into a password-protected subdirectory. An ordinary web crawler automates what a clicking human does. Copyright law is also relevant, in that the stuff I put on my webpage is not to be copied without permission. Any webpage access necessarily involves automatic copying from machine to machine: in putting stuff out there for the world to see, I am saying that the world can do that level of automatic copying that arises from normal html-and-click interactions. It does not mean that you can download and do stuff with my copyrighted content (i.e., it is not an abandonment of copyright: I did not put that stuff in the public domain). Putting a web page out there in an unrestricted fashion means that you've given a certain level of permission to "copy" (at least in the automatic server-to-browser viewing sense). I may want to impose conditions on peoples' access to my stuff, so I can impose terms on such material. For instance, I may require users to agree to certain conditions before accessing the CoolStuff subdirectory. Users then have to jump through a minor hoop and agree to those terms. In that case, my permission is conditional, and if you violate the terms of that agreement, I may be able to sue you for copyright infringement. It could then be a violation of my terms of service (TOS) if I say "you may not crawl my website" (in less vague language). A TOS gets its legal power from copyright law, because every webpage interaction involves copying (I assume that technical point is obvious), and copying can only be done with permission. You may technologically overcome my weak click-through technology so that the bot just says "sure whatever" and proceeds to illegally use my web page: I can sue you now for copyright infringement. The robot-specific methods of meta-tags and robots.txt have no legal force. Although there is a way to say "no you may not," which is tailored to automated access, the meaning and enforcement of these devices has not yet reached the law. If my page uses NOFOLLOW and your program doesn't know or care, you (your program) do not (yet) have a duty to understand, detect and respect that tag. Prior registration is also not a legal requirement, and very many pages that are on the master crawl list get there from being linked to by someone else's web page. Again, there is at present no legal requirement of pre-registration (and there is no effective mechanism for verifying that the site owner has registered the site). Archiving and especially re-displaying someone's content is, on the other hand, not legal. It would be plainly copyright infringement if you were to scoop up someone else's webpage and host it. You can analyze their material and somehow associate it with some search terms, and display a link to that page, but you cannot copy and republish their material. You can put very short snippets out there taken from a web page, under the "fair use" doctrine, but you can't wholesale republish a webpage. (It should be noted that the archive.org is an internationally recognized library, and libraries have extra statutory powers to archive).
a few quick notes that come to mind. As the commenters point out: DSGVO is indeed the German equivalent name to the GDPR (English term) "Imprint" isn't a privacy related topic that much, as such it's not really changed by the upcoming GDPR The GDPR changes many things, but the requirement for up front information isn't one of them - so it would've already been a rule to follow Sometimes the question who is responsible for privacy information might not be that easy to decide when you're on platforms. I take this situation as being pretty clear though. You are basically given a blank slate, you can do with that page whatever you want, and the visitor has no clue whatsoever that it might be hosted by Github. In addition to this, Github would be classified as a data processor (providing the tool) and you as the data controller who is in charge of practically everything except for the provision of the page. I hope this helps. Btw. it's not that hard anymore to write a privacy policy these days.
Simply use a cookie to store consent. First consider the opposite. If a user does not agree to store cookies, a cookie is the only way to remember this, as you want to avoid a new pop-up on every page-load. Because this use of a cookie is functional, you don't need permission to store that cookie. You seem have the impression that you have to prove towards the ICO if someone has provided consent. However I think it would be sufficient if you can demonstrate how your website works technically. In particular whether consent is handled properly. You might need to create screenshots or a screencast to do so. Add new proof after each major update of your website. Storing consent server-side would violate the data minimization principle of the GDPR I think. In particular because you would need to do something to be able to identify users. That would violate Art. 11(1) GDPR: If the purposes for which a controller processes personal data do not or do no longer require the identification of a data subject by the controller, the controller shall not be obliged to maintain, acquire or process additional information in order to identify the data subject for the sole purpose of complying with this Regulation. Notice it is possible to configure google analytics in a way so you don't need any consent. See my answer to this question. Just IP anonymization is not sufficient.
The notice has a lot to do with legacy requirements in the United States to claim the copyright to a work. Up until 1989, the copyright notice was required. Today, the statements are mainly maintained to protect against "innocent infringement" which might reduce what a content owner can get in court. What exactly do those terms entail? That the owner stated owns all rights and you may do nothing with the content. My biggest concern is this: by writing that, is the company claiming to own everything on the website, even potentially copyrighted user-submitted material? That's exactly what they are doing. Depending on the terms of the specific site, content contributors generally either assign copyright to the site owner or license the content in a way that allows the site owner to do exactly what they want with it. Site creators with the smarts or money to do it right/get someone to do it right usually state something like: Copyright [Site Owner] and contributors. Other sites (like this one) state specifically what they hold the copyright to: site design / logo © 2015 Stack Exchange Inc THIS IS NOT LEGAL ADVICE. CONSULT AN ATTORNEY REGARDING YOUR SPECIFIC SITUATION.
There is nothing in the GDPR requiring you to collect individual personally identifiable information. If the website has no need, and the website owner no desire, to collect such information, there is no requirement to do so. The GDPR requires that if such information is collected, that there is a lawful basis, and that it be handled appropriately and stored securely, and deleted when there is no longer a need to retain it, or on a proper request. If no such information is collected in the first place, all the rules about how to handle it do not apply. It is possible that some law of an individual country in the EU might mandate collection of some particular information, but I have not heard of any such requirement.
Laws against such actions are not stated in terms of popular and fluid concepts like "computer virus", they are stated in terms of clear concepts like "unauthorized access". There are federal and state laws against this. This web site lists and links to all of the state laws on the matter. There is also a federal law: a detailed legal analysis by DOJ is given here. There are some limits to federal jurisdiction, for example "protected computers" include "computers used in or affecting interstate or foreign commerce or communication". The term "affecting interstate or foreign commerce or communication" is widely used in federal law, and can be used to prohibit growing feed for your own animals. Anything that you "send" clearly affects interstate commerce (the internet is internationally connected). 18 USC 1030(a) says Whoever ... (2) intentionally accesses a computer without authorization or exceeds authorized access, and thereby obtains...(C) information from any protected computer Essentially, a computer connected to the outside world is protected. The key here is "without authorization". If you authorize MS to report back stuff about your computer, that is not unauthorized. It may not be possible to use their product without giving such authorization, in which case you can use a different product that doesn't require that you grant authorization. There is also the possibility that some software producer has technically violates the law because they think that it's okay for them to access the computer as long as they do no harm. Typically, people are not aware that they have granted software publishers access to their computer. The concept of "harm" is pretty much irrelevant to computer-crime criminal law. It would be relevant, though, if a plaintiff were to sue someone for sniffing around their computer: then you'd have to show that you were damaged.
You generally can request data for free, but keep in mind that the Act does not require every set of data to be published or released. For instance, it applies only to data that is subject to release under the Freedom of Information Act. This means that it has no effect on the Delaware Business Search database, because that data is not controlled by a federal agency, and therefore not subject to FOIA. There have been requests for bulk access to this data in the past, and as far as I know, they have all been rejected. Given the state's economic reliance on its very sketchy business laws, I would but expect that to change any time soon. You would suddenly have journalists and the like scouring the database making all sorts of connections and uncovering all manner of unsavory activity, and no one in a policy making role wants to have to deal with that.
As far as I can tell, whenever GitHub redistributes code, it adheres to all of the requirements of the GPL/BSD etc. For example, it never gives away code without including the license text, and it doesn't claim to add unpermitted restrictions to the code. The section you're concerned about is this: That means you're giving us the right to do things like reproduce your content (so we can do things like copy it to our database and make backups); display it (so we can do things like show it to you and other users); modify it (so our server can do things like parse it into a search index); distribute it (so we can do things like share it with other users); and perform it (in case your content is something like music or video). That section isn't worded very precisely, but I understand this to mean they want to reproduce your content in full, not just the code without the license text attached. They want to display your content in full, as you provided it. They want to modify it internally (i.e. they are not redistributing this modified form) in order to allow searching. To the extent that they display code-snippets as search results, that is likely defensible as fair use (search engines are frequently cited as examples of successful fair use defenses). GitHub appears to be acting consistently with the GPL when it reproduces, displays, modifies, and distributes code that you upload. If I'm correct about these facts then nothing that GitHub does with the code goes beyond what the many contributors to a GPL project have explicitly permitted, or what is otherwise permitted by fair use law. If you had some code with a more restrictive license, you might not be able to simultaneously comply with that restrictive license and GitHub's License Grant. Consider the scenario where you have some code that is licensed exclusively to you, and you are not given permission to reproduce or distribute it. In that scenario, the code's license prevents you from agreeing to GitHub's request for a license grant.