Scraping factual data off the web and re-organize to release under proprietary license? Lots of bits and pieces of information and chaotic data spread across the web and it does cost an arm and a leg to gather them together to form something larger and consistent that would prove to be much more useful. Is it legally viable to scrape these factual data / information from different websites, compile and re-organize them as databases / data sets that will then be licensed under a proprietary license? Thus far after some research, I find factual data can't be copyrighted, but the particular compilation / structure of data can. Does this mean I can do whatever with the scraped factual data as long as I have my own unique data organization / compilation / structure? For example, I want to make a business information database off hundreds of different websites and then radically re-organize / re-structure / re-index these different data sets into one database that is better than any of the sources. Can I proprietorially license this particular database I made? I know I can never own the data as they are straight facts. Right? But can I OWN the particular data organization and legally profit from it? We are operating in US, but the data could be from websites on other continents.
From your question(s), as well as your various comments, I understand you to have two general inquiries: 1. Is there any infringement of copyright laws if you use things like the titles of books, games, apps, names, address (and any other number of things) which you will then put into datasets that will be licensed for proprietary commercial purposes? You may freely put titles, names of people, places or things into datasets without fear that you are infringing on copyright or any other laws. That is clear. Copyright law does not protect names, titles, short phrases or expressions. Even if a name, title, or short phrase is novel or distinctive it cannot be protected by copyright. So, there is no point in discussing the doctrine of fair use in this context, because Fair Use is a defense, or a legal safe harbor that is merely an exception to copyright infringement allowing people to use a copyrighted works under specific circumstances. As I understand your intended endeavor, you will not be infringing on any copyrights to the extent that you are merely using factual data, like names of copyrighted things for the purpose of creating a dataset or an application to help access it. This is why I say you need not concern yourself with the test for Fair Use with regard to this issue. The Copyright Office states clearly, despite what people may think, that there are no exclusive rights in brief combinations of words such as: • Names of products or services • Names of businesses, organizations, or groups (including the names of performing groups) • Pseudonyms of individuals (including pen or stage names) • Titles of works • Catchwords, catchphrases, mottoes, slogans, or short advertising expressions • Listings of ingredients, as in recipes, labels, or formulas. When a recipe or formula is accompanied by an explanation or directions, the text directions may be copyrightable, but the recipe or formula itself remains uncopyrightable. Hence, these things are not registrable under a copyright. While something may be potentially attached to or included in copyrighted material, is not in and of itself subject to the protections of these laws. If it (whatever it is) cannot be registered for a copyright, it is not copyrightable. Because copyright registration/notices have been optional since 1989, when the U.S. attached itself to the Berne Convention, whereby copyright protection is automatic as soon as a work is “fixed in a tangible medium of expression” (written down, recorded, painted, etc.) it’s protected. No notice is required. Registration only becomes required for litigation or enforcement purposes. But this is really extraneous to your inquiry anyway, as far as it applies to the actual data. When you get into copying whole databases for your purpose, that analysis is different. 2. You want to "scrub" the internet for information that you intend to put into your proprietary datasets and use for commercial purposes, some or most of which is already in a database or some organized form, and you want to know if there is some sort of copyright or duty owned to the person who originally databased the materials? Since ideas, procedures, principles, discoveries, and devices are all specifically excluded from copyright protection, if you want to compile this type of information from the internet for the purpose of creating datasets, or searchable databases, this is permissible. That said, there are protections for existing databases under copyright law, provided under the concept of a "compilation copyright". A compilation copyright protects the collection and creative assembling of data or other materials. Compilation copyrights protect the collection and assembling of data or other materials, such that databases are generally protected by copyright law as compilations. Under the Copyright Act, a compilation is defined as a "collection and assembling of preexisting materials or of data that are selected in such a way that the resulting work as a whole constitutes an original work of authorship." 17. U.S.C. § 101. The preexisting materials or data may be protected by copyright since the selections of materials and the form they take in an existing database may be original enough to be subject to a copyright. However, the data itself is merely information and is not protectable. The Copyright Act specifically states that the copyright in a compilation extends only to the compilation itself, and not to the underlying materials or data. 17 U.S.C. § 103(b). As a result, "compilation copyrights" can't be used to place protection upon those things that are otherwise not protectable. In the case of Feist Publications, Inc. v. Rural Telephone Service Company, Inc., the U.S. Supreme Court ruled that a compilation work such as a database must contain a minimum level of creativity in order to be protectable under the Copyright Act. Feist makes clear that even a copyright protected database does not hold the right to prevent an individual from extracting factual data from the database (so long as you're not copying the entire database as a whole). If you take an already compiled and copyrighted dataset in its entirety, you must obtain a license for its use. However, if you are merely amassing great amounts of data to then put into your own dataset, that you are free to do. The big issue will be (and you seem to realize this) where you will amass this data from. Some websites have specific licenses in place that say you cannot use or rework their content. However, many times these websites simply throw these license requirements out there for users to see, despite the fact that they may not be (and some would argue) are not enforceable. The courts have heard arguments that "contracts" (the end-user licenses) that protect databases and information on websites is beyond the protection available through copyright law should be "preempted" by the Copyright Act itself. The preemption argument goes like this: Federal law controlling something that is subject to interstate commerce or use, should be controlled by the federal laws. So,since the federal government has enacted the Copyright Act to govern any protections to any original works, states should be (arguably are) prohibited from having contradictory laws. Because of the ability of a federal statute to preempt state law, and the fact that the Copyright Act at 17 U.S.C. § 301 sets forth specific preemptions, no state may create rights that are equivalent to any of the exclusive rights provided under the Act. It is this concept of preemption that prevents copyright protection from varying depending upon the state where a work of authorship is created. Arguably, the same is true for the internet, and supposed contractual relationship created through licenses that dictates how non-copyrightable material may be used. In the case of ProCD, Incorporated v. Matthew Zeidenberg and Silken Mountain Web Services, Inc. the court examined whether an end-user of a CD ROM phone database was subject to the license, when they extracted a large portion of the database and made it available over the Internet. The database was almost the same as the type of data in the Feist case-The lower court rejected all copyright claims and found that the shrinkwrap license that controlled the end user's right to use the data was both unenforceable (as a shrink wrap license) and preempted by the Copyright Act. As a result, there was no relief available to the creator of the phone database and the end-user was free to extract the data and use it as he saw fit. However, on appeal this decision was reversed (7th circuit). The appellate court did acknowledge that the database (on the CD) was not original enough to be protected by copyright (finding no copyright infringement by the end-user); However, they did find the end-user was breach of contract, since the shrink-wrap license prohibited the end-user's conduct. What this tells us is that these licenses (on websites) may or may not be enforceable. While the 7th Circuit found a contract right pursuant to the license, despite the preemption argument, another appellate court that is more liberal may find otherwise. Also, this was a disk, not the internet, which is the "wild west" of information, largely unregulated and unlitigated as it pertains to the legality and enforceability of (some) regulations that do exist. License agreements for site use on the internet are everywhere. If you take a database from some site that has a license saying you cannot take their work and add to it, or whatever, and you do add it to other databases that are not licensed and then make your own dataset - chances are you are NOT going to be infringing on anyone's copyright. That said, you may be in breach of contract (the license) if they find out about it, and sue you (using it doesn't put you in breach; only getting sued and having a court determine you're in breach puts you in breach. It may be a distinction worth contemplation, but that is up to you). The safest, bet would be to get a license from them to rework the materials. If the material is generic enough, and will be changed enough, that you are creating your own new (copyrightable) work - I'm not sure how they would know you "scrubbed the data in contravention of their license agreement ( I have NO CLUE if there is coding or metadata attached to it such that it's identifiable in that way. I have not tech background and do not endorse taking what's not yours). But if they can and do know, they could cause problems for you. Lastly, I will just say that the internet is littered with sites that claim copyrights, or impose unenforceable licenses on material that is ripe for public use. Just because it says it's theirs does not make it so. The inverse is also true. Just because a site does not claim copyright to something, does not mean it is in the public domain. I would recommend either sticking to public domain/use sites for your scrubbing endeavors, or seeking permissions from the sites who impose licensing requirements. Short of that, I would recommend (as I already have) seeking an formal legal opinion to say that you are not imposing on anyone's copyrights (this could only be done once you showed an attorney every place you took material from, as well as what the material is), and that the licenses from sites with generalized information that may try to limit use, are unenforceable. I would do this before you invest a lot of time or money into something that is largely based on the accumulation of other peoples work product. I wish there was an answer certain, but there just isn't without seeing everything in the end.
These are some thoughts about the state of affairs in the US, I do not know how it works in the UK. In the US it seems to be a legal gray area. Gray enough that I do not think any lawyer could say for certain that the use of the data is legal. The data is stolen. If possessing stolen property is illegal then possessing this data is also likely illegal. Of course experts disagree, Stuart Karle, an adjunct media professor at Columbia University and former general counsel for the Wall Street Journal says: ...the documents have been published by the hackers, they are now public by virtue of being put on the Internet. But Barrett Brown was charged with trafficking in stolen authentication when he forwarded a link to some stolen emails. He signed a plea for acting as an accessory after the fact. He spent more than a year in jail while they sorted it out. In the US there is no law banning the download of hacked documents. In fact Bartnicki v. Vopper 532 US 514 (2001) stands for the rule that journalists can report on illegally obtained information. But contrast that with the Barret Brown prosecution! And decide where a data scientist fits. Also there is the question of whether an illegally recorded conversation is of the same "illegal" nature as hacked personal information.
You Own The Code To answer your question on whether or not it is copyright infringement: Yes, you do own the rights to the written code but posting it on Github gives Github the right to store, archive, parse, and display Your Content, and make incidental copies, as necessary to provide the Service, including improving the Service over time. This license includes the right to do things like copy it to our database and make backups; show it to you and other users; parse it into a search index or otherwise analyze it on our servers; share it with other users; and perform it. To simply put it, no matter what license you use, you give GitHub the right to host your code and to use your code to improve their products and features. This license does not grant GitHub the right to sell Your Content. It also does not grant GitHub the right to otherwise distribute or use Your Content outside of our provision of the Service, except that as part of the right to archive your Content. So with respect to code that’s already on GitHub, I think the answer to the question of copyright infringement is fairly straightforward. Things aren’t quite as clear-cut in a scenario where Copilot is trained on code that is hosted outside of GitHub. In that situation, the copyright infringement question would hinge largely on the concept of fair use. If Copilot is being trained on code outside of GitHub, we accept that at least some of what they’re looking at is copyrightable work. So, the question then becomes if it’s fair use. Now, you ultimately can’t conclude definitively that something is fair use until you go to court and a judge agrees with your assessment. But I think there’s a strong case to be made that Copilot’s use of code is very transformative, a point which would favor the fair use argument. There is precedent for this sort of situation. Take the case of Google Books, for example. Google scanned millions of books, provided people who were doing research with the ability to search the book, and provided the user a small snippet of the text that the user was searching for in the book itself. The court did in fact find that was fair use. The use was very transformative. It allowed people to search millions of books. It didn’t substitute for the book itself. It didn’t really take away anything from the copyright holders; in fact, it made it easier for readers to access the work and actually opened a broader market for book authors. And, it was a huge value add on top of the copyrighted corpus. In the latter scenario, a lot depends on the thoroughness and the length of Copilot’s suggestions. The more complex and lengthy the suggestion, the more likely it has some sort of copyrightable expression. If a suggestion is short enough, the fact that it repeats something in someone else’s code may not make it copyrightable expression. There’s also the question of whether what’s being produced is actually a copy of what’s in the corpus. That’s a little unclear right now. GitHub reports that Copilot is mostly producing brand-new material, only regurgitating copies of learned code 0.1% of the time. But, we have seen certain examples online of the suggestions and those suggestions include fairly large amounts of code and code that clearly is being copied because it even includes comments from the original source code.
In all likelihood, the judge's order related to data collection and reselling is not legally enforceable. They weren't parties to the expungement action, so the judge doesn't have jurisdiction over them. And, the First Amendment protects the right to say truthful things pretty absolutely. Arguably, if the sites provided the information without making clear that it might not be current because records were expunged or corrected, there might be a claim for negligent misrepresentation, false light, or even defamation, but I seriously doubt that even those claims would hold up. The language in the order might cause sites to comply out of not legally justified concern, or just a desire to be accurate, even if it is not enforceable. So, it doesn't hurt to bring that information to the attention of such sites and ask them to take down the information. But, when push comes to shove, I very much doubt that you would prevail in court enforcing that order against them. Certainly, if you do nothing, they will do nothing, because they are not psychic and have no idea that the court order related to those records has been entered. Even a valid and enforceable order directed at a party over whom a court has jurisdiction is not effective until the person ordered to comply with it has notice of the order. And, there is no system that gives sites like that notice without you taking action to inform them of an order.
Question 1. Are technical identifiers personal data? Yes. If they identify a person. For example an IP Address is considered personal data, because a person or household can be identified by an IP. Yes, I know that technically there are a lot of exceptions. But in general, if you have an IP, you can identify the subscriber given the right databases. So if your identifier, lets say a generated GUID, identifies a customer or something the customer can be linked to, it is personal data. Question 2. Can technical identifiers be stored in measurement devices? Just because something is personal data, does not mean you are forbidden from using it. As long as you need it to do your job, you can store it. The internet would not work, if everybody was forbidden from storing any IP address. So to summarize it: yes, it is personal data since it identifies a subscriber. However, it is needed for the job your subscriber asked you to do. So for as long as the job takes, it is legal to store it. Once your job is done, you would be required to delete it. But consent trumps everything. If your subscriber consents to you keeping all data of such incidents to improve your network and handle future incidents better, then it's legal. Just let them sign it with your other legal paperwork. Done. No problem. You could just periodically delete all records that have no consent for long term storage beyond the current incident. Lets say every 24h or 48h. You should get the details on the wording of the paperwork and the period that it is legal to keep the data as &quot;current incedent related&quot; from your data protection officer or legal department.
Maybe, but probably not The geographic location of the organisation is immaterial: under Article 3.2: This Regulation applies to the processing of personal data of data subjects who are in the Union by a controller or processor not established in the Union, where the processing activities are related to: ... (b) the monitoring of their behaviour as far as their behaviour takes place within the Union. Posts anyone (not just EU citizens) make to Reddit (or anywhere else) while they are physically located in the EU or UK engage the GDPR. Pushift.io is therefore captured by the GDPR and any denial of that is just plain wrong. Given the denial, it is likely right out of the gate that they are non-complient. For example, they are unlikely to provided the required information under Article 14. More importantly, it seems that they have not determined the lawful basis for processing the data under Article 6 - they can possibly rely on the public interest basis (preserving deleted publication is arguably a public interest) or a legitimate interest but that requires a balancing of their interest against the data subject's. That said, the right to be forgotten is not absolute, the reasons that might be applicable here are: The data is being used to exercise the right of freedom of expression and information. The data is being used to perform a task that is being carried out in the public interest or when exercising an organization’s official authority. The data represents important information that serves the public interest, scientific research, historical research, or statistical purposes and where erasure of the data would likely to impair or halt progress towards the achievement that was the goal of the processing.
As far as I can tell, whenever GitHub redistributes code, it adheres to all of the requirements of the GPL/BSD etc. For example, it never gives away code without including the license text, and it doesn't claim to add unpermitted restrictions to the code. The section you're concerned about is this: That means you're giving us the right to do things like reproduce your content (so we can do things like copy it to our database and make backups); display it (so we can do things like show it to you and other users); modify it (so our server can do things like parse it into a search index); distribute it (so we can do things like share it with other users); and perform it (in case your content is something like music or video). That section isn't worded very precisely, but I understand this to mean they want to reproduce your content in full, not just the code without the license text attached. They want to display your content in full, as you provided it. They want to modify it internally (i.e. they are not redistributing this modified form) in order to allow searching. To the extent that they display code-snippets as search results, that is likely defensible as fair use (search engines are frequently cited as examples of successful fair use defenses). GitHub appears to be acting consistently with the GPL when it reproduces, displays, modifies, and distributes code that you upload. If I'm correct about these facts then nothing that GitHub does with the code goes beyond what the many contributors to a GPL project have explicitly permitted, or what is otherwise permitted by fair use law. If you had some code with a more restrictive license, you might not be able to simultaneously comply with that restrictive license and GitHub's License Grant. Consider the scenario where you have some code that is licensed exclusively to you, and you are not given permission to reproduce or distribute it. In that scenario, the code's license prevents you from agreeing to GitHub's request for a license grant.
First off: Legally, everything is copyrighted anyway. Licensing is not at all necessary. Hence, even if a court would disagree with # SPDX-License-Identifier: Apache-2.0, that would just make it closed source. Having said that, the law generally doesn't bother with trivialities such as &quot;file headers&quot;. Any commonly accepted way to state the copyright and license terms is OK. Your LICENSE is such a common convention. If you want to avoid all doubt what is covered under that license, put a reference to that LICENSE in each header. If you have just five files in one directory that are all licensed the same, I wouldn't even bother with that. Again, the default position is that everything is closed source.