Is it legal to share a script that scrapes a website and presents in a different format? Most news sites and tech blogs provide only a summary in their rss feeds. If I write a script that scrapes their website and extracts some useful data, can I share the script as open source. Does it depend on each and every website and needs prior permission? If scraping is not allowed without permission, who is guilty, the person who wrote the script or the one who used them? Note 1: I am not redistributing website's content, just sharing the script that helps to read the content in different form. I am specifically interested in Indian and US Law. Note 2: The script may be specific to particular website and may access data that is accessible only with user's login credentials. Note 3: script means a program or software application E.g nytimes offer subscription offer at $3.5 a week for web+smartphone access. But for tablet access, it is $5.00 a week. Can I sell an app for tablet that parses the page from the web interface and converts it to tablet friendly format. The app will use login credentials from the user to login using a virtual browser to download the content and format it such that it is better readable in tablet? Do I have to get permission from nytimes to sell the app? I don't care about this special case but in general can the provider legally restrict how the content is consumed by the end user.
Check the website's terms of service. Check to see if you're violating these terms, and check to see if the script you are making enables other people to violate them. Courts don't often look kindly on actions whose sole purpose is enabling someone else to do something that is prohibited. If you're making a script that helps people do something they're allowed to do, in a way that's better for at least somebody and makes nobody worse off, that's often a different story. Major websites will generally indicate whether or not you're allowed to do this. Some sites are fairly strict about prohibiting scraping (e.g. Craigslist, which at one point shut down Padmapper's alternative more-useful presentation of their content). Others, like Wikipedia, much more actively encourage reusing content from their sites as long as you meet certain conditions such as a link back to the original source.
The Right to Access is pretty absolute. However, there are some limitations: Is the service even the Data Controller for the data in question? Here, you're talking about notes of one user about another. Is the platform the controller for the notes, or would the note-taker be the controller? Or both, jointly? If the platform weren't a controller but merely a data processor for these notes, it would be illegal for them to disclose the information. Trish also correctly points out that the GDPR does not apply to processing for purely personal or household purposes, e.g. personal social media use. So GDPR would not provide a basis to compel a user to disclose their notes to the data subject, assuming that the note-taker is covered by this exception. Of course, this exception wouldn't apply if the notes are taken for other purposes, e.g. professional networking. Also, this exception doesn't affect the platform. There is an explicit limitation to the right of access in Art 15(4): The right to obtain a copy referred to in paragraph 3 shall not adversely affect the rights and freedoms of others. Disclosing user A's notes about user B to user B would likely violate the privacy rights of A. The notes are both A's and B's personal data. However, the correct balance depends on context. E.g. an employer probably can't refuse to provide access to a performance assessment merely because it was written by an identifiable manager. The UK ICO has provided detailed guidance on this aspect to the right of access. They propose a three-step test: Step one – Does the request require disclosing information that identifies another individual? For example, it might be possible to redact other people's information (but not in your Mastodon notes example). Step two – Has the other individual provided consent? Step three – Is it reasonable to disclose without consent? What is reasonable is highly context-dependent, but UK data protection law gives some concrete criteria to consider. The EU EDPB has draft guidance on the Right to Access. They note that the Art 15(4) can cover a wide range of rights, not just other people's privacy rights. But as in all things, the data controller is required to strike an appropriate balance between user A and B's conflicting rights. In the Mastodon user notes scenario, I think that the note author's rights to privacy should be considered more important than the data subject's right to access those notes, thus making it possible to reject that part of a DSAR under Art 15(4) GDPR. If we assume that the note-taker A is a (joint) controller for these notes, then it would also be necessary to consult them before making a decision about the access request.
The words "proprietary format" are important. Are you sure the format is proprietary? If it is, then it's likely protected in which case they might have grounds to sue (but that does not mean they would definetly win). If the format is not proprietary, and so long as you don't share data which is proprietary then I believe you fine. I'm not a lawyer - but I cite GIF files as an example. They were still protected by the US and some other countries until at least the late 90s and there were various threats to open source linux companies who shared code that used the file format. I'm not aware of anything other than threats and never heard of any company being sued, let alone winning or losing. Another example that comes to mind is the RedHat ISO format. My understanding is RedHat could not stop anyone from sharing open source, but they could prevent folks from sharing the format they assembled and shared the open source. Again, I am not sure if it was ever tested in any court of law anywhere.
From your question(s), as well as your various comments, I understand you to have two general inquiries: 1. Is there any infringement of copyright laws if you use things like the titles of books, games, apps, names, address (and any other number of things) which you will then put into datasets that will be licensed for proprietary commercial purposes? You may freely put titles, names of people, places or things into datasets without fear that you are infringing on copyright or any other laws. That is clear. Copyright law does not protect names, titles, short phrases or expressions. Even if a name, title, or short phrase is novel or distinctive it cannot be protected by copyright. So, there is no point in discussing the doctrine of fair use in this context, because Fair Use is a defense, or a legal safe harbor that is merely an exception to copyright infringement allowing people to use a copyrighted works under specific circumstances. As I understand your intended endeavor, you will not be infringing on any copyrights to the extent that you are merely using factual data, like names of copyrighted things for the purpose of creating a dataset or an application to help access it. This is why I say you need not concern yourself with the test for Fair Use with regard to this issue. The Copyright Office states clearly, despite what people may think, that there are no exclusive rights in brief combinations of words such as: • Names of products or services • Names of businesses, organizations, or groups (including the names of performing groups) • Pseudonyms of individuals (including pen or stage names) • Titles of works • Catchwords, catchphrases, mottoes, slogans, or short advertising expressions • Listings of ingredients, as in recipes, labels, or formulas. When a recipe or formula is accompanied by an explanation or directions, the text directions may be copyrightable, but the recipe or formula itself remains uncopyrightable. Hence, these things are not registrable under a copyright. While something may be potentially attached to or included in copyrighted material, is not in and of itself subject to the protections of these laws. If it (whatever it is) cannot be registered for a copyright, it is not copyrightable. Because copyright registration/notices have been optional since 1989, when the U.S. attached itself to the Berne Convention, whereby copyright protection is automatic as soon as a work is “fixed in a tangible medium of expression” (written down, recorded, painted, etc.) it’s protected. No notice is required. Registration only becomes required for litigation or enforcement purposes. But this is really extraneous to your inquiry anyway, as far as it applies to the actual data. When you get into copying whole databases for your purpose, that analysis is different. 2. You want to "scrub" the internet for information that you intend to put into your proprietary datasets and use for commercial purposes, some or most of which is already in a database or some organized form, and you want to know if there is some sort of copyright or duty owned to the person who originally databased the materials? Since ideas, procedures, principles, discoveries, and devices are all specifically excluded from copyright protection, if you want to compile this type of information from the internet for the purpose of creating datasets, or searchable databases, this is permissible. That said, there are protections for existing databases under copyright law, provided under the concept of a "compilation copyright". A compilation copyright protects the collection and creative assembling of data or other materials. Compilation copyrights protect the collection and assembling of data or other materials, such that databases are generally protected by copyright law as compilations. Under the Copyright Act, a compilation is defined as a "collection and assembling of preexisting materials or of data that are selected in such a way that the resulting work as a whole constitutes an original work of authorship." 17. U.S.C. § 101. The preexisting materials or data may be protected by copyright since the selections of materials and the form they take in an existing database may be original enough to be subject to a copyright. However, the data itself is merely information and is not protectable. The Copyright Act specifically states that the copyright in a compilation extends only to the compilation itself, and not to the underlying materials or data. 17 U.S.C. § 103(b). As a result, "compilation copyrights" can't be used to place protection upon those things that are otherwise not protectable. In the case of Feist Publications, Inc. v. Rural Telephone Service Company, Inc., the U.S. Supreme Court ruled that a compilation work such as a database must contain a minimum level of creativity in order to be protectable under the Copyright Act. Feist makes clear that even a copyright protected database does not hold the right to prevent an individual from extracting factual data from the database (so long as you're not copying the entire database as a whole). If you take an already compiled and copyrighted dataset in its entirety, you must obtain a license for its use. However, if you are merely amassing great amounts of data to then put into your own dataset, that you are free to do. The big issue will be (and you seem to realize this) where you will amass this data from. Some websites have specific licenses in place that say you cannot use or rework their content. However, many times these websites simply throw these license requirements out there for users to see, despite the fact that they may not be (and some would argue) are not enforceable. The courts have heard arguments that "contracts" (the end-user licenses) that protect databases and information on websites is beyond the protection available through copyright law should be "preempted" by the Copyright Act itself. The preemption argument goes like this: Federal law controlling something that is subject to interstate commerce or use, should be controlled by the federal laws. So,since the federal government has enacted the Copyright Act to govern any protections to any original works, states should be (arguably are) prohibited from having contradictory laws. Because of the ability of a federal statute to preempt state law, and the fact that the Copyright Act at 17 U.S.C. § 301 sets forth specific preemptions, no state may create rights that are equivalent to any of the exclusive rights provided under the Act. It is this concept of preemption that prevents copyright protection from varying depending upon the state where a work of authorship is created. Arguably, the same is true for the internet, and supposed contractual relationship created through licenses that dictates how non-copyrightable material may be used. In the case of ProCD, Incorporated v. Matthew Zeidenberg and Silken Mountain Web Services, Inc. the court examined whether an end-user of a CD ROM phone database was subject to the license, when they extracted a large portion of the database and made it available over the Internet. The database was almost the same as the type of data in the Feist case-The lower court rejected all copyright claims and found that the shrinkwrap license that controlled the end user's right to use the data was both unenforceable (as a shrink wrap license) and preempted by the Copyright Act. As a result, there was no relief available to the creator of the phone database and the end-user was free to extract the data and use it as he saw fit. However, on appeal this decision was reversed (7th circuit). The appellate court did acknowledge that the database (on the CD) was not original enough to be protected by copyright (finding no copyright infringement by the end-user); However, they did find the end-user was breach of contract, since the shrink-wrap license prohibited the end-user's conduct. What this tells us is that these licenses (on websites) may or may not be enforceable. While the 7th Circuit found a contract right pursuant to the license, despite the preemption argument, another appellate court that is more liberal may find otherwise. Also, this was a disk, not the internet, which is the "wild west" of information, largely unregulated and unlitigated as it pertains to the legality and enforceability of (some) regulations that do exist. License agreements for site use on the internet are everywhere. If you take a database from some site that has a license saying you cannot take their work and add to it, or whatever, and you do add it to other databases that are not licensed and then make your own dataset - chances are you are NOT going to be infringing on anyone's copyright. That said, you may be in breach of contract (the license) if they find out about it, and sue you (using it doesn't put you in breach; only getting sued and having a court determine you're in breach puts you in breach. It may be a distinction worth contemplation, but that is up to you). The safest, bet would be to get a license from them to rework the materials. If the material is generic enough, and will be changed enough, that you are creating your own new (copyrightable) work - I'm not sure how they would know you "scrubbed the data in contravention of their license agreement ( I have NO CLUE if there is coding or metadata attached to it such that it's identifiable in that way. I have not tech background and do not endorse taking what's not yours). But if they can and do know, they could cause problems for you. Lastly, I will just say that the internet is littered with sites that claim copyrights, or impose unenforceable licenses on material that is ripe for public use. Just because it says it's theirs does not make it so. The inverse is also true. Just because a site does not claim copyright to something, does not mean it is in the public domain. I would recommend either sticking to public domain/use sites for your scrubbing endeavors, or seeking permissions from the sites who impose licensing requirements. Short of that, I would recommend (as I already have) seeking an formal legal opinion to say that you are not imposing on anyone's copyrights (this could only be done once you showed an attorney every place you took material from, as well as what the material is), and that the licenses from sites with generalized information that may try to limit use, are unenforceable. I would do this before you invest a lot of time or money into something that is largely based on the accumulation of other peoples work product. I wish there was an answer certain, but there just isn't without seeing everything in the end.
Providing they meet the basic requirements (see What is a contract and what is required for them to be valid?) then they are binding contracts. Consideration is not an issue: the site provides the content, the user provides eyeballs on it. Consent is the major stumbling block. Online Terms of Service are either presented as browsewrap or clickwrap or sign-up wrap. A browsewrap provides notice of the terms of service but there is no specific user assent to them. A clickwrap requires the user to check a box specifically about agreeing to the terms (with or without user registration). A sign-up wrap presents user registration with a "Sign up" and provides notice to the terms at the point of service but doesn't have a check button specifically for the terms. As an example, Stack Exchange provides two types of wrap. For the casual user, there is a Legal link at the bottom of the page - a browsewrap. If you sign-up, you go to a page that says "By clicking "Sign up", you agree to our terms of service, ..." - a sign-up wrap. Whether a Terms of Service is an enforcable contract depends on whether the user provided notice, whether the user gave consent and whether enforcing the agreement is conscionable. Clickwraps and sign-up wraps have the advantage over browsewraps in the first two of these. Assuming that the contract terms are unremarkable (i.e. they are within the range of "normal" for that type of contract) a clickwrap will normally create an enforcable contract - Forrest v Verizon and Motise v America Online being the relevant case law. Browsewraps are more problematic: Specht v Netscape said no contract but where the browsewrap is shown prominently and repeatedly they can form an enforceable contract - Hubbert v Dell and Cairo v CrossMedia Services. Zaltz v Jdate was a sign-up wrap and did create an enforceable contract. All of these turn on the facts of how the information was was presented to the user. For example, in Meyer v Kalanick the Second Circuit said: Where there is no evidence that the offeree had actual notice of the terms of the agreement, the offeree will still be bound by the agreement if a reasonably prudent user would be on inquiry notice of the terms.[sic] So, in general, Terms of Service are enforcable as a contract if a reasonably prudent user could, on inquiry (e.g. by clicking a link), make themselves aware of the terms. This also explains why skrinkwraps (when software came on actual physical media in a shrink-wrapped box with the terms inside) were not enforcable - a reasonably prudent user could not inform themselves of the terms without unwrapping a product they didn't yet own.
Users grant StackExchange a licence: You agree that all Subscriber Content that You contribute to the Network is perpetually and irrevocably licensed to Stack Exchange under the Creative Commons Attribution Share Alike license. That should be determinative, but there was also some question as to whether StackExchange users can be deemed to have agreed to these terms. For this reason, I'll also review some case law relating to what are known as "click-wrap" agreements where the terms are made available via a hyperlink. In my opinion, StackExchange's way of displaying links to their Terms of Service during registration meets the requirements that have in the past been sufficient for courts to deem the user to have read and agreed to those Terms of Service. See for example, Schnabel v. Trilegiant Corp., 697 F. 3d 110 (2012), especially the section titled "Notice" for reference to other cases: A person can assent to terms even if he or she does not actually read them, but the "offer [must nonetheless] make clear to [a reasonable] consumer" both that terms are being presented and that they can be adopted through the conduct that the offeror alleges constituted assent. Specht v. Netscape Communications Corp., 306 F. 3d 17 (2002)1 frames the notice test in terms of a "reasonably prudent offeree" and whether they would "have known of the existence of license terms". In Guadagno v. E Trade Bank*, 592 F. Supp. 2d 1263 (2008), the court found that clicking on an acknowledgement icon near an underlined, highlighted link to an agreement was acceptance of that agreement: In the instant case, a highlighted, underlined link to the Agreement was directly above the acknowledgement box, along with notice that "The following contain important information about your account(s)." A reasonably prudent offeree would have noticed the link and reviewed the terms before clicking on the acknowledgment icon. I think the most similar case is Fteja v. Facebook, Inc., 841 F. Supp. 2d 829 Dist. Court, SD New York (2012), although not at an appellate level. In order to have obtained a Facebook account, Fteja must have clicked the second "Sign Up" button. Accordingly, if the phrase that appears below that button is given effect, when Fteja clicked "Sign Up," he "indicat[ed] that [he] ha[d] read and agree[d] to the Terms of Policy." This is very similar to StackExchange's sign-up process: That court outlined a lot of case law and concluded that "Fteja assented to the Terms of Use and therefore to the forum selection clause therein". A DMCA takedown could be successful if submitted by somebody other than the StackExchange user where that other party asserts copyright ownership in the contributed material. This could happen if a StackExchange user infringed copyright by posting material that they didn't have the right to reproduce. 1. Opinion authored by now Supreme Court Justice Sotomayor.
Generally, no, this will not violate copyright. Your end user might violate copyright, but that is their problem not yours. Your program is a tool, just like a word processor is. If I copy a Harry Potter book into a word processor that does not make the word processor maker liable. There is a concept of "contributory infringement", which I think is what you are concerned about. I don't know about Danish law in particular, but most countries would require evidence that you had reasonable knowledge that actual copyright violation was occuring, rather than just being aware that it might happen in theory. This is normally applied to file sharing services who have to have "notice and takedown" measures, file filters and the like. This isn't your concern as you never have any access to the files created by your users. You can also be found liable if you induce anyone to commit copyright violations. Don't draw your customers attention to the custom monster feature as a workaround for those copyrighted monsters, as that could be considered to be inducement. Also don't provide any "wizards" or similar tools which make it noticeably easier to duplicate the custom monsters (e.g. if you notice that the copyrighted custom dragon stats are all on a linear scale with dragon size, don't create a "dragon creation wizard" embodying that fact.) This assumes that your program runs entirely on the end user computer. Its different if you provide any kind of cloud storage for generated characters.
Probably not. There are potential problems on the levels of copyright, data protection, and the Youtube terms of service. You should assume that comments are typically covered by copyright. You do not have a license to these comments, only YouTube does. Therefore, YouTube can show the comments but you can't copy them – just like YouTube can stream your videos but others can't download them and host them on their own websites. To cover the copyright angle, you'd either need to obtain a license from the commenters, or get a sub-license from YouTube, or identify a suitable copyright exception. The comments are personal data within the meaning of the GDPR, so that your processing of these comments (including mere storage) would be subject to GDPR as well. You need a legal basis for processing personal data. Which legal basis is suitable would depend on the purpose of processing, and on your relationship with the commenters. Potentially applicable legal bases in this context: you have a contract with the commenters that requires you to show the comments on your website. For example, I could see such a contract if there were a “featured comment” perk for a Patreon subscription. But this is not going to fly with random commenters. you have a legitimate interest (LI) that allows the processing. A LI requires that you conduct a balancing test where you weigh this interest against the commenter's rights. This is very specific to the purposes for which you want to show the comments. However, a LI will generally only apply if you have an existing relationship with the commenters, making it possible for them to expect that this processing will occur – unlikely if you'll be scraping comments from YouTube. you have obtained consent from the data subject. Consent must be specific, informed, freely given, and unambiguous – you can't obtain consent by writing “by commenting under this video you consent to XXX” in the video description. Regardless of legal basis, you would have to inform the commenters under Art 14 GDPR when you scrape their comments from the platform. Finally, consider the platform terms of service. I have not read the YouTube ToS recently, so I don't know what their specific conditions are. But in general, such ToS will not allow you to scrape content from their platform in order to host it somewhere else. The ToS might allow certain actions like embedding a link/iframe to such videos on other sites, without allowing other actions such as copying other user's content to your site.