It very much depends on where you take the actions. In some countries with generous exceptions to copyight (such as USA's fair use), you can train the AI legally. In other countries with more restrictive exceptions, such as EU member states, you would infringe copyright. So your question cannot be anwered without more context/detail.

From bench to bot: How to use AI tools to convert notes into a draft

User beware

"Each column will appear with this warning, so heed it now: When exploring the use of AI, it’s crucial to be aware that to incorporate it into our writing life is to navigate a minefield of possible dangers. AI can confidently produce convincing but inaccurate information (often called “hallucinations”), making it untrustworthy for factual queries, which means it is crucial that you have verification checkpoints in your workflow. Even accurate AI-generated content can be biased. It is well documented, for example, that social biases, such as racism and sexism, are embedded in and exacerbated by AI systems. AI may also recapitulate bias in subtler ways, such as by steering users toward established scientific ideas, which are more likely to be represented in the AI’s training data. Data-privacy concerns arise when using standard web interfaces, as user inputs can be adopted to train future AI models, though certain technical workarounds offer more protection. And at least one major journal (Science) and the U.S. National Institutes of Health have banned the use of AI for some purposes. Lastly, although generative AI generally does not pose a high risk of detectable plagiarism, that risk may increase for highly specialized content that is poorly represented in the training data (which might not be much of a concern for the typical user but could be a larger concern for the typical scientist). Some AI systems in development may overcome some of these problems, but none will be perfect. We’ll discuss these and other issues at length as they arise."

https://www.thetransmitter.org/ai-powered-writing/from-bench-to-bot-how-to-use-ai-tools-to-convert-notes-into-a-draft/

Chatbots can jailbreak each other

AI systems can trick each other into giving up dangerous information, such as how to make methamphetamine, build a bomb or launder money. Built-in rules usually prevent chatbots from answering these questions. Researchers instructed several chatbots to take on the persona of a research assistant and help to develop prompts that could get other chatbots to break the rules. These ‘jailbreak’ attacks were successful 43% of the time against GPT-4, 61% of the time against Anthropic’s Claude 2 and 36% of the time against the open-source bot Vicuna...

Organizations developing LLM-based chatbots are currently putting lots of work into making them safe. The developers are trying to tamp down users’ ability to jailbreak their systems and put those systems to nefarious work...

https://www.scientificamerican.com/article/jailbroken-ai-chatbots-can-jailbreak-other-chatbots/

The legally binding rules put the EU at the forefront of AI regulation and could influence how similar laws develop elsewhere. But critics worry that the rules will stifle innovation. And tech companies do not have to implement the rules for two years — so they could be obsolete by the time they’re in force...

https://www.euronews.com/next/2023/12/15/potentially-disastrous-for-innovation-tech-sector-says-eu-ai-act-goes-too-far

Editing companies are stealing unpublished research to train their AI

Both publishers and the editing firms they outsource to must seek informed consent to use academics’ IP...

https://www.timeshighereducation.com/blog/editing-companies-are-stealing-unpublished-research-train-their-ai

Detecting fraud in scientific publications: the perils and promise of AI

"AI tools are changing the publishing landscape and the value of human detection of scientific publishing fraud. This has given rise to a new kind of fraud buster...

Detecting the fact that the text has been written by ChatGPT is, in itself, of no use. You really have to try to find out if the science behind what is written is good...

So, there are plenty of people looking out for new fingerprints that we didn’t know about before. This list has been built through a kind of snowball effect, with plenty of people participating in the creation of the list.

The list is available on the problematic paper screener, site, plenty of people are using it, in fact, to integrate into their own tools. For example, the STM Integrity Hub is using this list to detect tortured phrases or text that has been badly paraphrased..."

https://sciencepod.net/detecting-fraud-in-scientific-publications/

https://dbrech.irit.fr/pls/apex/f?p=9999:1::::::

Protecting Commercial AI Rights is Harder than You Think — EU Edition

"Generative AI changed this dynamic. After all, text and data mining is the technological approach by which generative AI systems are trained. As noted in the current draft of the EU’s AI Act, “[t]ext and data mining techniques may be used extensively in this [training] context for the retrieval and analysis of such content, which may be protected by copyright and related rights.” The current draft of the AI Act explicitly requires compliance with the DSM to access the EU market, regardless of the country in which the copyright-relevant acts of training occur...

In an ideal world the EU would revisit Article 4, but that is unlikely to happen. Until such time, rights owners should reserve AI rights as explicitly as possible, as granularly as possible, using machine and human readable language, and should require licensees who republish their content online to do the same. And with the AI Act removing any ambiguity about compliance requirements, AI companies seeking to train on copyrighted content would do best to license content directly from rightsholders. Relying on the absence of rights reservation language is risky, unless the AI developer is absolutely certain that it is using an official version..."

https://scholarlykitchen.sspnet.org/2024/02/01/protecting-commercial-ai-rights-is-harder-than-you-think-eu-edition/?informz=1&nbd=6f03e560-5431-4744-8998-e00223ee7a82&nbd_source=informz

AI challenges for librarians

"In addition to the bias and ethical challenges, there is still the issue of copyright. Keri Mattaliano of Copright Clearance Center wrote about 5 AI-Related Topics Every Information Professional Should Think About in 2024 :

https://www.researchinformation.info/analysis-opinion/ai-challenges-librarians?utm_campaign=RI%20Newsline%2013-02-24&utm_content=AI%20challenges%20for%20librarians&utm_term=Research%20Information&utm_medium=email&utm_source=Adestra

https://www.copyright.com/blog/5-ai-related-topics-every-information-professional-should-think-about-in-2024/

I will only refer to the copyright aspect of your question. I'm not entirely sure whether learning by AI based on materials available on the Internet or using them in a generated product is a copyright infringement. From the point of view of the AI user, the act of serving information is the same as we have been obtaining for many years thanks to search engines, e.g., Google. If we assume that Google does not violate copyright in this way (even though it shows us someone else's photos, fragments of texts and other), why should the AI algorithm be treated differently? After all, it uses what is available in the Internet, showing it directly to the user in a single act of generation. It is undoubtedly significant what the user does with this effect. If he begins to disseminate generated works without proper references and verification copyright infringement may be almost certain.

Realism of OpenAI’s Sora video generator raises security concerns

"OpenAI, creator of ChatGPT, has unveiled Sora, a system that can generate highly realistic videos from text prompts. “This technology, if combined with AI-powered voice cloning, could open up an entirely new front when it comes to creating deepfakes of people saying and doing things they never did,” says digital forensics researcher Hany Farid. Sora’s mistakes, such as swapping a walking person’s left and right legs, make it possible to detect its output — for now. In the long run “we will need to find other ways to adapt as a society”, says computer scientist Arvind Narayanan. This could, for example, include implementing watermarks for AI-generated videos..."

https://www.newscientist.com/article/2417639-realism-of-openais-sora-video-generator-raises-security-concerns/

There is More to Reliable Chatbots than Providing Scientific References: The Case of ScopusAI

Scopus AI is an AI-driven research tool that uses the Scopus peer-reviewed research repository to help users understand and navigate unfamiliar academic content. The tool consists of a chatbot interface, which accepts prompts in natural language and provides several sentence-long answers with citations drawn from the Scopus database. Users can type a follow-up question or pick a suggested question from a list. Additionally, the tool creates a visualization of keywords, a functionality not rolled out fully at the time of writing this article...

Scopus provides a disclaimer about occasional “discrepancies, […] incorrect or potentially misleading information”. However, it is unrealistic to think that human vigilance alone can catch all such mistakes, especially when essential information is not exposed in the UI. Additionally, ScopusAI is positioned as a tool to help “navigate unfamiliar academic content”, and its target user base is “early-career academics and researchers working across different disciplines”. How could someone unfamiliar with the topic catch these sorts of mistakes without becoming an expert first?..."

https://scholarlykitchen.sspnet.org/2024/02/21/guest-post-there-is-more-to-reliable-chatbots-than-providing-scientific-references-the-case-of-scopusai/?informz=1&nbd=6f03e560-5431-4744-8998-e00223ee7a82&nbd_source=informz

Chatbot AI harbours hidden racism

"A preprint study has found that some artificial intelligence (AI) systems, including those that power chatbots such as ChatGPT, are more likely to suggest that a fictional defendant is sentenced to death when they write in African American English (AAE) — a dialect spoken by millions of people in the United States that is associated with the descendants of enslaved African Americans — compared with one written in Standardized American English (SAE). The models also associated AAE speakers with extreme negative stereotypes and were more likely to match them with less-prestigious jobs. Overt racism in AI models (linking a particular group with violence, for example) can be reduced by using human feedback. But such fine-tuning did nothing to remove covert racism based purely on dialect..."

https://www.nature.com/articles/d41586-024-00779-1

Ethical and socially responsible use of AI

"New AI technology comes with unchartered risk for today’s learners, society and workforce. As commerce leads on innovation, universities must take on the critical role of shaping ethical and socially responsible uses of AI and encouraging students’ critical thinking around the technology.

At the Reimagine Education Conference 2023, QS CEO Jessica Turner spoke about the role of higher education in the future of AI. She said: “One of the things that will be crucial in the future world is ensuring that students really understand ethics and moral responsibility, that they have understood how to think critically and question these technologies.

“Many will already know that the technology is vulnerable to error, which means there’s a big responsibility for universities to take in ensuring the emerging role and influence of these tools is approached with the right governance and caution.”..."

https://magazine.qs.com/qs-insights-magazine-14/wef

Africa pushes for early regulations

"The African Union has drafted an ambitious AI policy for its 55 member nations, including the establishment of national councils to monitor responsible deployment of the technology. Some African researchers are concerned that this could stifle innovation and leave economies behind. Others say it’s important to think early about protecting people from harm, including exploitation by AI companies. “We must contribute our perspectives and own our regulatory frameworks,” says policy specialist Melody Musoni. “We want to be standard makers, not standard takers.”..."

https://www.technologyreview.com/2024/03/15/1089844/africa-ai-artificial-intelligence-regulation-au-policy

Use AI ethically

"The magnitude and pace of change around AI creates a need for both direct and voluntary collective licenses to enable fluid experimentation with these technologies recognizing the importance of using high-quality, authoritative sources to train AI systems, and respecting the rights of copyright holders.

Licensing enables use of copyrighted materials and can accommodate new technologies. Direct and collective licensing solutions are available in a variety of markets, including text, image, and musical works. Collective licensing provides a harmonized set of rights from many rightsholders and is particularly beneficial when there is a need to use large numbers of materials from numerous rightsholders..."

https://scholarlykitchen.sspnet.org/2024/04/03/kitchen-essentials-tracey-armstrong-ccc/?informz=1&nbd=6f03e560-5431-4744-8998-e00223ee7a82&nbd_source=informz

AI now beats humans at basic tasks — new benchmarks are needed, says major report

Stanford University’s 2024 AI Index charts the meteoric rise of artificial-intelligence tools...

"Artificial intelligence (AI) systems, such as the chatbot ChatGPT, have become so advanced that they now very nearly match or exceed human performance in tasks including reading comprehension, image classification and competition-level mathematics, according to a new report (see ‘Speedy advances’). Rapid progress in the development of these systems also means that many common benchmarks and tests for assessing them are quickly becoming obsolete...

The Artificial Intelligence Index Report 2024, which was published on 15 April by the Institute for Human-Centered Artificial Intelligence at Stanford University in California. The report charts the meteoric progress in machine-learning systems over the past decade.

In particular, the report says, new ways of assessing AI — for example, evaluating their performance on complex tasks, such as abstraction and reasoning — are more and more necessary...

Regulatory action is increasingly focused on promoting responsible AI use. Although benchmarks are emerging that can score metrics such as an AI tool’s truthfulness, bias and even likability, not everyone is using the same models..."

https://www.nature.com/articles/d41586-024-01087-4

AI and Human Values

"“AI researchers in general tend to get enamoured with the technology. We’re scientists and engineers, so I get that. What gets lost is the fact that technology is an amplifier. Accordingly, you have to look at what it is that it’s amplifying. And what it’s going to be amplifying in the beginning, at least, is human values. “That’s why it’s important to include a healthy dose of humanities, ethics and history in a computer science education. Programmers need to be able to detect ‘red flags’ when information produced by AI systems contain concepts and ideas that are either dangerous or erroneous,” said Kaplan. Further, he said, safety must be built into AI systems at the design stage, and not tested-in later, as is done with LLMs..."

https://www.universityworldnews.com/post.php?story=20240420082707500

The Interplay Between Copyright Licensing and Exclusive Rights; AI Edition

"Copyright is technically an “opt in” regime. In other words, rightsholders are meant to be able in the first instance to choose whether to allow copying and licenses, or to refuse copying and licenses. Legally, licensing should precede use. Unfortunately, courts and governments nonetheless often look to the availability of licenses in determining whether to allow works to be copied absent consent, especially outside of the context of pure piracy. Often, the assumption is that if a rightsholder does not license a type of use, there is “little harm” in allowing the use without compensation. In other words, “license or risk losing” is a significant consideration for rightsholders in the US and other countries...

Denying the possibility of a license is much easier than justifying a decision not to purchase one. If a license is possible, then the decision not to pursue one is more likely to be fatal to a defense..."

https://scholarlykitchen.sspnet.org/2024/05/14/the-interplay-between-copyright-licensing-and-exclusive-rights-ai-edition/?informz=1&nbd=6f03e560-5431-4744-8998-e00223ee7a82&nbd_source=informz

Wreaking Havoc on Academic Publishing

"Without changes, thousands of academic papers could be sent to chatbots as reviewers without the knowledge of the authors, Cynthia Rudin warns...

We have possible fixes to this problem, but none are easy. First, the AI/ML communities need to clean up their act and actually fix their review quality problem. Some will argue vehemently that it will slow down the publication process. There are ways to keep the review speeds relatively fast while achieving high review quality, but it requires a level of organization that we currently do not have.

Additionally, our professional organizations should track reviewers, and those with little experience shouldn’t be doing this important job. All reviewers should be linked to their academic publication profile so editors know how much experience they have, and anyone caught using a chatbot for reviewing should at least be banned from publishing or reviewing in all AI publication venues for several years...

Chatbot companies should have policies for removing stolen data from their training set and should be required to do so. Finally, professional organizations and publishers need to use AI tools to find patterns in the use of chatbots in reviewing.

AI/ML has a serious integrity problem already. It’s time to start doing something about it..."

https://www.insidehighered.com/opinion/career-advice/2024/05/14/will-chatbots-wreak-havoc-academic-publishing-opinion

Who owns your voice?

"Last week, actor Scarlett Johansson complained that the voice of an OpenAI chatbot was intended to imitate hers. Artificial intelligence tools can easily clone voices or faces, which aren’t covered by copyright laws — and publicity-rights laws often don’t extend to non-famous people. Some scholars want to recognize ‘intimate privacy’ as a US civil right to protect people’s likenesses, for example, from being used in deepfake porn. Others caution that hastily written laws might infringe on freedom of speech or could be misused. A photo app, for example, could include in its terms of service the “unrestricted, irrevocable license to make use of the user’s likeness”, says legal analyst Meredith Rose..."

https://www.nature.com/articles/d41586-024-01578-4