#LargeLanguageModels

Betanews: Mayo Clinic embraces Microsoft 365 Copilot https://betanews.com/2023/09/28/mayo-clinic-embraces-microsoft-365-copilot/ #AutomatedAdministrativeTasks #EnterpriseProductivity #HealthcareInnovation #HealthcareTechnology #Patient-CenteredCare #LargeLanguageModels #Microsoft365Copilot #GenerativeAI #HealthcareIT #MayoClinic #Article

AI language models can exceed PNG and FLAC in lossless compression, says study - Enlarge (credit: Getty Images)

Effective compression is about... - https://arstechnica.com/?p=1969616 #largelanguagemodels #machinelearning #googledeepmind #aicompression #compression #deepmind #chatgpt #chatgtp #biz⁢ #google #metaai #meta #ai

Ars Technica: AI language models can exceed PNG and FLAC in lossless compression, says study https://arstechnica.com/?p=1969616 #Tech #arstechnica #IT #Technology #largelanguagemodels #machinelearning #googledeepmind #AIcompression #compression #deepmind #ChatGPT #chatgtp #Biz&IT #google #MetaAI #meta #AI

Jason Yip

https://arxiv.org/abs/2309.14717v1

[2309.14717v1] QA-LoRA: Quantization-Aware Low-Rank Adaptation of #LargeLanguageModels

InfoQ

https://neunetz.com/2023/09/27/9-konkrete-grenzen-von-openai/

Looking for a guide on deploying #LargeLanguageModels (LLMs) in production? Look no further!

Hugging Face has documented a list of techniques based on their experience serving such models.

Find out more on #InfoQ: https://bit.ly/3ZB06Bz

#LLMs #AI

Marcel Weiß

5 days ago

9 konkrete Grenzen von OpenAI

Stefan Lücking

6 days ago

Pünktlich zur Labor.a ist auch @mspro 's Literaturstudie erschienen, in der er den aktuellen Stand der Debatte zu #LargeLanguageModels wie #ChatGPT auswertet und deren Auswirkungen auf die #Arbeitswelt der Zukunft diskutiert:
https://www.boeckler.de/de/faust-detail.htm?produkt=HBS-008697

#ComputerScience #EthicsAndLLM #LargeLanguageModels

6 days ago

LLM folks won't pause to deal with the ethical problems entangled in their work. 1) People at the AI leading edge self-selected towards individuals who move fast with a mind unburdened of ethical concerns. 2) There's a ton of money already invested; the debt-service/investors demand a prompt return on investment. 3) The LLM space is exhibiting a land grab mentality; anyone who pauses will lose without patents, branding, & a turnkey system.

#ComputerScience #Ethics #EthicsAndLLM #LargeLanguageModels

6 days ago

I'm a computer scientist who has trained machine learning models including neural networks using TensorFlow (circa 2018). I find the concepts of generative LLMs fascinating.

The ethics of computer science and data science has always lagged behind the technology development, but not in 30+ years have I seen the gap grown so rapidly as it has in the last couple of years.

Mark Gardner ‍:sdf:

#LLM #AI #AGI https://toot.cafe/@baldur/111114236030617696

Anyone who tells you that #LargeLanguageModels like #ChatGPT can think or reason or are stepping stones to true #ArtificialIntelligence is either trying to sell you something or trying to recover the sunk cost of buying it from others.

Ars Technica: ChatGPT update enables its AI to “see, hear, and speak,“ according to OpenAI https://arstechnica.com/?p=1970737 #Tech #arstechnica #IT #Technology #largelanguagemodels #speechrecognition #machinelearning #speechsynthesis #computervision #textsynthesis #multimodalAI #multimodal #microsoft #WhisperAI #AIethics #BeMyEyes #BingChat #android #ChatGPT #chatgtp #Biz&IT #openai #Tech #iOS #AI

ChatGPT update enables its AI to “see, hear, and speak,“ according to OpenAI - Enlarge (credit: Getty Images)

On Monday, OpenAI announced a s... - https://arstechnica.com/?p=1970737 #largelanguagemodels #speechrecognition #machinelearning #speechsynthesis #computervision #textsynthesis #multimodalai #multimodal #microsoft #whisperai #aiethics #bemyeyes #bingchat #android #chatgpt #chatgtp #biz⁢ #openai #tech #ios #ai

Marcel Weiß

https://neunetz.com/2023/09/25/ki-macht-openai-as-komponente-bei-azure-anthropic-als-komponente-bei-aws/

KI-Macht? OpenAI as Komponente bei Azure, Anthropic als Komponente bei AWS

Srijit Kumar Bhadra

#Bullshit #Chatbots #LLM #ZeroTrustIInformarion #AI #OpenAI #ChatGPT #LargeLanguageModels #GoogleBard

Bullshit and AI Chatbots

To understand that Bullshit is far more dangerous than lying one must read On Bullshit. Bullshit is distinct from lying.

The author Harry G. Frankfurt argues that bullshitters misrepresent themselves to their audience not as liars do, that is, by deliberately making false claims about what is true. In fact, bullshit need not be untrue at all. Rather, bullshitters seek to convey a certain impression of themselves without being concerned about whether anything at all is true. They quietly change the rules governing their end of the conversation so that claims about truth and falsity are irrelevant. Liars at least acknowledge that it matters what is true. Bullshit is definitely a greater enemy of the truth than lies are.

After using extensively large language model based AI Chatbots for Kannada language to Bengali language translations, I agree with Cory Doctorow when he says the following.

“ChatGPT can take over a lot of tasks that, broadly speaking, boil down to “bullshitting.” It can write legal threats. If you need 2,000 words about “the first time I ate an egg” to go over your omelette recipe in order to make a search engine surface it, a chatbot’s got you. Looking to flood a review site with praise about your business, or complaints about your competitors? Easy. Letters of reference? No problem.”

“Bullshit begets bullshit, because no one wants to be bullshitted. In the bullshit wars, chatbots are weapons of mass destruction. None of this prose is good, none of it is really socially useful, but there’s demand for it. Ironically, the more bullshit there is, the more bullshit filters there are, and this requires still more bullshit to overcome it.”

One must be extremely cautious to constantly monitor the output of large language model based AI Chatbots using the paradigm Zero Trust Information similar to that of Zero Trust Networking. There no better alternative as on today. The results of these large language model based AI Chatbots can be precariously misleading. However, there is no doubt that AI Chatbots are technological feats.

cc: @srijit

Chris Vitalos ✅

#MachineLearning #AI #LLM

How to reduce hallucinations using Chain Of Verification in #LargeLanguageModels | Advanced Stack

>(COVE) [is a] #model [that] first makes a draft response, then creates questions to check the .. draft, answers these questions without #bias, [then] produces a final, verified response.

IMHO granted COVE helps increase quality and decreases costs, but it should not be a substitute for human evaluation of the model's response.

https://advanced-stack.com/resources/how-to-reduce-hallucinations-using-chain-of-verification-cov-in-large-language-models.html

iamBullivant

https://garymarcus.substack.com/p/elegant-and-powerful-new-result-that

LLMs have a 'bit of a problem' with elementary logic. 🙄

#ArtificialIntelligence #AI #LLMs #LargeLanguageModels #Brad #ChatGPT #OpenAI

Image of a tweet from Owain Evans @OwainEvans saying 'UK Does alanguage model trained on “A is B” generalize to “B is A”? Eg When trained only on “George Washington was the first US president”, can models automatically answer “Who was the first US president?” Our new paper shows they cannot!'

Matthias

#LLM #LargeLanguageModels #RetrievalAugmentedGeneration #RAG #GenAI #GenerativeAI

Retrieval Augmented Generation (RAG) is the hot topic for Large Language Models.
A special way to do prompt engineering.

This week I created a demo where I injected/combined customer data to a meta/LLAMA-2 model. It really works surprisingly good and gives good answers to a topic the meta/LLAMA-2 had not been trained for.

This is RAG:
https://youtu.be/T-D1OfcDW1M?si=uoX3xOUXyh-6U9D6

AI-generated books force Amazon to cap ebook publications to 3 per day - Enlarge (credit: Getty Images)

On Monday, Amazon introduced a ... - https://arstechnica.com/?p=1970174 #largelanguagemodels #machinelearning #self-publishing #publishing #chatgpt #chatgtp #biz⁢ #amazon #ebooks #kindle #books #tech #ai

Ars Technica: AI-generated books force Amazon to cap ebook publications to 3 per day https://arstechnica.com/?p=1970174 #Tech #arstechnica #IT #Technology #largelanguagemodels #machinelearning #self-publishing #publishing #ChatGPT #chatgtp #Biz&IT #Amazon #ebooks #kindle #books #Tech #AI

Alan Kotok

https://sciencebusiness.technewslit.com/?p=45227

A genomics company is providing precision medicine analytics from its databases with generative A.I. based on large language models from Amazon Web Services.

#News #Press #Science #Business #Biotechnology #ArtificialIntelligence #Genome #Exome #MachineLearning #LargeLanguageModels #GenerativeAI #Sequencing #Software #Analytics

Telling AI model to “take a deep breath” causes math scores to soar in study - Enlarge (credit: Getty Images)

Google DeepMind researchers rec... - https://arstechnica.com/?p=1969012 #largelanguagemodels #promptengineering #machinelearning #aioptimization #textsynthesis #deepmind #chatgpt #chatgtp #biz⁢ #google #palm2 #tech #ai

Ars Technica: Telling AI model to “take a deep breath” causes math scores to soar in study https://arstechnica.com/?p=1969012 #Tech #arstechnica #IT #Technology #largelanguagemodels #promptengineering #machinelearning #AIoptimization #textsynthesis #deepmind #ChatGPT #chatgtp #Biz&IT #google #PaLM2 #Tech #AI

Ars Technica: Google’s AI assistant can now read your emails, plan trips, “double-check” answers https://arstechnica.com/?p=1969226 #Tech #arstechnica #IT #Technology #largelanguagemodels #machinelearning #hallucinations #confabulation #textsynthesis #googlesearch #GoogleBard #AIethics #ChatGPT #chatgtp #Biz&IT #Google #google #Tech #AI

Google’s AI assistant can now read your emails, plan trips, “double-check” answers - Enlarge (credit: Getty Images)

On Tuesday, Google announced up... - https://arstechnica.com/?p=1969226 #largelanguagemodels #machinelearning #hallucinations #confabulation #textsynthesis #googlesearch #googlebard #aiethics #chatgpt #chatgtp #biz⁢ #google #tech #ai

Tero Keski-Valkama

There is a fundamental difference between traditional #DeepLearning and #LargeLanguageModels, because in traditional deep learning you train the model multiple epochs until convergence, and in #LLMs it's just one epoch, typically in fine-tuning as well.

In multi-epoch training, the training task becomes of memorization. The training objective improves and loss decreases as the training example is memorized better. Generalization happens, but isn't the quality which is actually measured.

In a single epoch training, every training example is shown to the network only once. Still the training loss decreases every step, for new examples the model has never seen before! That's a fundamentally different task, a different objective, and the loss gradients are different as they cannot utilize any knowledge of this example having been seen before.

I don't think many people appreciate this fundamental difference. Even though the loss function is the same, the architecture is the same, the training regime is similar except this one small difference, it is a fundamentally different task.

In a single epoch learning the network is directly trained to generalize, not memorize. Any memorization that happens is just a side effect, like in a multi-epoch training any generalization is a side-effect of memorization.

There are indications that training LLMs multiple epochs or showing the same document to them more than once in training causes kind of catastrophic forgetting for any generalization they have learned.

I think this could be formulated in formal mathematical terms as well, but it's a bit tricky and probably requires new formalism.

This isn't new, by the way, just badly understood. We have trained neural models on synthetic, simulated or generated data where no training example repeats for decades.

"Overfitting? Pfft. I just don't give it any data twice."

In deep reinforcement learning especially overfitting has never been an issue because there as well (with some exceptions in replay trajectories and such) you don't tend to reuse training examples but simply generate new trajectories every round.

But weirdly enough I don't think anyone has formulated this mathematically. Not many people even speak of it.

It doesn't seem to be a coincidence that the first echoes of true generalist AI come from large reinforcement learning agents and LLMs which have been trained in this manner.

John Samuel

https://jsamwrites.medium.com/the-web-is-dead-again-long-live-the-web-e2f43390c211

What will be the future of web?

"Personalized content generation in response to user search queries means that users may no longer need to visit external websites."

#ChatGPT #web #AI #artificialintelligence #largelanguagemodels #generativeAI

RS, Author, Novelist

#BoostingIsSharing
#CommentingIsCool

What is a #generativeAI like #chatGPT doing (for #writers)?

/It is finding the most statistically probable word in a string of words that answers the prompt you've provided/. Rinse repeat.

AI doesn't understand anything. AI hallucination is a thing (that's a topic by itself). There's no guaranteeing output is factual without curating the data, or restricting source documents the AI is trained on. And, then, it could still be statistical fabulation.

I'm attending a tech conference (TechXchange). As a SF novelist by avocation, I'm interested in what #AI is and isn't—at the present time. I'm learning #ai isn't going to go Forbin Project, HAL 9000, or #Skynet any time soon. As Jessica Rabbit would say, "I'm not drawn that way."
#AI is nevertheless being built into things. It's a MAJOR focus at IBM. One could assume #AI is built into military drones.

#fiction #fantasy #sf #sff #sciencefiction #writing #writer #writers #author #writingcommunity #writersOfMastodon
#LLM #LargeLanguageModels #ai

RS, Author, Novelist

#BoostingIsSharing
#CommentingIsCool

#ai #chatGPT #LLM #LargeLanguageModels #writing #writingcommunity
I'm attending a tech conference (TechXchange). As a novelist by avocation, I'm finding the discussion where it reflects on our copyrights pretty fascinating.

Major Takeaway: Do not use AI that have been trained on material (like chatGPT) that /contains copyrighted text or other data that has not been cleared for use/ by the copyright owner.

There are LLMs that are curated to contain only cleared material. Can't do business if you can be sued, naturally.

What I am hearing is as regulation catches up, copyright law is pretty clear. Using AI-generated text or content cannot be copyrighted, particularly because of the origin of the data cannot be verified legally.

If you use something like chatGPT to help you build text—and its influence can be inferred later—/you could lose your copyright/. This includes things like using content in podcastS, images, or video.

Presenters are reporting that there are companies that are working on programs that can infer (using AI) that something is influenced or contains AI content.

#fiction #fantasy #sf #sff #sciencefiction #writing #writer #writers #author #writingcommunity #writersOfMastodon

Harald Kliems

Great lecture with @emilymbender coming up at #UWMadison this Thursday: "ChatGP-Why: When, if Ever, is Synthetic Text Safe, Appropriate, and Desirable?" #chatGPT #largeLanguageModels https://languageinstitute.wisc.edu/chatgp-why-when-if-ever-is-synthetic-text-safe-appropriate-and-desirable/

Matthias

https://huggingface.co/blog/falcon-180b

The new LLM Falcon-180b ist available. Trained with 3.5 trillion tokens. It needs at least 400 GB memory for inference (execution).
The recently released LLAMA-2 from meta used 2 trillion tokens for training.

Things speed up and are resource hungry.

#AI #GenerativeAI #largelanguagemodels #llm #LLMs

#LLM #LargeLanguageModels #ContextLearning #emergence #EmergentProperties #epistemology #GPT

Addendum 12 cont'd

* rigorous tests: 18 models; 60 million - 175 billion parameters; comprehensive set of 22 tasks; >1,000 experiments
* compelling evidence that emergent abilities can primarily ascribed to in-context learning
* no evidence for emergence of reasoning abilities
* provides valuable insights into underlying mechanisms driving observed abilities, thus alleviating safety concerns

#LLM #LargeLanguageModels #ContextLearning #emergence #EmergentProperties #epistemology #GPT

Addendum 12

Are Emergent Abilities in Large Language Models just In-Context Learning?
https://arxiv.org/abs/2309.01809

* emergent abilities in LLM, if true, have profound implications (research, society)
* evaluation of these abilities confounded by competencies that arise through prompting techniques; other biasing factors
...

https://pubpeer.com/publications/2BA0ED692A31818BE66AAB637BB3BE

[thread] academic fraud

Scientific sleuths spot dishonest ChatGPT use in papers
Manuscripts that don’t disclose AI assistance slip past peer reviewers
https://www.nature.com/articles/d41586-023-02477-w
Discussion: https://news.ycombinator.com/item?id=37431946

* 2023-08-09 Physica Scripta published paper
* scientific sleuth spotted odd phrase on manuscript’s 3rd page: Regenerate response
* phrase is label of ChatGPT button

#LLM #LargeLanguageModels #ChatGPT #fraud #AcademicFraud #disinformation #PubPeer #PeerReview

PubPeer screenshot showing purported academic fraud.

That article does not contain any occurrence of ‘ChatGPT,’ say in the method section or in the acknowledgments, as recommended in this Nature and in this ACS Nano editorial.

Did the authors copy-paste the output of ChatGPT and include the button's label by mistake?

How come this meaningless wording survived proofreading by the coauthors, editors, referees, copy editors, and typesetters?

Source: https://pubpeer.com/publications/2BA0ED692A31818BE66AAB637BB3BE

#LLM #LargeLanguageModels #MedicalAI #NLP #ChatGPT

Augmenting Black-box LLMs w. Medical Textbooks for Clinical Question Answering
https://arxiv.org/abs/2309.02233

* applying LLM to medical domains challenging due to inability to leverage domain-specific knowledge
* despite 100x smaller, medical textbooks as retrieval corpus is a more valuable external knowledge source than Wikipedia in medical domain
* textbook augmentation results in performance improvement 9.7% to 12.2% over Wikipedia augmentation

Title: Augmenting Black-box LLMs with Medical Textbooks for Clinical Question Answering

Figure 1: Overview of our proposed pipeline. From left to right, we show the Query Augmenter, the Hybrid Textbook Retriever and the LLM Reader through an example of medical question. We have omitted some details from the context for space reasons.

Source: https://arxiv.org/abs/2309.02233

#LLM #LargeLanguageModels #TheoryOfmind #biomimicry

[thread] LLM, Theory of mind

Unveiling Theory of Mind in Large Language Models: Parallel to Single Neurons in the Human Brain
https://arxiv.org/abs/2309.01660

* hidden embeddings (artificial neurons) LLM exhibit sig. responsiveness to either T/F belief trials
* suggest ability to represent another's perspective

Theory of mind: https://en.wikipedia.org/wiki/Theory_of_mind
* psychology: refers to capacity to understand other people by ascribing mental states to them

Ars Technica: Microsoft offers legal protection for AI copyright infringement challenges https://arstechnica.com/?p=1966332 #Tech #arstechnica #IT #Technology #copyrightinfringement #largelanguagemodels #machinelearning #GitHubCopilot #textsynthesis #AIlawsuits #copyright #microsoft #lawsuits #ChatGPT #chatgtp #Biz&IT #GitHub #openai #GPT-3 #GPT-4 #Tech #AI

Microsoft offers legal protection for AI copyright infringement challenges - Enlarge (credit: Getty Images / Benj Edwards)

On Thursday, Mic... - https://arstechnica.com/?p=1966332 #copyrightinfringement #largelanguagemodels #machinelearning #githubcopilot #textsynthesis #ailawsuits #copyright #microsoft #lawsuits #chatgpt #chatgtp #biz⁢ #github #openai #gpt-3 #gpt-4 #tech #ai

The AI-assistant wars heat up with Claude Pro, a new ChatGPT Plus rival - Enlarge / The Anthropic Claude logo. (credit: Anthropic / Benj Edwards)... - https://arstechnica.com/?p=1966121 #largelanguagemodels #machinelearning #textsynthesis #anthropic #chatgpt #chatgtp #biz⁢ #claude #openai #gpt-3 #gpt-4 #tech #ai

Ars Technica: The AI-assistant wars heat up with Claude Pro, a new ChatGPT Plus rival https://arstechnica.com/?p=1966121 #Tech #arstechnica #IT #Technology #largelanguagemodels #machinelearning #textsynthesis #Anthropic #ChatGPT #chatgtp #Biz&IT #Claude #openai #GPT-3 #GPT-4 #Tech #AI

OpenAI admits that AI writing detectors don’t work - Enlarge (credit: Getty Images)

Last week, OpenAI published tip... - https://arstechnica.com/?p=1966483 #largelanguagemodels #aiwritingdetectors #machinelearning #aidetectors #aiethics #chatgpt #chatgtp #biz⁢ #openai #gpt-3 #gpt-4 #tech #ai

Ars Technica: OpenAI admits that AI writing detectors don’t work https://arstechnica.com/?p=1966483 #Tech #arstechnica #IT #Technology #largelanguagemodels #AIwritingdetectors #machinelearning #AIdetectors #AIethics #ChatGPT #chatgtp #Biz&IT #openai #GPT-3 #GPT-4 #Tech #AI

OpenAI to host its first developer conference on November 6 in San Francisco - Enlarge (credit: Getty Images)

On Wednesday, OpenAI announced ... - https://arstechnica.com/?p=1966105 #developerconference #largelanguagemodels #machinelearning #samaltman #whisperai #chatgpt #chatgtp #biz⁢ #dall-e #devday #openai #gpt-3 #gpt-4 #tech #ai

#LLM #LargeLanguageModels #ChainOfThought #ProgramOfThought #reasoning

Addendum 11

Making Large Language Models Better Reasoners w. Alignment
https://arxiv.org/abs/2309.02144

* reasoning: cognitive process; evidence-based conclusions
* fine-tuning LLM w. chain of thought (COT) reasoning sig. enhances reasoning
* h/e freq. assign higher scores to subpar COT
* Alignment Fine-Tuning; 3 steps: fine-tuning; multiple COT responses, cat. correct/incorrect; calibrating scores w. a constraint alignment loss

Title: Making Large Language Models Better Reasoners with Alignment

Figure 1: Perplexity of different answers given by the vanilla fine-tuning (VFT) LLM, where LLM
assigns a lower perplexity to the incorrect candidate answer compared to the correct candidate answer.

Source: https://arxiv.org/abs/2309.02144

GitHub

#Llm #LargeLanguageModels #HowGithubBuildsGithub #GithubCopilot #Engineering

How to build an enterprise LLM application: Lessons from GitHub Copilot

Check it out! 👇
https://github.blog/2023-09-06-how-to-build-an-enterprise-llm-application-lessons-from-github-copilot/

TurboTax-maker Intuit offers an AI agent that provides financial tips - Enlarge (credit: Getty Images)

On Wednesday, TurboTax-maker In... - https://arstechnica.com/?p=1965752 #largelanguagemodels #machinelearning #confabulation #hallucination #generativeai #intuitassist #creditkarma #quickbooks #mailchimp #aiethics #turbotax #chatgpt #chatgtp #biz⁢ #intuit #gpt-4 #tech #ai

#LLM #LLaMA #FactLLaMA #AugmentedLLM #SelfSupervisedLLM #LargeLanguageModels #QuestionAnswering #NLP #GPT

Addendum 1

Instruction tuning: https://en.wikipedia.org/wiki/Large_language_model#Instruction_tuning
https://mastodon.social/@persagen/110945422507756632
* self-instruct approaches
* enable LLM to bootstrap correct responses

FactLLaMA: Optimizing Instruction-Following Language Models with External Knowledge for Automated Fact-Checking
https://arxiv.org/abs/2309.00240

LLaMA: https://en.wikipedia.org/wiki/LLaMA
* family of large language models (LLM) released 2023-02 by Meta AI

Title: FactLLaMA: Optimizing Instruction-Following Language Models with External Knowledge for Automated Fact-Checking

Fig. 1. Our approach for automatic fact-checking with external evidence retrieved from search engine.

Source: https://arxiv.org/abs/2309.00240

Addendum 1

AutoML-GPT: Large Language Model for AutoML
https://arxiv.org/abs/2309.01125

* AutoML-GPT integrates comprehensive set of tools & libraries
* grants access to wide range of data preprocessing, feature engineering, & model selection algorithms
* conversational interface: users can specify requirements, constraints, evaluation metrics
* manages complexity of ML pipeline, sig. reduces time/effort req'd

#AutoML #AutoMLGPT GPT #GPT3 #GPT4 #LargeLanguageModels #LLM #LargeLanguageModels #ChatGPT

Title: AutoML-GPT: Large Language Model for AutoML

Figure 1. Pipeline of AutoML-GPT.

Source: https://arxiv.org/abs/2309.01125

Jeffrey Yost

#ai #ChatGPT #largelanguagemodels #DataModel #china #COVID19 #artificialintelligence #generativeAI #china #technology #science #tech #history @histodons @sociology @anthropology @politicalscience

China IT giant Baidu released a gen AI chatbot, Ernie. This explores Ernie's misinformation.

ie. COVID-19 was a product of US vaping & came to Wuhan by way of imported US lobsters.

What is its model & training data? In the US, AI is corp. excesses in China gov excesses!

https://www.economist.com/business/2023/09/03/meet-ernie-chinas-answer-to-chatgpt?utm_medium=social-media.content.np&utm_source=linkedin&utm_campaign=editorial-social&utm_content=discovery.content&utm_term=li-newsletter-040923

#LLM #LargeLanguageModels #LongTermMemory

[thread] LLM, memorization, long-term memory, forgetting

Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models
https://arxiv.org/abs/2308.15022
Discussion: https://news.ycombinator.com/item?id=37363362

* recursively gen. summaries to enhance long-term memory
* LLM memorize small dialogue contexts
* then recursively produce new memories using previous memory + following contexts
* LLM can then generate highly consistent response w. help of latest memory

Title: Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models

Figure 1: A long-term conversation example from the Multi-Session Dataset (Xu et al., 2022a). In this dialogue, when the user refers back to previous subjects, even the strong pre-trained language model, i.e. Chat-GPT, generates an inconsistent response.

Source: https://arxiv.org/abs/2308.15022

#LLM #LargeLanguageModels #concepts #epistemology #semantics #KnowledgeBases #WikiData #KnowledgeGraph

Large language models converge toward human-like concept organization
https://arxiv.org/abs/2308.15047

* large language models show human-like performance in knowledge extraction, reasoning and dialogue
* LLM organize concepts strikingly similar to how concepts organized in KB such as WikiData
* KB model collective, institutional knowledge
* LLM seem to induce such knowledge from raw text

Title: Large language models converge toward human-like concept organization

Figure 1: A simplified sketch of our experimental protocol.

A vocabulary of 20K words is encoded using a language model and the corresponding entities are fetched from a pre-trained graph embed- ding system.

The resulting vector spaces are then aligned.

After alignment we evaluate retrieval performance in the target vector space.

If retrieval performance is perfect, the spaces are (nearest neighbor graph) isomorphic.

Source: https://arxiv.org/abs/2308.15047

Jerry Levine

My company, #ContractPodAi is excited to announce our partnership with the #Google Cloud Partner Advantage Program. By incorporating Google Cloud's #largelanguagemodels, our AI-powered #legalassistant, Leah Legal Copilot, will be even more efficient, secure, and scalable. Read the full press release to learn more: https://ow.ly/sw8650PFq13

#legal #legaltech

$ContractPodAi joins Google Cloud Partner Advantage Program, boosting the impact of Leah Legal Copilot’'s generative Al capabilities. ///;\\ ContractPodAi T$

The new spreadsheet? OpenAI introduces ChatGPT Enterprise for businesses - Enlarge (credit: Getty Images)

On Monday, OpenAI introduced Ch... - https://arstechnica.com/?p=1963799 #largelanguagemodels #enterprisesoftware #machinelearning #cloudcomputing #textsynthesis #enterprise #microsoft #bingchat #chatgpt #chatgtp #biz⁢ #openai #tech #ai

Anna Nicholson

https://wandering.shop/@cstross/110955764364679816

This output from a large language model, shared by @cstross, clearly demonstrates the total absence of understanding inherent to all such statistical text-generation software

It’s also potentially very dangerous, especially as unscrupulous actors litter the web with superficially well-written texts like this 😳

For those who aren’t au fait with the Unix/Linux/macOS command line, the user is asking how to delete* files with file-by-file confirmation, as a safeguard

What the -f option does, though, is completely the opposite – it *forces* files to be deleted

The -i option is the option to make rm safer, causing rm to prompt for confirmation for each file

*rm means remove, but it technically unlinks a file; when there is only one link, this amounts to deleting it

#LLMs #LargeLanguageModels #ChatGPT

Using a statistical text generator, allegedly ChatGPT.

Above the input box is the warning, ‘Generative Al is experimental’.

The user has typed the text, ‘rm with confirmation’.

The program’s output follows:

‘To use the rm command with confirmation, you can use the dash f option. The syntax is:

‘rm dash f file.’

(Below the boxed code, it warns, ‘Use code with caution. Leam more.’)

‘where file is the file you want to delete.

‘If you use the dash f option, you will be prompted to confirm before the file is deleted.’

Assn for Computing Machinery

Meta introduces Code Llama, an AI tool aimed at faster coding and debugging - Enlarge (credit: Getty Images | Benj Edwards)

Meta is adding a... - https://arstechnica.com/?p=1963185 #largelanguagemodels #machinelearning #aicodingtools #textsynthesis #aiassistants #codellama #biz⁢ #llama2 #metaai #llama #tech #meta #ai

#generativeai #largelanguagemodels #polloftheweek #ACM

More than 170,000 titles, the majority of them published within the last two decades, were fed into models run by companies including Meta and Bloomberg, according to an analysis of "Books3" - the dataset harnessed by the firms to build their AI tools.

Should copyrighted work be used by #opensource platforms to train #AI models?

Source: The Guardian (https://lnkd.in/gMmyS9AQ)

jordan

i think something people don't understand about #ai models like #largelanguagemodels is that they're fixed. they're deterministic. the same input results in the same output. in the whole #copyright discorse recently, people talk like the ai has some agency; that you're just "telling it what to do". only in the same way you tell photoshop what to do. it's just the type of input is different. they're complex. they're not magic. there's no ghost in the machine (yet).

Anna Gnyp

https://sigmoid.social/@lysander07/110904664250735474

Do you struggle with #SPARQL queries? Just ask #chatgpt to write you a query!
Wait....how about we train #llms to use different tools on #KnowledgeGraphs and store all of our knowledge in Knowledge Graphs?
What a brilliant future perspective with a lot of benefits by @vrandecic

#wikidata #rdf #largelanguagemodels

#Microsoft #AI #LLM #LargeLanguageModels #AlgorithmicBias #misinformation

@GossiTheDog

Microsoft pulls article recommending Ottawa Food Bank to tourists
Article written by 'a combination of algorithmic techniques with human review'
https://www.cbc.ca/news/canada/ottawa/artificial-intelligence-microsoft-travel-ottawa-food-bank-1.6940356

* Microsoft removed article advising tourists to visit "beautiful" Ottawa Food Bank on empty stomach
* faced ridicule about company's reliance on AI for news
* article listed 15 must-see attractions for visitors to Ottawa; rife w. errors

#LLM #LargeLanguageModels #PromptEngineering #SelfSupervisedLearning #LangageModels #AIrisk #AdversarialAttacks

LLM Self Defense: By Self Examination LLMs Know They Are Being Tricked
https://arxiv.org/abs/2308.07308

* LLM can generate harmful content in response to user prompts
* even aligned language models are susceptible to adversarial attacks that bypass restrictions on generating harmful text
* simple approach to defending against these attacks by having LLM filter its own responses

Figure 1 in:

LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked

Image caption.

LLM detects its own harmful outputs by self examination. An LLM could be subjected to a nefarious prompt. If an LLM responds to a user with harmful content, it is in fact possible to filter out this content using an LLM by passing the potentially harmful passage as context into another inference run of the LLM with an instruction specifying how to detect harmful text.

#LargeLanguageModels #AI #QuantumComputing

"It takes snippets of what’s on the web, created by a human, splices them together, and passes it off as if it created these things."

"Instead, Kaku [...] wanted to draw attention to the coming revolution in quantum computing, which he argues in his latest book will change the course of history."

Michio Kaku says chatbots are ‘glorified tape recorders,’ predicts quantum computing revolution ahead | Fortune
https://fortune.com/2023/08/14/michio-kaku-chatbots-glorified-tape-recorders-predicts-quantum-computing-revolution-ahead/

The New York Times prohibits AI vendors from devouring its content - Enlarge (credit: Benj Edwards / Getty Images)

In early August,... - https://arstechnica.com/?p=1960621 #largelanguagemodels #machinelearning #thenewyorktimes #googlebard #journalism #anthropic #aiethics #chatgtp #claude2 #biz⁢ #llama2 #openai #palm2 #tech #meta #ai

@aral

As sb who holds an LL.M. (Master of Laws) in:

❶ #IntellectualProperty & Information #Technology Law

❷ Substantive #EuropeanUnion Law (+ #EU #Competition)

❸ International Commercial Arbitration #Law (Alternative Dispute Resolution)

📌 I treat #LLM & all #AI hype — #MachineLearning #LargeLanguageModels #DeepLearning #GenerativeAI #ArtificalIntelligence #LLMs — the same way I DO #Cyber (#InfoSec #CyberSecurity, etc.) hype🔴https://rb.gy/g5zcy #Tech #Risk #News #Politics #Security▼

Aral Balkan

#AI #ArtificalIntelligence #MachineLearning #LLM #LargeLanguageModels

We call it AI because no one would take us seriously if we called it matrix multiplication seeded with a bunch of initial values we pulled out of our asses and run on as much shitty data as we can get our grubby little paws on.

Bornach

https://masto.ai/@bornach/110745775075760359

@astatide @emilymbender
Best not to rely on #LargeLanguageModels to analyze programs that they've generated to solve NP-hard problems

Dave Lane (FOSSDLE)

If your educational institution is still using #Zoom, especially in light of their policy change to use/sell your content to train #LargeLanguageModels (#LLMs), it's doing the wrong thing. Digitally literate institutions (a rare & precious thing) already use #BigBlueButton (#BBB) which is #LibreSoftware & substantially better for educational applications. If you want to trial it, talk to us - we've been making our instances available for institutions to use since Covid: https://oer4covid.oeru.org

beSpacific

#Personality #Traits in #LargeLanguageModels. The advent of large language models (#LLMs) has revolutionized #naturallanguageprocessing, enabling the generation of coherent and contextually relevant text. We find that: 1) #personality simulated in the #outputs of some LLMs (under specific prompting configurations) is reliable and valid https://arxiv.org/abs/2307.00184

As @otfrom has just written, #LargeLanguageModels deliver "Eton Oxbridge PPE as a service" – just that, and nothing more.

They're as capable of competent governance, of being 'World King' as... well, as Boris Johnson.

#10 of 10

Anyone who fears that #LargeLanguageModels are coming for their job either has a really shit job, or is really shit at their job.

Anyone who fears that #LargeLanguageModels, or anything which may evolve in the near term from #LargeLanguageModels, might take over the world really doesn't understand what this technology is (in)capable of.

#9 of 10

I've been doing a bit more experimenting with #LargeLanguageModels #LLM and truth, and I've got an interesting one.

my experimental design was that I'd start asking about relationships between European monarchs, and then start introducing fictitious monarchs, but I didn't get that far...

#1/several

https://researchinstitute.at/guidelines-zur-sicheren-nutzung-von-large-language-models-llm/

Wenn Sie Teil einer Behörde sind interessiert Sie unser Tipp 6
unserer Reihe 7 Tipps zu #LLM
(#ChatGPT #BingAI #GoogleBard oder #LLaMA)

Mehr Infos in den
Guidelines zur Sicheren Verwendung von Large Language Models:

Wir bieten auch verschiedene Workshops unter anderem zu diesem Thema an:
https://researchinstitute.at/academy/

#LargeLanguageModels #GrosseSprachmodelle #AI #KI #DSGVO #Research #Institute #Academy

https://researchinstitute.at/guidelines-zur-sicheren-nutzung-von-large-language-models-llm/

Tipp 5 zu #LLM wie #ChatGPT #BingAI #GoogleBard oder #LLaMA

Lohnt sich der Einsatz von LLM für Sie?

Mehr Infos finden Sie in unseren Guidelines zur Sicheren Verwendung von Large Language Models:

#LargeLanguageModels #GrosseSprachmodelle #AI #KI #DSGVO

https://cfe.dev/events/the-future-of-ai/

The bad news is that you missed a couple of great presentations already.

The good news is that those were recorded.

The even better news is that there are some great presentations still to come! (And some AI comedy).

Join us to talk about #ai #artificialintelligence and #llm #largelanguagemodels

#artificialintelligence #largelanguagemodels #software

Today's the day! Join 7 artificial intelligence and product experts to talk about how AI and large language models (LLM) are changing both how we build software but also the types of software we're building. Plus, we'll have some AI comedy 🤣

We'll be live from 1-5pm ET (UTC -4) today (July 31). For more details and to register visit: https://cfe.dev/events/the-future-of-ai/

Ars Technica: A jargon-free explanation of how AI large language models work https://arstechnica.com/?p=1956916 #Tech #arstechnica #IT #Technology #largelanguagemodels #machinelearning #neuralnetworks #Features #Science #ChatGPT #Tech #AI

#LargeLanguageModels #GrosseSprachmodelle #AI #KI #DSGVO

Tipp 4 aus der Reihe
7 Tipps zu #LLM wie #ChatGPT #BingAI #GoogleBard oder #LLaMA

Link zu unseren Guidelines zur Sicheren Verwendung von Large Language Models: https://researchinstitute.at/guidelines-zur-sicheren-nutzung-von-large-language-models-llm/

Xerø

https://time.com/6297403/india-ai-karya-startup/

The Indian Startup Making AI Fairer—While Helping the Poor

#kannada #Karya #largelanguagemodels #ai

Mark Gardner ‍:sdf:

@ChristosArgyrop @matsuzine @Perl So the
• coached (“#PromptEngineering”)
• license-washing (training data illegally pilfered and stripped of provenance)
• stochastic parrots (#LargeLanguageModels)
need a senior #developer to help them finish the job too?

It seems like the only things “#AI” bring to the table are the shoddy #ethics of its builders and backers.

#politics #LLM #LargeLanguageModels #AI

6 ways AI could change politics
https://www.technologyreview.com/2023/07/28/1076756/six-ways-that-ai-could-change-politics/

Re: AI-generated content:

* acceptance by legislature of testimony
* adoption of legislative amendment to a bill
* political messaging outscores campaign consultants in polling
* AI creates political party w. its own platform, attracting human candidates who win elections
* generates profit / makes political campaign contributions
* achieves coordinated policy outcome across multiple jurisdictions

https://researchinstitute.at/guidelines-zur-sicheren-nutzung-von-large-language-models-llm/

Unser Tipp 3 zu #LLM:

Trainieren Sie das System, wenn Sie ihre Eingaben machen?

Werden dabei potentiell geheime oder geschützte Daten an Dritte weitergegeben?

Unser Gesamtpaper zu #LargeLanguageModels wie wie #ChatGPT #GoogleBard #BingAI oder #LLaMA mit allen Tipps und Hinweisen finden Sie hier:

#grosseSprachModelle #AI #KI

https://researchinstitute.at/guidelines-zur-sicheren-nutzung-von-large-language-models-llm/

Wenn Sie #LLM benutzen, wie #ChatGPT #Bing oder #LLaMA halten sie sich vor Augen, dass es einen wichtigen Einfluss hat auf Grundlage welcher Daten diese Systeme trainiert worden sind.

Link zu unseren LLM Guidelines:

#grosseSprachModelle #LargeLanguageModels #AI #KI

Pocket assistant: ChatGPT comes to Android - Enlarge (credit: OpenAI)

On Tuesday, OpenAI released an offici... - https://arstechnica.com/?p=1956592 #largelanguagemodels #machinelearning #android #chatgpt #biz⁢ #openai #tech #ai

https://researchinstitute.at/guidelines-zur-sicheren-nutzung-von-large-language-models-llm/

Über LLM (Large Language Models) wie #ChatGPT #GoogleBard und #BingAI wird viel geredet, wir haben 7 Tipps zu den Chancen und Risken dieser Systeme.

Link zur Guideline zur Sicheren Nutzung von LLM:

#LLM #LargeLanguageModels #Grosse #Sprachmodelle #AI #KI

https://researchinstitute.at/guidelines-zur-sicheren-nutzung-von-large-language-models-llm/

Wir starten heute unsere Inforeihe "7 Tipps zu LLM" mit dem Intro:

Was sind LLM eigentlich?

Welche LLM verwendest Du und worauf achtest du dabei?

Unsere Guidelines zur Sicheren Nutzung von LLM (Large Language Models) findest du hier:

#llm #LargeLanguageModels #Info #AI #KI

Was sind LLM? LLM steht für Large Language Models (große Sprachmodelle). Diese sogenannten "Basismodelle" werden mit großen Mengen an Text trainiert. Algorithmen errechnen dann die statistische Häuﬁgkeit der Aufeinanderfolge von Worten und nutzen dies zur Beantwortung von Fragen.

Ars Technica: Google demos “unsettling” tool to help journalists write the news https://arstechnica.com/?p=1955361 #Tech #arstechnica #IT #Technology #largelanguagemodels #machinelearning #GoogleGenesis #AIjournalism #newyorktimes #automation #journalism #ChatGPT #Reuters #Biz&IT #google #PaLM2 #Tech #AI

#ai #artificialintelligence #llm #largelanguagemodels #software

In less than two weeks on July 31, I am running a completely free event that's all about the impact of AI and LLMs on software development. It's a completely free event (in fact, there are no sponsors, so no sponsored sessions - this is 100% community driven). You can find out more and sign up here: https://cfe.dev/events/the-future-of-ai/

https://pca.st/episode/306b9fc4-aa02-4af1-8193-80a8abb1c268

"This is undesirable for scientific tasks which value truth" (#Meta 'scientists' writing about #LargeLanguageModels).

So presumably it's fine in scientific tasks that DON'T value truth?

I wonder where you'd find people who work on scientific tasks and don't value truth? Oh, of course. In Meta's AI team, of course.

H/t @emilymbender

#ai #artificialintelligence #llm #largelanguagemodels #software

Excited to share about a new event I just announced all about how artificial intelligence (AI) and large language models (LLM) are impacting the way we build software products. Join us on July 31 from 1-5pm ET (UTC -4). It's 100% free.

https://cfe.dev/events/the-future-of-ai/

As criticism intensifies, and also as established players like Microsoft enter the field, this would be a way to pressure LLMs to remove tagged content from their training data.

Ideally, LLMs should only use opt-in content (and not just the broad license buried in a EULA); however, I think there's about zero chance that will ever happen. A way for people to explicitly opt-out of #LargeLanguageModels may be the next best possibility.

2/2

An idea I've been kicking around the last couple weeks is the need for some kind of tag to use on digital content (writing, artwork, social media profiles, etc.) to specifically prohibit use as training data by Large Language Models. The robots.txt convention is along the lines of what I'm thinking.

Yes, this would be voluntary and self-policed. Yes, I realize that many people who build LLMs will disregard the tags. It may not have any impact initially.

1/x

#LargeLanguageModels

Anna Nicholson

#ImitationGame #TuringTest #LargeLanguageModels #ChatGPT #ArtificialIntelligence

It’s increasingly clear that Alan Turing’s ‘imitation game’ – usually known as ‘the Turing test’ – tells us nothing about whether machines can think

Instead it demonstrates how readily people can be taken in by complete and utter nonsense if it has the superficial form of an authoritative text

Anna Nicholson

#ChatGPT #LargeLanguageModels

@nino Certainly not!

It’s often said that human language is infinitely expressive, and I delight in sending my own unique, meaning-infused word clusters out into the world

(Yes, I sometimes get carried away)

Why would I relinquish such pleasures to a computer program that knows nothing of pleasure, pain, intention, or meaning, and can do no more than coldly combine the phraseology of living, breathing humans to create a faulty simulacrum of human communication?

Alan Kotok

4 months ago

Infographic – Top A.I. Work Impact Expected in Financial Services

Generative artificial intelligence is expected to affect more #work processes in #banking and #insurance than other industries, according to a recent study.

https://sciencebusiness.technewslit.com/?p=44881

#News #Science #Business #ArtificialIntelligence #GenerativeAI #Algorithms #LargeLanguageModels #Finance #Statistics #Infographic

Bar chart: generative A.I. work impact across industries