NLP Wiki

User Tools

Site Tools

Trace: • rnn • human-in-the-loop • nn_tricks • seminars
ml:fine-tuning

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
ml:fine-tuning [2025/05/15 06:35] – [Parameter-Efficient Tuning (PET)] jmflanigml:fine-tuning [2025/07/14 07:37] (current) – [General Papers] jmflanig
Line 40: Line 40:
     * [[https://arxiv.org/pdf/2401.14556|2024 - Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling]] Says "LLMs fall short of achieving state-of-the-art results in information extraction (IE) tasks, many of which are formulated as sequence labeling (SL)"     * [[https://arxiv.org/pdf/2401.14556|2024 - Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling]] Says "LLMs fall short of achieving state-of-the-art results in information extraction (IE) tasks, many of which are formulated as sequence labeling (SL)"
     * [[https://arxiv.org/pdf/2404.05961|LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders]] Shows that Mistral was probably pre-trained using some bi-directional attention.     * [[https://arxiv.org/pdf/2404.05961|LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders]] Shows that Mistral was probably pre-trained using some bi-directional attention.
 +    * [[https://arxiv.org/pdf/2504.06225|Zhang et al 2025 - Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptation]] They seem to think they are the first to do this (adapt pretrained decoder-only LLMs to encoder-decoder), which is incorrect.
  
 ===== Parameter-Efficient Tuning (PET) ===== ===== Parameter-Efficient Tuning (PET) =====
ml/fine-tuning.1747290955.txt.gz · Last modified: 2025/05/15 06:35 by jmflanig
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki