Scribd
Upload
59 views

Community Session - PromptEngineering-Fine-Tuning

The document provides an outline for a community session discussing prompt engineering versus fine-tuning versus instruction-tuning of large language models. It includes an introductory talk on these topics and best practices, followed by community breakout discussions. It then describes a live interactive build demo of fine-tuning an LLM to produce SQL code from natural language.

Uploaded by

Sani Kamal
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
59 views

Community Session - PromptEngineering-Fine-Tuning

The document provides an outline for a community session discussing prompt engineering versus fine-tuning versus instruction-tuning of large language models. It includes an introductory talk on these topics and best practices, followed by community breakout discussions. It then describes a live interactive build demo of fine-tuning an LLM to produce SQL code from natural language.

Uploaded by

Sani Kamal
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 34

Prompt Engineering vs.

Fine-Tuning
Community Session
Outline
Introductory Talk
● Prompt Engineering and Best-Practices
● Fine-Tuning the Input-Output Schema of an LLM
● Instruction-Tuning

Community Breakout Discussion Activities

Live Interactive Build Demo!


● Fine-Tuning an LLM to produce SQL code from Natural Language

© 2023 FourthBrain
Meet Shoggoth

https://en.wikipedia.org/wiki/Shoggoth
A few LLM Renditions

https://twitter.com/anthrupad

© 2023 FourthBrain https://knowyourmeme.com/memes/shoggoth-with-smiley-face-artificial-intelligence


Prompt Engineer as Shoggoth Wrangler

“the machine is not the


shoggoth, we are”

Twitter: Prateek Arora

© 2023 FourthBrain
Consider Latent Space

https://twitter.com/i/status/1577061390720069632

convolutional image autoencoder model architecture


https://kopiustech.com/foundation-models/

© 2023 FourthBrain
Prompt Engineering & Best-Practices
Prompt Engineering
● “Prompt engineering is a relatively new discipline for developing and optimizing
prompts to efficiently use language models (LMs) for a wide variety of applications
and research topics.”
https://www.promptingguide.ai/

● Definitions
○ Zero-shot = zero examples provided
○ Few-shot = providing a few examples to serve to condition the model

© 2023 FourthBrain
Zero-Shot Prompting
● Classify the text into neutral, negative or positive.
Text: I think this community session is OK.
Sentiment:

● Instruction
Input
Completion indicator

© 2023 FourthBrain
Few-Shot Prompting
● A "whatpu" is a small, furry animal native to Tanzania.
An example of a sentence that uses the word whatpu is:
We were traveling in Africa and we saw these very cute whatpus.
To do a "farduddle" means to jump up and down really fast.
An example of a sentence that uses the word farduddle is:

● Example
Instruction
Input
Completion indicator

© 2023 FourthBrain Example from Brown, et al. https://arxiv.org/abs/2005.14165


Chain-of-Thought (CoT) Prompting
Other examples:

● Instructing the model with an


itemized format and solving
student solutions, in ChatGPT
Prompt Engineering for
Developers by Deeplearning.ai

● By bus, train, and bike by Chris


Alexiuk

● “Give the model time to think”

Q: Break down the steps that


you took to arrive at 27…
Image Source: Wei et al. (2022)

© 2023 FourthBrain
Self-Refinement
● A way to move from Standard Prompting to Chain-of-Thought Prompting

E.g., Ask the LLM


● To break down the task into component parts
● To try and improve upon its earlier solution
○ e.g., Fibonacci Sequence Refinement
● To ask it to explain it’s own reasoning, and then to incorporate that logic
into a new answer
● **For web-enabled LLMs** - ask to use guides as reference

© 2023 FourthBrain
Prompt Improvement: An example
Prompt:

A 3d visualization of ‘latent space’

© 2023 FourthBrain
Using Search to Improve Prompts

The Ultimate Midjourney Prompt Guide: (substack.com)

© 2023 FourthBrain
Iteration 1
A 3D visualization of latent space that looks like a 32-bit isometric video game

© 2023 FourthBrain
Iteration 2
A colorful and abstract 3D visualization of latent space with duotone effect

© 2023 FourthBrain
Iteration 3
A 3D visualization of latent space in the style of Salvador Dali

© 2023 FourthBrain
#1 Rule in Prompting
Write clear, specific, (even itemized) Instructions
Key Elements of Prompt Engineering
1. Instruction: Write clear and specific instructions
2. Context: Role/Voice/Character/Persona
3. *Self-Refinement*
4. Input: Few-shot/CoT prompting
5. Output: Format

© 2023 FourthBrain
Prompt Engineering =

Poking & Prodding the Shoggoth, or


Exploring Latent Spaces
Try out a few
https://github.com/f/awesome-ch
Breakouts! atgpt-prompts in Bing!

(8 minutes) Notice the specific “instruction”


and the “role” in most all cases!
5 per room Often, there is also an output
specified.
Awesome ChatGPT Prompts

© 2023 FourthBrain
Fine-Tuning
Unsupervised Pre-Training → Completion
● 🟢 Predicting the next word

● 🔴 Following instructions

● Note: the sheer scale of data and


compute involved in this process
is absurd.

© 2023 FourthBrain
Beyond Unsupervised Fine-Tuning
So what are we left with?

1. Supervised Fine-Tuning

2. Reinforcement Learning with Human Feedback

Both of these tools “unlock” capabilities that the pretrained model already
has, but are hard for users to access via prompting alone.
https://huyenchip.com/2023/05/02/rlhf.html

© 2023 FourthBrain
Prompting versus Fine-Tuning I-O Schema
Prompting: for each
sample, explicitly tell your
model how it should
respond.

Fine Tuning: train a model


on how to respond, so you
don’t have to specify that
in your prompt.

© 2023 FourthBrain Image Credit: https://huyenchip.com/2023/04/11/llm-engineering.html


Instruction-Tuning
Unsupervised Pre-Training → Completion
● 🟢 Predicting the next word

● 🔴 Following instructions

● Note: the sheer scale of data and


compute involved in this process
is absurd.

© 2023 FourthBrain
Instruction Tuning IS Fine-Tuning
● “InstructGPT is better than GPT-3 at following instructions”
● “We’ve trained language models that are much better at following user
intentions than GPT-3 while also making them more truthful and less toxic,
using techniques developed through our alignment research.”
○ https://openai.com/research/instruction-following

● “databricks-dolly-15k contains 15,000 high-quality human-generated


prompt/response pair specifically designed for instruction tuning large
language models”
○ https://github.com/databrickslabs/dolly/tree/master/data

© 2023 FourthBrain
© 2023 FourthBrain
Rule of Thumb:
Always pick up Instruct-Tuned Models off the Shelf
When should you use prompt

Breakouts! engineering versus fine-tuning


versus instruction-tuning?
(10 minutes)
10 per room Assign ONE person from your
room to take notes and share!
Prompt Engineering vs. Fine-Tuning

© 2023 FourthBrain
This week’s build - SQL from Natural Language
● Model: OpenLLaMA
○ LoRA: Low-Rank Adaptation of Large Language Models

● Dataset: wikisql
○ A large crowd-sourced dataset for developing natural language interfaces for relational databases.
○ WikiSQL is a dataset of 80654 hand-annotated examples of questions and SQL queries distributed across
24241 tables from Wikipedia.

© 2023 FourthBrain
Let’s Check it Out!
Thank You!

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy