Prompt Design For Beginners

A bonus section for those few folks who aren't ready for the workshop to end! 

• This module introduces you to the concept of few-shot prompting or a strategy where you show ChatGPT a few completed examples to guide the model in providing the answer in a desired format.
  • Learn how to make use of an intermediate reasoning step to guide the model through a complex thought process. 
     
• It outlines how to integrate interactive feedback into your prompt to prevent the model from making mistakes in its final response. 
   
• Gain experience working through a more challenging example. 

 

Image Caption: A cycle diagram depicting an overview of the approach used to build an effective prompt to format a list of song titles from a block of text. 

Note:

• The titles of the steps listed below sync with the steps shown in the diagram.
• For example: 
  • Step 1: The Block Of Text (in the blue bar below) syncs with Step 1 Block of Text - Artists & Song Titles in the diagram. 

The block of text used in this example is quoted below.

A tiny sample of popular songs with numbers in their title are “Eight Days a Week” and “When I’m Sixty Four” by the Beatles, “Edge of Seventeen” by Stevie Nicks, “Fifteen” and “22” by Taylor Swift, “99 Luftballons” by Nena, “10,000 Hours” by Dan + Shay and Justin Bieber, “99 Problems” by JAY-Z, “2 Become 1” by the Spice Girls, “Whalien 52” by BTS, “7/11″” by Beyonce, “Five Years” by David Bowie, “Three Times a Lady” by the Commodores and “7” by Prince.

Reference

Bonato, A. (2021, December 1). Adele’s ‘30’: A mathematician explores number patterns in album titles. The Conversation.
https://theconversation.com/adeles-30-a-mathematician-explores-number-patterns-in-album-titles-172677 

• Dr. Bonato is at Toronto Metropolitan University. 

In this step, I am asking ChatGPT to take a block of text and generate a list of artists and song titles in a specific format.

• Note that I provide the block of text and then ask the model to wait.

  • Giving the model time to process a chunk of text helps prevent it from making reasoning errors. 

• I apply the few-shot prompting strategy by providing the model with two or more examples to shape how model will format the list of artists and song titles. 

  • My prompt includes four concise examples. It's important to note that these examples don't necessarily have to originate from the  block of text.

• The prompts make use of delimiters such as Text: and ##.

  • Recall that delimiters are user selected symbols or words that help the model to understand the different sections of a prompt.
    For example, the use of Text: separates the text block from the instructional sentence of the prompt.

---

Prompt In ChatGPT-3.5: 

Laura:

Here is a block of Text. Please wait for instructions.

Text:

A tiny sample of popular songs with numbers in their title are “Eight Days a Week” and “When I’m Sixty Four” by the Beatles, “Edge of Seventeen” by Stevie Nicks, “Fifteen” and “22” by Taylor Swift, “99 Luftballons” by Nena, “10,000 Hours” by Dan + Shay and Justin Bieber, “99 Problems” by JAY-Z, “2 Become 1” by the Spice Girls, “Whalien 52” by BTS, “7/11″” by Beyonce, “Five Years” by David Bowie, “Three Times a Lady” by the Commodores and “7” by Prince.

ChatGPT:

Laura, I've read the text. Please provide me with the instructions whenever you're ready.

Laura:

For the block of Text, I want you to output a list of artists and song titles in the following format:
[Artist Name] => "Song Title"

Here are four examples:

[Jimmy Buffett] => "A Pirate Looks at Forty”
##
[Metallica] => “One”
##
[Ariana Grande] => “7 Rings”
##
[Matchbox Twenty] => “3 AM”

ChatGPT:
Certainly, Laura. Here's the list based on the format you provided:

[The Beatles] => "Eight Days a Week"

[The Beatles] => "When I'm Sixty Four"

[Stevie Nicks] => "Edge of Seventeen"

[Taylor Swift] => "Fifteen"

[Taylor Swift] => "22"

[Nena] => "99 Luftballons"

[Dan + Shay and Justin Bieber] => "10,000 Hours"

[JAY-Z] => "99 Problems"

[Spice Girls] => "2 Become 1"

[BTS] => "Whalien 52"

[Beyonce] => "7/11"

[David Bowie] => "Five Years"

[The Commodores] => "Three Times a Lady"

[Prince] => "7"

Next, I want to format the song titles into a consistent style by converting all of the numbers written as words into their numeric format.

For example: 

• [The Beatles] => "Eight Days a Week" converts to:
• [The Beatles] => "8 Days a Week"

However, converting Eight to 8 is a surprisingly complex task for ChatGPT to consistently grasp.

In this step, I am going to:

• Use two examples to guide ChatGPT through the process of converting song titles containing numbers written as words into their numeric formats. The process utilizes an intermediate reasoning step which "is crucial in assisting the model to work through the steps and generate the desired output logically" (Bhavsar, 2023). 
  • In the example below, "Think" is the intermediate reasoning step. 
    Input: [Jimmy Buffett] => "A Pirate Looks at Forty”
    Think: Forty to 40
    Output: [Jimmy Buffett] => "A Pirate Looks at 40”
     
• After showing the model two examples, I ask ChatGPT to apply the thought process to a third song title to test the model's understanding.
  • Example:
    Input: [David Bowie] => "Five Years"
    Think:
    Output:

Why Did I Use Two Detailed Examples to Guide the Model Instead Of Twenty?

• For complex tasks, breaking down an example into specific steps helps the model navigate through the thought process. While it might seem tempting to inundate the model with numerous examples, starting with two or three detailed ones and refining based on the output, tends to yield more desirable results.
   
• All Large Language Models, such as ChatGPT, have constraints called the context window.
  • This means that ChatGPT has limits to how much of a conversation it will pay attention to when generating a response, including both the input (prompt or conversation history) and the model's responses (Aryani, 2023).
  • To make your prompts more effective, it's important to understand that ChatGPT processes information in units called tokens, where each token can represent part of a word or a complete word.
    • The token limit for GPT-3.5 is 4,096 tokens, which works out to be approximately 800 to 1,000 words.
  • This is why you don't want to overwhelm the model with an excessive number of lengthy examples.

---

Prompt In ChatGPT-3.5: 

Laura:
Instructions:

These Song Title names have numbers written as words. I would like the numbers in the Song Title names to be in their numeric format.

Here are two examples:

Input: [Jimmy Buffett] => "A Pirate Looks at Forty”
Think: Forty to 40
Output: [Jimmy Buffett] => "A Pirate Looks at 40”
##
Input: [Bob Marley] => “Three Little Birds”
Think: Three to 3
Output: [Bob Marley] => “3 Little Birds”

Your Task:

Apply the two examples in the following format:
Input: [David Bowie] => "Five Years"
Think:
Output:

ChatGPT:
Certainly! 

Input: [David Bowie] => "Five Years"
Think: Five to 5
Output: [David Bowie] => "5 Years"

I hope this meets your expectations. If you have any more requests or questions, feel free to let me know!

References

Aryani, A. (2023, December 9). What is your context window? Medium.
https://medium.com/@amiraryani/what-is-your-context-window-772b1c65b881

Bhavsar, K. (2023, November 18). Prompt engineering for arithmetic reasoning problems. Towards Data Science.
https://medium.com/towards-data-science/prompt-engineering-for-arithmetic-reasoning-problems-28c8bcd5bf0e

In my original prompt design plan (fantasy?), I wanted ChatGPT to take a single prompt and correctly (and consistently!) convert all 14 song titles. I quickly learned that ChatGPT isn't capable of this type of execution yet (this module was written in December 2023). I realized that I had to guide the model through the thought process of converting numbers into their numeric formats and also needed to break down the entire procedure into individual prompts for each song title. This would allow me to closely monitor the model and correct any formatting errors as they occurred.

Thus, this step is designed to provide an interactive and controlled approach to the song title conversion process. 

• Note that this step represents a heavy cognitive load for ChatGPT so it can process the second prompt very slowly.

Let's break down the two prompts:

In Prompt 1:

1. I give the model the list of artists and song titles from Step 2 and label it, The List. I ask the model to wait. 

• I provide the list of song titles in the prompt since I was concerned about exceeding the model's context window. Recall that if a conversation exceeds the token limit then the model drops older parts of the conversation (Aryani, 2023).

In Prompt 2:

1. I repeat one of the guided examples from Step 4 in my prompt and label it.
   Like this:
   Thought Process:
   Input: [Jimmy Buffett] => "A Pirate Looks at Forty”
   Think: Forty to 40
   Output: [Jimmy Buffett] => "A Pirate Looks at 40”
    
2. I ask the model to apply the Thought Process to each song title in the list. 
    
3. I tell the model exactly how I want its response to be formatted.
   Like this:
   Input:
   Think: (**Note:** Specify "no conversion needed" only if the number is already in numeric format.)
   Output:
    
4. If the song title is already in a numeric format then the model displays "no conversion needed" for the Think step.
   For example:
   ChatGPT:
   Input: [Taylor Swift] => "22"
   Think: no conversion needed
   Output: [Taylor Swift] => "22"
    
5. Starting with the first song title, the model sequentially converts one title at a time and verifies its correctness before moving on to the next. 
   For example:
   ChatGPT:

   Input: [The Beatles] => "Eight Days a Week"
   Think: Eight to 8
   Output: [The Beatles] => "8 Days a Week"

   Is this conversion correct? Y or N?

6. I respond with Yes (correct conversion) or No (incorrect conversion). This allows me to correct any formatting mistakes or oversights as they occur on a title-by-title basis.  

• Note: I only show the process below for the first four song titles, as well as the last one, because it is so repetitive. 

---

Prompt In ChatGPT-3.5: 

Prompt 1

Laura:

Instructions:

Here is The List of artists and song titles. Please wait for further instructions.

The format is [Artist Name] => "Song Title"

The List:

[The Beatles] => "Eight Days a Week"
[The Beatles] => "When I'm Sixty Four"
[Stevie Nicks] => "Edge of Seventeen"
[Taylor Swift] => "Fifteen"
[Taylor Swift] => "22"
[Nena] => "99 Luftballons"
[Dan + Shay and Justin Bieber] => "10,000 Hours"
[JAY-Z] => "99 Problems"
[Spice Girls] => "2 Become 1"
[BTS] => "Whalien 52"
[Beyonce] => "7/11"
[David Bowie] => "Five Years"
[The Commodores] => "Three Times a Lady"
[Prince] => "7"

ChatGPT:

Thank you for providing the list. I'm here and ready for further instructions whenever you're ready to proceed.

Prompt 2

Laura:

Instructions:

The Song Title names have numbers written in words. I would like the numbers in the Song Title names to be written in their numeric format.
The Thought Process for converting numbers written as words into their numeric format.

Thought Process:
Input: [Jimmy Buffett] => "A Pirate Looks at Forty”
Think: Forty to 40
Output: [Jimmy Buffett] => "A Pirate Looks at 40”

Apply the Thought Process to The List.
Start with the first item in The List.

Your Response will be formatted as:
Input:
Think: (**Note:** Specify "no conversion needed" only if the number is already in numeric format.)
Output:

Confirm with me if the conversion is correct before you proceed to the next item in The List. 
Ask Y or N?

My Response will be formatted as:
Y: Correct conversion
N: Incorrect conversion

ChatGPT:

Input: [The Beatles] => "Eight Days a Week"
Think: Eight to 8
Output: [The Beatles] => "8 Days a Week"

Is this conversion correct? Y or N?

Laura:

Y

ChatGPT:

Great! Moving on to the next one:

Input: [The Beatles] => "When I'm Sixty Four"
Think: Sixty Four to 64
Output: [The Beatles] => "When I'm 64"

Is this conversion correct? Y or N?

Laura:

Y

ChatGPT:

Excellent! Let's proceed:

Input: [Stevie Nicks] => "Edge of Seventeen"
Think: Seventeen to 17
Output: [Stevie Nicks] => "Edge of 17"

Is this conversion correct? (Y/N)

Laura:

Y

ChatGPT:

Great! Let's continue:

Input: [Taylor Swift] => "22"
Think: no conversion needed
Output: [Taylor Swift] => "22"

Is this conversion correct? Y or N?

Laura:

Y

[Process continues for the remaining song titles in the list. I'm not going to show the entire process since it's so repetitive.]

ChatGPT:

Perfect! Let's proceed.

Input: [Prince] => "7"
Think: no conversion needed
Output: [Prince] => "7"

Is this conversion correct? Y or N?

Laura:

Y

• Whew! The final step is to tell ChatGPT to gather all of its Outputs into a numbered list. I ask the model to verify that it has all 14 song titles since it occasionally drops Prince's song. 

---

Prompt In ChatGPT-3.5: 

Laura:

Compile all of your Outputs into a numbered list.
Verify that you have 14 items.

ChatGPT:

Certainly! Here is the compiled list of outputs:

1. [The Beatles] => "8 Days a Week"
2. [The Beatles] => "When I'm 64"
3. [Stevie Nicks] => "Edge of 17"
4. [Taylor Swift] => "15"
5. [Taylor Swift] => "22"
6. [Nena] => "99 Luftballons"
7. [Dan + Shay and Justin Bieber] => "10,000 Hours"
8. [JAY-Z] => "99 Problems"
9. [Spice Girls] => "2 Become 1"
10. [BTS] => "Whalien 52"
11. [Beyonce] => "7/11"
12. [David Bowie] => "5 Years"
13. [The Commodores] => "3 Times a Lady"
14. [Prince] => "7"

There are 14 items in the list. If you have any further instructions or questions, feel free to let me know!

For the following block of text:

• Create a numbered list of bands and song titles in an output style of your choice.
• For those band names with numbers expressed in a numeric format, convert the numbers into a written format. Check that your prompt strategy properly changed all of the band names.
  For example, [Spacemen 3] => "Hey Man" is converted to [Spacemen Three] => "Hey Man".

Text:

Embark on a diverse musical journey with bands featuring numbers in their names. The legendary Jackson 5 introduced the world to soulful harmonies with "ABC." 30 Seconds To Mars, led by Jared Leto, offers anthemic rock vibes in hits like "City of Angels." Blink 182's punk-rock spirit shines in classics like "All The Small Things." 3 Doors Down captivates with the post-grunge anthem "Kryptonite." One Republic's pop-rock brilliance is evident in the chart-topper "Counting Stars." Matchbox Twenty's alternative rock sound resonates in "Real World." Three Days Grace brings intense energy with "The Good Life." One Direction's global charm is epitomized in the pop gem "What Makes You Beautiful." 5 Seconds Of Summer delivers infectious pop-punk energy with hits like "Teeth." These bands with numeric elements in their names have left an indelible mark on the music scene, each bringing a unique flavor to the industry.

Laura's Solution: You can read my prompt interaction with ChatGPT for the activity by clicking here.