Loïc Faugeron Technical Blog

Now that a couple of months have passed since its over hyped launch, surely ChatGPT has found some use cases where it could be of any actual use. Or is it all fluff? Let's find out.

In my quest to find a use for ChatGPT in my day to day developer activity, I've stumbled upon this video: ChatGPT Prompt Engineering: How to Write a Story, by Kris - All About AI).

Granted, story writing is a bit irrelevant for what I'm looking for. Yet the way the messages sent to ChatGPT (aka "prompts") were crafted, proved quite interesting!

Let me walk you through it, with some added references so we can learn a thing or two.

1st prompt: "You're an expert"

After opening a new conversation, the First Prompt (or user message) sent to ChatGPT is used to prime it:

You are a {Genre} author.
Your task is to write {Genre} stories in a vivid and intriguing language.
Answer with "..." if you acknowledge.
Don't write anything yet.

Genre = Sci-Fi

This is a variation of the You are [job], you have been [doing thing] for 20 years prompt, which helps ChatGPT narrow down the context that it's going to use to generate its replies.

Reference: "Give ChatGPT an identity" section from How To Use ChatGPT: Advanced Prompt Engineering

Interestingly, it uses a placeholder ({Genre}) and sets its value (Genre = Sci-Fi).

It also makes sure that ChatGPT doesn't generate any reply.

Could this be because ChatGPT will base its next replies on any text it will have already generated in the conversation?

2nd prompt: "Templates within Templates"

The Second Prompt is used to specify more context for chatGPT:

Title: [Insert story title here]
Setting: [Insert setting details here, including time period, location, and any relevant background information]
Protagonist: [Insert protagonist's name, age, and occupation, as well as a brief description of their personality and motivations]
Fill out the template above for a {Genre} story
Genre = Sci-Fi

The brilliance of this one is that it uses a second kind of placeholder ([Insert story title here]), one that is intended for ChatGPT to replace (Fill out template above).

If ChatGPT is indeed going to base its next reply on whatever it has already generated earlier in the conversation, then having it repeat the instructions with the added details is simply genius.

The list format might also not be trivial, as I've seen claims that usage of colons (Title: [insert story title here]) is supposed to help ChatGPT be more specific in its replies too.

Reference: "Prompt 2" chapter from Advanced ChatGPT Prompt Tutorial

3rd & 4th prompts: "Now, do your job"

While the First and Second prompts were about setting the context, the Third and Fourth ones are finally the calls to action.

Here's the Third Prompt:

Build a story outlines from the factors above:

ChatGPT is going to generate an ordered list based on the context previously given.

Then the Fourth Prompt:

Great, now create story chapters from the outlines above:

ChatGPT is going to reuse its reply for the Third Prompt, and expand on it.

I'm taking note of the use of the word Great, could its purpose be to give some positive feedback to ChatGPT?

Another thing worth mentioning is the use of the word now, which I see a lot from prompt engineers, and I'm not sure if it's delibarate to improve ChatGPT replies.

But most importantly, I'm wondering how from the factors above is intepreted by ChatGPT: is it going to consider every messages sent in the current conversation, or is it going to focus on the immediately prior message.

I've personally got a feeling that ChatGPT will focus mainly on its own replies from the conversation.

The usage of a final colon : is especially noteworthy, I'd be curious to know if this is another hack to help ChatGPT write more focused replies.

5th prompt: "Use given style"

The Fifth Prompt is yet another call to action:

Write Chapter 1 in depth and in great detail, in a intriguing writing style:

It asks ChatGPT to reuse its previous reply and expand on it, but this time only a section of it by referencing the chapter number (Write Chapter 1)!

The rest of the prompt is a variation of Write a [length] [content type] on [topic] in the style of [style].

Specifying some constraints such as these helps ChatGPT avoid irrelevant output.

Reference: "Be specific, descriptive and as detailed as possible about the desired context, outcome, length, format, style, etc " section from Best practices for prompt engineering with OpenAI API

5th prompt (bis): Prompt Editing!

In my opinion, the most important piece of advice from that video comes in play when writing the second to tenth chapters:

Write Chapter 2 in depth and in great detail, in a intriguing writing style:

Instead of copying the prompt, pasting it in the input message box, changing the chapter number and then sending it as the Sixth Prompt, the advice we're given is to edit the Fifth Prompt (through ChatGPT's UI), change the number and submit it!

ChatGPT's UI will still allow you to check the previous versions of the prompt, as well as their generated replies, so you don't lose anything here.

But this ties in with the idea that ChatGPT performs better with its immediately prior message, than asking it to expand on a reply it wrote 10 messages earlier.

Takeways

While the purpose of the video was to write a Sci-Fi story, it helped learn a lot of Prompt Engineering techniques (the art of crafting messages that will improve ChatGPT replies).

The First Prompt needs to prime ChatGPT for the targeted domain, and make sure that it doesn't reply anything:

You're [JOB].
You have been [DOING JOB] for 20 years.
Your task is now to [TASK].
Answer with "..." if you acknowledge.
Don't write anything yet.

It seems to me that ChatGPT bases its reply on its own immediately prior message, so the Second Prompt needs to make ChatGPT repeat it in its reply, to set up the context for the Third Prompt's reply.

The usage of colons (:) helps ChatGPT to be more specific in its replies, and it's possible to use placeholders that are going to be replaced with generated content in ChatGPT reply:

Title: [Insert title]
Setting: [Insert setting details here]
Fill out the template above for a Sci-Fi story

If the prompt is intended to be copy pasted and used in different scenarios, it's possible to use placeholders that act as variables, and to which the value is set further down the prompt:

You are a {Genre} author.
Your task is to write {Genre} stories

Genre = Sci-Fi

The Third Prompt is going to be a call to action, that's going to use the Second Prompt's reply for context.

Now, write [length] [content type] on [topic] in the style of [style], using [context] from above:

While we're on the topic of length it's very important to bear in mind that shared between the prompt you write and the reply it generates, ChatGPT can only handle 4000 tokens, which is approximately 3000 words (when it uses the text-davinci-003 model).

So for example, if your prompt contains 3000 tokens, it only leaves ChatGPT with a 1000 token left for its reply!

Reference: "Important: Prompt Size Matters" chapter from This Will Make You Better than 99% ChatGPT Users

The Fourth Prompt is going to be a call to action, that's going to use the Third Prompt's reply for context, and so on.

If for some reason the Fifth Prompt needs to use the Third Prompt's reply for context, then we should instead edit directly the Third Prompt, make the required changes, and submit them.

All in all, while I haven't found a practical use case for ChatGPT in my day to day developer activity, it still seems worth exploring its potential for generating content. The quest continues.