Prompt

Engineering

Author:LeeBoonstra

Google

PromptEngineering

Acknowledgements

Contentcontributors

MichaelShermanYuanCao

ErickArmbrust

AnantNawalgariaAntonioGulli

SimoneCammel

CuratorsandEditors

AntonioGulli

AnantNawalgariaGraceMollison

TechnicalWriter

JoeyHaymaker

Designer

MichaelLanning

February20252

Tableofcontents

Introduction6

Promptengineering··7

LLMoutputconfiguration8

Outputlength.8

Samplingcontrols9

Temperature9

Top-Kandtop-P··10

Puttingitalltogether11

Promptingtechniques·13

Generalprompting/zeroshot·13

One-shot&few-shot.15

System,contextualandroleprompting18

Systemprompting·19

Roleprompting·21

Contextualprompting23

Step-backprompting25

ChainofThought(CoT)·29

Self-consistency32

TreeofThoughts(ToT)·36

ReAct(reason&act)·37

AutomaticPromptEngineering·40

Codeprompting42

Promptsforwritingcode42

Promptsforexplainingcode··44

Promptsfortranslatingcode46

Promptsfordebuggingandreviewingcode48

Whataboutmultimodalprompting?54

BestPractices·54

Provideexamples54

Designwithsimplicity··55

Bespecificabouttheoutput56

UseInstructionsoverConstraints56

Controlthemaxtokenlength58

Usevariablesinprompts·58

Experimentwithinputformatsandwritingstyles··59

Forfew-shotpromptingwithclassificationtasks,mixuptheclasses59

Adapttomodelupdates·60

Experimentwithoutputformats·60

JSONRepair61

WorkingwithSchemas·62

Experimenttogetherwithotherpromptengineers63

CoTBestpractices·64

Documentthevariouspromptattempts64

Summary·66

Endnotes68

PromptEngineering

Youdon’tneedtobeadata

scientistoramachinelearningengineer–everyonecanwriteaprompt.

Introduction

Whenthinkingaboutalargelanguagemodelinputandoutput,atextprompt(sometimes

accompaniedbyothermodalitiessuchasimageprompts)istheinputthemodeluses

topredictaspecificoutput.Youdon’tneedtobeadatascientistoramachinelearning

engineer–everyonecanwriteaprompt.However,craftingthemosteffectivepromptcanbecomplicated.Manyaspectsofyourpromptaffectitsefficacy:themodelyouuse,themodel’strainingdata,themodelconfigurations,yourword-choice,styleandtone,structure,and

contextallmatter.Therefore,promptengineeringisaniterativeprocess.Inadequatepromptscanleadtoambiguous,inaccurateresponses,andcanhinderthemodel’sabilitytoprovidemeaningfuloutput.

February20256

PromptEngineering

February20257

WhenyouchatwiththeGeminichatbot,1youbasicallywriteprompts,howeverthis

whitepaperfocusesonwritingpromptsfortheGeminimodelwithinVertexAIorbyusingtheAPI,becausebypromptingthemodeldirectlyyouwillhaveaccesstotheconfigurationsuchastemperatureetc.

Thiswhitepaperdiscussespromptengineeringindetail.Wewilllookintothevariouspromptingtechniquestohelpyougettingstartedandsharetipsandbestpracticestobecomeapromptingexpert.Wewillalsodiscusssomeofthechallengesyoucanfacewhilecraftingprompts.

Promptengineering

RememberhowanLLMworks;it’sapredictionengine.Themodeltakessequentialtextas

aninputandthenpredictswhatthefollowingtokenshouldbe,basedonthedataitwas

trainedon.TheLLMisoperationalizedtodothisoverandoveragain,addingthepreviouslypredictedtokentotheendofthesequentialtextforpredictingthefollowingtoken.Thenexttokenpredictionisbasedontherelationshipbetweenwhat’sintheprevioustokensandwhattheLLMhasseenduringitstraining.

Whenyouwriteaprompt,youareattemptingtosetuptheLLMtopredicttherightsequenceoftokens.Promptengineeringistheprocessofdesigninghigh-qualitypromptsthatguide

LLMstoproduceaccurateoutputs.Thisprocessinvolvestinkeringtofindthebestprompt,optimizingpromptlength,andevaluatingaprompt’swritingstyleandstructureinrelationtothetask.InthecontextofnaturallanguageprocessingandLLMs,apromptisaninputprovidedtothemodeltogeneratearesponseorprediction.

PromptEngineering

February20258

Thesepromptscanbeusedtoachievevariouskindsofunderstandingandgenerationtaskssuchastextsummarization,informationextraction,questionandanswering,textclassification,languageorcodetranslation,codegeneration,andcodedocumentationorreasoning.

PleasefeelfreetorefertoGoogle’spromptingguides2,3withsimpleandeffectivepromptingexamples.

Whenpromptengineering,youwillstartbychoosingamodel.Promptsmightneedtobe

optimizedforyourspecificmodel,regardlessofwhetheryouuseGeminilanguagemodelsinVertexAI,GPT,Claude,oranopensourcemodellikeGemmaorLLaMA.

Besidestheprompt,youwillalsoneedtotinkerwiththevariousconfigurationsofaLLM.

LLMoutputconfiguration

Onceyouchooseyourmodelyouwillneedtofigureoutthemodelconfiguration.MostLLMscomewithvariousconfigurationoptionsthatcontroltheLLM’soutput.Effectiveprompt

engineeringrequiressettingtheseconfigurationsoptimallyforyourtask.

Outputlength

Animportantconfigurationsettingisthenumberoftokenstogenerateinaresponse.

GeneratingmoretokensrequiresmorecomputationfromtheLLM,leadingtohigherenergyconsumption,potentiallyslowerresponsetimes,andhighercosts.

PromptEngineering

February20259

ReducingtheoutputlengthoftheLLMdoesn’tcausetheLLMtobecomemorestylisticallyortextuallysuccinctintheoutputitcreates,itjustcausestheLLMtostoppredictingmoretokensoncethelimitisreached.Ifyourneedsrequireashortoutputlength,you’llalso

possiblyneedtoengineeryourprompttoaccommodate.

OutputlengthrestrictionisespeciallyimportantforsomeLLMpromptingtechniques,likeReAct,wheretheLLMwillkeepemittinguselesstokensaftertheresponseyouwant.

Beaware,generatingmoretokensrequiresmorecomputationfromtheLLM,leadingtohigherenergyconsumptionandpotentiallyslowerresponsetimes,whichleadstohighercosts.

Samplingcontrols

LLMsdonotformallypredictasingletoken.Rather,LLMspredictprobabilitiesforwhatthenexttokencouldbe,witheachtokenintheLLM’svocabularygettingaprobability.Thosetokenprobabilitiesarethensampledtodeterminewhatthenextproducedtokenwillbe.

Temperature,top-K,andtop-Parethemostcommonconfigurationsettingsthatdeterminehowpredictedtokenprobabilitiesareprocessedtochooseasingleoutputtoken.

Temperature

Temperaturecontrolsthedegreeofrandomnessintokenselection.Lowertemperatures

aregoodforpromptsthatexpectamoredeterministicresponse,whilehighertemperaturescanleadtomorediverseorunexpectedresults.Atemperatureof0(greedydecoding)is

PromptEngineering

February202510

deterministic:thehighestprobabilitytokenisalwaysselected(thoughnotethatiftwotokenshavethesamehighestpredictedprobability,dependingonhowtiebreakingisimplementedyoumaynotalwaysgetthesameoutputwithtemperature0).

Temperaturesclosetothemaxtendtocreatemorerandomoutput.Andastemperaturegetshigherandhigher,alltokensbecomeequallylikelytobethenextpredictedtoken.

TheGeminitemperaturecontrolcanbeunderstoodinasimilarwaytothesoftmaxfunction

usedinmachinelearning.Alowtemperaturesettingmirrorsalowsoftmaxtemperature(T),emphasizingasingle,preferredtemperaturewithhighcertainty.AhigherGeminitemperaturesettingislikeahighsoftmaxtemperature,makingawiderrangeoftemperaturesaround

theselectedsettingmoreacceptable.Thisincreaseduncertaintyaccommodatesscenarioswherearigid,precisetemperaturemaynotbeessentiallikeforexamplewhenexperimentingwithcreativeoutputs.

Top-Kandtop-P

Top-Kandtop-P(alsoknownasnucleussampling)4aretwosamplingsettingsusedinLLMstorestrictthepredictednexttokentocomefromtokenswiththetoppredictedprobabilities.Liketemperature,thesesamplingsettingscontroltherandomnessanddiversityof

generatedtext.

?Top-KsamplingselectsthetopKmostlikelytokensfromthemodel’spredicted

distribution.Thehighertop-K,themorecreativeandvariedthemodel’soutput;the

lowertop-K,themorerestiveandfactualthemodel’soutput.Atop-Kof1isequivalenttogreedydecoding.

PromptEngineering

February202511

?Top-Psamplingselectsthetoptokenswhosecumulativeprobabilitydoesnotexceedacertainvalue(P).ValuesforPrangefrom0(greedydecoding)to1(alltokensintheLLM’svocabulary).

Thebestwaytochoosebetweentop-Kandtop-Pistoexperimentwithbothmethods(orbothtogether)andseewhichoneproducestheresultsyouarelookingfor.

Puttingitalltogether

Choosingbetweentop-K,top-P,temperature,andthenumberoftokenstogenerate,

dependsonthespecificapplicationanddesiredoutcome,andthesettingsallimpactoneanother.It’salsoimportanttomakesureyouunderstandhowyourchosenmodelcombinesthedifferentsamplingsettingstogether.

Iftemperature,top-K,andtop-Pareallavailable(asinVertexStudio),tokensthatmeet

boththetop-Kandtop-Pcriteriaarecandidatesforthenextpredictedtoken,andthen

temperatureisappliedtosamplefromthetokensthatpassedthetop-Kandtop-Pcriteria.Ifonlytop-Kortop-Pisavailable,thebehavioristhesamebutonlytheonetop-KorPsettingisused.

Iftemperatureisnotavailable,whatevertokensmeetthetop-Kand/ortop-Pcriteriaarethenrandomlyselectedfromtoproduceasinglenextpredictedtoken.

Atextremesettingsofonesamplingconfigurationvalue,thatonesamplingsettingeithercancelsoutotherconfigurationsettingsorbecomesirrelevant.

PromptEngineering

February202512

?Ifyousettemperatureto0,top-Kandtop-Pbecomeirrelevant–themostprobable

tokenbecomesthenexttokenpredicted.Ifyousettemperatureextremelyhigh(above1–generallyintothe10s),temperaturebecomesirrelevantandwhatevertokensmakeitthroughthetop-Kand/ortop-Pcriteriaarethenrandomlysampledtochooseanextpredictedtoken.

?Ifyousettop-Kto1,temperatureandtop-Pbecomeirrelevant.Onlyonetokenpassesthetop-Kcriteria,andthattokenisthenextpredictedtoken.Ifyousettop-Kextremelyhigh,liketothesizeoftheLLM’svocabulary,anytokenwithanonzeroprobabilityofbeingthenexttokenwillmeetthetop-Kcriteriaandnoneareselectedout.

?Ifyousettop-Pto0(oraverysmallvalue),mostLLMsamplingimplementationswillthenonlyconsiderthemostprobabletokentomeetthetop-Pcriteria,makingtemperatureandtop-Kirrelevant.Ifyousettop-Pto1,anytokenwithanonzeroprobabilityofbeingthe

nexttokenwillmeetthetop-Pcriteria,andnoneareselectedout.

Asageneralstartingpoint,atemperatureof.2,top-Pof.95,andtop-Kof30willgiveyou

relativelycoherentresultsthatcanbecreativebutnotexcessivelyso.Ifyouwantespeciallycreativeresults,trystartingwithatemperatureof.9,top-Pof.99,andtop-Kof40.Andifyouwantlesscreativeresults,trystartingwithatemperatureof.1,top-Pof.9,andtop-Kof20.Finally,ifyourtaskalwayshasasinglecorrectanswer(e.g.,answeringamathproblem),startwithatemperatureof0.

NOTE:Withmorefreedom(highertemperature,top-K,top-P,andoutputtokens),theLLMmightgeneratetextthatislessrelevant.

WARNING:Haveyoueverseenaresponseendingwithalargeamountoffillerwords?Thisisalsoknownasthe"repetitionloopbug",whichisacommonissueinLargeLanguage

Modelswherethemodelgetsstuckinacycle,repeatedlygeneratingthesame(filler)word,phrase,orsentencestructure,oftenexacerbatedbyinappropriatetemperatureandtop-k/

PromptEngineering

February202513

top-psettings.Thiscanoccuratbothlowandhightemperaturesettings,thoughfordifferentreasons.Atlowtemperatures,themodelbecomesoverlydeterministic,stickingrigidlytothehighestprobabilitypath,whichcanleadtoaloopifthatpathrevisitspreviouslygenerated

text.Conversely,athightemperatures,themodel'soutputbecomesexcessivelyrandom,

increasingtheprobabilitythatarandomlychosenwordorphrasewill,bychance,leadbacktoapriorstate,creatingaloopduetothevastnumberofavailableoptions.Inbothcases,themodel'ssamplingprocessgets"stuck,"resultinginmonotonousandunhelpfuloutput

untiltheoutputwindowisfilled.Solvingthisoftenrequirescarefultinkeringwithtemperatureandtop-k/top-pvaluestofindtheoptimalbalancebetweendeterminismandrandomness.

Promptingtechniques

LLMsaretunedtofollowinstructionsandaretrainedonlargeamountsofdatasotheycan

understandapromptandgenerateananswer.ButLLMsaren’tperfect;thecleareryour

prompttext,thebetteritisfortheLLMtopredictthenextlikelytext.Additionally,specific

techniquesthattakeadvantageofhowLLMsaretrainedandhowLLMsworkwillhelpyougettherelevantresultsfromLLMs

Nowthatweunderstandwhatpromptengineeringisandwhatittakes,let’sdiveintosomeexamplesofthemostimportantpromptingtechniques.

Generalprompting/zeroshot

Azero-shot5promptisthesimplesttypeofprompt.ItonlyprovidesadescriptionofataskandsometextfortheLLMtogetstartedwith.Thisinputcouldbeanything:aquestion,astartofastory,orinstructions.Thenamezero-shotstandsfor’noexamples’.

PromptEngineering

February202514

Let’suseVertexAIStudio(forLanguage)inVertexAI,6whichprovidesaplaygroundtotestprompts.InTable1,youwillseeanexamplezero-shotprompttoclassifymoviereviews.

Thetableformatasusedbelowisagreatwayofdocumentingprompts.Yourpromptswilllikelygothroughmanyiterationsbeforetheyendupinacodebase,soit’simportanttokeeptrackofyourpromptengineeringworkinadisciplined,structuredway.Moreonthistable

format,theimportanceoftrackingpromptengineeringwork,andthepromptdevelopmentprocessisintheBestPracticessectionlaterinthischapter(“Documentthevariouspromptattempts”).

Themodeltemperatureshouldbesettoalownumber,sincenocreativityisneeded,andweusethegemini-prodefaulttop-Kandtop-Pvalues,whicheffectivelydisablebothsettings(see‘LLMOutputConfiguration’above).Payattentiontothegeneratedoutput.Thewordsdisturbingandmasterpieceshouldmakethepredictionalittlemorecomplicated,asboth

wordsareusedinthesamesentence.

PromptEngineering

February202515

Name

1_1_movie_classification

Goal

Classifymoviereviewsaspositive,neutralornegative.

Model

gemini-pro

Temperature

0.1

TokenLimit

5

Top-K

N/A

Top-P

1

Prompt

ClassifymoviereviewsasPOSITIVE,NEUTRALorNEGATIVE.

Review:"Her"isadisturbingstudyrevealingthedirection

humanityisheadedifAIisallowedtokeepevolving,

unchecked.Iwishthereweremoremovieslikethismasterpiece.Sentiment:

Output

POSITIVE

Table1.Anexampleofzero-shotprompting

Whenzero-shotdoesn’twork,youcanprovidedemonstrationsorexamplesintheprompt,whichleadsto“one-shot”and“few-shot”prompting.Generalprompting/zeroshot

One-shot&few-shot

WhencreatingpromptsforAImodels,itishelpfultoprovideexamples.Theseexamplescanhelpthemodelunderstandwhatyouareaskingfor.Examplesareespeciallyusefulwhenyouwanttosteerthemodeltoacertainoutputstructureorpattern.

Aone-shotprompt,providesasingleexample,hencethenameone-shot.Theideaisthemodelhasanexampleitcanimitatetobestcompletethetask.

Afew-shotprompt7providesmultipleexamplestothemodel.Thisapproachshowsthe

modelapatternthatitneedstofollow.Theideaissimilartoone-shot,butmultipleexamplesofthedesiredpatternincreasesthechancethemodelfollowsthepattern.

PromptEngineering

February202516

Thenumberofexamplesyouneedforfew-shotpromptingdependsonafewfactors,

includingthecomplexityofthetask,thequalityoftheexamples,andthecapabilitiesofthegenerativeAI(genAI)modelyouareusing.Asageneralruleofthumb,youshoulduseatleastthreetofiveexamplesforfew-shotprompting.However,youmayneedtousemoreexamplesformorecomplextasks,oryoumayneedtousefewerduetotheinputlength

limitationofyourmodel.

Table2showsafew-shotpromptexample,let’susethesamegemini-promodel

configurationsettingsasbefore,otherthanincreasingthetokenlimittoaccommodatetheneedforalongerresponse.

Goal

ParsepizzaorderstoJSON

Model

gemini-pro

Temperature

0.1

TokenLimit

250

Top-K

N/A

Top-P

1

Prompt

Parseacustomer'spizzaorderintovalidJSON:

EXAMPLE:

Iwantasmallpizzawithcheese,tomatosauce,andpepperoni.JSONResponse:

、、、

{

"size":"small","type":"normal",

"ingredients":[["cheese","tomatosauce","peperoni"]]}

、、、

Continuesnextpage...

PromptEngineering

February202517

Prompt

EXAMPLE:

CanIgetalargepizzawithtomatosauce,basilandmozzarella

{

"size":"large","type":"normal",

"ingredients":[["tomatosauce","bazel","mozzarella"]]}

Now,Iwouldlikealargepizza,withthefirsthalfcheeseandmozzarella.Andtheothertomatosauce,hamandpineapple.

JSONResponse:

Output

、、、

{

"size":"large",

"type":"half-half",

"ingredients":[["cheese","mozzarella"],["tomatosauce","ham","pineapple"]]

}

、、、

Table2.Anexampleoffew-shotprompting

Whenyouchooseexamplesforyourprompt,useexamplesthatarerelevanttothetaskyouwanttoperform.Theexamplesshouldbediverse,ofhighquality,andwellwritten.Onesmallmistakecanconfusethemodelandwillresultinundesiredoutput.

Ifyouaretryingtogenerateoutputthatisrobusttoavarietyofinputs,thenitisimportanttoincludeedgecasesinyourexamples.Edgecasesareinputsthatareunusualorunexpected,butthatthemodelshouldstillbeabletohandle.

PromptEngineering

February202518

System,contextualandroleprompting

System,contextuaIandroIepromptingareaIItechniquesusedtoguidehowLLMsgeneratetext,buttheyfocusondifferentaspects:

?SystempromptingsetstheoveraIIcontextandpurposefortheIanguagemodeI.It

definesthe,bigpicture,ofwhatthemodeIshouIdbedoing,IiketransIatingaIanguage,cIassifyingareviewetc.

?ContextualpromptingprovidesspecificdetaiIsorbackgroundinformationreIevanttothecurrentconversationortask.ItheIpsthemodeItounderstandthenuancesofwhat,sbeingaskedandtaiIortheresponseaccordingIy.

?RolepromptingassignsaspecificcharacteroridentityfortheIanguagemodeItoadopt.ThishelpsthemodelgenerateresponsesthatareconsistentwiththeassignedroleanditsassociatedknowIedgeandbehavior.

TherecanbeconsiderabIeoverIapbetweensystem,contextuaI,androIeprompting.E.g.apromptthatassignsaroIetothesystem,canaIsohaveacontext.

However,eachtypeofpromptservesasIightIydifferentprimarypurpose:

?Systemprompt:DefinesthemodeI,sfundamentaIcapabiIitiesandoverarchingpurpose.

?ContextuaIprompt:Providesimmediate,task-specificinformationtoguidetheresponse.It,shighIyspecifictothecurrenttaskorinput,whichisdynamic.

?RoIeprompt:FramesthemodeI,soutputstyIeandvoice.ItaddsaIayerofspecificityandpersonaIity.

PromptEngineering

February202519

Distinguishingbetweensystem,contextual,androlepromptsprovidesaframeworkfor

designingpromptswithcIearintent,aIIowingforflexibIecombinationsandmakingiteasiertoanaIyzehoweachprompttypeinfluencestheIanguagemodeI,soutput.

Let,sdiveintothesethreedifferentkindsofprompts.

Systemprompting

Table3containsasystemprompt,whereIspecifyadditionalinformationonhowtoreturntheoutput.IincreasedthetemperaturetogetahighercreativityI