AIdisruptionis

drivinginnovationinon-deviceinference

HowtheproliferationandevolutionofgenerativemodelswilltransformtheAIlandscapeandunlockvalue.

February2025

SnapdragonandQualcommbrandedproductsareproductsofQualcommTechnologies,Inc.and/oritssubsidiaries.

2

Contents

Executivesummary 3

QualityAImodelsarenowabundantanda<ordable 4

Innovationsboostmodelqualityandreducedevelopmenttimeandcost 4

Smallmodelsachievebigcapabilitiesattheedge 5

TheeraofAIinferenceinnovationishere 7

QualcommissettobealeaderintheAIinferenceera 8

Expandingacrossallkeyedgesegments 9

Mobile 9

PCs 10

Automotive 10

IndustrialIoT 11

Networking 11

Conclusion 11

3

Executivesummary

TheintroductionofDeepSeekR1,acutting-edgereasoningAImodel,hascausedripplesthroughoutthetechindustry.That’sbecauseitsperformanceisonparwithorbetterthanstate-of-the-artalternatives,disruptingtheconventionalwisdomaroundAIdevelopment.

Thispivotalmomentispartofabroadertrendthatunderscorestheinnovationincreatinghigh-qualitysmalllanguageandmultimodalreasoningmodels,andhowthey’repreparingAIforcommercialapplicationsandon-deviceinference.Thefactthatthesenewmodelscanrunondevicesacceleratesscaleandcreatesdemandforpowerfulchipsattheedge.

Drivingthisshiftarefourmajortrendsthatareleadingtoadramaticimprovementinthequality,performance,ande<iciencyofAImodelsthatcannowrunondevice:

?Today’sstate-of-the-artsmallerAImodelshavesuperiorperformance.NewtechniqueslikemodeldistillationandnovelAInetworkarchitecturessimplifythedevelopmentprocesswithoutsacri?cingquality,allowingnewmodelsto

outperformlargeronesfromayearago,whichcouldonlyoperateonthecloud.

?Modelsizesaredecreasingrapidly.State-of-the-artquantizationandpruning

techniquesallowdeveloperstoreducethesizeofmodelswithnomaterialimpactinaccuracy.

?Developershavemoretoworkwith.Therapidproliferationofhigh-qualityAI

modelsmeansfeaturesliketextsummarization,codingassistantsandlive

translationarecommonindeviceslikesmartphones,makingAIreadyfor

commercialapplicationsatscaleacrosstheedge.

?AIisbecomingthenewuserinterface.PersonalizedmultimodalAIagentswillsimplifyinteractionsandpro?cientlycompletetasksacrossvariousapplications.

QualcommTechnologiesisstrategicallypositionedtoleadandcapitalizeonthetransitionfromAItrainingtolarge-scaleinference,aswellastheexpansionofAIcomputational

processingfromthecloudtotheedge.Thecompanyhasanextensivetrackrecordin

developingcustomcentralprocessingunits(CPUs),neuralprocessingunits(NPUs),

graphicsprocessingunits(GPUs),andlow-powersubsystems.Thecompany’s

collaborationwithmodelmakers,alongwithtools,frameworks,andSDKsfordeployingmodelsacrossvariousedgedevicesegments,enablesdeveloperstoacceleratethe

adoptionofAIagentsandapplicationsattheedge.

TherecentdisruptionandreassessmentofhowAImodelsaretrainedvalidatesthe

imminentAIlandscapeshifttowardslarge-scaleinference.Itwillcreateanewcycleof

innovationandupgradeofinferencecomputingattheedge.Whiletrainingwillcontinueinthecloud,inferencewillbenefitfromthescaleofdevicesrunningonQualcomm?

technologyandcreatedemandformoreAI-enabledprocessorsattheedge.

4

QualityAImodelsarenowabundantanda9ordable

Innovationsboostmodelqualityandreducedevelopmenttimeandcost

AIhasreachedthepointwherethedropinthecostoftrainingAImodels,combinedwithopen-sourcecollaboration,ismakingthedevelopmentofhigh-qualitymodelsaccessibletomorepeopleandorganizations.

Thisshiftisdrivenbyvarioustechnicaladvancements.Usageoflongercontextlength,

alongwithsimplificationofsomeofthetrainingsteps,savescomputationalcosts.Newernetworkarchitecturesrangingfrommixture-of-experts(MoE)tostate-spacemodels(SSM)arepushingtheboundaryofwhatcanbeaccomplishedwithreducedcomputational

overheadandpowerconsumption.

NewerAImodelsalsointegrateadvancedmethodssuchaschain-of-thoughtreasoningandself-verification,enablingthemtoperformwellacrossvariouschallengingdomainslikemathematics,coding,andscientificreasoning.

Distillationisakeytechniqueinthedevelopmentofcapablesmallmodels.Itallowslargemodelsto"teach"smallermodels,transferringknowledgewhilemaintainingaccuracy.Theuseofdistillationhasledtoasurgeinsmallerfoundationmodels—manyofthemfine-

tunedforspecializedtasks.

Thepowerofdistillationisexemplifiedinfigure1.ThispresentsaverageLiveBenchresultscomparingtheLlama3.370BmodelwithitsdistilledDeepSeekR1counterpart.Thechartshowshowdistillationsignificantlyenhancesperformanceinreasoning,coding,and

mathematicstasksforthesamenumberofparameters.

5

Figure1:LiveBenchAIaveragebenchmarkresultscomparingMetaLlama70Bmodelwithitsdistilled

counterpartbyDeepSeek.Source:LiveBench.ai,Feb.2025.

Smallmodelsachievebigcapabilitiesattheedge

Smallermodelsareapproachingthequalityoflargefrontiermodelsduetodistillationandothertechniquesdescribedabove.Figure2showsbenchmarksfortheDeepSeekR1

distilledmodelscomparedtoleading-edgealternatives.DeepSeek-distilledversions

basedonQwenandLlamamodelsshowareasofsigni?cantsuperiority,particularlyintheGPQAbenchmark–achievingsuperiororsimilarscorescomparedtostate-of-the-art

modelssuchasGPT-4o,Claude3.5Sonnet,andGPT-o1mini.GPQAisacriticalmetricbecauseitinvolvesdeep,multi-stepreasoningtosolvecomplexqueries,whichmanymodels?ndchallenging.

6

Figure2:Mathematicandcodingbenchmarks.Source:DeepSeek,Jan.2025.

ManypopularmodelfamiliesincludingDeepSeekR1,MetaLlama,IBMGranite,Mistral

Ministralfeaturesmallvariantswhichoverdeliverintermsofperformanceand

benchmarksforspecifictasks,regardlessoftheirsize.Thereductionoflarge,foundationalmodelsintosmaller,efficientversionsenablesfasterinference,smallermemoryfootprintandlowerspowerconsumption–allwhilemaintainingahighbaronperformance,allowingdeploymentofsuchmodelswithindeviceslikesmartphones,PCs,andautomobiles.

Furtheroptimizations,likequantization,compressionandpruninghelpreducemodel

sizes.Quantizationlowerspowerconsumptionandspeedsupoperationsbyreducing

precisionwithoutsignificantlysacrificingaccuracy,whilepruningeliminatesunnecessaryparameters.

Thesetechnicaldevelopmentshaveledtoaproliferationofhigh-qualitygenerativeAI

models.AccordingtodatacompiledbyEpochAI(Figure3),morethan75%oflarge-scaleAImodelspublishedin2024featurelessthan100billionparameters.

7

Figure3:Numberoflarge-scaleAImodelspublishedbyyear,categorizedbynumberofparameters.Source:

EpochAI,Jan.2025.

TheeraofAIinferenceinnovationishere

Theabundanceofhigh-quality,smallermodelsisbringingrenewedattentiontoinferenceworkloads–whichiswhereapplicationsandservicesmakeuseofthemodelstoprovidevaluetobusinessesandconsumers.

QualcommTechnologieshasworkedontheoptimizationofnumerousAImodelsto

supportthecommercializationofthenewgenerationofAI-orientedCopilot+PCs.

Similarly,thecompanyhascollaboratedwithOEMssuchasSamsungandXiaomiinthelaunchof?agshipsmartphonesequippedwithmanyAI-enabledfeatures.

TheproliferationofAIinferencingcapabilitiesacrossdeviceshasenabledthecreationofgenerativeAIapplicationsandassistants.Documentsummarization,AI-imagegenerationandediting,andreal-timelanguagetranslationarenowcommonfeatures.CameraappsleverageAIforcomputationalphotography,objectrecognitionandreal-timescene

optimization.

Nextupisthedevelopmentofmultimodalapplicationswhichcombinemultipletypesofdata—text,vision,audioandsensorinput—todeliverricher,morecontext-awareand

personalizedexperiences.TheQualcommAIEnginecombinesthecapabilitiesofcustom-builtNPUs,CPUsandGPUstooptimizesuchtaskson-device,enablingAIassistantsto

switchbetweencommunicationmodesandgeneratemultimodaloutputs.

AgenticAIispositionedattheheartofthenextgenerationofuserinterfaces.AIsystems

8

arecapableofdecision-makingandtaskmanagementbypredictinguserneedsand

proactivelyexecutingcomplexworkflowswithindevicesandapplications.QualcommTechnologies’emphasisonefficient,real-timeAIprocessingallowstheseagentsto

functioncontinuouslyandsecurelywithinthedevices,whilerelyinguponapersonal

knowledgegraphthataccuratelyde?nestheuser’spreferencesandneeds,withoutanyclouddependency.Overtime,theseadvancementsarelayingthegroundworkforAItobecometheprimaryUI,withnaturallanguageandimage,videoandgesture-based

interactionssimplifyinghowpeopleengagewithtechnology.

Lookingahead,QualcommTechnologiesisalsopositionedfortheeraofembodiedAI,inwhichAIcapabilitiesareintegratedintorobotics.Byleveragingitsexpertiseininferenceoptimization,QualcommTechnologiesaimstopowerreal-timedecision-makingfor

robots,dronesandotherautonomousdevices,enablingpreciseinteractionsindynamic,real-worldenvironments.

WhilenumerousAImodelsaretrainedinthecloud,distilledsmallermodelsareavailableforoperationandrunondevicesoftenwithinweeksordays.Forexample,withinlessthanaweek,DeepSeekR1-distilledmodelswererunningon

PCs

and

smartphones

poweredbySnapdragon?platforms.

Deployinginferencewithindevicesaddressesimmediacythroughreducedlatency,

enhancesprivacy,reliesonlocaldatatoprovideadditionalcontextandenables

continuousfunctionalityofAIfeaturesandapplications.Italsoreducescostsforusersand/ordevelopersbyavoidingfeesassociatedwithcloudinferenceservices.AllofthiscreatesincentivesforsoftwareandserviceproviderstodeployAIinferenceattheedge.

QualcommissettobealeaderintheAIinferenceera

Asaleaderinon-deviceAI,QualcommTechnologiesisstrategicallypositionedtoadvancetheAIinferenceerawithitsindustry-leadinghardwareandsoftwaresolutionsforedge

devices.Thesesolutionsencompassbillionsofsmartphones,automobiles,XRheadsetsandglasses,PCs,industrialIoTdevices,andmore.

QualcommTechnologieshasalonghistoryofdevelopingcustomCPUs,NPUs,GPUsandlow-powersubsystems,which,whencombinedwithexpertiseinpackagingandthermal

design,formthefoundationofitsindustry-leadingsystem-on-chip(SoC)products.

TheseSoCsdeliverhigh-performance,energy-efficientAIinferencedirectlyon-device.Bytightlyintegratingthesecores,QualcommTechnologies’platformscanhandlecomplexAItaskswhilemaintainingbatterylifeandoverallpowerefficiency—criticalforedgeuse

cases.

TounlockthefullpotentialofAIonitsplatforms,QualcommTechnologieshasbuiltarobustAIsoftwarestackdesignedtoempowersoftwaredevelopers.TheQualcommAI

9

Stackincludeslibraries,SDKs,andoptimizationtoolsthatstreamlinemodeldeploymentandenhanceperformance.DeveloperscanleveragetheseresourcestoefficientlyadaptmodelsforQualcommplatforms,reducingtime-to-marketforAI-poweredapplications.QualcommTechnologies’developer-focusedapproachacceleratesinnovationby

simplifyingtheintegrationofcutting-edgeAIfeaturesintoconsumerandenterpriseproducts.

Lastly,thecompany’scollaborationwithAImodelmakersacrosstheglobeandits

provisionofservicesliketheQualcommAIHubarecentraltoitsstrategyforscalingAI

acrossindustries.OntheQualcommAIHub,inthreesimplesteps,adevelopercan1)pickamodelorbringtheirownmodelorcreateamodelbasedontheirdata;2)pickany

frameworkandruntime,writeandtesttheirAIappsonacloud-basedphysicaldevice

farm;and3)usetoolstodeploytheirappscommercially.TheQualcommAIHubsupportsmajorlargelanguageandmultimodalmodel(LLM,LMM)families,allowingdeveloperstodeploy,optimize,andmanageinferenceondevicespoweredbyQualcommplatforms.

Withfeatureslikepre-optimizedmodellibrariesandsupportforcustommodel

optimizationandintegration,QualcommTechnologiesenablesrapiddevelopmentcycleswhileenhancingcompatibilitywithdiverseAIecosystems.Thiscollaborativeapproach

strengthensQualcommTechnologies’positionasaleaderinenablingscalable,real-timeAIapplications.

Expandingacrossallkeyedgesegments

QualcommTechnologiesuseson-deviceAItosupportmanyindustries,unlocking

businessvalueandsupportingnewuserexperiences,allenabledbyenhanced

performance,efficiency,responsivenessandprivacybyprocessingAIlocallyondevices.

Mobile

Snapdragonmobileplatforms,suchasthelatestSnapdragon8elite,areadvancingthe

capabilitiesofon-deviceAIbyenablingseveralcutting-edgemultimodalgenerativemodelsandagenticAItooperatenativelyonsmartphones.AIhasenhancedsmartphonefeaturesacrossvariouscategoriessuchascommunicationimprovement,generativeimageeditingtools,personalization,andaccessibility.On-devicegenerativeAIisbeingutilizedto

developmoreintuitive,user-centricfeaturesandtoautomatetasksinmobiledevices.

ThistrendtowardsAI-drivenfunctionalitiesisevidentinthelatest?agshipsmartphonereleasesfrommajormanufacturersutilizingSnapdragonplatforms,includingSamsung,ASUS,Xiaomi,Oppo,Vivo,andHonor.

10

PCs

SnapdragonXSeriesplatformswereinstrumentalindefiningthenewcategoryofAIPCs,

withbest-in-classcustomNPUcoresthatwerebuiltfromground-upforhighperformance,energyefficientgenerativeAIinference.ThisNPUisturbo-chargingWindowsapps,addingnewfeatures,boostingperformance,andenhancingprivacyandbatterylife.Developers

canrungenerativeAIinferenceon-device,offeringcutting-edgeCopilot+PCfeatureswhichdebutedontheSnapdragonXSeries.

Popularthird-partyappslikeZoom,Affinity,DjayPro,CapCut,MoisesLive,and

BlackmagicDesign’sDaVinciResolvetakeadvantageoftheNPUtoofferspecificAI-poweredcapabilitiesonSnapdragonXSeriesplatforms.

Automotive

Snapdragon?DigitalChassis?solutionuseson-deviceAIinitscontext-awareintelligentcockpitsystemdesignedtoenhancevehiclesafetyanddriverexperience.Thissystem

leveragesadvancedcameras,biometricandenvironmentalsensors,andstate-of-the-artmultimodalAInetworkstoprovidereal-timefeedbackandfunctionalitytailoredtothe

driver'sstateandenvironmentalconditions.

Forautomateddrivingandassistancesystems,QualcommTechnologieshasdevelopedanend-to-endarchitecturewhichuseslargetrainingdatasets,fastre-trainingusingreal-worldandAI-augmenteddata,over-the-airupdates,andastate-of-the-artstackincludingmultimodalAImodelsandcausalreasoninginthevehicletohandlemodernautomated

drivingandassistancecomplexities.

Example:LLMAgentlistenstheconversationsinthecabin,onepassengermentionscoffee,

afterafewminsPOIshowsacoffeehouse,LLMAgentproposesastopforcoffee)

Perception-to-IVI

LLMAgent

(AIAssistant)

EnhancedARHUD

Perception

In-VehicleSensors

IntuitiveHMI

Driving

MultimodalLLM

DeepPoints,POI

PlanningProposals&DriverStatus

HumanDriver

Effectivesceneunderstandingandcognition

Decision

ADAS

ADSensors

Transformer

Tokenized

Environment

EnvironmentTokenization

Improvedspatialreasoningandreal-timeplanningcapabilities

Perception-to-ADAS

11

Figure4:Simplifiedin-vehicleAIsystemarchitecturetosupportintelligentcockpitandautonomousand

advanceddrivingassistance.Source:QualcommTechnologies,Jan.2025,

IndustrialIoT

ForindustrialIoTandenterpriseapplications,QualcommTechnologiesrecently

introduceditstheQualcomm?AIOn-PremApplianceSolution,anon-premisesdesktoporwall-mountedhardwaresolution,andQualcomm?AIInferenceSuite,asetofsoftwareandservicesforAIinferencingspanningfromnear-edgetocloud.

ThisedgeAIapproachallowssensitivecustomerdata,fine-tunedmodels,andinference

loadstoremainonpremises,enhancingprivacy,control,energyefficiency,andlow

latency.That’scriticalforAI-enabledbusinessapplicationssuchasintelligentmulti-

lingualsearch,customAIassistantsandagents,codegeneration,andcomputervisionforsecurity,safetyandsitemonitoring.

Networking

QualcommTechnologieshasintroducedanAI-enabledWi-Finetworkingplatform–the

Qualcomm?NetworkingProA7Elite.ThesolutionintegratesWi-Fi7andedgeAItoallow

accesspointsandrouterstorungenerativeAIinferenceonbehalfofconnecteddevicesinthenetwork.Itsupportsinnovativeapplicationsinareaslikesecurity,energymanagement,virtualassistants,andhealthmonitoringbyprocessingdataonthegatewayforenhancedprivacyandrea