TechnicalReport

ModernisationofHigherEducationInstitutions

ProcedureinBuIgaria

InterimEvaIuationandLessonsforeducationprogramming

》OECD

BETTERPOLlcIESFORBETTERLIVES

★

★★

★

★

★

★

★

★★

★

★

Co-fundedby

theEuropeanunion

ModernisationofHigherEducationInstitutionsProcedureinBulgaria

InterimEvaluationandLessonsforEducationalProgramming

ThisworkispublishedundertheresponsibilityoftheSecretary-GeneraloftheOECD.

TheopinionsexpressedandargumentsemployedhereindonotnecessarilyreflecttheofficialviewsoftheMembercountriesoftheOECD.

Thisdocument,aswellasanydataandmapincludedherein,arewithoutprejudicetothestatusoforsovereigntyoveranyterritory,tothedelimitationofinternationalfrontiersandboundariesandtothenameofanyterritory,cityorarea.

ThestatisticaldataforIsraelaresuppliedbyandundertheresponsibilityoftherelevantIsraeliauthorities.TheuseofsuchdatabytheOECDiswithoutprejudicetothestatusoftheGolanHeights,EastJerusalemandIsraelisettlementsintheWestBankunderthetermsofinternationallaw.

ThisdocumentwasproducedwiththefinancialassistanceoftheEuropeanUnion.TheviewsexpressedhereincaninnowaybetakentoreflecttheofficialopinionoftheEuropeanUnion.

Pleasecitethispublicationas:

OECD(2025),ModernisationofHigherEducationInstitutionsProcedureinBulgaria:InterimEvaluationandLessonsfor

EducationalProgramming,OECDPublishing,Paris,

/10.1787/cddcd19e-en

.

Photocredits:Cover?alphaspirit.it/Shutterstock

CorrigendatoOECDpublicationsmaybefoundat:

/en/publications/support/corrigenda.html

.?OECD2025

Attribution4.0International(CCBY4.0)

ThisworkismadeavailableundertheCreativeCommonsAttribution4.0Internationallicence.Byusingthiswork,youaccepttobeboundbythetermsofthislicence(

/licenses/by/4.0/

).

Attribution–youmustcitethework.

Translations–youmustcitetheoriginalwork,identifychangestotheoriginalandaddthefollowingtext:Intheeventofanydiscrepancybetweentheoriginalworkandthetranslation,onlythetextoforiginalworkshouldbeconsideredvalid.

Adaptations–youmustcitetheoriginalworkandaddthefollowingtext:ThisisanadaptationofanoriginalworkbytheOECD.TheopinionsexpressedandargumentsemployedinthisadaptationshouldnotbereportedasrepresentingtheofficialviewsoftheOECDorofitsMembercountries.

Third-partymaterial–thelicencedoesnotapplytothird-partymaterialinthework.Ifusingsuchmaterial,youareresponsibleforobtainingpermissionfromthethirdpartyandforanyclaimsofinfringement.

YoumustnotusetheOECDlogo,visualidentityorcoverimagewithoutexpresspermissionorsuggesttheOECDendorsesyouruseofthework.

AnydisputearisingunderthislicenceshallbesettledbyarbitrationinaccordancewiththePermanentCourtofArbitration(PCA)ArbitrationRules2012.TheseatofarbitrationshallbeParis(France).Thenumberofarbitratorsshallbeone.

l3

MODERNISATIONOFHIGHEREDUCATIONINSTITUTIONSPROCEDUREINBULGARIA?OECD2025

Foreword

ThisreportisanoutputundertheOECD’sagreementtosupporttheEvaluationPlanofBulgaria’sProgramme“Education”2021-2027.

ThomasWekowastheleadevaluatorandauthorofthisreport.ZhasminAvetisyanconductedtheresearchandanalysisandcontributedasco-authortoseveralsectionsofthereport.CassieHagueledandcoordinatedtheprocess,offeringeditorialguidanceandco-authoringsomesectionsofthereport.AdministrativesupportwasprovidedbyDaianaTorresLimaandChristinaMitrakos.SophieLimogescoordinatedthereport’sproduction.

ThereportwaspreparedwithoversightfromElizabethFordham,DeputyHead,andPauloSantiago,HeadofthePolicyAdviceandImplementation(PAI)DivisionwithintheDirectorateforEducationandSkillsattheOECD,undertheleadershipofAndreasSchleicher.TheauthorsaregratefulfortheoversightprovidedbytheOECD’sEducationPolicyCommittee(EDPC).

Externalevaluationexpert,ProfessorManuelSouto-OterofromtheUniversityofBristol,isgratefullyacknowledgedforhisinvaluableinsightsandfeedbackonthedraftreport.

TheauthorswishtothanktheExecutiveAgency“ProgrammeEducation”fortheirassistanceincoordinatingtheevaluationlogisticsandfacilitatingproductivediscussionswithstakeholders.Theirviewofevaluationasalearningtoolwasadrivingforcebehindtheevaluationanditsareasoffocus.

TheprojectwasfundedbytheEuropeanUnionandEuropeanCommissionstaffwhoprovidedinputaregratefullyacknowledged.

TheauthorsextendtheirgratitudetoallindividualsinBulgariawhoparticipatedinthedatacollectionprocess,includingintervieweesandfocusgroupparticipantsfromHigherEducationInstitutions.Theirinputandinsightsshapethefindingsandrecommendationsofthisevaluation.

4l

MODERNISATIONOFHIGHEREDUCATIONINSTITUTIONSPROCEDUREINBULGARIA?OECD2025

Tableofcontents

Foreword3

ExecutiveSummary6

1Overview7

Keyfindingsbyevaluationcriteria7

Promisingpractices9

Lessonsandrecommendations9

Definitions10

References11

Notes11

2Introduction,BackgroundandContext12

Theevaluationcontext12

The"ModernisationofHigherEducationInstitutions"procedure15

ThehighereducationactivitiesandproceduresofProgramme“Education”19

Evaluationapproach21

References25

Notes26

3Mainfindings28

1.Objectivesanddesignofactivities28

2.Selectionofprojects32

3.Implementationofprojectsbybeneficiaries36

4.Monitoring,reporting,andlearning43

5.Outcomes,sustainability,andEUaddedvalue45

References52

Notes54

4Implicationsforcurrentandfutureeducationprogramming55

Conclusions55

Promisingpractices56

Lessonsandrecommendations56

Notes61

FIGURES

Figure2.1.BulgariahasahigherqualificationsmismatchthanmanyotherEUcountries14

l5

MODERNISATIONOFHIGHEREDUCATIONINSTITUTIONSPROCEDUREINBULGARIA?OECD2025

Figure2.2.CreditmobilityamongBulgarianundergraduatestudentsremainslow15

Figure2.3.OverviewoftheModernisationactivity,includingprojectparticipants,activities,andrequirements17

Figure2.4.75%ofthetechnicalandfinancialevaluationisbasedoncriteriausingpositionalmeanvaluesand

mathematicalmodels18

Figure3.1.Largeruniversitiesplannedtointroduceupto15newjointprogrammes,whilesmallerHEIsmet

theminimumrequirementof241

Figure3.2.ThenumberofcreditmobilegraduatesunderErasmus+hasdeclinedinrecentyears51

TABLES

Table2.1.“Regionalsignificance”criterionandfulfilmentrequirements19

Table2.2.PlannedandongoinghighereducationactivitiesunderPE(2021-2027)20

Table2.3.Evaluationcriteriaanddifferentstagesoftheprocedure22

Table3.1.HEIsareawardedoneoffourratings:Gold,Silver,Bronze,orRequiresImprovement,basedonthe

evidencetheysubmitandtheirperformanceagainstnationalindicators49

BOXES

Box2.1.Exampleofacriterionusedtoevaluateandselectprojectproposals19

Box2.2.PlannedactivitiesundertheStrengtheningtheCompetenceApproachinHigherEducationoperation21

Box3.1.Multi-institutionalpartnershipsandregionalconsortiainscalinguphighereducationinitiatives30

Box3.2.Internationalpracticesonjointandnewdegrees38

Box3.3.Countrypracticesonincorporatingthecompetencemodelinhighereducation49

6l

MODERNISATIONOFHIGHEREDUCATIONINSTITUTIONSPROCEDUREINBULGARIA?OECD2025

ExecutiveSummary

ThisreportprovidesanevaluationofBulgaria’s“ModernisationofHigherEducationInstitutions”procedure,whichranbetween21August2020and31December2023.Theprocedurewasco-financedbytheEuropeanSocialFund(ESF)andthenationalbudgetandimplementedundertheOperationalProgramme“ScienceandEducationforSmartGrowth”2014-2020(OPSESG).TheprimarygoalofthisreportistoderivelessonsfromthiscompletedactivitytoinformtheimplementationofhighereducationinitiativesundercurrentandfutureEuropeanfundingprogrammes.

Insummary,theevaluationfindsthattheModernisationofHigherEducationInstitutionsprocedure,withatotalfundingofBGN52million(EUR26.6million),wasbroadlyresponsivetokeyneedswithinBulgaria’shighereducationsystemandaimedtodrivemuch-neededchangesinitshighereducationinstitutions(HEIs).Itseffortstodrivechangewerepartiallysuccessful.ItexpandedtheavailabilityofcontinuingprofessionaldevelopmentinHEIs,widenedtheadoptionofdigitaltechnologies,stimulatedtheadoptionofcompetency-basedprogrammes,andencouragedHEIstorecognisethebenefitsofcollaborationincurriculumandteaching.

However,statutoryandfundingprovisionscreatedchallengesforeffectiveimplementation.Fewer,clearerprioritiescouldhavereducedtheadministrativeburdenandbyconcentratinggrantstowardstheachievementoffeweraims,mayhaveledtomoresystemicimpact.Addressingabroadrangeofprioritieswithinoneactivityintroducedcomplexitythroughouttheactivity’slifecycle,fromtheassessmentofapplicationstoperformancereporting.DiligentadherencetoESFadministrativeproceduresensuredtransparencyandimpartiality,butmorein-depthandsustainedinputfromhighereducationexpertsandstakeholdersmighthavepreventedsomeimplementationchallenges.ThereismuchtobelearnedfromtheexperienceoftheModernisationactivitythatcanbeusedtostrengthenthehighereducationactivitiesofbothcurrentandfutureeducationprogramminginBulgaria.

|7

MODERNISATIONOFHIGHEREDUCATIONINSTITUTIONSPROCEDUREINBULGARIA?OECD2025

1Overview

Keyfindingsbyevaluationcriteria

Thissectionprovidesthekeymessagesofthisreportundertheevaluationcriteriaofrelevance,coherence,effectiveness,efficiency,sustainability,andEUaddedvalue.

Relevance

?ThemainaimsoftheModernisationactivity—advancingdigitalisation,competency-basededucation,internationalisation,collaborationinthedevelopmentofjointdegreeprogrammes,andtheprofessionaldevelopmentofacademicstaff—werecloselyalignedwithkeyobjectivesofBulgaria’sStrategyfortheDevelopmentofHigherEducationinBulgaria2021-2030andbuiltonothernational(e.g.theNationalprogrammefordevelopment“Bulgaria2020”)andEuropeaninitiatives.1Forexample,Modernisationaimedtocontributetothebroadergoalofestablishingacompetency-basedapproachinhighereducation2(MinistryofEducationandScience,2021[1]),helpingtodevelopcompetencyprofilesforeachspecialityandfieldofstudythatreflectthedemandsofdigitaltransformation,mobility,andadynamiclabourmarket.Italsopromotedkeycompetencies,suchasforeignlanguageproficiencyanddigitalliteracyamongacademicstaff,andaimedtostrengthengraduates’abilitiesincriticalthinking,teamwork,leadership,andproblem-solving-.Inaddition,theproceduresupportedtheobjectiveofimprovingteachingandlearningthroughtheuseofdigitaltechnologiesandstrengtheninglinksacrossdisciplines3,includingthedevelopmentofhybriddisciplinesthatbringtogetherexpertisefrommultipleacademicfields.Italsoaimedtoreinforceuniversity-businesscooperation4,particularlythroughindustryinvolvementincurriculumdevelopmentandcontributedtotheexpansionofdigitallearning,e-learningplatforms,andinteractiveeducationalresources.Whilerelativelysmallinscale,representingonly3%oftotalpublicHEinstitutionalrevenues,ModernisationcontributedtomanyoftheobjectivessetintheStrategy5.

Coherence

?TheManagingAuthoritydevelopedtheModernisationactivitywithcarefuladherencetoEuropeanSocialFund(ESF)andnationalproceduralguidelines,asoutlinedindecreesissuedbytheBulgarianCouncilofMinistersandapplicableacrossgovernment.Theseprocessesensuredstrongtransparencyandfairnessinapplication,evaluation,andprojectselection.Whiletheplanninganddesignoftheactivityincludedformalpublicconsultations,therewerelimitedopportunitiestoincorporatespecialisedhighereducationexpertisebeyondtheseprocesses.Asaresult,theprocedurecouldhavebenefitedfrommoreexpertadvice,particularlyaroundtheconditions,constraints,andopportunitiesthatwouldsurroundtheimplementationofModernisationactivities.TherewassomeincoherencebetweenBulgaria’spolicyframeworkandkeyactivitiesoftheModernisationactivity,andmoreexpertadvicemayhavepreventedboththisincoherenceandtheresultingdifficultiesintheimplementationofkeyprojectactivities.

8|

MODERNISATIONOFHIGHEREDUCATIONINSTITUTIONSPROCEDUREINBULGARIA?OECD2025

Effectivenessandefficiency

?TheModernisationprocedureincludedabroadandheterogenoussetofprioritiesandprojectactivities.Itswidescope,combinedwiththeManagingAuthority’scommitmenttoanimpartialprocessofassessingandselectingproposals,ledtotheadoptionofamethodologythatreliedonquantitativeindicatorsandmathematicalformulaetoassessproposals.Thiseffectivelyminimisedopportunitiesforpartiality.However,therelianceonmostlyquantitativeindicatorsweakenedtheconnectionbetweentheassessmentprocessandtheprocedure’sbroaderquality-orientatedgoals.Incorporatingmorequalitativeindicatorscouldhavefacilitatedabroaderassessmentoftheoveralllogicandqualityofprojectproposals.

?Theactivitiestowhichbeneficiariescommittedthemselvesintheirprojectproposalswere,insomecases,slowertobeimplementedthananticipatedandsomeoftheplannedactivities(e.g.mobility)wereeventuallycarriedoutbelowinitiallyplannedlevels.BeneficiariesunanimouslyreportedthatprojectimplementationwasaidedbytheattentivesupportofManagingAuthoritystaff,andthatimplementationproblemsaroseinsteadfromimpedimentsposedbythelegalandbudgetframeworkwithinwhichtheyworked,aswellasbythedifficultyofadaptingtheirprojectstonewandunanticipatedchallengesandopportunities.Inaddition,theCOVID-19pandemiclimitedtheabilityofbeneficiariestoimplementplannedactivitiesasoriginallyenvisioned,constrainedmobilityandface-to-facecollaboration,andnecessitatedadjustmentstoworkingmethodswhichposedadditionalchallengesforbothplanningandexecutionoftheModernisationactivities.

?MonitoringdatawerecollectedtoensurethatthefundsprovidedtobeneficiarieswerecorrectlyspentandtomonitortheachievementofperformanceindicatorsagreedbetweentheManagingAuthorityandbeneficiaries.Performancedatameasuredlevelsofactivity,suchasthenumberofpersonsengagedintraining,orthepreliminaryresultofactivities(coursecompletion,advancementtofurthertrainingetc.).However,planswerenotmadeforthecreation,collection,orretentionofdataaboutthebroaderoutcomesofprojectactivitiestobeusedforevaluation,suchassurveysoftrainingparticipantsorassessmentsoflearninggains.ThismakesitmoredifficulttoevaluatetheoutcomesoftheModernisationprocedure.

?Itispossibletoreachsometentativeassessmentsoftheprocedurebasedoninterviewswithhighereducationprojectmanagersandacademicstaffwhoimplementedprojects.TogethertheircommentssuggestModernisationwasonlypartiallysuccessfulinachievingitsmostambitiousaimoftransformingHEIsthroughcollaborationinthedevelopmentofdegreeprogrammesandtheadoptionofcompetency-basedteachingandlearning.Incontrast,theModernisationactivitiesachievedtheirfullestsuccesswherethecountry’spolicylandscapeposednoimpedimentstotheirimplementation.Forexample,whereactivitieshelpedtoadvanceinitiativesalreadyunderwayinHEIs,suchasdigitalisationofeducationalcontentandintegrationofthecloudenvironmentorprovidedHEIswithanopportunitytoofferservicesandsupportliketrainingandmobilityprogrammes,whichareoftenunder-providedrelativetodemandfromfaculty.

Sustainability/EUaddedvalue

?ThesustainabilityofsomeModernisationactivitiesremainsunknown.HEIsdeveloped95newjointprogrammesbutthereissomeuncertaintyabouthowlongsomeofthesearelikelytocontinueduetolegal,fundingandmotivationconstraints.Whilesupportforprofessionaldevelopmentandmobilitybroughtmeaningfulbenefits,particularlyforsmalleruniversities,furtheradaptationmaybeneededtoensuretheirlongevity.However,evenwhenactivitiesdidnotcontinue,theycontributedtopositiveshiftsincollaborationacrossinstitutions,andEUfundingplayedakeyroleinenablingthescaleofimplementationforcertaininitiatives,suchascompetency-basedandjointprogrammes.

|9

MODERNISATIONOFHIGHEREDUCATIONINSTITUTIONSPROCEDUREINBULGARIA?OECD2025

Promisingpractices

Theproceduredemonstratedseveralpromisingpracticesthatcouldbebuiltuponinfutureinitiatives.

Publicparticipationandfeedback:Theprocedureprovidedampleopportunitiestosubmitpubliccommentsonthemethodologyandcriteriaforselectionofbeneficiaries,aswellastheguidelinesforapplicants,withdetailedwrittenfeedbackprovidedtoallcommenters.

Expandedprofessionaldevelopmentforacademicstaff:Therewasanimportantexpansionofprofessionaldevelopmentopportunitiesforacademicsthatfocusedonenhancinglanguageanddigitalskills,beneficialtotheirresponsibilitiesasteachersandresearchers.Theseskillsarecrucialinanincreasinglydigitalteachingandlearningenvironment.

Investmentsindigitalinfrastructure:Theproceduresupportedinvestmentsintechnologiesrangingfromcloudcomputingtotheacquisitionofspecialisedsoftware.ThishelpedtoexpandthecapacityofHEIsfordigitally-enhancedteachingandresearch,layingsomefoundationsforlonger-termdigitaltransformation.

Inter-institutionalcollaboration:TheproceduremadeanimportantstartindemonstratingthefeasibilityandmutualbenefitofcollaborationbetweenHEIsinsharingresponsibilityforcurriculumdevelopmentandteaching.Thisapproachallowedinstitutionstoformpartnershipsthroughwhichtheycanshareexpertise,resources,andbestpractices.

Competency-basedcurriculumdesign:Theproceduresupportedmovestowardsacompetency-focusedredesignofcurriculumandpedagogymorealignedwiththeskilldemandsofprofessionalpractice,whichprovedattractivetostudents,instructorsandemployers.

Lessonsandrecommendations

TheexperienceoftheModernisationprocedureisthebasisfortwosetsofrecommendationsandactionstepsattheendofthisreporttofurtherstrengthenproceduresinhighereducation,bothforProgramme“Education”(2021-2027)andfutureeducationalprogrammingandpolicy-makinginBulgaria.

?Thefirstsetofrecommendationsaremadeacrossthesequenceofstepsinvolvedinlaunchingaprocedure,including:(1)identifyingthefocusandscopeoftheactivity;(2)engagingexpertsandstakeholders;(3)identifyingevaluationneeds;(4)developingtheselectioncriteriaandmethodology;(5)establishingasupportiveimplementationenvironment;and(6)learningthroughoutthelifeoftheactivity.

?ThesecondandfinalsetofrecommendationsarerelatedspecificallytotwoupcominghighereducationactivitieswithinthecurrentESF+fundedProgramme"Education"(2021-2027).Thereportnotesthatthe“AccesstoHigherEducation”procedure(aspresentlyplanned)isacomparativelysimpleactivitywithanarrowscopeofwell-understoodtasks,andlikelytobewell-servedbyacompetitiveprocedure.Conversely,itsuggeststhattheupcomingPE“StrengtheningtheCompetenceApproachinHigherEducation”(Competence)activitymaybemoreappropriateforadirectawardprocedure,owingtoitsscopeandcomplexity.However,bothactivitieswouldbenefitfromincorporatingkeylessonsfromtheModernisationprocedure,including,amongothers,improvingthedepthandspanofconsultation,reducingthescopeoftasksincludedwithintheprocedure,anddesigningtheactivitywithevaluationinmind.

10l

MODERNISATIONOFHIGHEREDUCATIONINSTITUTIONSPROCEDUREINBULGARIA?OECD2025

Definitions

Thissectionoutlineshowanumberofkeytermsareusedinthisreport:

Programme–AnOperationalProgramme(OP)isadocumentthatoutlineshowamemberstatewilluseEuropeanStructuralandInvestmentFunds(ESIF)resourceswithinaspecificprogrammingperiod.ItservesasadetailedplanforimplementingthestrategicobjectivessetforthinthebroaderNationalStrategicReferenceFramework(NSRF)andthePartnershipAgreementwiththeEuropeanCommission.UnderitspartnershipagreementwiththeEuropeanCommission,BulgariaisimplementingESF+viathreenationalprogrammeswithatotalofEUR2.6billioninESF+(EUR3.14billionwithnationalco-funding).

Programmingperiod–ThedurationofanagreedEuropeanSocialFund(ESF)programmetypicallyalignswiththeEU'smultiannualfinancialframework(MFF)andcanvarydependingonthespecificprogrammeanditsobjectives.OperationalProgrammestypicallyhaveadurationofsevenyears.Thecurrentprogrammingcyclecovers2021to2027,followingtheEU'sMFF.

Programme“Education”2021-2027–Programme“Education”(PE)istheProgrammefortheperiod2021-2027agreedbetweentheBulgarianGovernmentandtheEuropeanCommission,withabudgetofnearlyEUR1billion(ESF+andnationalco-funding).Itwasprecededbythe2014-2020OperationalProgramme“ScienceandEducationforSmartGrowth”(OPSESG).

Procedure–OperationalProgrammesareimplementedthroughgrantprocedures,i.e.,protocolsgoverningprojectapplicationandselectionfinancialmanagement,monitoring,andevaluation.Anannouncement