AlgorithmsIlluminatedPart4:AlgorithmsforNP-HardProblemsTimRoughgarden
c 2020byTimRoughgardenAllrightsreserved.Noportionofthisbookmaybereproducedinanyformwithoutpermissionfromthepublisher,exceptaspermittedbyU.S.copyrightlaw.FirstEditionCoverimage:NormanZammitt,BlueBurning,1982acryliconcanvas;84x1681/2in.(213.36x427.99cm)SanFranciscoMuseumofModernArt,GiftofJudgeandMrs.WilliamJ.Lasarowc EstateofNormanZammittPhotograph:KatherineDuTielISBN:978-0-9992829-6-0(Paperback)ISBN:978-0-9992829-7-7(ebook)LibraryofCongressControlNumber:2017914282SoundlikeyourselfPublishing,LLCNewYork,NYsoundlikeyourselfpublishing@gmail.comwww.algorithmsilluminated.org
ContentsPrefacev19WhatIsNP-Hardness?119.1MSTvs.TSP:AnAlgorithmicMystery119.2PossibleLevelsofExpertise719.3EasyandHardProblems919.4AlgorithmicStrategiesforNP-HardProblems1719.5ProvingNP-Hardness:ASimpleRecipe2319.6RookieMistakesandAcceptableInaccuracies33Problems3720CompromisingonCorrectness:EfficientInexactAlgorithms4120.1MakespanMinimization4120.2MaximumCoverage55*20.3InfluenceMaximization6620.4The2-OPTHeuristicAlgorithmfortheTSP7520.5PrinciplesofLocalSearch82Problems9421CompromisingonSpeed:ExactInefficientAlgorithms10521.1TheBellman-Held-KarpAlgorithmfortheTSP105*21.2FindingLongPathsbyColorCoding11421.3Problem-SpecificAlgorithmsvs.MagicBoxes12721.4MixedIntegerProgrammingSolvers12921.5SatisfiabilitySolvers134Problems140iii
ivContents22ProvingProblemsNP-Hard14822.1ReductionsRevisited14822.23-SATandtheCook-LevinTheorem15022.3TheBigPicture15222.4ATemplateforReductions15622.5IndependentSetIsNP-Hard158*22.6DirectedHamiltonianPathIsNP-Hard16322.7TheTSPIsNP-Hard16922.8SubsetSumIsNP-Hard172Problems17823P,NP,andAllThat182*23.1AmassingEvidenceofIntractability183*23.2Decision,Search,andOptimization185*23.3NP:ProblemswithEasilyRecognizedSolutions186*23.4TheP6=NPConjecture192*23.5TheExponentialTimeHypothesis195*23.6NP-Completeness200Problems20524CaseStudy:TheFCCIncentiveAuction20824.1RepurposingWirelessSpectrum20824.2GreedyHeuristicsforBuyingBackLicenses21124.3FeasibilityChecking21924.4ImplementationasaDescendingClockAuction22624.5TheFinalOutcome231Problems233Epilogue:AFieldGuidetoAlgorithmDesign236HintsandSolutions239Index251
PrefaceThisbookisthefourthinaseriesbasedonmyonlinealgorithmscoursesthathavebeenrunningregularlysince2012,whichinturnarebasedonanundergraduatecoursethatItaughtmanytimesatStanfordUniversity.Part4assumesatleastsomefamiliaritywithasymptoticanalysisandbig-Onotation,graphsearchandshortest-pathalgorithms,greedyalgorithms,anddynamicprogramming(allcoveredinParts1–3).WhatWe’llCoverinThisBookAlgorithmsIlluminated,Part4isallaboutNP-hardproblemsandwhattodoaboutthem.AlgorithmictoolsfortacklingNP-hardproblems.Manyreal-worldproblemsare“NP-hard”andappearunsolvablebythetypesofalways-correctandalways-fastalgorithmsthathavestarredinthefirstthreepartsofthisbookseries.WhenanNP-hardproblemshowsupinyourownwork,youmustcompromiseoneithercorrect-nessorspeed.We’llseetechniquesold(likegreedyalgorithms)andnew(likelocalsearch)fordevisingfastheuristicalgorithmsthatare“approximatelycorrect,”withapplicationstoscheduling,influencemaximizationinsocialnetworks,andthetravelingsalesmanproblem.We’llalsocovertechniquesold(likedynamicprogramming)andnew(likeMIPandSATsolvers)fordevelopingcorrectalgorithmsthatimprovedramaticallyonexhaustivesearch;applicationshereincludethetravelingsalesmanproblem(again),findingsignalingpathwaysinbiologicalnetworks,andtelevisionstationrepackinginarecentandhigh-stakesspectrumauctionintheUnitedStates.RecognizingNP-hardproblems.ThisbookwillalsotrainyoutoquicklyrecognizeanNP-hardproblemsothatyoudon’tinadver-v
viPrefacetentlywastetimetryingtodesignatoo-good-to-be-truealgorithmforit.You’llacquirefamiliaritywithmanyfamousandbasicNP-hardproblems,rangingfromsatisfiabilitytographcoloringtotheHamil-tonianpathproblem.Throughpractice,you’lllearnthetricksofthetradeinprovingproblemsNP-hardviareductions.Foramoredetailedlookintothebook’scontents,checkoutthe“Upshot”sectionsthatconcludeeachchapterandhighlightthemostimportantpoints.The“FieldGuidetoAlgorithmDesign”onpage236providesabird’s-eyeviewofhowthetopicsofthisbookfitintothebiggeralgorithmicpicture.Thestarredsectionsofthebookarethemostadvancedones.Thetime-constrainedreadercanskipthesesectionsonafirstreadingwithoutanylossofcontinuity.Topicscoveredinthefirstthreeparts.AlgorithmsIllumi-nated,Part1coversasymptoticnotation(big-Onotationanditsclosecousins),divide-and-conqueralgorithmsandthemastermethod,randomizedQuickSortanditsanalysis,andlinear-timeselectionalgo-rithms.Part2isaboutdatastructures(heaps,balancedsearchtrees,hashtables,bloomfilters),graphprimitives(breadth-anddepth-firstsearch,connectivity,shortestpaths),andtheirapplications(rang-ingfromdeduplicationtosocialnetworkanalysis).Part3focusesongreedyalgorithms(scheduling,minimumspanningtrees,cluster-ing,Huffmancodes)anddynamicprogramming(knapsack,sequencealignment,shortestpaths,optimalsearchtrees).SkillsYou’llLearnFromThisBookSeriesMasteringalgorithmstakestimeandeffort.Whybother?Becomeabetterprogrammer.You’lllearnseveralblazinglyfastsubroutinesforprocessingdataaswellasseveralusefuldatastructuresfororganizingdatathatyoucandeploydirectlyinyourownprograms.Implementingandusingthesealgorithmswillstretchandimproveyourprogrammingskills.You’llalsolearngeneralalgorithmdesignparadigmsthatarerelevanttomanydifferentproblemsacrossdifferentdomains,aswellastoolsforpredictingtheperformanceofsuchalgorithms.These“algorithmicdesignpatterns”canhelpyoucomeupwithnewalgorithmsforproblemsthatariseinyourownwork.