Details | Last modification | View Log | RSS feed
| Rev | Author | Line No. | Line | 
|---|---|---|---|
| 14 | pmbaty | 1 | //===- CGSCCPassManager.h - Call graph pass management ----------*- C++ -*-===// | 
| 2 | // | ||
| 3 | // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. | ||
| 4 | // See https://llvm.org/LICENSE.txt for license information. | ||
| 5 | // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception | ||
| 6 | // | ||
| 7 | //===----------------------------------------------------------------------===// | ||
| 8 | /// \file | ||
| 9 | /// | ||
| 10 | /// This header provides classes for managing passes over SCCs of the call | ||
| 11 | /// graph. These passes form an important component of LLVM's interprocedural | ||
| 12 | /// optimizations. Because they operate on the SCCs of the call graph, and they | ||
| 13 | /// traverse the graph in post-order, they can effectively do pair-wise | ||
| 14 | /// interprocedural optimizations for all call edges in the program while | ||
| 15 | /// incrementally refining it and improving the context of these pair-wise | ||
| 16 | /// optimizations. At each call site edge, the callee has already been | ||
| 17 | /// optimized as much as is possible. This in turn allows very accurate | ||
| 18 | /// analysis of it for IPO. | ||
| 19 | /// | ||
| 20 | /// A secondary more general goal is to be able to isolate optimization on | ||
| 21 | /// unrelated parts of the IR module. This is useful to ensure our | ||
| 22 | /// optimizations are principled and don't miss oportunities where refinement | ||
| 23 | /// of one part of the module influences transformations in another part of the | ||
| 24 | /// module. But this is also useful if we want to parallelize the optimizations | ||
| 25 | /// across common large module graph shapes which tend to be very wide and have | ||
| 26 | /// large regions of unrelated cliques. | ||
| 27 | /// | ||
| 28 | /// To satisfy these goals, we use the LazyCallGraph which provides two graphs | ||
| 29 | /// nested inside each other (and built lazily from the bottom-up): the call | ||
| 30 | /// graph proper, and a reference graph. The reference graph is super set of | ||
| 31 | /// the call graph and is a conservative approximation of what could through | ||
| 32 | /// scalar or CGSCC transforms *become* the call graph. Using this allows us to | ||
| 33 | /// ensure we optimize functions prior to them being introduced into the call | ||
| 34 | /// graph by devirtualization or other technique, and thus ensures that | ||
| 35 | /// subsequent pair-wise interprocedural optimizations observe the optimized | ||
| 36 | /// form of these functions. The (potentially transitive) reference | ||
| 37 | /// reachability used by the reference graph is a conservative approximation | ||
| 38 | /// that still allows us to have independent regions of the graph. | ||
| 39 | /// | ||
| 40 | /// FIXME: There is one major drawback of the reference graph: in its naive | ||
| 41 | /// form it is quadratic because it contains a distinct edge for each | ||
| 42 | /// (potentially indirect) reference, even if are all through some common | ||
| 43 | /// global table of function pointers. This can be fixed in a number of ways | ||
| 44 | /// that essentially preserve enough of the normalization. While it isn't | ||
| 45 | /// expected to completely preclude the usability of this, it will need to be | ||
| 46 | /// addressed. | ||
| 47 | /// | ||
| 48 | /// | ||
| 49 | /// All of these issues are made substantially more complex in the face of | ||
| 50 | /// mutations to the call graph while optimization passes are being run. When | ||
| 51 | /// mutations to the call graph occur we want to achieve two different things: | ||
| 52 | /// | ||
| 53 | /// - We need to update the call graph in-flight and invalidate analyses | ||
| 54 | ///   cached on entities in the graph. Because of the cache-based analysis | ||
| 55 | ///   design of the pass manager, it is essential to have stable identities for | ||
| 56 | ///   the elements of the IR that passes traverse, and to invalidate any | ||
| 57 | ///   analyses cached on these elements as the mutations take place. | ||
| 58 | /// | ||
| 59 | /// - We want to preserve the incremental and post-order traversal of the | ||
| 60 | ///   graph even as it is refined and mutated. This means we want optimization | ||
| 61 | ///   to observe the most refined form of the call graph and to do so in | ||
| 62 | ///   post-order. | ||
| 63 | /// | ||
| 64 | /// To address this, the CGSCC manager uses both worklists that can be expanded | ||
| 65 | /// by passes which transform the IR, and provides invalidation tests to skip | ||
| 66 | /// entries that become dead. This extra data is provided to every SCC pass so | ||
| 67 | /// that it can carefully update the manager's traversal as the call graph | ||
| 68 | /// mutates. | ||
| 69 | /// | ||
| 70 | /// We also provide support for running function passes within the CGSCC walk, | ||
| 71 | /// and there we provide automatic update of the call graph including of the | ||
| 72 | /// pass manager to reflect call graph changes that fall out naturally as part | ||
| 73 | /// of scalar transformations. | ||
| 74 | /// | ||
| 75 | /// The patterns used to ensure the goals of post-order visitation of the fully | ||
| 76 | /// refined graph: | ||
| 77 | /// | ||
| 78 | /// 1) Sink toward the "bottom" as the graph is refined. This means that any | ||
| 79 | ///    iteration continues in some valid post-order sequence after the mutation | ||
| 80 | ///    has altered the structure. | ||
| 81 | /// | ||
| 82 | /// 2) Enqueue in post-order, including the current entity. If the current | ||
| 83 | ///    entity's shape changes, it and everything after it in post-order needs | ||
| 84 | ///    to be visited to observe that shape. | ||
| 85 | /// | ||
| 86 | //===----------------------------------------------------------------------===// | ||
| 87 | |||
| 88 | #ifndef LLVM_ANALYSIS_CGSCCPASSMANAGER_H | ||
| 89 | #define LLVM_ANALYSIS_CGSCCPASSMANAGER_H | ||
| 90 | |||
| 91 | #include "llvm/ADT/MapVector.h" | ||
| 92 | #include "llvm/Analysis/LazyCallGraph.h" | ||
| 93 | #include "llvm/IR/PassManager.h" | ||
| 94 | #include "llvm/IR/ValueHandle.h" | ||
| 95 | #include "llvm/Support/raw_ostream.h" | ||
| 96 | #include <cassert> | ||
| 97 | #include <utility> | ||
| 98 | |||
| 99 | namespace llvm { | ||
| 100 | |||
| 101 | class Function; | ||
| 102 | class Value; | ||
| 103 | template <typename T, unsigned int N> class SmallPriorityWorklist; | ||
| 104 | struct CGSCCUpdateResult; | ||
| 105 | |||
| 106 | class Module; | ||
| 107 | |||
| 108 | // Allow debug logging in this inline function. | ||
| 109 | #define DEBUG_TYPE "cgscc" | ||
| 110 | |||
| 111 | /// Extern template declaration for the analysis set for this IR unit. | ||
| 112 | extern template class AllAnalysesOn<LazyCallGraph::SCC>; | ||
| 113 | |||
| 114 | extern template class AnalysisManager<LazyCallGraph::SCC, LazyCallGraph &>; | ||
| 115 | |||
| 116 | /// The CGSCC analysis manager. | ||
| 117 | /// | ||
| 118 | /// See the documentation for the AnalysisManager template for detail | ||
| 119 | /// documentation. This type serves as a convenient way to refer to this | ||
| 120 | /// construct in the adaptors and proxies used to integrate this into the larger | ||
| 121 | /// pass manager infrastructure. | ||
| 122 | using CGSCCAnalysisManager = | ||
| 123 | AnalysisManager<LazyCallGraph::SCC, LazyCallGraph &>; | ||
| 124 | |||
| 125 | // Explicit specialization and instantiation declarations for the pass manager. | ||
| 126 | // See the comments on the definition of the specialization for details on how | ||
| 127 | // it differs from the primary template. | ||
| 128 | template <> | ||
| 129 | PreservedAnalyses | ||
| 130 | PassManager<LazyCallGraph::SCC, CGSCCAnalysisManager, LazyCallGraph &, | ||
| 131 | CGSCCUpdateResult &>::run(LazyCallGraph::SCC &InitialC, | ||
| 132 |                                       CGSCCAnalysisManager &AM, | ||
| 133 | LazyCallGraph &G, CGSCCUpdateResult &UR); | ||
| 134 | extern template class PassManager<LazyCallGraph::SCC, CGSCCAnalysisManager, | ||
| 135 | LazyCallGraph &, CGSCCUpdateResult &>; | ||
| 136 | |||
| 137 | /// The CGSCC pass manager. | ||
| 138 | /// | ||
| 139 | /// See the documentation for the PassManager template for details. It runs | ||
| 140 | /// a sequence of SCC passes over each SCC that the manager is run over. This | ||
| 141 | /// type serves as a convenient way to refer to this construct. | ||
| 142 | using CGSCCPassManager = | ||
| 143 | PassManager<LazyCallGraph::SCC, CGSCCAnalysisManager, LazyCallGraph &, | ||
| 144 | CGSCCUpdateResult &>; | ||
| 145 | |||
| 146 | /// An explicit specialization of the require analysis template pass. | ||
| 147 | template <typename AnalysisT> | ||
| 148 | struct RequireAnalysisPass<AnalysisT, LazyCallGraph::SCC, CGSCCAnalysisManager, | ||
| 149 | LazyCallGraph &, CGSCCUpdateResult &> | ||
| 150 | : PassInfoMixin<RequireAnalysisPass<AnalysisT, LazyCallGraph::SCC, | ||
| 151 |                                         CGSCCAnalysisManager, LazyCallGraph &, | ||
| 152 | CGSCCUpdateResult &>> { | ||
| 153 | PreservedAnalyses run(LazyCallGraph::SCC &C, CGSCCAnalysisManager &AM, | ||
| 154 | LazyCallGraph &CG, CGSCCUpdateResult &) { | ||
| 155 | (void)AM.template getResult<AnalysisT>(C, CG); | ||
| 156 | return PreservedAnalyses::all(); | ||
| 157 |   } | ||
| 158 | void printPipeline(raw_ostream &OS, | ||
| 159 | function_ref<StringRef(StringRef)> MapClassName2PassName) { | ||
| 160 | auto ClassName = AnalysisT::name(); | ||
| 161 | auto PassName = MapClassName2PassName(ClassName); | ||
| 162 | OS << "require<" << PassName << ">"; | ||
| 163 |   } | ||
| 164 | }; | ||
| 165 | |||
| 166 | /// A proxy from a \c CGSCCAnalysisManager to a \c Module. | ||
| 167 | using CGSCCAnalysisManagerModuleProxy = | ||
| 168 | InnerAnalysisManagerProxy<CGSCCAnalysisManager, Module>; | ||
| 169 | |||
| 170 | /// We need a specialized result for the \c CGSCCAnalysisManagerModuleProxy so | ||
| 171 | /// it can have access to the call graph in order to walk all the SCCs when | ||
| 172 | /// invalidating things. | ||
| 173 | template <> class CGSCCAnalysisManagerModuleProxy::Result { | ||
| 174 | public: | ||
| 175 | explicit Result(CGSCCAnalysisManager &InnerAM, LazyCallGraph &G) | ||
| 176 | : InnerAM(&InnerAM), G(&G) {} | ||
| 177 | |||
| 178 |   /// Accessor for the analysis manager. | ||
| 179 | CGSCCAnalysisManager &getManager() { return *InnerAM; } | ||
| 180 | |||
| 181 |   /// Handler for invalidation of the Module. | ||
| 182 |   /// | ||
| 183 |   /// If the proxy analysis itself is preserved, then we assume that the set of | ||
| 184 |   /// SCCs in the Module hasn't changed. Thus any pointers to SCCs in the | ||
| 185 |   /// CGSCCAnalysisManager are still valid, and we don't need to call \c clear | ||
| 186 |   /// on the CGSCCAnalysisManager. | ||
| 187 |   /// | ||
| 188 |   /// Regardless of whether this analysis is marked as preserved, all of the | ||
| 189 |   /// analyses in the \c CGSCCAnalysisManager are potentially invalidated based | ||
| 190 |   /// on the set of preserved analyses. | ||
| 191 | bool invalidate(Module &M, const PreservedAnalyses &PA, | ||
| 192 | ModuleAnalysisManager::Invalidator &Inv); | ||
| 193 | |||
| 194 | private: | ||
| 195 | CGSCCAnalysisManager *InnerAM; | ||
| 196 | LazyCallGraph *G; | ||
| 197 | }; | ||
| 198 | |||
| 199 | /// Provide a specialized run method for the \c CGSCCAnalysisManagerModuleProxy | ||
| 200 | /// so it can pass the lazy call graph to the result. | ||
| 201 | template <> | ||
| 202 | CGSCCAnalysisManagerModuleProxy::Result | ||
| 203 | CGSCCAnalysisManagerModuleProxy::run(Module &M, ModuleAnalysisManager &AM); | ||
| 204 | |||
| 205 | // Ensure the \c CGSCCAnalysisManagerModuleProxy is provided as an extern | ||
| 206 | // template. | ||
| 207 | extern template class InnerAnalysisManagerProxy<CGSCCAnalysisManager, Module>; | ||
| 208 | |||
| 209 | extern template class OuterAnalysisManagerProxy< | ||
| 210 | ModuleAnalysisManager, LazyCallGraph::SCC, LazyCallGraph &>; | ||
| 211 | |||
| 212 | /// A proxy from a \c ModuleAnalysisManager to an \c SCC. | ||
| 213 | using ModuleAnalysisManagerCGSCCProxy = | ||
| 214 | OuterAnalysisManagerProxy<ModuleAnalysisManager, LazyCallGraph::SCC, | ||
| 215 | LazyCallGraph &>; | ||
| 216 | |||
| 217 | /// Support structure for SCC passes to communicate updates the call graph back | ||
| 218 | /// to the CGSCC pass manager infrastructure. | ||
| 219 | /// | ||
| 220 | /// The CGSCC pass manager runs SCC passes which are allowed to update the call | ||
| 221 | /// graph and SCC structures. This means the structure the pass manager works | ||
| 222 | /// on is mutating underneath it. In order to support that, there needs to be | ||
| 223 | /// careful communication about the precise nature and ramifications of these | ||
| 224 | /// updates to the pass management infrastructure. | ||
| 225 | /// | ||
| 226 | /// All SCC passes will have to accept a reference to the management layer's | ||
| 227 | /// update result struct and use it to reflect the results of any CG updates | ||
| 228 | /// performed. | ||
| 229 | /// | ||
| 230 | /// Passes which do not change the call graph structure in any way can just | ||
| 231 | /// ignore this argument to their run method. | ||
| 232 | struct CGSCCUpdateResult { | ||
| 233 |   /// Worklist of the RefSCCs queued for processing. | ||
| 234 |   /// | ||
| 235 |   /// When a pass refines the graph and creates new RefSCCs or causes them to | ||
| 236 |   /// have a different shape or set of component SCCs it should add the RefSCCs | ||
| 237 |   /// to this worklist so that we visit them in the refined form. | ||
| 238 |   /// | ||
| 239 |   /// This worklist is in reverse post-order, as we pop off the back in order | ||
| 240 |   /// to observe RefSCCs in post-order. When adding RefSCCs, clients should add | ||
| 241 |   /// them in reverse post-order. | ||
| 242 | SmallPriorityWorklist<LazyCallGraph::RefSCC *, 1> &RCWorklist; | ||
| 243 | |||
| 244 |   /// Worklist of the SCCs queued for processing. | ||
| 245 |   /// | ||
| 246 |   /// When a pass refines the graph and creates new SCCs or causes them to have | ||
| 247 |   /// a different shape or set of component functions it should add the SCCs to | ||
| 248 |   /// this worklist so that we visit them in the refined form. | ||
| 249 |   /// | ||
| 250 |   /// Note that if the SCCs are part of a RefSCC that is added to the \c | ||
| 251 |   /// RCWorklist, they don't need to be added here as visiting the RefSCC will | ||
| 252 |   /// be sufficient to re-visit the SCCs within it. | ||
| 253 |   /// | ||
| 254 |   /// This worklist is in reverse post-order, as we pop off the back in order | ||
| 255 |   /// to observe SCCs in post-order. When adding SCCs, clients should add them | ||
| 256 |   /// in reverse post-order. | ||
| 257 | SmallPriorityWorklist<LazyCallGraph::SCC *, 1> &CWorklist; | ||
| 258 | |||
| 259 |   /// The set of invalidated RefSCCs which should be skipped if they are found | ||
| 260 |   /// in \c RCWorklist. | ||
| 261 |   /// | ||
| 262 |   /// This is used to quickly prune out RefSCCs when they get deleted and | ||
| 263 |   /// happen to already be on the worklist. We use this primarily to avoid | ||
| 264 |   /// scanning the list and removing entries from it. | ||
| 265 | SmallPtrSetImpl<LazyCallGraph::RefSCC *> &InvalidatedRefSCCs; | ||
| 266 | |||
| 267 |   /// The set of invalidated SCCs which should be skipped if they are found | ||
| 268 |   /// in \c CWorklist. | ||
| 269 |   /// | ||
| 270 |   /// This is used to quickly prune out SCCs when they get deleted and happen | ||
| 271 |   /// to already be on the worklist. We use this primarily to avoid scanning | ||
| 272 |   /// the list and removing entries from it. | ||
| 273 | SmallPtrSetImpl<LazyCallGraph::SCC *> &InvalidatedSCCs; | ||
| 274 | |||
| 275 |   /// If non-null, the updated current \c SCC being processed. | ||
| 276 |   /// | ||
| 277 |   /// This is set when a graph refinement takes place and the "current" point | ||
| 278 |   /// in the graph moves "down" or earlier in the post-order walk. This will | ||
| 279 |   /// often cause the "current" SCC to be a newly created SCC object and the | ||
| 280 |   /// old one to be added to the above worklist. When that happens, this | ||
| 281 |   /// pointer is non-null and can be used to continue processing the "top" of | ||
| 282 |   /// the post-order walk. | ||
| 283 | LazyCallGraph::SCC *UpdatedC; | ||
| 284 | |||
| 285 |   /// Preserved analyses across SCCs. | ||
| 286 |   /// | ||
| 287 |   /// We specifically want to allow CGSCC passes to mutate ancestor IR | ||
| 288 |   /// (changing both the CG structure and the function IR itself). However, | ||
| 289 |   /// this means we need to take special care to correctly mark what analyses | ||
| 290 |   /// are preserved *across* SCCs. We have to track this out-of-band here | ||
| 291 |   /// because within the main `PassManager` infrastructure we need to mark | ||
| 292 |   /// everything within an SCC as preserved in order to avoid repeatedly | ||
| 293 |   /// invalidating the same analyses as we unnest pass managers and adaptors. | ||
| 294 |   /// So we track the cross-SCC version of the preserved analyses here from any | ||
| 295 |   /// code that does direct invalidation of SCC analyses, and then use it | ||
| 296 |   /// whenever we move forward in the post-order walk of SCCs before running | ||
| 297 |   /// passes over the new SCC. | ||
| 298 |   PreservedAnalyses CrossSCCPA; | ||
| 299 | |||
| 300 |   /// A hacky area where the inliner can retain history about inlining | ||
| 301 |   /// decisions that mutated the call graph's SCC structure in order to avoid | ||
| 302 |   /// infinite inlining. See the comments in the inliner's CG update logic. | ||
| 303 |   /// | ||
| 304 |   /// FIXME: Keeping this here seems like a big layering issue, we should look | ||
| 305 |   /// for a better technique. | ||
| 306 | SmallDenseSet<std::pair<LazyCallGraph::Node *, LazyCallGraph::SCC *>, 4> | ||
| 307 | &InlinedInternalEdges; | ||
| 308 | |||
| 309 |   /// Weak VHs to keep track of indirect calls for the purposes of detecting | ||
| 310 |   /// devirtualization. | ||
| 311 |   /// | ||
| 312 |   /// This is a map to avoid having duplicate entries. If a Value is | ||
| 313 |   /// deallocated, its corresponding WeakTrackingVH will be nulled out. When | ||
| 314 |   /// checking if a Value is in the map or not, also check if the corresponding | ||
| 315 |   /// WeakTrackingVH is null to avoid issues with a new Value sharing the same | ||
| 316 |   /// address as a deallocated one. | ||
| 317 | SmallMapVector<Value *, WeakTrackingVH, 16> IndirectVHs; | ||
| 318 | }; | ||
| 319 | |||
| 320 | /// The core module pass which does a post-order walk of the SCCs and | ||
| 321 | /// runs a CGSCC pass over each one. | ||
| 322 | /// | ||
| 323 | /// Designed to allow composition of a CGSCCPass(Manager) and | ||
| 324 | /// a ModulePassManager. Note that this pass must be run with a module analysis | ||
| 325 | /// manager as it uses the LazyCallGraph analysis. It will also run the | ||
| 326 | /// \c CGSCCAnalysisManagerModuleProxy analysis prior to running the CGSCC | ||
| 327 | /// pass over the module to enable a \c FunctionAnalysisManager to be used | ||
| 328 | /// within this run safely. | ||
| 329 | class ModuleToPostOrderCGSCCPassAdaptor | ||
| 330 | : public PassInfoMixin<ModuleToPostOrderCGSCCPassAdaptor> { | ||
| 331 | public: | ||
| 332 | using PassConceptT = | ||
| 333 | detail::PassConcept<LazyCallGraph::SCC, CGSCCAnalysisManager, | ||
| 334 | LazyCallGraph &, CGSCCUpdateResult &>; | ||
| 335 | |||
| 336 | explicit ModuleToPostOrderCGSCCPassAdaptor(std::unique_ptr<PassConceptT> Pass) | ||
| 337 | : Pass(std::move(Pass)) {} | ||
| 338 | |||
| 339 | ModuleToPostOrderCGSCCPassAdaptor(ModuleToPostOrderCGSCCPassAdaptor &&Arg) | ||
| 340 | : Pass(std::move(Arg.Pass)) {} | ||
| 341 | |||
| 342 | friend void swap(ModuleToPostOrderCGSCCPassAdaptor &LHS, | ||
| 343 | ModuleToPostOrderCGSCCPassAdaptor &RHS) { | ||
| 344 | std::swap(LHS.Pass, RHS.Pass); | ||
| 345 |   } | ||
| 346 | |||
| 347 |   ModuleToPostOrderCGSCCPassAdaptor & | ||
| 348 | operator=(ModuleToPostOrderCGSCCPassAdaptor RHS) { | ||
| 349 | swap(*this, RHS); | ||
| 350 | return *this; | ||
| 351 |   } | ||
| 352 | |||
| 353 |   /// Runs the CGSCC pass across every SCC in the module. | ||
| 354 | PreservedAnalyses run(Module &M, ModuleAnalysisManager &AM); | ||
| 355 | |||
| 356 | void printPipeline(raw_ostream &OS, | ||
| 357 | function_ref<StringRef(StringRef)> MapClassName2PassName) { | ||
| 358 | OS << "cgscc("; | ||
| 359 | Pass->printPipeline(OS, MapClassName2PassName); | ||
| 360 | OS << ")"; | ||
| 361 |   } | ||
| 362 | |||
| 363 | static bool isRequired() { return true; } | ||
| 364 | |||
| 365 | private: | ||
| 366 | std::unique_ptr<PassConceptT> Pass; | ||
| 367 | }; | ||
| 368 | |||
| 369 | /// A function to deduce a function pass type and wrap it in the | ||
| 370 | /// templated adaptor. | ||
| 371 | template <typename CGSCCPassT> | ||
| 372 | ModuleToPostOrderCGSCCPassAdaptor | ||
| 373 | createModuleToPostOrderCGSCCPassAdaptor(CGSCCPassT &&Pass) { | ||
| 374 | using PassModelT = detail::PassModel<LazyCallGraph::SCC, CGSCCPassT, | ||
| 375 | PreservedAnalyses, CGSCCAnalysisManager, | ||
| 376 | LazyCallGraph &, CGSCCUpdateResult &>; | ||
| 377 |   // Do not use make_unique, it causes too many template instantiations, | ||
| 378 |   // causing terrible compile times. | ||
| 379 | return ModuleToPostOrderCGSCCPassAdaptor( | ||
| 380 | std::unique_ptr<ModuleToPostOrderCGSCCPassAdaptor::PassConceptT>( | ||
| 381 | new PassModelT(std::forward<CGSCCPassT>(Pass)))); | ||
| 382 | } | ||
| 383 | |||
| 384 | /// A proxy from a \c FunctionAnalysisManager to an \c SCC. | ||
| 385 | /// | ||
| 386 | /// When a module pass runs and triggers invalidation, both the CGSCC and | ||
| 387 | /// Function analysis manager proxies on the module get an invalidation event. | ||
| 388 | /// We don't want to fully duplicate responsibility for most of the | ||
| 389 | /// invalidation logic. Instead, this layer is only responsible for SCC-local | ||
| 390 | /// invalidation events. We work with the module's FunctionAnalysisManager to | ||
| 391 | /// invalidate function analyses. | ||
| 392 | class FunctionAnalysisManagerCGSCCProxy | ||
| 393 | : public AnalysisInfoMixin<FunctionAnalysisManagerCGSCCProxy> { | ||
| 394 | public: | ||
| 395 | class Result { | ||
| 396 | public: | ||
| 397 | explicit Result() : FAM(nullptr) {} | ||
| 398 | explicit Result(FunctionAnalysisManager &FAM) : FAM(&FAM) {} | ||
| 399 | |||
| 400 | void updateFAM(FunctionAnalysisManager &FAM) { this->FAM = &FAM; } | ||
| 401 |     /// Accessor for the analysis manager. | ||
| 402 | FunctionAnalysisManager &getManager() { | ||
| 403 | assert(FAM); | ||
| 404 | return *FAM; | ||
| 405 |     } | ||
| 406 | |||
| 407 | bool invalidate(LazyCallGraph::SCC &C, const PreservedAnalyses &PA, | ||
| 408 | CGSCCAnalysisManager::Invalidator &Inv); | ||
| 409 | |||
| 410 | private: | ||
| 411 | FunctionAnalysisManager *FAM; | ||
| 412 | }; | ||
| 413 | |||
| 414 |   /// Computes the \c FunctionAnalysisManager and stores it in the result proxy. | ||
| 415 | Result run(LazyCallGraph::SCC &C, CGSCCAnalysisManager &AM, LazyCallGraph &); | ||
| 416 | |||
| 417 | private: | ||
| 418 | friend AnalysisInfoMixin<FunctionAnalysisManagerCGSCCProxy>; | ||
| 419 | |||
| 420 | static AnalysisKey Key; | ||
| 421 | }; | ||
| 422 | |||
| 423 | extern template class OuterAnalysisManagerProxy<CGSCCAnalysisManager, Function>; | ||
| 424 | |||
| 425 | /// A proxy from a \c CGSCCAnalysisManager to a \c Function. | ||
| 426 | using CGSCCAnalysisManagerFunctionProxy = | ||
| 427 | OuterAnalysisManagerProxy<CGSCCAnalysisManager, Function>; | ||
| 428 | |||
| 429 | /// Helper to update the call graph after running a function pass. | ||
| 430 | /// | ||
| 431 | /// Function passes can only mutate the call graph in specific ways. This | ||
| 432 | /// routine provides a helper that updates the call graph in those ways | ||
| 433 | /// including returning whether any changes were made and populating a CG | ||
| 434 | /// update result struct for the overall CGSCC walk. | ||
| 435 | LazyCallGraph::SCC &updateCGAndAnalysisManagerForFunctionPass( | ||
| 436 | LazyCallGraph &G, LazyCallGraph::SCC &C, LazyCallGraph::Node &N, | ||
| 437 | CGSCCAnalysisManager &AM, CGSCCUpdateResult &UR, | ||
| 438 | FunctionAnalysisManager &FAM); | ||
| 439 | |||
| 440 | /// Helper to update the call graph after running a CGSCC pass. | ||
| 441 | /// | ||
| 442 | /// CGSCC passes can only mutate the call graph in specific ways. This | ||
| 443 | /// routine provides a helper that updates the call graph in those ways | ||
| 444 | /// including returning whether any changes were made and populating a CG | ||
| 445 | /// update result struct for the overall CGSCC walk. | ||
| 446 | LazyCallGraph::SCC &updateCGAndAnalysisManagerForCGSCCPass( | ||
| 447 | LazyCallGraph &G, LazyCallGraph::SCC &C, LazyCallGraph::Node &N, | ||
| 448 | CGSCCAnalysisManager &AM, CGSCCUpdateResult &UR, | ||
| 449 | FunctionAnalysisManager &FAM); | ||
| 450 | |||
| 451 | /// Adaptor that maps from a SCC to its functions. | ||
| 452 | /// | ||
| 453 | /// Designed to allow composition of a FunctionPass(Manager) and | ||
| 454 | /// a CGSCCPassManager. Note that if this pass is constructed with a pointer | ||
| 455 | /// to a \c CGSCCAnalysisManager it will run the | ||
| 456 | /// \c FunctionAnalysisManagerCGSCCProxy analysis prior to running the function | ||
| 457 | /// pass over the SCC to enable a \c FunctionAnalysisManager to be used | ||
| 458 | /// within this run safely. | ||
| 459 | class CGSCCToFunctionPassAdaptor | ||
| 460 | : public PassInfoMixin<CGSCCToFunctionPassAdaptor> { | ||
| 461 | public: | ||
| 462 | using PassConceptT = detail::PassConcept<Function, FunctionAnalysisManager>; | ||
| 463 | |||
| 464 | explicit CGSCCToFunctionPassAdaptor(std::unique_ptr<PassConceptT> Pass, | ||
| 465 | bool EagerlyInvalidate, bool NoRerun) | ||
| 466 | : Pass(std::move(Pass)), EagerlyInvalidate(EagerlyInvalidate), | ||
| 467 | NoRerun(NoRerun) {} | ||
| 468 | |||
| 469 | CGSCCToFunctionPassAdaptor(CGSCCToFunctionPassAdaptor &&Arg) | ||
| 470 | : Pass(std::move(Arg.Pass)), EagerlyInvalidate(Arg.EagerlyInvalidate), | ||
| 471 | NoRerun(Arg.NoRerun) {} | ||
| 472 | |||
| 473 | friend void swap(CGSCCToFunctionPassAdaptor &LHS, | ||
| 474 | CGSCCToFunctionPassAdaptor &RHS) { | ||
| 475 | std::swap(LHS.Pass, RHS.Pass); | ||
| 476 |   } | ||
| 477 | |||
| 478 | CGSCCToFunctionPassAdaptor &operator=(CGSCCToFunctionPassAdaptor RHS) { | ||
| 479 | swap(*this, RHS); | ||
| 480 | return *this; | ||
| 481 |   } | ||
| 482 | |||
| 483 |   /// Runs the function pass across every function in the module. | ||
| 484 | PreservedAnalyses run(LazyCallGraph::SCC &C, CGSCCAnalysisManager &AM, | ||
| 485 | LazyCallGraph &CG, CGSCCUpdateResult &UR); | ||
| 486 | |||
| 487 | void printPipeline(raw_ostream &OS, | ||
| 488 | function_ref<StringRef(StringRef)> MapClassName2PassName) { | ||
| 489 | OS << "function"; | ||
| 490 | if (EagerlyInvalidate) | ||
| 491 | OS << "<eager-inv>"; | ||
| 492 | OS << "("; | ||
| 493 | Pass->printPipeline(OS, MapClassName2PassName); | ||
| 494 | OS << ")"; | ||
| 495 |   } | ||
| 496 | |||
| 497 | static bool isRequired() { return true; } | ||
| 498 | |||
| 499 | private: | ||
| 500 | std::unique_ptr<PassConceptT> Pass; | ||
| 501 | bool EagerlyInvalidate; | ||
| 502 | bool NoRerun; | ||
| 503 | }; | ||
| 504 | |||
| 505 | /// A function to deduce a function pass type and wrap it in the | ||
| 506 | /// templated adaptor. | ||
| 507 | template <typename FunctionPassT> | ||
| 508 | CGSCCToFunctionPassAdaptor | ||
| 509 | createCGSCCToFunctionPassAdaptor(FunctionPassT &&Pass, | ||
| 510 | bool EagerlyInvalidate = false, | ||
| 511 | bool NoRerun = false) { | ||
| 512 | using PassModelT = | ||
| 513 | detail::PassModel<Function, FunctionPassT, PreservedAnalyses, | ||
| 514 | FunctionAnalysisManager>; | ||
| 515 |   // Do not use make_unique, it causes too many template instantiations, | ||
| 516 |   // causing terrible compile times. | ||
| 517 | return CGSCCToFunctionPassAdaptor( | ||
| 518 | std::unique_ptr<CGSCCToFunctionPassAdaptor::PassConceptT>( | ||
| 519 | new PassModelT(std::forward<FunctionPassT>(Pass))), | ||
| 520 | EagerlyInvalidate, NoRerun); | ||
| 521 | } | ||
| 522 | |||
| 523 | // A marker to determine if function passes should be run on a function within a | ||
| 524 | // CGSCCToFunctionPassAdaptor. This is used to prevent running an expensive | ||
| 525 | // function pass (manager) on a function multiple times if SCC mutations cause a | ||
| 526 | // function to be visited multiple times and the function is not modified by | ||
| 527 | // other SCC passes. | ||
| 528 | class ShouldNotRunFunctionPassesAnalysis | ||
| 529 | : public AnalysisInfoMixin<ShouldNotRunFunctionPassesAnalysis> { | ||
| 530 | public: | ||
| 531 | static AnalysisKey Key; | ||
| 532 | struct Result {}; | ||
| 533 | |||
| 534 | Result run(Function &F, FunctionAnalysisManager &FAM) { return Result(); } | ||
| 535 | }; | ||
| 536 | |||
| 537 | /// A helper that repeats an SCC pass each time an indirect call is refined to | ||
| 538 | /// a direct call by that pass. | ||
| 539 | /// | ||
| 540 | /// While the CGSCC pass manager works to re-visit SCCs and RefSCCs as they | ||
| 541 | /// change shape, we may also want to repeat an SCC pass if it simply refines | ||
| 542 | /// an indirect call to a direct call, even if doing so does not alter the | ||
| 543 | /// shape of the graph. Note that this only pertains to direct calls to | ||
| 544 | /// functions where IPO across the SCC may be able to compute more precise | ||
| 545 | /// results. For intrinsics, we assume scalar optimizations already can fully | ||
| 546 | /// reason about them. | ||
| 547 | /// | ||
| 548 | /// This repetition has the potential to be very large however, as each one | ||
| 549 | /// might refine a single call site. As a consequence, in practice we use an | ||
| 550 | /// upper bound on the number of repetitions to limit things. | ||
| 551 | class DevirtSCCRepeatedPass : public PassInfoMixin<DevirtSCCRepeatedPass> { | ||
| 552 | public: | ||
| 553 | using PassConceptT = | ||
| 554 | detail::PassConcept<LazyCallGraph::SCC, CGSCCAnalysisManager, | ||
| 555 | LazyCallGraph &, CGSCCUpdateResult &>; | ||
| 556 | |||
| 557 | explicit DevirtSCCRepeatedPass(std::unique_ptr<PassConceptT> Pass, | ||
| 558 | int MaxIterations) | ||
| 559 | : Pass(std::move(Pass)), MaxIterations(MaxIterations) {} | ||
| 560 | |||
| 561 |   /// Runs the wrapped pass up to \c MaxIterations on the SCC, iterating | ||
| 562 |   /// whenever an indirect call is refined. | ||
| 563 | PreservedAnalyses run(LazyCallGraph::SCC &InitialC, CGSCCAnalysisManager &AM, | ||
| 564 | LazyCallGraph &CG, CGSCCUpdateResult &UR); | ||
| 565 | |||
| 566 | void printPipeline(raw_ostream &OS, | ||
| 567 | function_ref<StringRef(StringRef)> MapClassName2PassName) { | ||
| 568 | OS << "devirt<" << MaxIterations << ">("; | ||
| 569 | Pass->printPipeline(OS, MapClassName2PassName); | ||
| 570 | OS << ")"; | ||
| 571 |   } | ||
| 572 | |||
| 573 | private: | ||
| 574 | std::unique_ptr<PassConceptT> Pass; | ||
| 575 | int MaxIterations; | ||
| 576 | }; | ||
| 577 | |||
| 578 | /// A function to deduce a function pass type and wrap it in the | ||
| 579 | /// templated adaptor. | ||
| 580 | template <typename CGSCCPassT> | ||
| 581 | DevirtSCCRepeatedPass createDevirtSCCRepeatedPass(CGSCCPassT &&Pass, | ||
| 582 | int MaxIterations) { | ||
| 583 | using PassModelT = detail::PassModel<LazyCallGraph::SCC, CGSCCPassT, | ||
| 584 | PreservedAnalyses, CGSCCAnalysisManager, | ||
| 585 | LazyCallGraph &, CGSCCUpdateResult &>; | ||
| 586 |   // Do not use make_unique, it causes too many template instantiations, | ||
| 587 |   // causing terrible compile times. | ||
| 588 | return DevirtSCCRepeatedPass( | ||
| 589 | std::unique_ptr<DevirtSCCRepeatedPass::PassConceptT>( | ||
| 590 | new PassModelT(std::forward<CGSCCPassT>(Pass))), | ||
| 591 | MaxIterations); | ||
| 592 | } | ||
| 593 | |||
| 594 | // Clear out the debug logging macro. | ||
| 595 | #undef DEBUG_TYPE | ||
| 596 | |||
| 597 | } // end namespace llvm | ||
| 598 | |||
| 599 | #endif // LLVM_ANALYSIS_CGSCCPASSMANAGER_H |