Subversion Repositories QNX 8.QNX8 LLVM/Clang compiler suite

Rev

Details | Last modification | View Log | RSS feed

Rev Author Line No. Line
14 pmbaty 1
//===- Attributor.h --- Module-wide attribute deduction ---------*- C++ -*-===//
2
//
3
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
4
// See https://llvm.org/LICENSE.txt for license information.
5
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
6
//
7
//===----------------------------------------------------------------------===//
8
//
9
// Attributor: An inter procedural (abstract) "attribute" deduction framework.
10
//
11
// The Attributor framework is an inter procedural abstract analysis (fixpoint
12
// iteration analysis). The goal is to allow easy deduction of new attributes as
13
// well as information exchange between abstract attributes in-flight.
14
//
15
// The Attributor class is the driver and the link between the various abstract
16
// attributes. The Attributor will iterate until a fixpoint state is reached by
17
// all abstract attributes in-flight, or until it will enforce a pessimistic fix
18
// point because an iteration limit is reached.
19
//
20
// Abstract attributes, derived from the AbstractAttribute class, actually
21
// describe properties of the code. They can correspond to actual LLVM-IR
22
// attributes, or they can be more general, ultimately unrelated to LLVM-IR
23
// attributes. The latter is useful when an abstract attributes provides
24
// information to other abstract attributes in-flight but we might not want to
25
// manifest the information. The Attributor allows to query in-flight abstract
26
// attributes through the `Attributor::getAAFor` method (see the method
27
// description for an example). If the method is used by an abstract attribute
28
// P, and it results in an abstract attribute Q, the Attributor will
29
// automatically capture a potential dependence from Q to P. This dependence
30
// will cause P to be reevaluated whenever Q changes in the future.
31
//
32
// The Attributor will only reevaluate abstract attributes that might have
33
// changed since the last iteration. That means that the Attribute will not
34
// revisit all instructions/blocks/functions in the module but only query
35
// an update from a subset of the abstract attributes.
36
//
37
// The update method `AbstractAttribute::updateImpl` is implemented by the
38
// specific "abstract attribute" subclasses. The method is invoked whenever the
39
// currently assumed state (see the AbstractState class) might not be valid
40
// anymore. This can, for example, happen if the state was dependent on another
41
// abstract attribute that changed. In every invocation, the update method has
42
// to adjust the internal state of an abstract attribute to a point that is
43
// justifiable by the underlying IR and the current state of abstract attributes
44
// in-flight. Since the IR is given and assumed to be valid, the information
45
// derived from it can be assumed to hold. However, information derived from
46
// other abstract attributes is conditional on various things. If the justifying
47
// state changed, the `updateImpl` has to revisit the situation and potentially
48
// find another justification or limit the optimistic assumes made.
49
//
50
// Change is the key in this framework. Until a state of no-change, thus a
51
// fixpoint, is reached, the Attributor will query the abstract attributes
52
// in-flight to re-evaluate their state. If the (current) state is too
53
// optimistic, hence it cannot be justified anymore through other abstract
54
// attributes or the state of the IR, the state of the abstract attribute will
55
// have to change. Generally, we assume abstract attribute state to be a finite
56
// height lattice and the update function to be monotone. However, these
57
// conditions are not enforced because the iteration limit will guarantee
58
// termination. If an optimistic fixpoint is reached, or a pessimistic fix
59
// point is enforced after a timeout, the abstract attributes are tasked to
60
// manifest their result in the IR for passes to come.
61
//
62
// Attribute manifestation is not mandatory. If desired, there is support to
63
// generate a single or multiple LLVM-IR attributes already in the helper struct
64
// IRAttribute. In the simplest case, a subclass inherits from IRAttribute with
65
// a proper Attribute::AttrKind as template parameter. The Attributor
66
// manifestation framework will then create and place a new attribute if it is
67
// allowed to do so (based on the abstract state). Other use cases can be
68
// achieved by overloading AbstractAttribute or IRAttribute methods.
69
//
70
//
71
// The "mechanics" of adding a new "abstract attribute":
72
// - Define a class (transitively) inheriting from AbstractAttribute and one
73
//   (which could be the same) that (transitively) inherits from AbstractState.
74
//   For the latter, consider the already available BooleanState and
75
//   {Inc,Dec,Bit}IntegerState if they fit your needs, e.g., you require only a
76
//   number tracking or bit-encoding.
77
// - Implement all pure methods. Also use overloading if the attribute is not
78
//   conforming with the "default" behavior: A (set of) LLVM-IR attribute(s) for
79
//   an argument, call site argument, function return value, or function. See
80
//   the class and method descriptions for more information on the two
81
//   "Abstract" classes and their respective methods.
82
// - Register opportunities for the new abstract attribute in the
83
//   `Attributor::identifyDefaultAbstractAttributes` method if it should be
84
//   counted as a 'default' attribute.
85
// - Add sufficient tests.
86
// - Add a Statistics object for bookkeeping. If it is a simple (set of)
87
//   attribute(s) manifested through the Attributor manifestation framework, see
88
//   the bookkeeping function in Attributor.cpp.
89
// - If instructions with a certain opcode are interesting to the attribute, add
90
//   that opcode to the switch in `Attributor::identifyAbstractAttributes`. This
91
//   will make it possible to query all those instructions through the
92
//   `InformationCache::getOpcodeInstMapForFunction` interface and eliminate the
93
//   need to traverse the IR repeatedly.
94
//
95
//===----------------------------------------------------------------------===//
96
 
97
#ifndef LLVM_TRANSFORMS_IPO_ATTRIBUTOR_H
98
#define LLVM_TRANSFORMS_IPO_ATTRIBUTOR_H
99
 
100
#include "llvm/ADT/DenseSet.h"
101
#include "llvm/ADT/GraphTraits.h"
102
#include "llvm/ADT/MapVector.h"
103
#include "llvm/ADT/STLExtras.h"
104
#include "llvm/ADT/SetOperations.h"
105
#include "llvm/ADT/SetVector.h"
106
#include "llvm/ADT/Triple.h"
107
#include "llvm/ADT/iterator.h"
108
#include "llvm/Analysis/AssumeBundleQueries.h"
109
#include "llvm/Analysis/CFG.h"
110
#include "llvm/Analysis/CGSCCPassManager.h"
111
#include "llvm/Analysis/LazyCallGraph.h"
112
#include "llvm/Analysis/LoopInfo.h"
113
#include "llvm/Analysis/MemoryLocation.h"
114
#include "llvm/Analysis/MustExecute.h"
115
#include "llvm/Analysis/OptimizationRemarkEmitter.h"
116
#include "llvm/Analysis/PostDominators.h"
117
#include "llvm/Analysis/TargetLibraryInfo.h"
118
#include "llvm/IR/AbstractCallSite.h"
119
#include "llvm/IR/ConstantRange.h"
120
#include "llvm/IR/Constants.h"
121
#include "llvm/IR/InstIterator.h"
122
#include "llvm/IR/Instruction.h"
123
#include "llvm/IR/PassManager.h"
124
#include "llvm/IR/Value.h"
125
#include "llvm/Support/Alignment.h"
126
#include "llvm/Support/Allocator.h"
127
#include "llvm/Support/Casting.h"
128
#include "llvm/Support/DOTGraphTraits.h"
129
#include "llvm/Support/TimeProfiler.h"
130
#include "llvm/Transforms/Utils/CallGraphUpdater.h"
131
 
132
#include <limits>
133
#include <map>
134
#include <optional>
135
 
136
namespace llvm {
137
 
138
class DataLayout;
139
class LLVMContext;
140
class Pass;
141
template <typename Fn> class function_ref;
142
struct AADepGraphNode;
143
struct AADepGraph;
144
struct Attributor;
145
struct AbstractAttribute;
146
struct InformationCache;
147
struct AAIsDead;
148
struct AttributorCallGraph;
149
struct IRPosition;
150
 
151
class AAResults;
152
class Function;
153
 
154
/// Abstract Attribute helper functions.
155
namespace AA {
156
using InstExclusionSetTy = SmallPtrSet<Instruction *, 4>;
157
 
158
enum class GPUAddressSpace : unsigned {
159
  Generic = 0,
160
  Global = 1,
161
  Shared = 3,
162
  Constant = 4,
163
  Local = 5,
164
};
165
 
166
/// Flags to distinguish intra-procedural queries from *potentially*
167
/// inter-procedural queries. Not that information can be valid for both and
168
/// therefore both bits might be set.
169
enum ValueScope : uint8_t {
170
  Intraprocedural = 1,
171
  Interprocedural = 2,
172
  AnyScope = Intraprocedural | Interprocedural,
173
};
174
 
175
struct ValueAndContext : public std::pair<Value *, const Instruction *> {
176
  using Base = std::pair<Value *, const Instruction *>;
177
  ValueAndContext(const Base &B) : Base(B) {}
178
  ValueAndContext(Value &V, const Instruction *CtxI) : Base(&V, CtxI) {}
179
  ValueAndContext(Value &V, const Instruction &CtxI) : Base(&V, &CtxI) {}
180
 
181
  Value *getValue() const { return this->first; }
182
  const Instruction *getCtxI() const { return this->second; }
183
};
184
 
185
/// Return true if \p I is a `nosync` instruction. Use generic reasoning and
186
/// potentially the corresponding AANoSync.
187
bool isNoSyncInst(Attributor &A, const Instruction &I,
188
                  const AbstractAttribute &QueryingAA);
189
 
190
/// Return true if \p V is dynamically unique, that is, there are no two
191
/// "instances" of \p V at runtime with different values.
192
/// Note: If \p ForAnalysisOnly is set we only check that the Attributor will
193
/// never use \p V to represent two "instances" not that \p V could not
194
/// technically represent them.
195
bool isDynamicallyUnique(Attributor &A, const AbstractAttribute &QueryingAA,
196
                         const Value &V, bool ForAnalysisOnly = true);
197
 
198
/// Return true if \p V is a valid value in \p Scope, that is a constant or an
199
/// instruction/argument of \p Scope.
200
bool isValidInScope(const Value &V, const Function *Scope);
201
 
202
/// Return true if the value of \p VAC is a valid at the position of \p VAC,
203
/// that is a constant, an argument of the same function, or an instruction in
204
/// that function that dominates the position.
205
bool isValidAtPosition(const ValueAndContext &VAC, InformationCache &InfoCache);
206
 
207
/// Try to convert \p V to type \p Ty without introducing new instructions. If
208
/// this is not possible return `nullptr`. Note: this function basically knows
209
/// how to cast various constants.
210
Value *getWithType(Value &V, Type &Ty);
211
 
212
/// Return the combination of \p A and \p B such that the result is a possible
213
/// value of both. \p B is potentially casted to match the type \p Ty or the
214
/// type of \p A if \p Ty is null.
215
///
216
/// Examples:
217
///        X + none  => X
218
/// not_none + undef => not_none
219
///          V1 + V2 => nullptr
220
std::optional<Value *>
221
combineOptionalValuesInAAValueLatice(const std::optional<Value *> &A,
222
                                     const std::optional<Value *> &B, Type *Ty);
223
 
224
/// Helper to represent an access offset and size, with logic to deal with
225
/// uncertainty and check for overlapping accesses.
226
struct RangeTy {
227
  int64_t Offset = Unassigned;
228
  int64_t Size = Unassigned;
229
 
230
  RangeTy(int64_t Offset, int64_t Size) : Offset(Offset), Size(Size) {}
231
  RangeTy() = default;
232
  static RangeTy getUnknown() { return RangeTy{Unknown, Unknown}; }
233
 
234
  /// Return true if offset or size are unknown.
235
  bool offsetOrSizeAreUnknown() const {
236
    return Offset == RangeTy::Unknown || Size == RangeTy::Unknown;
237
  }
238
 
239
  /// Return true if offset and size are unknown, thus this is the default
240
  /// unknown object.
241
  bool offsetAndSizeAreUnknown() const {
242
    return Offset == RangeTy::Unknown && Size == RangeTy::Unknown;
243
  }
244
 
245
  /// Return true if the offset and size are unassigned.
246
  bool isUnassigned() const {
247
    assert((Offset == RangeTy::Unassigned) == (Size == RangeTy::Unassigned) &&
248
           "Inconsistent state!");
249
    return Offset == RangeTy::Unassigned;
250
  }
251
 
252
  /// Return true if this offset and size pair might describe an address that
253
  /// overlaps with \p Range.
254
  bool mayOverlap(const RangeTy &Range) const {
255
    // Any unknown value and we are giving up -> overlap.
256
    if (offsetOrSizeAreUnknown() || Range.offsetOrSizeAreUnknown())
257
      return true;
258
 
259
    // Check if one offset point is in the other interval [offset,
260
    // offset+size].
261
    return Range.Offset + Range.Size > Offset && Range.Offset < Offset + Size;
262
  }
263
 
264
  RangeTy &operator&=(const RangeTy &R) {
265
    if (Offset == Unassigned)
266
      Offset = R.Offset;
267
    else if (R.Offset != Unassigned && R.Offset != Offset)
268
      Offset = Unknown;
269
 
270
    if (Size == Unassigned)
271
      Size = R.Size;
272
    else if (Size == Unknown || R.Size == Unknown)
273
      Size = Unknown;
274
    else if (R.Size != Unassigned)
275
      Size = std::max(Size, R.Size);
276
 
277
    return *this;
278
  }
279
 
280
  /// Comparison for sorting ranges by offset.
281
  ///
282
  /// Returns true if the offset \p L is less than that of \p R.
283
  inline static bool OffsetLessThan(const RangeTy &L, const RangeTy &R) {
284
    return L.Offset < R.Offset;
285
  }
286
 
287
  /// Constants used to represent special offsets or sizes.
288
  /// - We cannot assume that Offsets and Size are non-negative.
289
  /// - The constants should not clash with DenseMapInfo, such as EmptyKey
290
  ///   (INT64_MAX) and TombstoneKey (INT64_MIN).
291
  /// We use values "in the middle" of the 64 bit range to represent these
292
  /// special cases.
293
  static constexpr int64_t Unassigned = std::numeric_limits<int32_t>::min();
294
  static constexpr int64_t Unknown = std::numeric_limits<int32_t>::max();
295
};
296
 
297
inline raw_ostream &operator<<(raw_ostream &OS, const RangeTy &R) {
298
  OS << "[" << R.Offset << ", " << R.Size << "]";
299
  return OS;
300
}
301
 
302
inline bool operator==(const RangeTy &A, const RangeTy &B) {
303
  return A.Offset == B.Offset && A.Size == B.Size;
304
}
305
 
306
inline bool operator!=(const RangeTy &A, const RangeTy &B) { return !(A == B); }
307
 
308
/// Return the initial value of \p Obj with type \p Ty if that is a constant.
309
Constant *getInitialValueForObj(Value &Obj, Type &Ty,
310
                                const TargetLibraryInfo *TLI,
311
                                const DataLayout &DL,
312
                                RangeTy *RangePtr = nullptr);
313
 
314
/// Collect all potential values \p LI could read into \p PotentialValues. That
315
/// is, the only values read by \p LI are assumed to be known and all are in
316
/// \p PotentialValues. \p PotentialValueOrigins will contain all the
317
/// instructions that might have put a potential value into \p PotentialValues.
318
/// Dependences onto \p QueryingAA are properly tracked, \p
319
/// UsedAssumedInformation will inform the caller if assumed information was
320
/// used.
321
///
322
/// \returns True if the assumed potential copies are all in \p PotentialValues,
323
///          false if something went wrong and the copies could not be
324
///          determined.
325
bool getPotentiallyLoadedValues(
326
    Attributor &A, LoadInst &LI, SmallSetVector<Value *, 4> &PotentialValues,
327
    SmallSetVector<Instruction *, 4> &PotentialValueOrigins,
328
    const AbstractAttribute &QueryingAA, bool &UsedAssumedInformation,
329
    bool OnlyExact = false);
330
 
331
/// Collect all potential values of the one stored by \p SI into
332
/// \p PotentialCopies. That is, the only copies that were made via the
333
/// store are assumed to be known and all are in \p PotentialCopies. Dependences
334
/// onto \p QueryingAA are properly tracked, \p UsedAssumedInformation will
335
/// inform the caller if assumed information was used.
336
///
337
/// \returns True if the assumed potential copies are all in \p PotentialCopies,
338
///          false if something went wrong and the copies could not be
339
///          determined.
340
bool getPotentialCopiesOfStoredValue(
341
    Attributor &A, StoreInst &SI, SmallSetVector<Value *, 4> &PotentialCopies,
342
    const AbstractAttribute &QueryingAA, bool &UsedAssumedInformation,
343
    bool OnlyExact = false);
344
 
345
/// Return true if \p IRP is readonly. This will query respective AAs that
346
/// deduce the information and introduce dependences for \p QueryingAA.
347
bool isAssumedReadOnly(Attributor &A, const IRPosition &IRP,
348
                       const AbstractAttribute &QueryingAA, bool &IsKnown);
349
 
350
/// Return true if \p IRP is readnone. This will query respective AAs that
351
/// deduce the information and introduce dependences for \p QueryingAA.
352
bool isAssumedReadNone(Attributor &A, const IRPosition &IRP,
353
                       const AbstractAttribute &QueryingAA, bool &IsKnown);
354
 
355
/// Return true if \p ToI is potentially reachable from \p FromI without running
356
/// into any instruction in \p ExclusionSet The two instructions do not need to
357
/// be in the same function. \p GoBackwardsCB can be provided to convey domain
358
/// knowledge about the "lifespan" the user is interested in. By default, the
359
/// callers of \p FromI are checked as well to determine if \p ToI can be
360
/// reached. If the query is not interested in callers beyond a certain point,
361
/// e.g., a GPU kernel entry or the function containing an alloca, the
362
/// \p GoBackwardsCB should return false.
363
bool isPotentiallyReachable(
364
    Attributor &A, const Instruction &FromI, const Instruction &ToI,
365
    const AbstractAttribute &QueryingAA,
366
    const AA::InstExclusionSetTy *ExclusionSet = nullptr,
367
    std::function<bool(const Function &F)> GoBackwardsCB = nullptr);
368
 
369
/// Same as above but it is sufficient to reach any instruction in \p ToFn.
370
bool isPotentiallyReachable(
371
    Attributor &A, const Instruction &FromI, const Function &ToFn,
372
    const AbstractAttribute &QueryingAA,
373
    const AA::InstExclusionSetTy *ExclusionSet = nullptr,
374
    std::function<bool(const Function &F)> GoBackwardsCB = nullptr);
375
 
376
/// Return true if \p Obj is assumed to be a thread local object.
377
bool isAssumedThreadLocalObject(Attributor &A, Value &Obj,
378
                                const AbstractAttribute &QueryingAA);
379
 
380
/// Return true if \p I is potentially affected by a barrier.
381
bool isPotentiallyAffectedByBarrier(Attributor &A, const Instruction &I,
382
                                    const AbstractAttribute &QueryingAA);
383
bool isPotentiallyAffectedByBarrier(Attributor &A, ArrayRef<const Value *> Ptrs,
384
                                    const AbstractAttribute &QueryingAA,
385
                                    const Instruction *CtxI);
386
} // namespace AA
387
 
388
template <>
389
struct DenseMapInfo<AA::ValueAndContext>
390
    : public DenseMapInfo<AA::ValueAndContext::Base> {
391
  using Base = DenseMapInfo<AA::ValueAndContext::Base>;
392
  static inline AA::ValueAndContext getEmptyKey() {
393
    return Base::getEmptyKey();
394
  }
395
  static inline AA::ValueAndContext getTombstoneKey() {
396
    return Base::getTombstoneKey();
397
  }
398
  static unsigned getHashValue(const AA::ValueAndContext &VAC) {
399
    return Base::getHashValue(VAC);
400
  }
401
 
402
  static bool isEqual(const AA::ValueAndContext &LHS,
403
                      const AA::ValueAndContext &RHS) {
404
    return Base::isEqual(LHS, RHS);
405
  }
406
};
407
 
408
template <>
409
struct DenseMapInfo<AA::ValueScope> : public DenseMapInfo<unsigned char> {
410
  using Base = DenseMapInfo<unsigned char>;
411
  static inline AA::ValueScope getEmptyKey() {
412
    return AA::ValueScope(Base::getEmptyKey());
413
  }
414
  static inline AA::ValueScope getTombstoneKey() {
415
    return AA::ValueScope(Base::getTombstoneKey());
416
  }
417
  static unsigned getHashValue(const AA::ValueScope &S) {
418
    return Base::getHashValue(S);
419
  }
420
 
421
  static bool isEqual(const AA::ValueScope &LHS, const AA::ValueScope &RHS) {
422
    return Base::isEqual(LHS, RHS);
423
  }
424
};
425
 
426
template <>
427
struct DenseMapInfo<const AA::InstExclusionSetTy *>
428
    : public DenseMapInfo<void *> {
429
  using super = DenseMapInfo<void *>;
430
  static inline const AA::InstExclusionSetTy *getEmptyKey() {
431
    return static_cast<const AA::InstExclusionSetTy *>(super::getEmptyKey());
432
  }
433
  static inline const AA::InstExclusionSetTy *getTombstoneKey() {
434
    return static_cast<const AA::InstExclusionSetTy *>(
435
        super::getTombstoneKey());
436
  }
437
  static unsigned getHashValue(const AA::InstExclusionSetTy *BES) {
438
    unsigned H = 0;
439
    if (BES)
440
      for (const auto *II : *BES)
441
        H += DenseMapInfo<const Instruction *>::getHashValue(II);
442
    return H;
443
  }
444
  static bool isEqual(const AA::InstExclusionSetTy *LHS,
445
                      const AA::InstExclusionSetTy *RHS) {
446
    if (LHS == RHS)
447
      return true;
448
    if (LHS == getEmptyKey() || RHS == getEmptyKey() ||
449
        LHS == getTombstoneKey() || RHS == getTombstoneKey())
450
      return false;
451
    if (!LHS || !RHS)
452
      return ((LHS && LHS->empty()) || (RHS && RHS->empty()));
453
    if (LHS->size() != RHS->size())
454
      return false;
455
    return llvm::set_is_subset(*LHS, *RHS);
456
  }
457
};
458
 
459
/// The value passed to the line option that defines the maximal initialization
460
/// chain length.
461
extern unsigned MaxInitializationChainLength;
462
 
463
///{
464
enum class ChangeStatus {
465
  CHANGED,
466
  UNCHANGED,
467
};
468
 
469
ChangeStatus operator|(ChangeStatus l, ChangeStatus r);
470
ChangeStatus &operator|=(ChangeStatus &l, ChangeStatus r);
471
ChangeStatus operator&(ChangeStatus l, ChangeStatus r);
472
ChangeStatus &operator&=(ChangeStatus &l, ChangeStatus r);
473
 
474
enum class DepClassTy {
475
  REQUIRED, ///< The target cannot be valid if the source is not.
476
  OPTIONAL, ///< The target may be valid if the source is not.
477
  NONE,     ///< Do not track a dependence between source and target.
478
};
479
///}
480
 
481
/// The data structure for the nodes of a dependency graph
482
struct AADepGraphNode {
483
public:
484
  virtual ~AADepGraphNode() = default;
485
  using DepTy = PointerIntPair<AADepGraphNode *, 1>;
486
 
487
protected:
488
  /// Set of dependency graph nodes which should be updated if this one
489
  /// is updated. The bit encodes if it is optional.
490
  TinyPtrVector<DepTy> Deps;
491
 
492
  static AADepGraphNode *DepGetVal(DepTy &DT) { return DT.getPointer(); }
493
  static AbstractAttribute *DepGetValAA(DepTy &DT) {
494
    return cast<AbstractAttribute>(DT.getPointer());
495
  }
496
 
497
  operator AbstractAttribute *() { return cast<AbstractAttribute>(this); }
498
 
499
public:
500
  using iterator =
501
      mapped_iterator<TinyPtrVector<DepTy>::iterator, decltype(&DepGetVal)>;
502
  using aaiterator =
503
      mapped_iterator<TinyPtrVector<DepTy>::iterator, decltype(&DepGetValAA)>;
504
 
505
  aaiterator begin() { return aaiterator(Deps.begin(), &DepGetValAA); }
506
  aaiterator end() { return aaiterator(Deps.end(), &DepGetValAA); }
507
  iterator child_begin() { return iterator(Deps.begin(), &DepGetVal); }
508
  iterator child_end() { return iterator(Deps.end(), &DepGetVal); }
509
 
510
  virtual void print(raw_ostream &OS) const { OS << "AADepNode Impl\n"; }
511
  TinyPtrVector<DepTy> &getDeps() { return Deps; }
512
 
513
  friend struct Attributor;
514
  friend struct AADepGraph;
515
};
516
 
517
/// The data structure for the dependency graph
518
///
519
/// Note that in this graph if there is an edge from A to B (A -> B),
520
/// then it means that B depends on A, and when the state of A is
521
/// updated, node B should also be updated
522
struct AADepGraph {
523
  AADepGraph() = default;
524
  ~AADepGraph() = default;
525
 
526
  using DepTy = AADepGraphNode::DepTy;
527
  static AADepGraphNode *DepGetVal(DepTy &DT) { return DT.getPointer(); }
528
  using iterator =
529
      mapped_iterator<TinyPtrVector<DepTy>::iterator, decltype(&DepGetVal)>;
530
 
531
  /// There is no root node for the dependency graph. But the SCCIterator
532
  /// requires a single entry point, so we maintain a fake("synthetic") root
533
  /// node that depends on every node.
534
  AADepGraphNode SyntheticRoot;
535
  AADepGraphNode *GetEntryNode() { return &SyntheticRoot; }
536
 
537
  iterator begin() { return SyntheticRoot.child_begin(); }
538
  iterator end() { return SyntheticRoot.child_end(); }
539
 
540
  void viewGraph();
541
 
542
  /// Dump graph to file
543
  void dumpGraph();
544
 
545
  /// Print dependency graph
546
  void print();
547
};
548
 
549
/// Helper to describe and deal with positions in the LLVM-IR.
550
///
551
/// A position in the IR is described by an anchor value and an "offset" that
552
/// could be the argument number, for call sites and arguments, or an indicator
553
/// of the "position kind". The kinds, specified in the Kind enum below, include
554
/// the locations in the attribute list, i.a., function scope and return value,
555
/// as well as a distinction between call sites and functions. Finally, there
556
/// are floating values that do not have a corresponding attribute list
557
/// position.
558
struct IRPosition {
559
  // NOTE: In the future this definition can be changed to support recursive
560
  // functions.
561
  using CallBaseContext = CallBase;
562
 
563
  /// The positions we distinguish in the IR.
564
  enum Kind : char {
565
    IRP_INVALID,  ///< An invalid position.
566
    IRP_FLOAT,    ///< A position that is not associated with a spot suitable
567
                  ///< for attributes. This could be any value or instruction.
568
    IRP_RETURNED, ///< An attribute for the function return value.
569
    IRP_CALL_SITE_RETURNED, ///< An attribute for a call site return value.
570
    IRP_FUNCTION,           ///< An attribute for a function (scope).
571
    IRP_CALL_SITE,          ///< An attribute for a call site (function scope).
572
    IRP_ARGUMENT,           ///< An attribute for a function argument.
573
    IRP_CALL_SITE_ARGUMENT, ///< An attribute for a call site argument.
574
  };
575
 
576
  /// Default constructor available to create invalid positions implicitly. All
577
  /// other positions need to be created explicitly through the appropriate
578
  /// static member function.
579
  IRPosition() : Enc(nullptr, ENC_VALUE) { verify(); }
580
 
581
  /// Create a position describing the value of \p V.
582
  static const IRPosition value(const Value &V,
583
                                const CallBaseContext *CBContext = nullptr) {
584
    if (auto *Arg = dyn_cast<Argument>(&V))
585
      return IRPosition::argument(*Arg, CBContext);
586
    if (auto *CB = dyn_cast<CallBase>(&V))
587
      return IRPosition::callsite_returned(*CB);
588
    return IRPosition(const_cast<Value &>(V), IRP_FLOAT, CBContext);
589
  }
590
 
591
  /// Create a position describing the instruction \p I. This is different from
592
  /// the value version because call sites are treated as intrusctions rather
593
  /// than their return value in this function.
594
  static const IRPosition inst(const Instruction &I,
595
                               const CallBaseContext *CBContext = nullptr) {
596
    return IRPosition(const_cast<Instruction &>(I), IRP_FLOAT, CBContext);
597
  }
598
 
599
  /// Create a position describing the function scope of \p F.
600
  /// \p CBContext is used for call base specific analysis.
601
  static const IRPosition function(const Function &F,
602
                                   const CallBaseContext *CBContext = nullptr) {
603
    return IRPosition(const_cast<Function &>(F), IRP_FUNCTION, CBContext);
604
  }
605
 
606
  /// Create a position describing the returned value of \p F.
607
  /// \p CBContext is used for call base specific analysis.
608
  static const IRPosition returned(const Function &F,
609
                                   const CallBaseContext *CBContext = nullptr) {
610
    return IRPosition(const_cast<Function &>(F), IRP_RETURNED, CBContext);
611
  }
612
 
613
  /// Create a position describing the argument \p Arg.
614
  /// \p CBContext is used for call base specific analysis.
615
  static const IRPosition argument(const Argument &Arg,
616
                                   const CallBaseContext *CBContext = nullptr) {
617
    return IRPosition(const_cast<Argument &>(Arg), IRP_ARGUMENT, CBContext);
618
  }
619
 
620
  /// Create a position describing the function scope of \p CB.
621
  static const IRPosition callsite_function(const CallBase &CB) {
622
    return IRPosition(const_cast<CallBase &>(CB), IRP_CALL_SITE);
623
  }
624
 
625
  /// Create a position describing the returned value of \p CB.
626
  static const IRPosition callsite_returned(const CallBase &CB) {
627
    return IRPosition(const_cast<CallBase &>(CB), IRP_CALL_SITE_RETURNED);
628
  }
629
 
630
  /// Create a position describing the argument of \p CB at position \p ArgNo.
631
  static const IRPosition callsite_argument(const CallBase &CB,
632
                                            unsigned ArgNo) {
633
    return IRPosition(const_cast<Use &>(CB.getArgOperandUse(ArgNo)),
634
                      IRP_CALL_SITE_ARGUMENT);
635
  }
636
 
637
  /// Create a position describing the argument of \p ACS at position \p ArgNo.
638
  static const IRPosition callsite_argument(AbstractCallSite ACS,
639
                                            unsigned ArgNo) {
640
    if (ACS.getNumArgOperands() <= ArgNo)
641
      return IRPosition();
642
    int CSArgNo = ACS.getCallArgOperandNo(ArgNo);
643
    if (CSArgNo >= 0)
644
      return IRPosition::callsite_argument(
645
          cast<CallBase>(*ACS.getInstruction()), CSArgNo);
646
    return IRPosition();
647
  }
648
 
649
  /// Create a position with function scope matching the "context" of \p IRP.
650
  /// If \p IRP is a call site (see isAnyCallSitePosition()) then the result
651
  /// will be a call site position, otherwise the function position of the
652
  /// associated function.
653
  static const IRPosition
654
  function_scope(const IRPosition &IRP,
655
                 const CallBaseContext *CBContext = nullptr) {
656
    if (IRP.isAnyCallSitePosition()) {
657
      return IRPosition::callsite_function(
658
          cast<CallBase>(IRP.getAnchorValue()));
659
    }
660
    assert(IRP.getAssociatedFunction());
661
    return IRPosition::function(*IRP.getAssociatedFunction(), CBContext);
662
  }
663
 
664
  bool operator==(const IRPosition &RHS) const {
665
    return Enc == RHS.Enc && RHS.CBContext == CBContext;
666
  }
667
  bool operator!=(const IRPosition &RHS) const { return !(*this == RHS); }
668
 
669
  /// Return the value this abstract attribute is anchored with.
670
  ///
671
  /// The anchor value might not be the associated value if the latter is not
672
  /// sufficient to determine where arguments will be manifested. This is, so
673
  /// far, only the case for call site arguments as the value is not sufficient
674
  /// to pinpoint them. Instead, we can use the call site as an anchor.
675
  Value &getAnchorValue() const {
676
    switch (getEncodingBits()) {
677
    case ENC_VALUE:
678
    case ENC_RETURNED_VALUE:
679
    case ENC_FLOATING_FUNCTION:
680
      return *getAsValuePtr();
681
    case ENC_CALL_SITE_ARGUMENT_USE:
682
      return *(getAsUsePtr()->getUser());
683
    default:
684
      llvm_unreachable("Unkown encoding!");
685
    };
686
  }
687
 
688
  /// Return the associated function, if any.
689
  Function *getAssociatedFunction() const {
690
    if (auto *CB = dyn_cast<CallBase>(&getAnchorValue())) {
691
      // We reuse the logic that associates callback calles to arguments of a
692
      // call site here to identify the callback callee as the associated
693
      // function.
694
      if (Argument *Arg = getAssociatedArgument())
695
        return Arg->getParent();
696
      return CB->getCalledFunction();
697
    }
698
    return getAnchorScope();
699
  }
700
 
701
  /// Return the associated argument, if any.
702
  Argument *getAssociatedArgument() const;
703
 
704
  /// Return true if the position refers to a function interface, that is the
705
  /// function scope, the function return, or an argument.
706
  bool isFnInterfaceKind() const {
707
    switch (getPositionKind()) {
708
    case IRPosition::IRP_FUNCTION:
709
    case IRPosition::IRP_RETURNED:
710
    case IRPosition::IRP_ARGUMENT:
711
      return true;
712
    default:
713
      return false;
714
    }
715
  }
716
 
717
  /// Return the Function surrounding the anchor value.
718
  Function *getAnchorScope() const {
719
    Value &V = getAnchorValue();
720
    if (isa<Function>(V))
721
      return &cast<Function>(V);
722
    if (isa<Argument>(V))
723
      return cast<Argument>(V).getParent();
724
    if (isa<Instruction>(V))
725
      return cast<Instruction>(V).getFunction();
726
    return nullptr;
727
  }
728
 
729
  /// Return the context instruction, if any.
730
  Instruction *getCtxI() const {
731
    Value &V = getAnchorValue();
732
    if (auto *I = dyn_cast<Instruction>(&V))
733
      return I;
734
    if (auto *Arg = dyn_cast<Argument>(&V))
735
      if (!Arg->getParent()->isDeclaration())
736
        return &Arg->getParent()->getEntryBlock().front();
737
    if (auto *F = dyn_cast<Function>(&V))
738
      if (!F->isDeclaration())
739
        return &(F->getEntryBlock().front());
740
    return nullptr;
741
  }
742
 
743
  /// Return the value this abstract attribute is associated with.
744
  Value &getAssociatedValue() const {
745
    if (getCallSiteArgNo() < 0 || isa<Argument>(&getAnchorValue()))
746
      return getAnchorValue();
747
    assert(isa<CallBase>(&getAnchorValue()) && "Expected a call base!");
748
    return *cast<CallBase>(&getAnchorValue())
749
                ->getArgOperand(getCallSiteArgNo());
750
  }
751
 
752
  /// Return the type this abstract attribute is associated with.
753
  Type *getAssociatedType() const {
754
    if (getPositionKind() == IRPosition::IRP_RETURNED)
755
      return getAssociatedFunction()->getReturnType();
756
    return getAssociatedValue().getType();
757
  }
758
 
759
  /// Return the callee argument number of the associated value if it is an
760
  /// argument or call site argument, otherwise a negative value. In contrast to
761
  /// `getCallSiteArgNo` this method will always return the "argument number"
762
  /// from the perspective of the callee. This may not the same as the call site
763
  /// if this is a callback call.
764
  int getCalleeArgNo() const {
765
    return getArgNo(/* CallbackCalleeArgIfApplicable */ true);
766
  }
767
 
768
  /// Return the call site argument number of the associated value if it is an
769
  /// argument or call site argument, otherwise a negative value. In contrast to
770
  /// `getCalleArgNo` this method will always return the "operand number" from
771
  /// the perspective of the call site. This may not the same as the callee
772
  /// perspective if this is a callback call.
773
  int getCallSiteArgNo() const {
774
    return getArgNo(/* CallbackCalleeArgIfApplicable */ false);
775
  }
776
 
777
  /// Return the index in the attribute list for this position.
778
  unsigned getAttrIdx() const {
779
    switch (getPositionKind()) {
780
    case IRPosition::IRP_INVALID:
781
    case IRPosition::IRP_FLOAT:
782
      break;
783
    case IRPosition::IRP_FUNCTION:
784
    case IRPosition::IRP_CALL_SITE:
785
      return AttributeList::FunctionIndex;
786
    case IRPosition::IRP_RETURNED:
787
    case IRPosition::IRP_CALL_SITE_RETURNED:
788
      return AttributeList::ReturnIndex;
789
    case IRPosition::IRP_ARGUMENT:
790
    case IRPosition::IRP_CALL_SITE_ARGUMENT:
791
      return getCallSiteArgNo() + AttributeList::FirstArgIndex;
792
    }
793
    llvm_unreachable(
794
        "There is no attribute index for a floating or invalid position!");
795
  }
796
 
797
  /// Return the associated position kind.
798
  Kind getPositionKind() const {
799
    char EncodingBits = getEncodingBits();
800
    if (EncodingBits == ENC_CALL_SITE_ARGUMENT_USE)
801
      return IRP_CALL_SITE_ARGUMENT;
802
    if (EncodingBits == ENC_FLOATING_FUNCTION)
803
      return IRP_FLOAT;
804
 
805
    Value *V = getAsValuePtr();
806
    if (!V)
807
      return IRP_INVALID;
808
    if (isa<Argument>(V))
809
      return IRP_ARGUMENT;
810
    if (isa<Function>(V))
811
      return isReturnPosition(EncodingBits) ? IRP_RETURNED : IRP_FUNCTION;
812
    if (isa<CallBase>(V))
813
      return isReturnPosition(EncodingBits) ? IRP_CALL_SITE_RETURNED
814
                                            : IRP_CALL_SITE;
815
    return IRP_FLOAT;
816
  }
817
 
818
  /// TODO: Figure out if the attribute related helper functions should live
819
  ///       here or somewhere else.
820
 
821
  /// Return true if any kind in \p AKs existing in the IR at a position that
822
  /// will affect this one. See also getAttrs(...).
823
  /// \param IgnoreSubsumingPositions Flag to determine if subsuming positions,
824
  ///                                 e.g., the function position if this is an
825
  ///                                 argument position, should be ignored.
826
  bool hasAttr(ArrayRef<Attribute::AttrKind> AKs,
827
               bool IgnoreSubsumingPositions = false,
828
               Attributor *A = nullptr) const;
829
 
830
  /// Return the attributes of any kind in \p AKs existing in the IR at a
831
  /// position that will affect this one. While each position can only have a
832
  /// single attribute of any kind in \p AKs, there are "subsuming" positions
833
  /// that could have an attribute as well. This method returns all attributes
834
  /// found in \p Attrs.
835
  /// \param IgnoreSubsumingPositions Flag to determine if subsuming positions,
836
  ///                                 e.g., the function position if this is an
837
  ///                                 argument position, should be ignored.
838
  void getAttrs(ArrayRef<Attribute::AttrKind> AKs,
839
                SmallVectorImpl<Attribute> &Attrs,
840
                bool IgnoreSubsumingPositions = false,
841
                Attributor *A = nullptr) const;
842
 
843
  /// Remove the attribute of kind \p AKs existing in the IR at this position.
844
  void removeAttrs(ArrayRef<Attribute::AttrKind> AKs) const {
845
    if (getPositionKind() == IRP_INVALID || getPositionKind() == IRP_FLOAT)
846
      return;
847
 
848
    AttributeList AttrList;
849
    auto *CB = dyn_cast<CallBase>(&getAnchorValue());
850
    if (CB)
851
      AttrList = CB->getAttributes();
852
    else
853
      AttrList = getAssociatedFunction()->getAttributes();
854
 
855
    LLVMContext &Ctx = getAnchorValue().getContext();
856
    for (Attribute::AttrKind AK : AKs)
857
      AttrList = AttrList.removeAttributeAtIndex(Ctx, getAttrIdx(), AK);
858
 
859
    if (CB)
860
      CB->setAttributes(AttrList);
861
    else
862
      getAssociatedFunction()->setAttributes(AttrList);
863
  }
864
 
865
  bool isAnyCallSitePosition() const {
866
    switch (getPositionKind()) {
867
    case IRPosition::IRP_CALL_SITE:
868
    case IRPosition::IRP_CALL_SITE_RETURNED:
869
    case IRPosition::IRP_CALL_SITE_ARGUMENT:
870
      return true;
871
    default:
872
      return false;
873
    }
874
  }
875
 
876
  /// Return true if the position is an argument or call site argument.
877
  bool isArgumentPosition() const {
878
    switch (getPositionKind()) {
879
    case IRPosition::IRP_ARGUMENT:
880
    case IRPosition::IRP_CALL_SITE_ARGUMENT:
881
      return true;
882
    default:
883
      return false;
884
    }
885
  }
886
 
887
  /// Return the same position without the call base context.
888
  IRPosition stripCallBaseContext() const {
889
    IRPosition Result = *this;
890
    Result.CBContext = nullptr;
891
    return Result;
892
  }
893
 
894
  /// Get the call base context from the position.
895
  const CallBaseContext *getCallBaseContext() const { return CBContext; }
896
 
897
  /// Check if the position has any call base context.
898
  bool hasCallBaseContext() const { return CBContext != nullptr; }
899
 
900
  /// Special DenseMap key values.
901
  ///
902
  ///{
903
  static const IRPosition EmptyKey;
904
  static const IRPosition TombstoneKey;
905
  ///}
906
 
907
  /// Conversion into a void * to allow reuse of pointer hashing.
908
  operator void *() const { return Enc.getOpaqueValue(); }
909
 
910
private:
911
  /// Private constructor for special values only!
912
  explicit IRPosition(void *Ptr, const CallBaseContext *CBContext = nullptr)
913
      : CBContext(CBContext) {
914
    Enc.setFromOpaqueValue(Ptr);
915
  }
916
 
917
  /// IRPosition anchored at \p AnchorVal with kind/argument numbet \p PK.
918
  explicit IRPosition(Value &AnchorVal, Kind PK,
919
                      const CallBaseContext *CBContext = nullptr)
920
      : CBContext(CBContext) {
921
    switch (PK) {
922
    case IRPosition::IRP_INVALID:
923
      llvm_unreachable("Cannot create invalid IRP with an anchor value!");
924
      break;
925
    case IRPosition::IRP_FLOAT:
926
      // Special case for floating functions.
927
      if (isa<Function>(AnchorVal) || isa<CallBase>(AnchorVal))
928
        Enc = {&AnchorVal, ENC_FLOATING_FUNCTION};
929
      else
930
        Enc = {&AnchorVal, ENC_VALUE};
931
      break;
932
    case IRPosition::IRP_FUNCTION:
933
    case IRPosition::IRP_CALL_SITE:
934
      Enc = {&AnchorVal, ENC_VALUE};
935
      break;
936
    case IRPosition::IRP_RETURNED:
937
    case IRPosition::IRP_CALL_SITE_RETURNED:
938
      Enc = {&AnchorVal, ENC_RETURNED_VALUE};
939
      break;
940
    case IRPosition::IRP_ARGUMENT:
941
      Enc = {&AnchorVal, ENC_VALUE};
942
      break;
943
    case IRPosition::IRP_CALL_SITE_ARGUMENT:
944
      llvm_unreachable(
945
          "Cannot create call site argument IRP with an anchor value!");
946
      break;
947
    }
948
    verify();
949
  }
950
 
951
  /// Return the callee argument number of the associated value if it is an
952
  /// argument or call site argument. See also `getCalleeArgNo` and
953
  /// `getCallSiteArgNo`.
954
  int getArgNo(bool CallbackCalleeArgIfApplicable) const {
955
    if (CallbackCalleeArgIfApplicable)
956
      if (Argument *Arg = getAssociatedArgument())
957
        return Arg->getArgNo();
958
    switch (getPositionKind()) {
959
    case IRPosition::IRP_ARGUMENT:
960
      return cast<Argument>(getAsValuePtr())->getArgNo();
961
    case IRPosition::IRP_CALL_SITE_ARGUMENT: {
962
      Use &U = *getAsUsePtr();
963
      return cast<CallBase>(U.getUser())->getArgOperandNo(&U);
964
    }
965
    default:
966
      return -1;
967
    }
968
  }
969
 
970
  /// IRPosition for the use \p U. The position kind \p PK needs to be
971
  /// IRP_CALL_SITE_ARGUMENT, the anchor value is the user, the associated value
972
  /// the used value.
973
  explicit IRPosition(Use &U, Kind PK) {
974
    assert(PK == IRP_CALL_SITE_ARGUMENT &&
975
           "Use constructor is for call site arguments only!");
976
    Enc = {&U, ENC_CALL_SITE_ARGUMENT_USE};
977
    verify();
978
  }
979
 
980
  /// Verify internal invariants.
981
  void verify();
982
 
983
  /// Return the attributes of kind \p AK existing in the IR as attribute.
984
  bool getAttrsFromIRAttr(Attribute::AttrKind AK,
985
                          SmallVectorImpl<Attribute> &Attrs) const;
986
 
987
  /// Return the attributes of kind \p AK existing in the IR as operand bundles
988
  /// of an llvm.assume.
989
  bool getAttrsFromAssumes(Attribute::AttrKind AK,
990
                           SmallVectorImpl<Attribute> &Attrs,
991
                           Attributor &A) const;
992
 
993
  /// Return the underlying pointer as Value *, valid for all positions but
994
  /// IRP_CALL_SITE_ARGUMENT.
995
  Value *getAsValuePtr() const {
996
    assert(getEncodingBits() != ENC_CALL_SITE_ARGUMENT_USE &&
997
           "Not a value pointer!");
998
    return reinterpret_cast<Value *>(Enc.getPointer());
999
  }
1000
 
1001
  /// Return the underlying pointer as Use *, valid only for
1002
  /// IRP_CALL_SITE_ARGUMENT positions.
1003
  Use *getAsUsePtr() const {
1004
    assert(getEncodingBits() == ENC_CALL_SITE_ARGUMENT_USE &&
1005
           "Not a value pointer!");
1006
    return reinterpret_cast<Use *>(Enc.getPointer());
1007
  }
1008
 
1009
  /// Return true if \p EncodingBits describe a returned or call site returned
1010
  /// position.
1011
  static bool isReturnPosition(char EncodingBits) {
1012
    return EncodingBits == ENC_RETURNED_VALUE;
1013
  }
1014
 
1015
  /// Return true if the encoding bits describe a returned or call site returned
1016
  /// position.
1017
  bool isReturnPosition() const { return isReturnPosition(getEncodingBits()); }
1018
 
1019
  /// The encoding of the IRPosition is a combination of a pointer and two
1020
  /// encoding bits. The values of the encoding bits are defined in the enum
1021
  /// below. The pointer is either a Value* (for the first three encoding bit
1022
  /// combinations) or Use* (for ENC_CALL_SITE_ARGUMENT_USE).
1023
  ///
1024
  ///{
1025
  enum {
1026
    ENC_VALUE = 0b00,
1027
    ENC_RETURNED_VALUE = 0b01,
1028
    ENC_FLOATING_FUNCTION = 0b10,
1029
    ENC_CALL_SITE_ARGUMENT_USE = 0b11,
1030
  };
1031
 
1032
  // Reserve the maximal amount of bits so there is no need to mask out the
1033
  // remaining ones. We will not encode anything else in the pointer anyway.
1034
  static constexpr int NumEncodingBits =
1035
      PointerLikeTypeTraits<void *>::NumLowBitsAvailable;
1036
  static_assert(NumEncodingBits >= 2, "At least two bits are required!");
1037
 
1038
  /// The pointer with the encoding bits.
1039
  PointerIntPair<void *, NumEncodingBits, char> Enc;
1040
  ///}
1041
 
1042
  /// Call base context. Used for callsite specific analysis.
1043
  const CallBaseContext *CBContext = nullptr;
1044
 
1045
  /// Return the encoding bits.
1046
  char getEncodingBits() const { return Enc.getInt(); }
1047
};
1048
 
1049
/// Helper that allows IRPosition as a key in a DenseMap.
1050
template <> struct DenseMapInfo<IRPosition> {
1051
  static inline IRPosition getEmptyKey() { return IRPosition::EmptyKey; }
1052
  static inline IRPosition getTombstoneKey() {
1053
    return IRPosition::TombstoneKey;
1054
  }
1055
  static unsigned getHashValue(const IRPosition &IRP) {
1056
    return (DenseMapInfo<void *>::getHashValue(IRP) << 4) ^
1057
           (DenseMapInfo<Value *>::getHashValue(IRP.getCallBaseContext()));
1058
  }
1059
 
1060
  static bool isEqual(const IRPosition &a, const IRPosition &b) {
1061
    return a == b;
1062
  }
1063
};
1064
 
1065
/// A visitor class for IR positions.
1066
///
1067
/// Given a position P, the SubsumingPositionIterator allows to visit "subsuming
1068
/// positions" wrt. attributes/information. Thus, if a piece of information
1069
/// holds for a subsuming position, it also holds for the position P.
1070
///
1071
/// The subsuming positions always include the initial position and then,
1072
/// depending on the position kind, additionally the following ones:
1073
/// - for IRP_RETURNED:
1074
///   - the function (IRP_FUNCTION)
1075
/// - for IRP_ARGUMENT:
1076
///   - the function (IRP_FUNCTION)
1077
/// - for IRP_CALL_SITE:
1078
///   - the callee (IRP_FUNCTION), if known
1079
/// - for IRP_CALL_SITE_RETURNED:
1080
///   - the callee (IRP_RETURNED), if known
1081
///   - the call site (IRP_FUNCTION)
1082
///   - the callee (IRP_FUNCTION), if known
1083
/// - for IRP_CALL_SITE_ARGUMENT:
1084
///   - the argument of the callee (IRP_ARGUMENT), if known
1085
///   - the callee (IRP_FUNCTION), if known
1086
///   - the position the call site argument is associated with if it is not
1087
///     anchored to the call site, e.g., if it is an argument then the argument
1088
///     (IRP_ARGUMENT)
1089
class SubsumingPositionIterator {
1090
  SmallVector<IRPosition, 4> IRPositions;
1091
  using iterator = decltype(IRPositions)::iterator;
1092
 
1093
public:
1094
  SubsumingPositionIterator(const IRPosition &IRP);
1095
  iterator begin() { return IRPositions.begin(); }
1096
  iterator end() { return IRPositions.end(); }
1097
};
1098
 
1099
/// Wrapper for FunctionAnalysisManager.
1100
struct AnalysisGetter {
1101
  // The client may be running the old pass manager, in which case, we need to
1102
  // map the requested Analysis to its equivalent wrapper in the old pass
1103
  // manager. The scheme implemented here does not require every Analysis to be
1104
  // updated. Only those new analyses that the client cares about in the old
1105
  // pass manager need to expose a LegacyWrapper type, and that wrapper should
1106
  // support a getResult() method that matches the new Analysis.
1107
  //
1108
  // We need SFINAE to check for the LegacyWrapper, but function templates don't
1109
  // allow partial specialization, which is needed in this case. So instead, we
1110
  // use a constexpr bool to perform the SFINAE, and then use this information
1111
  // inside the function template.
1112
  template <typename, typename = void> static constexpr bool HasLegacyWrapper = false;
1113
 
1114
  template <typename Analysis>
1115
  typename Analysis::Result *getAnalysis(const Function &F) {
1116
    if (FAM)
1117
      return &FAM->getResult<Analysis>(const_cast<Function &>(F));
1118
    if constexpr (HasLegacyWrapper<Analysis>)
1119
      if (LegacyPass)
1120
        return &LegacyPass
1121
                    ->getAnalysis<typename Analysis::LegacyWrapper>(
1122
                        const_cast<Function &>(F))
1123
                    .getResult();
1124
    return nullptr;
1125
  }
1126
 
1127
  AnalysisGetter(FunctionAnalysisManager &FAM) : FAM(&FAM) {}
1128
  AnalysisGetter(Pass *P) : LegacyPass(P) {}
1129
  AnalysisGetter() = default;
1130
 
1131
private:
1132
  FunctionAnalysisManager *FAM = nullptr;
1133
  Pass *LegacyPass = nullptr;
1134
};
1135
 
1136
template <typename Analysis>
1137
constexpr bool AnalysisGetter::HasLegacyWrapper<
1138
      Analysis, std::void_t<typename Analysis::LegacyWrapper>> = true;
1139
 
1140
/// Data structure to hold cached (LLVM-IR) information.
1141
///
1142
/// All attributes are given an InformationCache object at creation time to
1143
/// avoid inspection of the IR by all of them individually. This default
1144
/// InformationCache will hold information required by 'default' attributes,
1145
/// thus the ones deduced when Attributor::identifyDefaultAbstractAttributes(..)
1146
/// is called.
1147
///
1148
/// If custom abstract attributes, registered manually through
1149
/// Attributor::registerAA(...), need more information, especially if it is not
1150
/// reusable, it is advised to inherit from the InformationCache and cast the
1151
/// instance down in the abstract attributes.
1152
struct InformationCache {
1153
  InformationCache(const Module &M, AnalysisGetter &AG,
1154
                   BumpPtrAllocator &Allocator, SetVector<Function *> *CGSCC)
1155
      : DL(M.getDataLayout()), Allocator(Allocator),
1156
        Explorer(
1157
            /* ExploreInterBlock */ true, /* ExploreCFGForward */ true,
1158
            /* ExploreCFGBackward */ true,
1159
            /* LIGetter */
1160
            [&](const Function &F) { return AG.getAnalysis<LoopAnalysis>(F); },
1161
            /* DTGetter */
1162
            [&](const Function &F) {
1163
              return AG.getAnalysis<DominatorTreeAnalysis>(F);
1164
            },
1165
            /* PDTGetter */
1166
            [&](const Function &F) {
1167
              return AG.getAnalysis<PostDominatorTreeAnalysis>(F);
1168
            }),
1169
        AG(AG), TargetTriple(M.getTargetTriple()) {
1170
    if (CGSCC)
1171
      initializeModuleSlice(*CGSCC);
1172
  }
1173
 
1174
  ~InformationCache() {
1175
    // The FunctionInfo objects are allocated via a BumpPtrAllocator, we call
1176
    // the destructor manually.
1177
    for (auto &It : FuncInfoMap)
1178
      It.getSecond()->~FunctionInfo();
1179
    // Same is true for the instruction exclusions sets.
1180
    using AA::InstExclusionSetTy;
1181
    for (auto *BES : BESets)
1182
      BES->~InstExclusionSetTy();
1183
  }
1184
 
1185
  /// Apply \p CB to all uses of \p F. If \p LookThroughConstantExprUses is
1186
  /// true, constant expression users are not given to \p CB but their uses are
1187
  /// traversed transitively.
1188
  template <typename CBTy>
1189
  static void foreachUse(Function &F, CBTy CB,
1190
                         bool LookThroughConstantExprUses = true) {
1191
    SmallVector<Use *, 8> Worklist(make_pointer_range(F.uses()));
1192
 
1193
    for (unsigned Idx = 0; Idx < Worklist.size(); ++Idx) {
1194
      Use &U = *Worklist[Idx];
1195
 
1196
      // Allow use in constant bitcasts and simply look through them.
1197
      if (LookThroughConstantExprUses && isa<ConstantExpr>(U.getUser())) {
1198
        for (Use &CEU : cast<ConstantExpr>(U.getUser())->uses())
1199
          Worklist.push_back(&CEU);
1200
        continue;
1201
      }
1202
 
1203
      CB(U);
1204
    }
1205
  }
1206
 
1207
  /// Initialize the ModuleSlice member based on \p SCC. ModuleSlices contains
1208
  /// (a subset of) all functions that we can look at during this SCC traversal.
1209
  /// This includes functions (transitively) called from the SCC and the
1210
  /// (transitive) callers of SCC functions. We also can look at a function if
1211
  /// there is a "reference edge", i.a., if the function somehow uses (!=calls)
1212
  /// a function in the SCC or a caller of a function in the SCC.
1213
  void initializeModuleSlice(SetVector<Function *> &SCC) {
1214
    ModuleSlice.insert(SCC.begin(), SCC.end());
1215
 
1216
    SmallPtrSet<Function *, 16> Seen;
1217
    SmallVector<Function *, 16> Worklist(SCC.begin(), SCC.end());
1218
    while (!Worklist.empty()) {
1219
      Function *F = Worklist.pop_back_val();
1220
      ModuleSlice.insert(F);
1221
 
1222
      for (Instruction &I : instructions(*F))
1223
        if (auto *CB = dyn_cast<CallBase>(&I))
1224
          if (Function *Callee = CB->getCalledFunction())
1225
            if (Seen.insert(Callee).second)
1226
              Worklist.push_back(Callee);
1227
    }
1228
 
1229
    Seen.clear();
1230
    Worklist.append(SCC.begin(), SCC.end());
1231
    while (!Worklist.empty()) {
1232
      Function *F = Worklist.pop_back_val();
1233
      ModuleSlice.insert(F);
1234
 
1235
      // Traverse all transitive uses.
1236
      foreachUse(*F, [&](Use &U) {
1237
        if (auto *UsrI = dyn_cast<Instruction>(U.getUser()))
1238
          if (Seen.insert(UsrI->getFunction()).second)
1239
            Worklist.push_back(UsrI->getFunction());
1240
      });
1241
    }
1242
  }
1243
 
1244
  /// The slice of the module we are allowed to look at.
1245
  SmallPtrSet<Function *, 8> ModuleSlice;
1246
 
1247
  /// A vector type to hold instructions.
1248
  using InstructionVectorTy = SmallVector<Instruction *, 8>;
1249
 
1250
  /// A map type from opcodes to instructions with this opcode.
1251
  using OpcodeInstMapTy = DenseMap<unsigned, InstructionVectorTy *>;
1252
 
1253
  /// Return the map that relates "interesting" opcodes with all instructions
1254
  /// with that opcode in \p F.
1255
  OpcodeInstMapTy &getOpcodeInstMapForFunction(const Function &F) {
1256
    return getFunctionInfo(F).OpcodeInstMap;
1257
  }
1258
 
1259
  /// Return the instructions in \p F that may read or write memory.
1260
  InstructionVectorTy &getReadOrWriteInstsForFunction(const Function &F) {
1261
    return getFunctionInfo(F).RWInsts;
1262
  }
1263
 
1264
  /// Return MustBeExecutedContextExplorer
1265
  MustBeExecutedContextExplorer &getMustBeExecutedContextExplorer() {
1266
    return Explorer;
1267
  }
1268
 
1269
  /// Return TargetLibraryInfo for function \p F.
1270
  TargetLibraryInfo *getTargetLibraryInfoForFunction(const Function &F) {
1271
    return AG.getAnalysis<TargetLibraryAnalysis>(F);
1272
  }
1273
 
1274
  /// Return AliasAnalysis Result for function \p F.
1275
  AAResults *getAAResultsForFunction(const Function &F);
1276
 
1277
  /// Return true if \p Arg is involved in a must-tail call, thus the argument
1278
  /// of the caller or callee.
1279
  bool isInvolvedInMustTailCall(const Argument &Arg) {
1280
    FunctionInfo &FI = getFunctionInfo(*Arg.getParent());
1281
    return FI.CalledViaMustTail || FI.ContainsMustTailCall;
1282
  }
1283
 
1284
  bool isOnlyUsedByAssume(const Instruction &I) const {
1285
    return AssumeOnlyValues.contains(&I);
1286
  }
1287
 
1288
  /// Return the analysis result from a pass \p AP for function \p F.
1289
  template <typename AP>
1290
  typename AP::Result *getAnalysisResultForFunction(const Function &F) {
1291
    return AG.getAnalysis<AP>(F);
1292
  }
1293
 
1294
  /// Return datalayout used in the module.
1295
  const DataLayout &getDL() { return DL; }
1296
 
1297
  /// Return the map conaining all the knowledge we have from `llvm.assume`s.
1298
  const RetainedKnowledgeMap &getKnowledgeMap() const { return KnowledgeMap; }
1299
 
1300
  /// Given \p BES, return a uniqued version. \p BES is destroyed in the
1301
  /// process.
1302
  const AA::InstExclusionSetTy *
1303
  getOrCreateUniqueBlockExecutionSet(const AA::InstExclusionSetTy *BES) {
1304
    auto It = BESets.find(BES);
1305
    if (It != BESets.end())
1306
      return *It;
1307
    auto *UniqueBES = new (Allocator) AA::InstExclusionSetTy(*BES);
1308
    BESets.insert(UniqueBES);
1309
    return UniqueBES;
1310
  }
1311
 
1312
  /// Check whether \p F is part of module slice.
1313
  bool isInModuleSlice(const Function &F) {
1314
    return ModuleSlice.empty() || ModuleSlice.count(const_cast<Function *>(&F));
1315
  }
1316
 
1317
  /// Return true if the stack (llvm::Alloca) can be accessed by other threads.
1318
  bool stackIsAccessibleByOtherThreads() { return !targetIsGPU(); }
1319
 
1320
  /// Return true if the target is a GPU.
1321
  bool targetIsGPU() {
1322
    return TargetTriple.isAMDGPU() || TargetTriple.isNVPTX();
1323
  }
1324
 
1325
private:
1326
  struct FunctionInfo {
1327
    ~FunctionInfo();
1328
 
1329
    /// A nested map that remembers all instructions in a function with a
1330
    /// certain instruction opcode (Instruction::getOpcode()).
1331
    OpcodeInstMapTy OpcodeInstMap;
1332
 
1333
    /// A map from functions to their instructions that may read or write
1334
    /// memory.
1335
    InstructionVectorTy RWInsts;
1336
 
1337
    /// Function is called by a `musttail` call.
1338
    bool CalledViaMustTail;
1339
 
1340
    /// Function contains a `musttail` call.
1341
    bool ContainsMustTailCall;
1342
  };
1343
 
1344
  /// A map type from functions to informatio about it.
1345
  DenseMap<const Function *, FunctionInfo *> FuncInfoMap;
1346
 
1347
  /// Return information about the function \p F, potentially by creating it.
1348
  FunctionInfo &getFunctionInfo(const Function &F) {
1349
    FunctionInfo *&FI = FuncInfoMap[&F];
1350
    if (!FI) {
1351
      FI = new (Allocator) FunctionInfo();
1352
      initializeInformationCache(F, *FI);
1353
    }
1354
    return *FI;
1355
  }
1356
 
1357
  /// Initialize the function information cache \p FI for the function \p F.
1358
  ///
1359
  /// This method needs to be called for all function that might be looked at
1360
  /// through the information cache interface *prior* to looking at them.
1361
  void initializeInformationCache(const Function &F, FunctionInfo &FI);
1362
 
1363
  /// The datalayout used in the module.
1364
  const DataLayout &DL;
1365
 
1366
  /// The allocator used to allocate memory, e.g. for `FunctionInfo`s.
1367
  BumpPtrAllocator &Allocator;
1368
 
1369
  /// MustBeExecutedContextExplorer
1370
  MustBeExecutedContextExplorer Explorer;
1371
 
1372
  /// A map with knowledge retained in `llvm.assume` instructions.
1373
  RetainedKnowledgeMap KnowledgeMap;
1374
 
1375
  /// A container for all instructions that are only used by `llvm.assume`.
1376
  SetVector<const Instruction *> AssumeOnlyValues;
1377
 
1378
  /// Cache for block sets to allow reuse.
1379
  DenseSet<AA::InstExclusionSetTy *> BESets;
1380
 
1381
  /// Getters for analysis.
1382
  AnalysisGetter &AG;
1383
 
1384
  /// Set of inlineable functions
1385
  SmallPtrSet<const Function *, 8> InlineableFunctions;
1386
 
1387
  /// The triple describing the target machine.
1388
  Triple TargetTriple;
1389
 
1390
  /// Give the Attributor access to the members so
1391
  /// Attributor::identifyDefaultAbstractAttributes(...) can initialize them.
1392
  friend struct Attributor;
1393
};
1394
 
1395
/// Configuration for the Attributor.
1396
struct AttributorConfig {
1397
 
1398
  AttributorConfig(CallGraphUpdater &CGUpdater) : CGUpdater(CGUpdater) {}
1399
 
1400
  /// Is the user of the Attributor a module pass or not. This determines what
1401
  /// IR we can look at and modify. If it is a module pass we might deduce facts
1402
  /// outside the initial function set and modify functions outside that set,
1403
  /// but only as part of the optimization of the functions in the initial
1404
  /// function set. For CGSCC passes we can look at the IR of the module slice
1405
  /// but never run any deduction, or perform any modification, outside the
1406
  /// initial function set (which we assume is the SCC).
1407
  bool IsModulePass = true;
1408
 
1409
  /// Flag to determine if we can delete functions or keep dead ones around.
1410
  bool DeleteFns = true;
1411
 
1412
  /// Flag to determine if we rewrite function signatures.
1413
  bool RewriteSignatures = true;
1414
 
1415
  /// Flag to determine if we want to initialize all default AAs for an internal
1416
  /// function marked live. See also: InitializationCallback>
1417
  bool DefaultInitializeLiveInternals = true;
1418
 
1419
  /// Callback function to be invoked on internal functions marked live.
1420
  std::function<void(Attributor &A, const Function &F)> InitializationCallback =
1421
      nullptr;
1422
 
1423
  /// Helper to update an underlying call graph and to delete functions.
1424
  CallGraphUpdater &CGUpdater;
1425
 
1426
  /// If not null, a set limiting the attribute opportunities.
1427
  DenseSet<const char *> *Allowed = nullptr;
1428
 
1429
  /// Maximum number of iterations to run until fixpoint.
1430
  std::optional<unsigned> MaxFixpointIterations;
1431
 
1432
  /// A callback function that returns an ORE object from a Function pointer.
1433
  ///{
1434
  using OptimizationRemarkGetter =
1435
      function_ref<OptimizationRemarkEmitter &(Function *)>;
1436
  OptimizationRemarkGetter OREGetter = nullptr;
1437
  ///}
1438
 
1439
  /// The name of the pass running the attributor, used to emit remarks.
1440
  const char *PassName = nullptr;
1441
};
1442
 
1443
/// The fixpoint analysis framework that orchestrates the attribute deduction.
1444
///
1445
/// The Attributor provides a general abstract analysis framework (guided
1446
/// fixpoint iteration) as well as helper functions for the deduction of
1447
/// (LLVM-IR) attributes. However, also other code properties can be deduced,
1448
/// propagated, and ultimately manifested through the Attributor framework. This
1449
/// is particularly useful if these properties interact with attributes and a
1450
/// co-scheduled deduction allows to improve the solution. Even if not, thus if
1451
/// attributes/properties are completely isolated, they should use the
1452
/// Attributor framework to reduce the number of fixpoint iteration frameworks
1453
/// in the code base. Note that the Attributor design makes sure that isolated
1454
/// attributes are not impacted, in any way, by others derived at the same time
1455
/// if there is no cross-reasoning performed.
1456
///
1457
/// The public facing interface of the Attributor is kept simple and basically
1458
/// allows abstract attributes to one thing, query abstract attributes
1459
/// in-flight. There are two reasons to do this:
1460
///    a) The optimistic state of one abstract attribute can justify an
1461
///       optimistic state of another, allowing to framework to end up with an
1462
///       optimistic (=best possible) fixpoint instead of one based solely on
1463
///       information in the IR.
1464
///    b) This avoids reimplementing various kinds of lookups, e.g., to check
1465
///       for existing IR attributes, in favor of a single lookups interface
1466
///       provided by an abstract attribute subclass.
1467
///
1468
/// NOTE: The mechanics of adding a new "concrete" abstract attribute are
1469
///       described in the file comment.
1470
struct Attributor {
1471
 
1472
  /// Constructor
1473
  ///
1474
  /// \param Functions The set of functions we are deriving attributes for.
1475
  /// \param InfoCache Cache to hold various information accessible for
1476
  ///                  the abstract attributes.
1477
  /// \param Configuration The Attributor configuration which determines what
1478
  ///                      generic features to use.
1479
  Attributor(SetVector<Function *> &Functions, InformationCache &InfoCache,
1480
             AttributorConfig Configuration)
1481
      : Allocator(InfoCache.Allocator), Functions(Functions),
1482
        InfoCache(InfoCache), Configuration(Configuration) {}
1483
 
1484
  ~Attributor();
1485
 
1486
  /// Run the analyses until a fixpoint is reached or enforced (timeout).
1487
  ///
1488
  /// The attributes registered with this Attributor can be used after as long
1489
  /// as the Attributor is not destroyed (it owns the attributes now).
1490
  ///
1491
  /// \Returns CHANGED if the IR was changed, otherwise UNCHANGED.
1492
  ChangeStatus run();
1493
 
1494
  /// Lookup an abstract attribute of type \p AAType at position \p IRP. While
1495
  /// no abstract attribute is found equivalent positions are checked, see
1496
  /// SubsumingPositionIterator. Thus, the returned abstract attribute
1497
  /// might be anchored at a different position, e.g., the callee if \p IRP is a
1498
  /// call base.
1499
  ///
1500
  /// This method is the only (supported) way an abstract attribute can retrieve
1501
  /// information from another abstract attribute. As an example, take an
1502
  /// abstract attribute that determines the memory access behavior for a
1503
  /// argument (readnone, readonly, ...). It should use `getAAFor` to get the
1504
  /// most optimistic information for other abstract attributes in-flight, e.g.
1505
  /// the one reasoning about the "captured" state for the argument or the one
1506
  /// reasoning on the memory access behavior of the function as a whole.
1507
  ///
1508
  /// If the DepClass enum is set to `DepClassTy::None` the dependence from
1509
  /// \p QueryingAA to the return abstract attribute is not automatically
1510
  /// recorded. This should only be used if the caller will record the
1511
  /// dependence explicitly if necessary, thus if it the returned abstract
1512
  /// attribute is used for reasoning. To record the dependences explicitly use
1513
  /// the `Attributor::recordDependence` method.
1514
  template <typename AAType>
1515
  const AAType &getAAFor(const AbstractAttribute &QueryingAA,
1516
                         const IRPosition &IRP, DepClassTy DepClass) {
1517
    return getOrCreateAAFor<AAType>(IRP, &QueryingAA, DepClass,
1518
                                    /* ForceUpdate */ false);
1519
  }
1520
 
1521
  /// Similar to getAAFor but the return abstract attribute will be updated (via
1522
  /// `AbstractAttribute::update`) even if it is found in the cache. This is
1523
  /// especially useful for AAIsDead as changes in liveness can make updates
1524
  /// possible/useful that were not happening before as the abstract attribute
1525
  /// was assumed dead.
1526
  template <typename AAType>
1527
  const AAType &getAndUpdateAAFor(const AbstractAttribute &QueryingAA,
1528
                                  const IRPosition &IRP, DepClassTy DepClass) {
1529
    return getOrCreateAAFor<AAType>(IRP, &QueryingAA, DepClass,
1530
                                    /* ForceUpdate */ true);
1531
  }
1532
 
1533
  /// The version of getAAFor that allows to omit a querying abstract
1534
  /// attribute. Using this after Attributor started running is restricted to
1535
  /// only the Attributor itself. Initial seeding of AAs can be done via this
1536
  /// function.
1537
  /// NOTE: ForceUpdate is ignored in any stage other than the update stage.
1538
  template <typename AAType>
1539
  const AAType &getOrCreateAAFor(IRPosition IRP,
1540
                                 const AbstractAttribute *QueryingAA,
1541
                                 DepClassTy DepClass, bool ForceUpdate = false,
1542
                                 bool UpdateAfterInit = true) {
1543
    if (!shouldPropagateCallBaseContext(IRP))
1544
      IRP = IRP.stripCallBaseContext();
1545
 
1546
    if (AAType *AAPtr = lookupAAFor<AAType>(IRP, QueryingAA, DepClass,
1547
                                            /* AllowInvalidState */ true)) {
1548
      if (ForceUpdate && Phase == AttributorPhase::UPDATE)
1549
        updateAA(*AAPtr);
1550
      return *AAPtr;
1551
    }
1552
 
1553
    // No matching attribute found, create one.
1554
    // Use the static create method.
1555
    auto &AA = AAType::createForPosition(IRP, *this);
1556
 
1557
    // Always register a new attribute to make sure we clean up the allocated
1558
    // memory properly.
1559
    registerAA(AA);
1560
 
1561
    // If we are currenty seeding attributes, enforce seeding rules.
1562
    if (Phase == AttributorPhase::SEEDING && !shouldSeedAttribute(AA)) {
1563
      AA.getState().indicatePessimisticFixpoint();
1564
      return AA;
1565
    }
1566
 
1567
    // For now we ignore naked and optnone functions.
1568
    bool Invalidate =
1569
        Configuration.Allowed && !Configuration.Allowed->count(&AAType::ID);
1570
    const Function *AnchorFn = IRP.getAnchorScope();
1571
    if (AnchorFn) {
1572
      Invalidate |=
1573
          AnchorFn->hasFnAttribute(Attribute::Naked) ||
1574
          AnchorFn->hasFnAttribute(Attribute::OptimizeNone) ||
1575
          (!isModulePass() && !getInfoCache().isInModuleSlice(*AnchorFn));
1576
    }
1577
 
1578
    // Avoid too many nested initializations to prevent a stack overflow.
1579
    Invalidate |= InitializationChainLength > MaxInitializationChainLength;
1580
 
1581
    // Bootstrap the new attribute with an initial update to propagate
1582
    // information, e.g., function -> call site. If it is not on a given
1583
    // Allowed we will not perform updates at all.
1584
    if (Invalidate) {
1585
      AA.getState().indicatePessimisticFixpoint();
1586
      return AA;
1587
    }
1588
 
1589
    {
1590
      TimeTraceScope TimeScope(AA.getName() + "::initialize");
1591
      ++InitializationChainLength;
1592
      AA.initialize(*this);
1593
      --InitializationChainLength;
1594
    }
1595
 
1596
    // We update only AAs associated with functions in the Functions set or
1597
    // call sites of them.
1598
    if ((AnchorFn && !isRunOn(const_cast<Function *>(AnchorFn))) &&
1599
        !isRunOn(IRP.getAssociatedFunction())) {
1600
      AA.getState().indicatePessimisticFixpoint();
1601
      return AA;
1602
    }
1603
 
1604
    // If this is queried in the manifest stage, we force the AA to indicate
1605
    // pessimistic fixpoint immediately.
1606
    if (Phase == AttributorPhase::MANIFEST ||
1607
        Phase == AttributorPhase::CLEANUP) {
1608
      AA.getState().indicatePessimisticFixpoint();
1609
      return AA;
1610
    }
1611
 
1612
    // Allow seeded attributes to declare dependencies.
1613
    // Remember the seeding state.
1614
    if (UpdateAfterInit) {
1615
      AttributorPhase OldPhase = Phase;
1616
      Phase = AttributorPhase::UPDATE;
1617
 
1618
      updateAA(AA);
1619
 
1620
      Phase = OldPhase;
1621
    }
1622
 
1623
    if (QueryingAA && AA.getState().isValidState())
1624
      recordDependence(AA, const_cast<AbstractAttribute &>(*QueryingAA),
1625
                       DepClass);
1626
    return AA;
1627
  }
1628
  template <typename AAType>
1629
  const AAType &getOrCreateAAFor(const IRPosition &IRP) {
1630
    return getOrCreateAAFor<AAType>(IRP, /* QueryingAA */ nullptr,
1631
                                    DepClassTy::NONE);
1632
  }
1633
 
1634
  /// Return the attribute of \p AAType for \p IRP if existing and valid. This
1635
  /// also allows non-AA users lookup.
1636
  template <typename AAType>
1637
  AAType *lookupAAFor(const IRPosition &IRP,
1638
                      const AbstractAttribute *QueryingAA = nullptr,
1639
                      DepClassTy DepClass = DepClassTy::OPTIONAL,
1640
                      bool AllowInvalidState = false) {
1641
    static_assert(std::is_base_of<AbstractAttribute, AAType>::value,
1642
                  "Cannot query an attribute with a type not derived from "
1643
                  "'AbstractAttribute'!");
1644
    // Lookup the abstract attribute of type AAType. If found, return it after
1645
    // registering a dependence of QueryingAA on the one returned attribute.
1646
    AbstractAttribute *AAPtr = AAMap.lookup({&AAType::ID, IRP});
1647
    if (!AAPtr)
1648
      return nullptr;
1649
 
1650
    AAType *AA = static_cast<AAType *>(AAPtr);
1651
 
1652
    // Do not register a dependence on an attribute with an invalid state.
1653
    if (DepClass != DepClassTy::NONE && QueryingAA &&
1654
        AA->getState().isValidState())
1655
      recordDependence(*AA, const_cast<AbstractAttribute &>(*QueryingAA),
1656
                       DepClass);
1657
 
1658
    // Return nullptr if this attribute has an invalid state.
1659
    if (!AllowInvalidState && !AA->getState().isValidState())
1660
      return nullptr;
1661
    return AA;
1662
  }
1663
 
1664
  /// Allows a query AA to request an update if a new query was received.
1665
  void registerForUpdate(AbstractAttribute &AA);
1666
 
1667
  /// Explicitly record a dependence from \p FromAA to \p ToAA, that is if
1668
  /// \p FromAA changes \p ToAA should be updated as well.
1669
  ///
1670
  /// This method should be used in conjunction with the `getAAFor` method and
1671
  /// with the DepClass enum passed to the method set to None. This can
1672
  /// be beneficial to avoid false dependences but it requires the users of
1673
  /// `getAAFor` to explicitly record true dependences through this method.
1674
  /// The \p DepClass flag indicates if the dependence is striclty necessary.
1675
  /// That means for required dependences, if \p FromAA changes to an invalid
1676
  /// state, \p ToAA can be moved to a pessimistic fixpoint because it required
1677
  /// information from \p FromAA but none are available anymore.
1678
  void recordDependence(const AbstractAttribute &FromAA,
1679
                        const AbstractAttribute &ToAA, DepClassTy DepClass);
1680
 
1681
  /// Introduce a new abstract attribute into the fixpoint analysis.
1682
  ///
1683
  /// Note that ownership of the attribute is given to the Attributor. It will
1684
  /// invoke delete for the Attributor on destruction of the Attributor.
1685
  ///
1686
  /// Attributes are identified by their IR position (AAType::getIRPosition())
1687
  /// and the address of their static member (see AAType::ID).
1688
  template <typename AAType> AAType &registerAA(AAType &AA) {
1689
    static_assert(std::is_base_of<AbstractAttribute, AAType>::value,
1690
                  "Cannot register an attribute with a type not derived from "
1691
                  "'AbstractAttribute'!");
1692
    // Put the attribute in the lookup map structure and the container we use to
1693
    // keep track of all attributes.
1694
    const IRPosition &IRP = AA.getIRPosition();
1695
    AbstractAttribute *&AAPtr = AAMap[{&AAType::ID, IRP}];
1696
 
1697
    assert(!AAPtr && "Attribute already in map!");
1698
    AAPtr = &AA;
1699
 
1700
    // Register AA with the synthetic root only before the manifest stage.
1701
    if (Phase == AttributorPhase::SEEDING || Phase == AttributorPhase::UPDATE)
1702
      DG.SyntheticRoot.Deps.push_back(
1703
          AADepGraphNode::DepTy(&AA, unsigned(DepClassTy::REQUIRED)));
1704
 
1705
    return AA;
1706
  }
1707
 
1708
  /// Return the internal information cache.
1709
  InformationCache &getInfoCache() { return InfoCache; }
1710
 
1711
  /// Return true if this is a module pass, false otherwise.
1712
  bool isModulePass() const { return Configuration.IsModulePass; }
1713
 
1714
  /// Return true if we derive attributes for \p Fn
1715
  bool isRunOn(Function &Fn) const { return isRunOn(&Fn); }
1716
  bool isRunOn(Function *Fn) const {
1717
    return Functions.empty() || Functions.count(Fn);
1718
  }
1719
 
1720
  /// Determine opportunities to derive 'default' attributes in \p F and create
1721
  /// abstract attribute objects for them.
1722
  ///
1723
  /// \param F The function that is checked for attribute opportunities.
1724
  ///
1725
  /// Note that abstract attribute instances are generally created even if the
1726
  /// IR already contains the information they would deduce. The most important
1727
  /// reason for this is the single interface, the one of the abstract attribute
1728
  /// instance, which can be queried without the need to look at the IR in
1729
  /// various places.
1730
  void identifyDefaultAbstractAttributes(Function &F);
1731
 
1732
  /// Determine whether the function \p F is IPO amendable
1733
  ///
1734
  /// If a function is exactly defined or it has alwaysinline attribute
1735
  /// and is viable to be inlined, we say it is IPO amendable
1736
  bool isFunctionIPOAmendable(const Function &F) {
1737
    return F.hasExactDefinition() || InfoCache.InlineableFunctions.count(&F);
1738
  }
1739
 
1740
  /// Mark the internal function \p F as live.
1741
  ///
1742
  /// This will trigger the identification and initialization of attributes for
1743
  /// \p F.
1744
  void markLiveInternalFunction(const Function &F) {
1745
    assert(F.hasLocalLinkage() &&
1746
           "Only local linkage is assumed dead initially.");
1747
 
1748
    if (Configuration.DefaultInitializeLiveInternals)
1749
      identifyDefaultAbstractAttributes(const_cast<Function &>(F));
1750
    if (Configuration.InitializationCallback)
1751
      Configuration.InitializationCallback(*this, F);
1752
  }
1753
 
1754
  /// Helper function to remove callsite.
1755
  void removeCallSite(CallInst *CI) {
1756
    if (!CI)
1757
      return;
1758
 
1759
    Configuration.CGUpdater.removeCallSite(*CI);
1760
  }
1761
 
1762
  /// Record that \p U is to be replaces with \p NV after information was
1763
  /// manifested. This also triggers deletion of trivially dead istructions.
1764
  bool changeUseAfterManifest(Use &U, Value &NV) {
1765
    Value *&V = ToBeChangedUses[&U];
1766
    if (V && (V->stripPointerCasts() == NV.stripPointerCasts() ||
1767
              isa_and_nonnull<UndefValue>(V)))
1768
      return false;
1769
    assert((!V || V == &NV || isa<UndefValue>(NV)) &&
1770
           "Use was registered twice for replacement with different values!");
1771
    V = &NV;
1772
    return true;
1773
  }
1774
 
1775
  /// Helper function to replace all uses associated with \p IRP with \p NV.
1776
  /// Return true if there is any change. The flag \p ChangeDroppable indicates
1777
  /// if dropppable uses should be changed too.
1778
  bool changeAfterManifest(const IRPosition IRP, Value &NV,
1779
                           bool ChangeDroppable = true) {
1780
    if (IRP.getPositionKind() == IRPosition::IRP_CALL_SITE_ARGUMENT) {
1781
      auto *CB = cast<CallBase>(IRP.getCtxI());
1782
      return changeUseAfterManifest(
1783
          CB->getArgOperandUse(IRP.getCallSiteArgNo()), NV);
1784
    }
1785
    Value &V = IRP.getAssociatedValue();
1786
    auto &Entry = ToBeChangedValues[&V];
1787
    Value *CurNV = get<0>(Entry);
1788
    if (CurNV && (CurNV->stripPointerCasts() == NV.stripPointerCasts() ||
1789
                  isa<UndefValue>(CurNV)))
1790
      return false;
1791
    assert((!CurNV || CurNV == &NV || isa<UndefValue>(NV)) &&
1792
           "Value replacement was registered twice with different values!");
1793
    Entry = {&NV, ChangeDroppable};
1794
    return true;
1795
  }
1796
 
1797
  /// Record that \p I is to be replaced with `unreachable` after information
1798
  /// was manifested.
1799
  void changeToUnreachableAfterManifest(Instruction *I) {
1800
    ToBeChangedToUnreachableInsts.insert(I);
1801
  }
1802
 
1803
  /// Record that \p II has at least one dead successor block. This information
1804
  /// is used, e.g., to replace \p II with a call, after information was
1805
  /// manifested.
1806
  void registerInvokeWithDeadSuccessor(InvokeInst &II) {
1807
    InvokeWithDeadSuccessor.insert(&II);
1808
  }
1809
 
1810
  /// Record that \p I is deleted after information was manifested. This also
1811
  /// triggers deletion of trivially dead istructions.
1812
  void deleteAfterManifest(Instruction &I) { ToBeDeletedInsts.insert(&I); }
1813
 
1814
  /// Record that \p BB is deleted after information was manifested. This also
1815
  /// triggers deletion of trivially dead istructions.
1816
  void deleteAfterManifest(BasicBlock &BB) { ToBeDeletedBlocks.insert(&BB); }
1817
 
1818
  // Record that \p BB is added during the manifest of an AA. Added basic blocks
1819
  // are preserved in the IR.
1820
  void registerManifestAddedBasicBlock(BasicBlock &BB) {
1821
    ManifestAddedBlocks.insert(&BB);
1822
  }
1823
 
1824
  /// Record that \p F is deleted after information was manifested.
1825
  void deleteAfterManifest(Function &F) {
1826
    if (Configuration.DeleteFns)
1827
      ToBeDeletedFunctions.insert(&F);
1828
  }
1829
 
1830
  /// If \p IRP is assumed to be a constant, return it, if it is unclear yet,
1831
  /// return std::nullopt, otherwise return `nullptr`.
1832
  std::optional<Constant *> getAssumedConstant(const IRPosition &IRP,
1833
                                               const AbstractAttribute &AA,
1834
                                               bool &UsedAssumedInformation);
1835
  std::optional<Constant *> getAssumedConstant(const Value &V,
1836
                                               const AbstractAttribute &AA,
1837
                                               bool &UsedAssumedInformation) {
1838
    return getAssumedConstant(IRPosition::value(V), AA, UsedAssumedInformation);
1839
  }
1840
 
1841
  /// If \p V is assumed simplified, return it, if it is unclear yet,
1842
  /// return std::nullopt, otherwise return `nullptr`.
1843
  std::optional<Value *> getAssumedSimplified(const IRPosition &IRP,
1844
                                              const AbstractAttribute &AA,
1845
                                              bool &UsedAssumedInformation,
1846
                                              AA::ValueScope S) {
1847
    return getAssumedSimplified(IRP, &AA, UsedAssumedInformation, S);
1848
  }
1849
  std::optional<Value *> getAssumedSimplified(const Value &V,
1850
                                              const AbstractAttribute &AA,
1851
                                              bool &UsedAssumedInformation,
1852
                                              AA::ValueScope S) {
1853
    return getAssumedSimplified(IRPosition::value(V), AA,
1854
                                UsedAssumedInformation, S);
1855
  }
1856
 
1857
  /// If \p V is assumed simplified, return it, if it is unclear yet,
1858
  /// return std::nullopt, otherwise return `nullptr`. Same as the public
1859
  /// version except that it can be used without recording dependences on any \p
1860
  /// AA.
1861
  std::optional<Value *> getAssumedSimplified(const IRPosition &V,
1862
                                              const AbstractAttribute *AA,
1863
                                              bool &UsedAssumedInformation,
1864
                                              AA::ValueScope S);
1865
 
1866
  /// Try to simplify \p IRP and in the scope \p S. If successful, true is
1867
  /// returned and all potential values \p IRP can take are put into \p Values.
1868
  /// If the result in \p Values contains select or PHI instructions it means
1869
  /// those could not be simplified to a single value. Recursive calls with
1870
  /// these instructions will yield their respective potential values. If false
1871
  /// is returned no other information is valid.
1872
  bool getAssumedSimplifiedValues(const IRPosition &IRP,
1873
                                  const AbstractAttribute *AA,
1874
                                  SmallVectorImpl<AA::ValueAndContext> &Values,
1875
                                  AA::ValueScope S,
1876
                                  bool &UsedAssumedInformation);
1877
 
1878
  /// Register \p CB as a simplification callback.
1879
  /// `Attributor::getAssumedSimplified` will use these callbacks before
1880
  /// we it will ask `AAValueSimplify`. It is important to ensure this
1881
  /// is called before `identifyDefaultAbstractAttributes`, assuming the
1882
  /// latter is called at all.
1883
  using SimplifictionCallbackTy = std::function<std::optional<Value *>(
1884
      const IRPosition &, const AbstractAttribute *, bool &)>;
1885
  void registerSimplificationCallback(const IRPosition &IRP,
1886
                                      const SimplifictionCallbackTy &CB) {
1887
    SimplificationCallbacks[IRP].emplace_back(CB);
1888
  }
1889
 
1890
  /// Return true if there is a simplification callback for \p IRP.
1891
  bool hasSimplificationCallback(const IRPosition &IRP) {
1892
    return SimplificationCallbacks.count(IRP);
1893
  }
1894
 
1895
  using VirtualUseCallbackTy =
1896
      std::function<bool(Attributor &, const AbstractAttribute *)>;
1897
  void registerVirtualUseCallback(const Value &V,
1898
                                  const VirtualUseCallbackTy &CB) {
1899
    VirtualUseCallbacks[&V].emplace_back(CB);
1900
  }
1901
 
1902
private:
1903
  /// The vector with all simplification callbacks registered by outside AAs.
1904
  DenseMap<IRPosition, SmallVector<SimplifictionCallbackTy, 1>>
1905
      SimplificationCallbacks;
1906
 
1907
  DenseMap<const Value *, SmallVector<VirtualUseCallbackTy, 1>>
1908
      VirtualUseCallbacks;
1909
 
1910
public:
1911
  /// Translate \p V from the callee context into the call site context.
1912
  std::optional<Value *>
1913
  translateArgumentToCallSiteContent(std::optional<Value *> V, CallBase &CB,
1914
                                     const AbstractAttribute &AA,
1915
                                     bool &UsedAssumedInformation);
1916
 
1917
  /// Return true if \p AA (or its context instruction) is assumed dead.
1918
  ///
1919
  /// If \p LivenessAA is not provided it is queried.
1920
  bool isAssumedDead(const AbstractAttribute &AA, const AAIsDead *LivenessAA,
1921
                     bool &UsedAssumedInformation,
1922
                     bool CheckBBLivenessOnly = false,
1923
                     DepClassTy DepClass = DepClassTy::OPTIONAL);
1924
 
1925
  /// Return true if \p I is assumed dead.
1926
  ///
1927
  /// If \p LivenessAA is not provided it is queried.
1928
  bool isAssumedDead(const Instruction &I, const AbstractAttribute *QueryingAA,
1929
                     const AAIsDead *LivenessAA, bool &UsedAssumedInformation,
1930
                     bool CheckBBLivenessOnly = false,
1931
                     DepClassTy DepClass = DepClassTy::OPTIONAL,
1932
                     bool CheckForDeadStore = false);
1933
 
1934
  /// Return true if \p U is assumed dead.
1935
  ///
1936
  /// If \p FnLivenessAA is not provided it is queried.
1937
  bool isAssumedDead(const Use &U, const AbstractAttribute *QueryingAA,
1938
                     const AAIsDead *FnLivenessAA, bool &UsedAssumedInformation,
1939
                     bool CheckBBLivenessOnly = false,
1940
                     DepClassTy DepClass = DepClassTy::OPTIONAL);
1941
 
1942
  /// Return true if \p IRP is assumed dead.
1943
  ///
1944
  /// If \p FnLivenessAA is not provided it is queried.
1945
  bool isAssumedDead(const IRPosition &IRP, const AbstractAttribute *QueryingAA,
1946
                     const AAIsDead *FnLivenessAA, bool &UsedAssumedInformation,
1947
                     bool CheckBBLivenessOnly = false,
1948
                     DepClassTy DepClass = DepClassTy::OPTIONAL);
1949
 
1950
  /// Return true if \p BB is assumed dead.
1951
  ///
1952
  /// If \p LivenessAA is not provided it is queried.
1953
  bool isAssumedDead(const BasicBlock &BB, const AbstractAttribute *QueryingAA,
1954
                     const AAIsDead *FnLivenessAA,
1955
                     DepClassTy DepClass = DepClassTy::OPTIONAL);
1956
 
1957
  /// Check \p Pred on all (transitive) uses of \p V.
1958
  ///
1959
  /// This method will evaluate \p Pred on all (transitive) uses of the
1960
  /// associated value and return true if \p Pred holds every time.
1961
  /// If uses are skipped in favor of equivalent ones, e.g., if we look through
1962
  /// memory, the \p EquivalentUseCB will be used to give the caller an idea
1963
  /// what original used was replaced by a new one (or new ones). The visit is
1964
  /// cut short if \p EquivalentUseCB returns false and the function will return
1965
  /// false as well.
1966
  bool checkForAllUses(function_ref<bool(const Use &, bool &)> Pred,
1967
                       const AbstractAttribute &QueryingAA, const Value &V,
1968
                       bool CheckBBLivenessOnly = false,
1969
                       DepClassTy LivenessDepClass = DepClassTy::OPTIONAL,
1970
                       bool IgnoreDroppableUses = true,
1971
                       function_ref<bool(const Use &OldU, const Use &NewU)>
1972
                           EquivalentUseCB = nullptr);
1973
 
1974
  /// Emit a remark generically.
1975
  ///
1976
  /// This template function can be used to generically emit a remark. The
1977
  /// RemarkKind should be one of the following:
1978
  ///   - OptimizationRemark to indicate a successful optimization attempt
1979
  ///   - OptimizationRemarkMissed to report a failed optimization attempt
1980
  ///   - OptimizationRemarkAnalysis to provide additional information about an
1981
  ///     optimization attempt
1982
  ///
1983
  /// The remark is built using a callback function \p RemarkCB that takes a
1984
  /// RemarkKind as input and returns a RemarkKind.
1985
  template <typename RemarkKind, typename RemarkCallBack>
1986
  void emitRemark(Instruction *I, StringRef RemarkName,
1987
                  RemarkCallBack &&RemarkCB) const {
1988
    if (!Configuration.OREGetter)
1989
      return;
1990
 
1991
    Function *F = I->getFunction();
1992
    auto &ORE = Configuration.OREGetter(F);
1993
 
1994
    if (RemarkName.startswith("OMP"))
1995
      ORE.emit([&]() {
1996
        return RemarkCB(RemarkKind(Configuration.PassName, RemarkName, I))
1997
               << " [" << RemarkName << "]";
1998
      });
1999
    else
2000
      ORE.emit([&]() {
2001
        return RemarkCB(RemarkKind(Configuration.PassName, RemarkName, I));
2002
      });
2003
  }
2004
 
2005
  /// Emit a remark on a function.
2006
  template <typename RemarkKind, typename RemarkCallBack>
2007
  void emitRemark(Function *F, StringRef RemarkName,
2008
                  RemarkCallBack &&RemarkCB) const {
2009
    if (!Configuration.OREGetter)
2010
      return;
2011
 
2012
    auto &ORE = Configuration.OREGetter(F);
2013
 
2014
    if (RemarkName.startswith("OMP"))
2015
      ORE.emit([&]() {
2016
        return RemarkCB(RemarkKind(Configuration.PassName, RemarkName, F))
2017
               << " [" << RemarkName << "]";
2018
      });
2019
    else
2020
      ORE.emit([&]() {
2021
        return RemarkCB(RemarkKind(Configuration.PassName, RemarkName, F));
2022
      });
2023
  }
2024
 
2025
  /// Helper struct used in the communication between an abstract attribute (AA)
2026
  /// that wants to change the signature of a function and the Attributor which
2027
  /// applies the changes. The struct is partially initialized with the
2028
  /// information from the AA (see the constructor). All other members are
2029
  /// provided by the Attributor prior to invoking any callbacks.
2030
  struct ArgumentReplacementInfo {
2031
    /// Callee repair callback type
2032
    ///
2033
    /// The function repair callback is invoked once to rewire the replacement
2034
    /// arguments in the body of the new function. The argument replacement info
2035
    /// is passed, as build from the registerFunctionSignatureRewrite call, as
2036
    /// well as the replacement function and an iteratore to the first
2037
    /// replacement argument.
2038
    using CalleeRepairCBTy = std::function<void(
2039
        const ArgumentReplacementInfo &, Function &, Function::arg_iterator)>;
2040
 
2041
    /// Abstract call site (ACS) repair callback type
2042
    ///
2043
    /// The abstract call site repair callback is invoked once on every abstract
2044
    /// call site of the replaced function (\see ReplacedFn). The callback needs
2045
    /// to provide the operands for the call to the new replacement function.
2046
    /// The number and type of the operands appended to the provided vector
2047
    /// (second argument) is defined by the number and types determined through
2048
    /// the replacement type vector (\see ReplacementTypes). The first argument
2049
    /// is the ArgumentReplacementInfo object registered with the Attributor
2050
    /// through the registerFunctionSignatureRewrite call.
2051
    using ACSRepairCBTy =
2052
        std::function<void(const ArgumentReplacementInfo &, AbstractCallSite,
2053
                           SmallVectorImpl<Value *> &)>;
2054
 
2055
    /// Simple getters, see the corresponding members for details.
2056
    ///{
2057
 
2058
    Attributor &getAttributor() const { return A; }
2059
    const Function &getReplacedFn() const { return ReplacedFn; }
2060
    const Argument &getReplacedArg() const { return ReplacedArg; }
2061
    unsigned getNumReplacementArgs() const { return ReplacementTypes.size(); }
2062
    const SmallVectorImpl<Type *> &getReplacementTypes() const {
2063
      return ReplacementTypes;
2064
    }
2065
 
2066
    ///}
2067
 
2068
  private:
2069
    /// Constructor that takes the argument to be replaced, the types of
2070
    /// the replacement arguments, as well as callbacks to repair the call sites
2071
    /// and new function after the replacement happened.
2072
    ArgumentReplacementInfo(Attributor &A, Argument &Arg,
2073
                            ArrayRef<Type *> ReplacementTypes,
2074
                            CalleeRepairCBTy &&CalleeRepairCB,
2075
                            ACSRepairCBTy &&ACSRepairCB)
2076
        : A(A), ReplacedFn(*Arg.getParent()), ReplacedArg(Arg),
2077
          ReplacementTypes(ReplacementTypes.begin(), ReplacementTypes.end()),
2078
          CalleeRepairCB(std::move(CalleeRepairCB)),
2079
          ACSRepairCB(std::move(ACSRepairCB)) {}
2080
 
2081
    /// Reference to the attributor to allow access from the callbacks.
2082
    Attributor &A;
2083
 
2084
    /// The "old" function replaced by ReplacementFn.
2085
    const Function &ReplacedFn;
2086
 
2087
    /// The "old" argument replaced by new ones defined via ReplacementTypes.
2088
    const Argument &ReplacedArg;
2089
 
2090
    /// The types of the arguments replacing ReplacedArg.
2091
    const SmallVector<Type *, 8> ReplacementTypes;
2092
 
2093
    /// Callee repair callback, see CalleeRepairCBTy.
2094
    const CalleeRepairCBTy CalleeRepairCB;
2095
 
2096
    /// Abstract call site (ACS) repair callback, see ACSRepairCBTy.
2097
    const ACSRepairCBTy ACSRepairCB;
2098
 
2099
    /// Allow access to the private members from the Attributor.
2100
    friend struct Attributor;
2101
  };
2102
 
2103
  /// Check if we can rewrite a function signature.
2104
  ///
2105
  /// The argument \p Arg is replaced with new ones defined by the number,
2106
  /// order, and types in \p ReplacementTypes.
2107
  ///
2108
  /// \returns True, if the replacement can be registered, via
2109
  /// registerFunctionSignatureRewrite, false otherwise.
2110
  bool isValidFunctionSignatureRewrite(Argument &Arg,
2111
                                       ArrayRef<Type *> ReplacementTypes);
2112
 
2113
  /// Register a rewrite for a function signature.
2114
  ///
2115
  /// The argument \p Arg is replaced with new ones defined by the number,
2116
  /// order, and types in \p ReplacementTypes. The rewiring at the call sites is
2117
  /// done through \p ACSRepairCB and at the callee site through
2118
  /// \p CalleeRepairCB.
2119
  ///
2120
  /// \returns True, if the replacement was registered, false otherwise.
2121
  bool registerFunctionSignatureRewrite(
2122
      Argument &Arg, ArrayRef<Type *> ReplacementTypes,
2123
      ArgumentReplacementInfo::CalleeRepairCBTy &&CalleeRepairCB,
2124
      ArgumentReplacementInfo::ACSRepairCBTy &&ACSRepairCB);
2125
 
2126
  /// Check \p Pred on all function call sites.
2127
  ///
2128
  /// This method will evaluate \p Pred on call sites and return
2129
  /// true if \p Pred holds in every call sites. However, this is only possible
2130
  /// all call sites are known, hence the function has internal linkage.
2131
  /// If true is returned, \p UsedAssumedInformation is set if assumed
2132
  /// information was used to skip or simplify potential call sites.
2133
  bool checkForAllCallSites(function_ref<bool(AbstractCallSite)> Pred,
2134
                            const AbstractAttribute &QueryingAA,
2135
                            bool RequireAllCallSites,
2136
                            bool &UsedAssumedInformation);
2137
 
2138
  /// Check \p Pred on all call sites of \p Fn.
2139
  ///
2140
  /// This method will evaluate \p Pred on call sites and return
2141
  /// true if \p Pred holds in every call sites. However, this is only possible
2142
  /// all call sites are known, hence the function has internal linkage.
2143
  /// If true is returned, \p UsedAssumedInformation is set if assumed
2144
  /// information was used to skip or simplify potential call sites.
2145
  bool checkForAllCallSites(function_ref<bool(AbstractCallSite)> Pred,
2146
                            const Function &Fn, bool RequireAllCallSites,
2147
                            const AbstractAttribute *QueryingAA,
2148
                            bool &UsedAssumedInformation,
2149
                            bool CheckPotentiallyDead = false);
2150
 
2151
  /// Check \p Pred on all values potentially returned by \p F.
2152
  ///
2153
  /// This method will evaluate \p Pred on all values potentially returned by
2154
  /// the function associated with \p QueryingAA. The returned values are
2155
  /// matched with their respective return instructions. Returns true if \p Pred
2156
  /// holds on all of them.
2157
  bool checkForAllReturnedValuesAndReturnInsts(
2158
      function_ref<bool(Value &, const SmallSetVector<ReturnInst *, 4> &)> Pred,
2159
      const AbstractAttribute &QueryingAA);
2160
 
2161
  /// Check \p Pred on all values potentially returned by the function
2162
  /// associated with \p QueryingAA.
2163
  ///
2164
  /// This is the context insensitive version of the method above.
2165
  bool checkForAllReturnedValues(function_ref<bool(Value &)> Pred,
2166
                                 const AbstractAttribute &QueryingAA);
2167
 
2168
  /// Check \p Pred on all instructions in \p Fn with an opcode present in
2169
  /// \p Opcodes.
2170
  ///
2171
  /// This method will evaluate \p Pred on all instructions with an opcode
2172
  /// present in \p Opcode and return true if \p Pred holds on all of them.
2173
  bool checkForAllInstructions(function_ref<bool(Instruction &)> Pred,
2174
                               const Function *Fn,
2175
                               const AbstractAttribute &QueryingAA,
2176
                               const ArrayRef<unsigned> &Opcodes,
2177
                               bool &UsedAssumedInformation,
2178
                               bool CheckBBLivenessOnly = false,
2179
                               bool CheckPotentiallyDead = false);
2180
 
2181
  /// Check \p Pred on all instructions with an opcode present in \p Opcodes.
2182
  ///
2183
  /// This method will evaluate \p Pred on all instructions with an opcode
2184
  /// present in \p Opcode and return true if \p Pred holds on all of them.
2185
  bool checkForAllInstructions(function_ref<bool(Instruction &)> Pred,
2186
                               const AbstractAttribute &QueryingAA,
2187
                               const ArrayRef<unsigned> &Opcodes,
2188
                               bool &UsedAssumedInformation,
2189
                               bool CheckBBLivenessOnly = false,
2190
                               bool CheckPotentiallyDead = false);
2191
 
2192
  /// Check \p Pred on all call-like instructions (=CallBased derived).
2193
  ///
2194
  /// See checkForAllCallLikeInstructions(...) for more information.
2195
  bool checkForAllCallLikeInstructions(function_ref<bool(Instruction &)> Pred,
2196
                                       const AbstractAttribute &QueryingAA,
2197
                                       bool &UsedAssumedInformation,
2198
                                       bool CheckBBLivenessOnly = false,
2199
                                       bool CheckPotentiallyDead = false) {
2200
    return checkForAllInstructions(
2201
        Pred, QueryingAA,
2202
        {(unsigned)Instruction::Invoke, (unsigned)Instruction::CallBr,
2203
         (unsigned)Instruction::Call},
2204
        UsedAssumedInformation, CheckBBLivenessOnly, CheckPotentiallyDead);
2205
  }
2206
 
2207
  /// Check \p Pred on all Read/Write instructions.
2208
  ///
2209
  /// This method will evaluate \p Pred on all instructions that read or write
2210
  /// to memory present in the information cache and return true if \p Pred
2211
  /// holds on all of them.
2212
  bool checkForAllReadWriteInstructions(function_ref<bool(Instruction &)> Pred,
2213
                                        AbstractAttribute &QueryingAA,
2214
                                        bool &UsedAssumedInformation);
2215
 
2216
  /// Create a shallow wrapper for \p F such that \p F has internal linkage
2217
  /// afterwards. It also sets the original \p F 's name to anonymous
2218
  ///
2219
  /// A wrapper is a function with the same type (and attributes) as \p F
2220
  /// that will only call \p F and return the result, if any.
2221
  ///
2222
  /// Assuming the declaration of looks like:
2223
  ///   rty F(aty0 arg0, ..., atyN argN);
2224
  ///
2225
  /// The wrapper will then look as follows:
2226
  ///   rty wrapper(aty0 arg0, ..., atyN argN) {
2227
  ///     return F(arg0, ..., argN);
2228
  ///   }
2229
  ///
2230
  static void createShallowWrapper(Function &F);
2231
 
2232
  /// Returns true if the function \p F can be internalized. i.e. it has a
2233
  /// compatible linkage.
2234
  static bool isInternalizable(Function &F);
2235
 
2236
  /// Make another copy of the function \p F such that the copied version has
2237
  /// internal linkage afterwards and can be analysed. Then we replace all uses
2238
  /// of the original function to the copied one
2239
  ///
2240
  /// Only non-locally linked functions that have `linkonce_odr` or `weak_odr`
2241
  /// linkage can be internalized because these linkages guarantee that other
2242
  /// definitions with the same name have the same semantics as this one.
2243
  ///
2244
  /// This will only be run if the `attributor-allow-deep-wrappers` option is
2245
  /// set, or if the function is called with \p Force set to true.
2246
  ///
2247
  /// If the function \p F failed to be internalized the return value will be a
2248
  /// null pointer.
2249
  static Function *internalizeFunction(Function &F, bool Force = false);
2250
 
2251
  /// Make copies of each function in the set \p FnSet such that the copied
2252
  /// version has internal linkage afterwards and can be analysed. Then we
2253
  /// replace all uses of the original function to the copied one. The map
2254
  /// \p FnMap contains a mapping of functions to their internalized versions.
2255
  ///
2256
  /// Only non-locally linked functions that have `linkonce_odr` or `weak_odr`
2257
  /// linkage can be internalized because these linkages guarantee that other
2258
  /// definitions with the same name have the same semantics as this one.
2259
  ///
2260
  /// This version will internalize all the functions in the set \p FnSet at
2261
  /// once and then replace the uses. This prevents internalized functions being
2262
  /// called by external functions when there is an internalized version in the
2263
  /// module.
2264
  static bool internalizeFunctions(SmallPtrSetImpl<Function *> &FnSet,
2265
                                   DenseMap<Function *, Function *> &FnMap);
2266
 
2267
  /// Return the data layout associated with the anchor scope.
2268
  const DataLayout &getDataLayout() const { return InfoCache.DL; }
2269
 
2270
  /// The allocator used to allocate memory, e.g. for `AbstractAttribute`s.
2271
  BumpPtrAllocator &Allocator;
2272
 
2273
private:
2274
  /// This method will do fixpoint iteration until fixpoint or the
2275
  /// maximum iteration count is reached.
2276
  ///
2277
  /// If the maximum iteration count is reached, This method will
2278
  /// indicate pessimistic fixpoint on attributes that transitively depend
2279
  /// on attributes that were scheduled for an update.
2280
  void runTillFixpoint();
2281
 
2282
  /// Gets called after scheduling, manifests attributes to the LLVM IR.
2283
  ChangeStatus manifestAttributes();
2284
 
2285
  /// Gets called after attributes have been manifested, cleans up the IR.
2286
  /// Deletes dead functions, blocks and instructions.
2287
  /// Rewrites function signitures and updates the call graph.
2288
  ChangeStatus cleanupIR();
2289
 
2290
  /// Identify internal functions that are effectively dead, thus not reachable
2291
  /// from a live entry point. The functions are added to ToBeDeletedFunctions.
2292
  void identifyDeadInternalFunctions();
2293
 
2294
  /// Run `::update` on \p AA and track the dependences queried while doing so.
2295
  /// Also adjust the state if we know further updates are not necessary.
2296
  ChangeStatus updateAA(AbstractAttribute &AA);
2297
 
2298
  /// Remember the dependences on the top of the dependence stack such that they
2299
  /// may trigger further updates. (\see DependenceStack)
2300
  void rememberDependences();
2301
 
2302
  /// Determine if CallBase context in \p IRP should be propagated.
2303
  bool shouldPropagateCallBaseContext(const IRPosition &IRP);
2304
 
2305
  /// Apply all requested function signature rewrites
2306
  /// (\see registerFunctionSignatureRewrite) and return Changed if the module
2307
  /// was altered.
2308
  ChangeStatus
2309
  rewriteFunctionSignatures(SmallSetVector<Function *, 8> &ModifiedFns);
2310
 
2311
  /// Check if the Attribute \p AA should be seeded.
2312
  /// See getOrCreateAAFor.
2313
  bool shouldSeedAttribute(AbstractAttribute &AA);
2314
 
2315
  /// A nested map to lookup abstract attributes based on the argument position
2316
  /// on the outer level, and the addresses of the static member (AAType::ID) on
2317
  /// the inner level.
2318
  ///{
2319
  using AAMapKeyTy = std::pair<const char *, IRPosition>;
2320
  DenseMap<AAMapKeyTy, AbstractAttribute *> AAMap;
2321
  ///}
2322
 
2323
  /// Map to remember all requested signature changes (= argument replacements).
2324
  DenseMap<Function *, SmallVector<std::unique_ptr<ArgumentReplacementInfo>, 8>>
2325
      ArgumentReplacementMap;
2326
 
2327
  /// The set of functions we are deriving attributes for.
2328
  SetVector<Function *> &Functions;
2329
 
2330
  /// The information cache that holds pre-processed (LLVM-IR) information.
2331
  InformationCache &InfoCache;
2332
 
2333
  /// Abstract Attribute dependency graph
2334
  AADepGraph DG;
2335
 
2336
  /// Set of functions for which we modified the content such that it might
2337
  /// impact the call graph.
2338
  SmallSetVector<Function *, 8> CGModifiedFunctions;
2339
 
2340
  /// Information about a dependence. If FromAA is changed ToAA needs to be
2341
  /// updated as well.
2342
  struct DepInfo {
2343
    const AbstractAttribute *FromAA;
2344
    const AbstractAttribute *ToAA;
2345
    DepClassTy DepClass;
2346
  };
2347
 
2348
  /// The dependence stack is used to track dependences during an
2349
  /// `AbstractAttribute::update` call. As `AbstractAttribute::update` can be
2350
  /// recursive we might have multiple vectors of dependences in here. The stack
2351
  /// size, should be adjusted according to the expected recursion depth and the
2352
  /// inner dependence vector size to the expected number of dependences per
2353
  /// abstract attribute. Since the inner vectors are actually allocated on the
2354
  /// stack we can be generous with their size.
2355
  using DependenceVector = SmallVector<DepInfo, 8>;
2356
  SmallVector<DependenceVector *, 16> DependenceStack;
2357
 
2358
  /// A set to remember the functions we already assume to be live and visited.
2359
  DenseSet<const Function *> VisitedFunctions;
2360
 
2361
  /// Uses we replace with a new value after manifest is done. We will remove
2362
  /// then trivially dead instructions as well.
2363
  SmallMapVector<Use *, Value *, 32> ToBeChangedUses;
2364
 
2365
  /// Values we replace with a new value after manifest is done. We will remove
2366
  /// then trivially dead instructions as well.
2367
  SmallMapVector<Value *, PointerIntPair<Value *, 1, bool>, 32>
2368
      ToBeChangedValues;
2369
 
2370
  /// Instructions we replace with `unreachable` insts after manifest is done.
2371
  SmallSetVector<WeakVH, 16> ToBeChangedToUnreachableInsts;
2372
 
2373
  /// Invoke instructions with at least a single dead successor block.
2374
  SmallSetVector<WeakVH, 16> InvokeWithDeadSuccessor;
2375
 
2376
  /// A flag that indicates which stage of the process we are in. Initially, the
2377
  /// phase is SEEDING. Phase is changed in `Attributor::run()`
2378
  enum class AttributorPhase {
2379
    SEEDING,
2380
    UPDATE,
2381
    MANIFEST,
2382
    CLEANUP,
2383
  } Phase = AttributorPhase::SEEDING;
2384
 
2385
  /// The current initialization chain length. Tracked to avoid stack overflows.
2386
  unsigned InitializationChainLength = 0;
2387
 
2388
  /// Functions, blocks, and instructions we delete after manifest is done.
2389
  ///
2390
  ///{
2391
  SmallPtrSet<BasicBlock *, 8> ManifestAddedBlocks;
2392
  SmallSetVector<Function *, 8> ToBeDeletedFunctions;
2393
  SmallSetVector<BasicBlock *, 8> ToBeDeletedBlocks;
2394
  SmallSetVector<WeakVH, 8> ToBeDeletedInsts;
2395
  ///}
2396
 
2397
  /// Container with all the query AAs that requested an update via
2398
  /// registerForUpdate.
2399
  SmallSetVector<AbstractAttribute *, 16> QueryAAsAwaitingUpdate;
2400
 
2401
  /// User provided configuration for this Attributor instance.
2402
  const AttributorConfig Configuration;
2403
 
2404
  friend AADepGraph;
2405
  friend AttributorCallGraph;
2406
};
2407
 
2408
/// An interface to query the internal state of an abstract attribute.
2409
///
2410
/// The abstract state is a minimal interface that allows the Attributor to
2411
/// communicate with the abstract attributes about their internal state without
2412
/// enforcing or exposing implementation details, e.g., the (existence of an)
2413
/// underlying lattice.
2414
///
2415
/// It is sufficient to be able to query if a state is (1) valid or invalid, (2)
2416
/// at a fixpoint, and to indicate to the state that (3) an optimistic fixpoint
2417
/// was reached or (4) a pessimistic fixpoint was enforced.
2418
///
2419
/// All methods need to be implemented by the subclass. For the common use case,
2420
/// a single boolean state or a bit-encoded state, the BooleanState and
2421
/// {Inc,Dec,Bit}IntegerState classes are already provided. An abstract
2422
/// attribute can inherit from them to get the abstract state interface and
2423
/// additional methods to directly modify the state based if needed. See the
2424
/// class comments for help.
2425
struct AbstractState {
2426
  virtual ~AbstractState() = default;
2427
 
2428
  /// Return if this abstract state is in a valid state. If false, no
2429
  /// information provided should be used.
2430
  virtual bool isValidState() const = 0;
2431
 
2432
  /// Return if this abstract state is fixed, thus does not need to be updated
2433
  /// if information changes as it cannot change itself.
2434
  virtual bool isAtFixpoint() const = 0;
2435
 
2436
  /// Indicate that the abstract state should converge to the optimistic state.
2437
  ///
2438
  /// This will usually make the optimistically assumed state the known to be
2439
  /// true state.
2440
  ///
2441
  /// \returns ChangeStatus::UNCHANGED as the assumed value should not change.
2442
  virtual ChangeStatus indicateOptimisticFixpoint() = 0;
2443
 
2444
  /// Indicate that the abstract state should converge to the pessimistic state.
2445
  ///
2446
  /// This will usually revert the optimistically assumed state to the known to
2447
  /// be true state.
2448
  ///
2449
  /// \returns ChangeStatus::CHANGED as the assumed value may change.
2450
  virtual ChangeStatus indicatePessimisticFixpoint() = 0;
2451
};
2452
 
2453
/// Simple state with integers encoding.
2454
///
2455
/// The interface ensures that the assumed bits are always a subset of the known
2456
/// bits. Users can only add known bits and, except through adding known bits,
2457
/// they can only remove assumed bits. This should guarantee monotoniticy and
2458
/// thereby the existence of a fixpoint (if used corretly). The fixpoint is
2459
/// reached when the assumed and known state/bits are equal. Users can
2460
/// force/inidicate a fixpoint. If an optimistic one is indicated, the known
2461
/// state will catch up with the assumed one, for a pessimistic fixpoint it is
2462
/// the other way around.
2463
template <typename base_ty, base_ty BestState, base_ty WorstState>
2464
struct IntegerStateBase : public AbstractState {
2465
  using base_t = base_ty;
2466
 
2467
  IntegerStateBase() = default;
2468
  IntegerStateBase(base_t Assumed) : Assumed(Assumed) {}
2469
 
2470
  /// Return the best possible representable state.
2471
  static constexpr base_t getBestState() { return BestState; }
2472
  static constexpr base_t getBestState(const IntegerStateBase &) {
2473
    return getBestState();
2474
  }
2475
 
2476
  /// Return the worst possible representable state.
2477
  static constexpr base_t getWorstState() { return WorstState; }
2478
  static constexpr base_t getWorstState(const IntegerStateBase &) {
2479
    return getWorstState();
2480
  }
2481
 
2482
  /// See AbstractState::isValidState()
2483
  /// NOTE: For now we simply pretend that the worst possible state is invalid.
2484
  bool isValidState() const override { return Assumed != getWorstState(); }
2485
 
2486
  /// See AbstractState::isAtFixpoint()
2487
  bool isAtFixpoint() const override { return Assumed == Known; }
2488
 
2489
  /// See AbstractState::indicateOptimisticFixpoint(...)
2490
  ChangeStatus indicateOptimisticFixpoint() override {
2491
    Known = Assumed;
2492
    return ChangeStatus::UNCHANGED;
2493
  }
2494
 
2495
  /// See AbstractState::indicatePessimisticFixpoint(...)
2496
  ChangeStatus indicatePessimisticFixpoint() override {
2497
    Assumed = Known;
2498
    return ChangeStatus::CHANGED;
2499
  }
2500
 
2501
  /// Return the known state encoding
2502
  base_t getKnown() const { return Known; }
2503
 
2504
  /// Return the assumed state encoding.
2505
  base_t getAssumed() const { return Assumed; }
2506
 
2507
  /// Equality for IntegerStateBase.
2508
  bool
2509
  operator==(const IntegerStateBase<base_t, BestState, WorstState> &R) const {
2510
    return this->getAssumed() == R.getAssumed() &&
2511
           this->getKnown() == R.getKnown();
2512
  }
2513
 
2514
  /// Inequality for IntegerStateBase.
2515
  bool
2516
  operator!=(const IntegerStateBase<base_t, BestState, WorstState> &R) const {
2517
    return !(*this == R);
2518
  }
2519
 
2520
  /// "Clamp" this state with \p R. The result is subtype dependent but it is
2521
  /// intended that only information assumed in both states will be assumed in
2522
  /// this one afterwards.
2523
  void operator^=(const IntegerStateBase<base_t, BestState, WorstState> &R) {
2524
    handleNewAssumedValue(R.getAssumed());
2525
  }
2526
 
2527
  /// "Clamp" this state with \p R. The result is subtype dependent but it is
2528
  /// intended that information known in either state will be known in
2529
  /// this one afterwards.
2530
  void operator+=(const IntegerStateBase<base_t, BestState, WorstState> &R) {
2531
    handleNewKnownValue(R.getKnown());
2532
  }
2533
 
2534
  void operator|=(const IntegerStateBase<base_t, BestState, WorstState> &R) {
2535
    joinOR(R.getAssumed(), R.getKnown());
2536
  }
2537
 
2538
  void operator&=(const IntegerStateBase<base_t, BestState, WorstState> &R) {
2539
    joinAND(R.getAssumed(), R.getKnown());
2540
  }
2541
 
2542
protected:
2543
  /// Handle a new assumed value \p Value. Subtype dependent.
2544
  virtual void handleNewAssumedValue(base_t Value) = 0;
2545
 
2546
  /// Handle a new known value \p Value. Subtype dependent.
2547
  virtual void handleNewKnownValue(base_t Value) = 0;
2548
 
2549
  /// Handle a  value \p Value. Subtype dependent.
2550
  virtual void joinOR(base_t AssumedValue, base_t KnownValue) = 0;
2551
 
2552
  /// Handle a new assumed value \p Value. Subtype dependent.
2553
  virtual void joinAND(base_t AssumedValue, base_t KnownValue) = 0;
2554
 
2555
  /// The known state encoding in an integer of type base_t.
2556
  base_t Known = getWorstState();
2557
 
2558
  /// The assumed state encoding in an integer of type base_t.
2559
  base_t Assumed = getBestState();
2560
};
2561
 
2562
/// Specialization of the integer state for a bit-wise encoding.
2563
template <typename base_ty = uint32_t, base_ty BestState = ~base_ty(0),
2564
          base_ty WorstState = 0>
2565
struct BitIntegerState
2566
    : public IntegerStateBase<base_ty, BestState, WorstState> {
2567
  using base_t = base_ty;
2568
 
2569
  /// Return true if the bits set in \p BitsEncoding are "known bits".
2570
  bool isKnown(base_t BitsEncoding) const {
2571
    return (this->Known & BitsEncoding) == BitsEncoding;
2572
  }
2573
 
2574
  /// Return true if the bits set in \p BitsEncoding are "assumed bits".
2575
  bool isAssumed(base_t BitsEncoding) const {
2576
    return (this->Assumed & BitsEncoding) == BitsEncoding;
2577
  }
2578
 
2579
  /// Add the bits in \p BitsEncoding to the "known bits".
2580
  BitIntegerState &addKnownBits(base_t Bits) {
2581
    // Make sure we never miss any "known bits".
2582
    this->Assumed |= Bits;
2583
    this->Known |= Bits;
2584
    return *this;
2585
  }
2586
 
2587
  /// Remove the bits in \p BitsEncoding from the "assumed bits" if not known.
2588
  BitIntegerState &removeAssumedBits(base_t BitsEncoding) {
2589
    return intersectAssumedBits(~BitsEncoding);
2590
  }
2591
 
2592
  /// Remove the bits in \p BitsEncoding from the "known bits".
2593
  BitIntegerState &removeKnownBits(base_t BitsEncoding) {
2594
    this->Known = (this->Known & ~BitsEncoding);
2595
    return *this;
2596
  }
2597
 
2598
  /// Keep only "assumed bits" also set in \p BitsEncoding but all known ones.
2599
  BitIntegerState &intersectAssumedBits(base_t BitsEncoding) {
2600
    // Make sure we never loose any "known bits".
2601
    this->Assumed = (this->Assumed & BitsEncoding) | this->Known;
2602
    return *this;
2603
  }
2604
 
2605
private:
2606
  void handleNewAssumedValue(base_t Value) override {
2607
    intersectAssumedBits(Value);
2608
  }
2609
  void handleNewKnownValue(base_t Value) override { addKnownBits(Value); }
2610
  void joinOR(base_t AssumedValue, base_t KnownValue) override {
2611
    this->Known |= KnownValue;
2612
    this->Assumed |= AssumedValue;
2613
  }
2614
  void joinAND(base_t AssumedValue, base_t KnownValue) override {
2615
    this->Known &= KnownValue;
2616
    this->Assumed &= AssumedValue;
2617
  }
2618
};
2619
 
2620
/// Specialization of the integer state for an increasing value, hence ~0u is
2621
/// the best state and 0 the worst.
2622
template <typename base_ty = uint32_t, base_ty BestState = ~base_ty(0),
2623
          base_ty WorstState = 0>
2624
struct IncIntegerState
2625
    : public IntegerStateBase<base_ty, BestState, WorstState> {
2626
  using super = IntegerStateBase<base_ty, BestState, WorstState>;
2627
  using base_t = base_ty;
2628
 
2629
  IncIntegerState() : super() {}
2630
  IncIntegerState(base_t Assumed) : super(Assumed) {}
2631
 
2632
  /// Return the best possible representable state.
2633
  static constexpr base_t getBestState() { return BestState; }
2634
  static constexpr base_t
2635
  getBestState(const IncIntegerState<base_ty, BestState, WorstState> &) {
2636
    return getBestState();
2637
  }
2638
 
2639
  /// Take minimum of assumed and \p Value.
2640
  IncIntegerState &takeAssumedMinimum(base_t Value) {
2641
    // Make sure we never loose "known value".
2642
    this->Assumed = std::max(std::min(this->Assumed, Value), this->Known);
2643
    return *this;
2644
  }
2645
 
2646
  /// Take maximum of known and \p Value.
2647
  IncIntegerState &takeKnownMaximum(base_t Value) {
2648
    // Make sure we never loose "known value".
2649
    this->Assumed = std::max(Value, this->Assumed);
2650
    this->Known = std::max(Value, this->Known);
2651
    return *this;
2652
  }
2653
 
2654
private:
2655
  void handleNewAssumedValue(base_t Value) override {
2656
    takeAssumedMinimum(Value);
2657
  }
2658
  void handleNewKnownValue(base_t Value) override { takeKnownMaximum(Value); }
2659
  void joinOR(base_t AssumedValue, base_t KnownValue) override {
2660
    this->Known = std::max(this->Known, KnownValue);
2661
    this->Assumed = std::max(this->Assumed, AssumedValue);
2662
  }
2663
  void joinAND(base_t AssumedValue, base_t KnownValue) override {
2664
    this->Known = std::min(this->Known, KnownValue);
2665
    this->Assumed = std::min(this->Assumed, AssumedValue);
2666
  }
2667
};
2668
 
2669
/// Specialization of the integer state for a decreasing value, hence 0 is the
2670
/// best state and ~0u the worst.
2671
template <typename base_ty = uint32_t>
2672
struct DecIntegerState : public IntegerStateBase<base_ty, 0, ~base_ty(0)> {
2673
  using base_t = base_ty;
2674
 
2675
  /// Take maximum of assumed and \p Value.
2676
  DecIntegerState &takeAssumedMaximum(base_t Value) {
2677
    // Make sure we never loose "known value".
2678
    this->Assumed = std::min(std::max(this->Assumed, Value), this->Known);
2679
    return *this;
2680
  }
2681
 
2682
  /// Take minimum of known and \p Value.
2683
  DecIntegerState &takeKnownMinimum(base_t Value) {
2684
    // Make sure we never loose "known value".
2685
    this->Assumed = std::min(Value, this->Assumed);
2686
    this->Known = std::min(Value, this->Known);
2687
    return *this;
2688
  }
2689
 
2690
private:
2691
  void handleNewAssumedValue(base_t Value) override {
2692
    takeAssumedMaximum(Value);
2693
  }
2694
  void handleNewKnownValue(base_t Value) override { takeKnownMinimum(Value); }
2695
  void joinOR(base_t AssumedValue, base_t KnownValue) override {
2696
    this->Assumed = std::min(this->Assumed, KnownValue);
2697
    this->Assumed = std::min(this->Assumed, AssumedValue);
2698
  }
2699
  void joinAND(base_t AssumedValue, base_t KnownValue) override {
2700
    this->Assumed = std::max(this->Assumed, KnownValue);
2701
    this->Assumed = std::max(this->Assumed, AssumedValue);
2702
  }
2703
};
2704
 
2705
/// Simple wrapper for a single bit (boolean) state.
2706
struct BooleanState : public IntegerStateBase<bool, true, false> {
2707
  using super = IntegerStateBase<bool, true, false>;
2708
  using base_t = IntegerStateBase::base_t;
2709
 
2710
  BooleanState() = default;
2711
  BooleanState(base_t Assumed) : super(Assumed) {}
2712
 
2713
  /// Set the assumed value to \p Value but never below the known one.
2714
  void setAssumed(bool Value) { Assumed &= (Known | Value); }
2715
 
2716
  /// Set the known and asssumed value to \p Value.
2717
  void setKnown(bool Value) {
2718
    Known |= Value;
2719
    Assumed |= Value;
2720
  }
2721
 
2722
  /// Return true if the state is assumed to hold.
2723
  bool isAssumed() const { return getAssumed(); }
2724
 
2725
  /// Return true if the state is known to hold.
2726
  bool isKnown() const { return getKnown(); }
2727
 
2728
private:
2729
  void handleNewAssumedValue(base_t Value) override {
2730
    if (!Value)
2731
      Assumed = Known;
2732
  }
2733
  void handleNewKnownValue(base_t Value) override {
2734
    if (Value)
2735
      Known = (Assumed = Value);
2736
  }
2737
  void joinOR(base_t AssumedValue, base_t KnownValue) override {
2738
    Known |= KnownValue;
2739
    Assumed |= AssumedValue;
2740
  }
2741
  void joinAND(base_t AssumedValue, base_t KnownValue) override {
2742
    Known &= KnownValue;
2743
    Assumed &= AssumedValue;
2744
  }
2745
};
2746
 
2747
/// State for an integer range.
2748
struct IntegerRangeState : public AbstractState {
2749
 
2750
  /// Bitwidth of the associated value.
2751
  uint32_t BitWidth;
2752
 
2753
  /// State representing assumed range, initially set to empty.
2754
  ConstantRange Assumed;
2755
 
2756
  /// State representing known range, initially set to [-inf, inf].
2757
  ConstantRange Known;
2758
 
2759
  IntegerRangeState(uint32_t BitWidth)
2760
      : BitWidth(BitWidth), Assumed(ConstantRange::getEmpty(BitWidth)),
2761
        Known(ConstantRange::getFull(BitWidth)) {}
2762
 
2763
  IntegerRangeState(const ConstantRange &CR)
2764
      : BitWidth(CR.getBitWidth()), Assumed(CR),
2765
        Known(getWorstState(CR.getBitWidth())) {}
2766
 
2767
  /// Return the worst possible representable state.
2768
  static ConstantRange getWorstState(uint32_t BitWidth) {
2769
    return ConstantRange::getFull(BitWidth);
2770
  }
2771
 
2772
  /// Return the best possible representable state.
2773
  static ConstantRange getBestState(uint32_t BitWidth) {
2774
    return ConstantRange::getEmpty(BitWidth);
2775
  }
2776
  static ConstantRange getBestState(const IntegerRangeState &IRS) {
2777
    return getBestState(IRS.getBitWidth());
2778
  }
2779
 
2780
  /// Return associated values' bit width.
2781
  uint32_t getBitWidth() const { return BitWidth; }
2782
 
2783
  /// See AbstractState::isValidState()
2784
  bool isValidState() const override {
2785
    return BitWidth > 0 && !Assumed.isFullSet();
2786
  }
2787
 
2788
  /// See AbstractState::isAtFixpoint()
2789
  bool isAtFixpoint() const override { return Assumed == Known; }
2790
 
2791
  /// See AbstractState::indicateOptimisticFixpoint(...)
2792
  ChangeStatus indicateOptimisticFixpoint() override {
2793
    Known = Assumed;
2794
    return ChangeStatus::CHANGED;
2795
  }
2796
 
2797
  /// See AbstractState::indicatePessimisticFixpoint(...)
2798
  ChangeStatus indicatePessimisticFixpoint() override {
2799
    Assumed = Known;
2800
    return ChangeStatus::CHANGED;
2801
  }
2802
 
2803
  /// Return the known state encoding
2804
  ConstantRange getKnown() const { return Known; }
2805
 
2806
  /// Return the assumed state encoding.
2807
  ConstantRange getAssumed() const { return Assumed; }
2808
 
2809
  /// Unite assumed range with the passed state.
2810
  void unionAssumed(const ConstantRange &R) {
2811
    // Don't loose a known range.
2812
    Assumed = Assumed.unionWith(R).intersectWith(Known);
2813
  }
2814
 
2815
  /// See IntegerRangeState::unionAssumed(..).
2816
  void unionAssumed(const IntegerRangeState &R) {
2817
    unionAssumed(R.getAssumed());
2818
  }
2819
 
2820
  /// Intersect known range with the passed state.
2821
  void intersectKnown(const ConstantRange &R) {
2822
    Assumed = Assumed.intersectWith(R);
2823
    Known = Known.intersectWith(R);
2824
  }
2825
 
2826
  /// See IntegerRangeState::intersectKnown(..).
2827
  void intersectKnown(const IntegerRangeState &R) {
2828
    intersectKnown(R.getKnown());
2829
  }
2830
 
2831
  /// Equality for IntegerRangeState.
2832
  bool operator==(const IntegerRangeState &R) const {
2833
    return getAssumed() == R.getAssumed() && getKnown() == R.getKnown();
2834
  }
2835
 
2836
  /// "Clamp" this state with \p R. The result is subtype dependent but it is
2837
  /// intended that only information assumed in both states will be assumed in
2838
  /// this one afterwards.
2839
  IntegerRangeState operator^=(const IntegerRangeState &R) {
2840
    // NOTE: `^=` operator seems like `intersect` but in this case, we need to
2841
    // take `union`.
2842
    unionAssumed(R);
2843
    return *this;
2844
  }
2845
 
2846
  IntegerRangeState operator&=(const IntegerRangeState &R) {
2847
    // NOTE: `&=` operator seems like `intersect` but in this case, we need to
2848
    // take `union`.
2849
    Known = Known.unionWith(R.getKnown());
2850
    Assumed = Assumed.unionWith(R.getAssumed());
2851
    return *this;
2852
  }
2853
};
2854
 
2855
/// Simple state for a set.
2856
///
2857
/// This represents a state containing a set of values. The interface supports
2858
/// modelling sets that contain all possible elements. The state's internal
2859
/// value is modified using union or intersection operations.
2860
template <typename BaseTy> struct SetState : public AbstractState {
2861
  /// A wrapper around a set that has semantics for handling unions and
2862
  /// intersections with a "universal" set that contains all elements.
2863
  struct SetContents {
2864
    /// Creates a universal set with no concrete elements or an empty set.
2865
    SetContents(bool Universal) : Universal(Universal) {}
2866
 
2867
    /// Creates a non-universal set with concrete values.
2868
    SetContents(const DenseSet<BaseTy> &Assumptions)
2869
        : Universal(false), Set(Assumptions) {}
2870
 
2871
    SetContents(bool Universal, const DenseSet<BaseTy> &Assumptions)
2872
        : Universal(Universal), Set(Assumptions) {}
2873
 
2874
    const DenseSet<BaseTy> &getSet() const { return Set; }
2875
 
2876
    bool isUniversal() const { return Universal; }
2877
 
2878
    bool empty() const { return Set.empty() && !Universal; }
2879
 
2880
    /// Finds A := A ^ B where A or B could be the "Universal" set which
2881
    /// contains every possible attribute. Returns true if changes were made.
2882
    bool getIntersection(const SetContents &RHS) {
2883
      bool IsUniversal = Universal;
2884
      unsigned Size = Set.size();
2885
 
2886
      // A := A ^ U = A
2887
      if (RHS.isUniversal())
2888
        return false;
2889
 
2890
      // A := U ^ B = B
2891
      if (Universal)
2892
        Set = RHS.getSet();
2893
      else
2894
        set_intersect(Set, RHS.getSet());
2895
 
2896
      Universal &= RHS.isUniversal();
2897
      return IsUniversal != Universal || Size != Set.size();
2898
    }
2899
 
2900
    /// Finds A := A u B where A or B could be the "Universal" set which
2901
    /// contains every possible attribute. returns true if changes were made.
2902
    bool getUnion(const SetContents &RHS) {
2903
      bool IsUniversal = Universal;
2904
      unsigned Size = Set.size();
2905
 
2906
      // A := A u U = U = U u B
2907
      if (!RHS.isUniversal() && !Universal)
2908
        set_union(Set, RHS.getSet());
2909
 
2910
      Universal |= RHS.isUniversal();
2911
      return IsUniversal != Universal || Size != Set.size();
2912
    }
2913
 
2914
  private:
2915
    /// Indicates if this set is "universal", containing every possible element.
2916
    bool Universal;
2917
 
2918
    /// The set of currently active assumptions.
2919
    DenseSet<BaseTy> Set;
2920
  };
2921
 
2922
  SetState() : Known(false), Assumed(true), IsAtFixedpoint(false) {}
2923
 
2924
  /// Initializes the known state with an initial set and initializes the
2925
  /// assumed state as universal.
2926
  SetState(const DenseSet<BaseTy> &Known)
2927
      : Known(Known), Assumed(true), IsAtFixedpoint(false) {}
2928
 
2929
  /// See AbstractState::isValidState()
2930
  bool isValidState() const override { return !Assumed.empty(); }
2931
 
2932
  /// See AbstractState::isAtFixpoint()
2933
  bool isAtFixpoint() const override { return IsAtFixedpoint; }
2934
 
2935
  /// See AbstractState::indicateOptimisticFixpoint(...)
2936
  ChangeStatus indicateOptimisticFixpoint() override {
2937
    IsAtFixedpoint = true;
2938
    Known = Assumed;
2939
    return ChangeStatus::UNCHANGED;
2940
  }
2941
 
2942
  /// See AbstractState::indicatePessimisticFixpoint(...)
2943
  ChangeStatus indicatePessimisticFixpoint() override {
2944
    IsAtFixedpoint = true;
2945
    Assumed = Known;
2946
    return ChangeStatus::CHANGED;
2947
  }
2948
 
2949
  /// Return the known state encoding.
2950
  const SetContents &getKnown() const { return Known; }
2951
 
2952
  /// Return the assumed state encoding.
2953
  const SetContents &getAssumed() const { return Assumed; }
2954
 
2955
  /// Returns if the set state contains the element.
2956
  bool setContains(const BaseTy &Elem) const {
2957
    return Assumed.getSet().contains(Elem) || Known.getSet().contains(Elem);
2958
  }
2959
 
2960
  /// Performs the set intersection between this set and \p RHS. Returns true if
2961
  /// changes were made.
2962
  bool getIntersection(const SetContents &RHS) {
2963
    unsigned SizeBefore = Assumed.getSet().size();
2964
 
2965
    // Get intersection and make sure that the known set is still a proper
2966
    // subset of the assumed set. A := K u (A ^ R).
2967
    Assumed.getIntersection(RHS);
2968
    Assumed.getUnion(Known);
2969
 
2970
    return SizeBefore != Assumed.getSet().size();
2971
  }
2972
 
2973
  /// Performs the set union between this set and \p RHS. Returns true if
2974
  /// changes were made.
2975
  bool getUnion(const SetContents &RHS) { return Assumed.getUnion(RHS); }
2976
 
2977
private:
2978
  /// The set of values known for this state.
2979
  SetContents Known;
2980
 
2981
  /// The set of assumed values for this state.
2982
  SetContents Assumed;
2983
 
2984
  bool IsAtFixedpoint;
2985
};
2986
 
2987
/// Helper struct necessary as the modular build fails if the virtual method
2988
/// IRAttribute::manifest is defined in the Attributor.cpp.
2989
struct IRAttributeManifest {
2990
  static ChangeStatus manifestAttrs(Attributor &A, const IRPosition &IRP,
2991
                                    const ArrayRef<Attribute> &DeducedAttrs,
2992
                                    bool ForceReplace = false);
2993
};
2994
 
2995
/// Helper to tie a abstract state implementation to an abstract attribute.
2996
template <typename StateTy, typename BaseType, class... Ts>
2997
struct StateWrapper : public BaseType, public StateTy {
2998
  /// Provide static access to the type of the state.
2999
  using StateType = StateTy;
3000
 
3001
  StateWrapper(const IRPosition &IRP, Ts... Args)
3002
      : BaseType(IRP), StateTy(Args...) {}
3003
 
3004
  /// See AbstractAttribute::getState(...).
3005
  StateType &getState() override { return *this; }
3006
 
3007
  /// See AbstractAttribute::getState(...).
3008
  const StateType &getState() const override { return *this; }
3009
};
3010
 
3011
/// Helper class that provides common functionality to manifest IR attributes.
3012
template <Attribute::AttrKind AK, typename BaseType>
3013
struct IRAttribute : public BaseType {
3014
  IRAttribute(const IRPosition &IRP) : BaseType(IRP) {}
3015
 
3016
  /// See AbstractAttribute::initialize(...).
3017
  void initialize(Attributor &A) override {
3018
    const IRPosition &IRP = this->getIRPosition();
3019
    if (isa<UndefValue>(IRP.getAssociatedValue()) ||
3020
        this->hasAttr(getAttrKind(), /* IgnoreSubsumingPositions */ false,
3021
                      &A)) {
3022
      this->getState().indicateOptimisticFixpoint();
3023
      return;
3024
    }
3025
 
3026
    bool IsFnInterface = IRP.isFnInterfaceKind();
3027
    const Function *FnScope = IRP.getAnchorScope();
3028
    // TODO: Not all attributes require an exact definition. Find a way to
3029
    //       enable deduction for some but not all attributes in case the
3030
    //       definition might be changed at runtime, see also
3031
    //       http://lists.llvm.org/pipermail/llvm-dev/2018-February/121275.html.
3032
    // TODO: We could always determine abstract attributes and if sufficient
3033
    //       information was found we could duplicate the functions that do not
3034
    //       have an exact definition.
3035
    if (IsFnInterface && (!FnScope || !A.isFunctionIPOAmendable(*FnScope)))
3036
      this->getState().indicatePessimisticFixpoint();
3037
  }
3038
 
3039
  /// See AbstractAttribute::manifest(...).
3040
  ChangeStatus manifest(Attributor &A) override {
3041
    if (isa<UndefValue>(this->getIRPosition().getAssociatedValue()))
3042
      return ChangeStatus::UNCHANGED;
3043
    SmallVector<Attribute, 4> DeducedAttrs;
3044
    getDeducedAttributes(this->getAnchorValue().getContext(), DeducedAttrs);
3045
    return IRAttributeManifest::manifestAttrs(A, this->getIRPosition(),
3046
                                              DeducedAttrs);
3047
  }
3048
 
3049
  /// Return the kind that identifies the abstract attribute implementation.
3050
  Attribute::AttrKind getAttrKind() const { return AK; }
3051
 
3052
  /// Return the deduced attributes in \p Attrs.
3053
  virtual void getDeducedAttributes(LLVMContext &Ctx,
3054
                                    SmallVectorImpl<Attribute> &Attrs) const {
3055
    Attrs.emplace_back(Attribute::get(Ctx, getAttrKind()));
3056
  }
3057
};
3058
 
3059
/// Base struct for all "concrete attribute" deductions.
3060
///
3061
/// The abstract attribute is a minimal interface that allows the Attributor to
3062
/// orchestrate the abstract/fixpoint analysis. The design allows to hide away
3063
/// implementation choices made for the subclasses but also to structure their
3064
/// implementation and simplify the use of other abstract attributes in-flight.
3065
///
3066
/// To allow easy creation of new attributes, most methods have default
3067
/// implementations. The ones that do not are generally straight forward, except
3068
/// `AbstractAttribute::updateImpl` which is the location of most reasoning
3069
/// associated with the abstract attribute. The update is invoked by the
3070
/// Attributor in case the situation used to justify the current optimistic
3071
/// state might have changed. The Attributor determines this automatically
3072
/// by monitoring the `Attributor::getAAFor` calls made by abstract attributes.
3073
///
3074
/// The `updateImpl` method should inspect the IR and other abstract attributes
3075
/// in-flight to justify the best possible (=optimistic) state. The actual
3076
/// implementation is, similar to the underlying abstract state encoding, not
3077
/// exposed. In the most common case, the `updateImpl` will go through a list of
3078
/// reasons why its optimistic state is valid given the current information. If
3079
/// any combination of them holds and is sufficient to justify the current
3080
/// optimistic state, the method shall return UNCHAGED. If not, the optimistic
3081
/// state is adjusted to the situation and the method shall return CHANGED.
3082
///
3083
/// If the manifestation of the "concrete attribute" deduced by the subclass
3084
/// differs from the "default" behavior, which is a (set of) LLVM-IR
3085
/// attribute(s) for an argument, call site argument, function return value, or
3086
/// function, the `AbstractAttribute::manifest` method should be overloaded.
3087
///
3088
/// NOTE: If the state obtained via getState() is INVALID, thus if
3089
///       AbstractAttribute::getState().isValidState() returns false, no
3090
///       information provided by the methods of this class should be used.
3091
/// NOTE: The Attributor currently has certain limitations to what we can do.
3092
///       As a general rule of thumb, "concrete" abstract attributes should *for
3093
///       now* only perform "backward" information propagation. That means
3094
///       optimistic information obtained through abstract attributes should
3095
///       only be used at positions that precede the origin of the information
3096
///       with regards to the program flow. More practically, information can
3097
///       *now* be propagated from instructions to their enclosing function, but
3098
///       *not* from call sites to the called function. The mechanisms to allow
3099
///       both directions will be added in the future.
3100
/// NOTE: The mechanics of adding a new "concrete" abstract attribute are
3101
///       described in the file comment.
3102
struct AbstractAttribute : public IRPosition, public AADepGraphNode {
3103
  using StateType = AbstractState;
3104
 
3105
  AbstractAttribute(const IRPosition &IRP) : IRPosition(IRP) {}
3106
 
3107
  /// Virtual destructor.
3108
  virtual ~AbstractAttribute() = default;
3109
 
3110
  /// This function is used to identify if an \p DGN is of type
3111
  /// AbstractAttribute so that the dyn_cast and cast can use such information
3112
  /// to cast an AADepGraphNode to an AbstractAttribute.
3113
  ///
3114
  /// We eagerly return true here because all AADepGraphNodes except for the
3115
  /// Synthethis Node are of type AbstractAttribute
3116
  static bool classof(const AADepGraphNode *DGN) { return true; }
3117
 
3118
  /// Initialize the state with the information in the Attributor \p A.
3119
  ///
3120
  /// This function is called by the Attributor once all abstract attributes
3121
  /// have been identified. It can and shall be used for task like:
3122
  ///  - identify existing knowledge in the IR and use it for the "known state"
3123
  ///  - perform any work that is not going to change over time, e.g., determine
3124
  ///    a subset of the IR, or attributes in-flight, that have to be looked at
3125
  ///    in the `updateImpl` method.
3126
  virtual void initialize(Attributor &A) {}
3127
 
3128
  /// A query AA is always scheduled as long as we do updates because it does
3129
  /// lazy computation that cannot be determined to be done from the outside.
3130
  /// However, while query AAs will not be fixed if they do not have outstanding
3131
  /// dependences, we will only schedule them like other AAs. If a query AA that
3132
  /// received a new query it needs to request an update via
3133
  /// `Attributor::requestUpdateForAA`.
3134
  virtual bool isQueryAA() const { return false; }
3135
 
3136
  /// Return the internal abstract state for inspection.
3137
  virtual StateType &getState() = 0;
3138
  virtual const StateType &getState() const = 0;
3139
 
3140
  /// Return an IR position, see struct IRPosition.
3141
  const IRPosition &getIRPosition() const { return *this; };
3142
  IRPosition &getIRPosition() { return *this; };
3143
 
3144
  /// Helper functions, for debug purposes only.
3145
  ///{
3146
  void print(raw_ostream &OS) const override;
3147
  virtual void printWithDeps(raw_ostream &OS) const;
3148
  void dump() const { print(dbgs()); }
3149
 
3150
  /// This function should return the "summarized" assumed state as string.
3151
  virtual const std::string getAsStr() const = 0;
3152
 
3153
  /// This function should return the name of the AbstractAttribute
3154
  virtual const std::string getName() const = 0;
3155
 
3156
  /// This function should return the address of the ID of the AbstractAttribute
3157
  virtual const char *getIdAddr() const = 0;
3158
  ///}
3159
 
3160
  /// Allow the Attributor access to the protected methods.
3161
  friend struct Attributor;
3162
 
3163
protected:
3164
  /// Hook for the Attributor to trigger an update of the internal state.
3165
  ///
3166
  /// If this attribute is already fixed, this method will return UNCHANGED,
3167
  /// otherwise it delegates to `AbstractAttribute::updateImpl`.
3168
  ///
3169
  /// \Return CHANGED if the internal state changed, otherwise UNCHANGED.
3170
  ChangeStatus update(Attributor &A);
3171
 
3172
  /// Hook for the Attributor to trigger the manifestation of the information
3173
  /// represented by the abstract attribute in the LLVM-IR.
3174
  ///
3175
  /// \Return CHANGED if the IR was altered, otherwise UNCHANGED.
3176
  virtual ChangeStatus manifest(Attributor &A) {
3177
    return ChangeStatus::UNCHANGED;
3178
  }
3179
 
3180
  /// Hook to enable custom statistic tracking, called after manifest that
3181
  /// resulted in a change if statistics are enabled.
3182
  ///
3183
  /// We require subclasses to provide an implementation so we remember to
3184
  /// add statistics for them.
3185
  virtual void trackStatistics() const = 0;
3186
 
3187
  /// The actual update/transfer function which has to be implemented by the
3188
  /// derived classes.
3189
  ///
3190
  /// If it is called, the environment has changed and we have to determine if
3191
  /// the current information is still valid or adjust it otherwise.
3192
  ///
3193
  /// \Return CHANGED if the internal state changed, otherwise UNCHANGED.
3194
  virtual ChangeStatus updateImpl(Attributor &A) = 0;
3195
};
3196
 
3197
/// Forward declarations of output streams for debug purposes.
3198
///
3199
///{
3200
raw_ostream &operator<<(raw_ostream &OS, const AbstractAttribute &AA);
3201
raw_ostream &operator<<(raw_ostream &OS, ChangeStatus S);
3202
raw_ostream &operator<<(raw_ostream &OS, IRPosition::Kind);
3203
raw_ostream &operator<<(raw_ostream &OS, const IRPosition &);
3204
raw_ostream &operator<<(raw_ostream &OS, const AbstractState &State);
3205
template <typename base_ty, base_ty BestState, base_ty WorstState>
3206
raw_ostream &
3207
operator<<(raw_ostream &OS,
3208
           const IntegerStateBase<base_ty, BestState, WorstState> &S) {
3209
  return OS << "(" << S.getKnown() << "-" << S.getAssumed() << ")"
3210
            << static_cast<const AbstractState &>(S);
3211
}
3212
raw_ostream &operator<<(raw_ostream &OS, const IntegerRangeState &State);
3213
///}
3214
 
3215
struct AttributorPass : public PassInfoMixin<AttributorPass> {
3216
  PreservedAnalyses run(Module &M, ModuleAnalysisManager &AM);
3217
};
3218
struct AttributorCGSCCPass : public PassInfoMixin<AttributorCGSCCPass> {
3219
  PreservedAnalyses run(LazyCallGraph::SCC &C, CGSCCAnalysisManager &AM,
3220
                        LazyCallGraph &CG, CGSCCUpdateResult &UR);
3221
};
3222
 
3223
Pass *createAttributorLegacyPass();
3224
Pass *createAttributorCGSCCLegacyPass();
3225
 
3226
/// Helper function to clamp a state \p S of type \p StateType with the
3227
/// information in \p R and indicate/return if \p S did change (as-in update is
3228
/// required to be run again).
3229
template <typename StateType>
3230
ChangeStatus clampStateAndIndicateChange(StateType &S, const StateType &R) {
3231
  auto Assumed = S.getAssumed();
3232
  S ^= R;
3233
  return Assumed == S.getAssumed() ? ChangeStatus::UNCHANGED
3234
                                   : ChangeStatus::CHANGED;
3235
}
3236
 
3237
/// ----------------------------------------------------------------------------
3238
///                       Abstract Attribute Classes
3239
/// ----------------------------------------------------------------------------
3240
 
3241
/// An abstract attribute for the returned values of a function.
3242
struct AAReturnedValues
3243
    : public IRAttribute<Attribute::Returned, AbstractAttribute> {
3244
  AAReturnedValues(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
3245
 
3246
  /// Check \p Pred on all returned values.
3247
  ///
3248
  /// This method will evaluate \p Pred on returned values and return
3249
  /// true if (1) all returned values are known, and (2) \p Pred returned true
3250
  /// for all returned values.
3251
  ///
3252
  /// Note: Unlike the Attributor::checkForAllReturnedValuesAndReturnInsts
3253
  /// method, this one will not filter dead return instructions.
3254
  virtual bool checkForAllReturnedValuesAndReturnInsts(
3255
      function_ref<bool(Value &, const SmallSetVector<ReturnInst *, 4> &)> Pred)
3256
      const = 0;
3257
 
3258
  using iterator =
3259
      MapVector<Value *, SmallSetVector<ReturnInst *, 4>>::iterator;
3260
  using const_iterator =
3261
      MapVector<Value *, SmallSetVector<ReturnInst *, 4>>::const_iterator;
3262
  virtual llvm::iterator_range<iterator> returned_values() = 0;
3263
  virtual llvm::iterator_range<const_iterator> returned_values() const = 0;
3264
 
3265
  virtual size_t getNumReturnValues() const = 0;
3266
 
3267
  /// Create an abstract attribute view for the position \p IRP.
3268
  static AAReturnedValues &createForPosition(const IRPosition &IRP,
3269
                                             Attributor &A);
3270
 
3271
  /// See AbstractAttribute::getName()
3272
  const std::string getName() const override { return "AAReturnedValues"; }
3273
 
3274
  /// See AbstractAttribute::getIdAddr()
3275
  const char *getIdAddr() const override { return &ID; }
3276
 
3277
  /// This function should return true if the type of the \p AA is
3278
  /// AAReturnedValues
3279
  static bool classof(const AbstractAttribute *AA) {
3280
    return (AA->getIdAddr() == &ID);
3281
  }
3282
 
3283
  /// Unique ID (due to the unique address)
3284
  static const char ID;
3285
};
3286
 
3287
struct AANoUnwind
3288
    : public IRAttribute<Attribute::NoUnwind,
3289
                         StateWrapper<BooleanState, AbstractAttribute>> {
3290
  AANoUnwind(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
3291
 
3292
  /// Returns true if nounwind is assumed.
3293
  bool isAssumedNoUnwind() const { return getAssumed(); }
3294
 
3295
  /// Returns true if nounwind is known.
3296
  bool isKnownNoUnwind() const { return getKnown(); }
3297
 
3298
  /// Create an abstract attribute view for the position \p IRP.
3299
  static AANoUnwind &createForPosition(const IRPosition &IRP, Attributor &A);
3300
 
3301
  /// See AbstractAttribute::getName()
3302
  const std::string getName() const override { return "AANoUnwind"; }
3303
 
3304
  /// See AbstractAttribute::getIdAddr()
3305
  const char *getIdAddr() const override { return &ID; }
3306
 
3307
  /// This function should return true if the type of the \p AA is AANoUnwind
3308
  static bool classof(const AbstractAttribute *AA) {
3309
    return (AA->getIdAddr() == &ID);
3310
  }
3311
 
3312
  /// Unique ID (due to the unique address)
3313
  static const char ID;
3314
};
3315
 
3316
struct AANoSync
3317
    : public IRAttribute<Attribute::NoSync,
3318
                         StateWrapper<BooleanState, AbstractAttribute>> {
3319
  AANoSync(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
3320
 
3321
  /// Returns true if "nosync" is assumed.
3322
  bool isAssumedNoSync() const { return getAssumed(); }
3323
 
3324
  /// Returns true if "nosync" is known.
3325
  bool isKnownNoSync() const { return getKnown(); }
3326
 
3327
  /// Helper function used to determine whether an instruction is non-relaxed
3328
  /// atomic. In other words, if an atomic instruction does not have unordered
3329
  /// or monotonic ordering
3330
  static bool isNonRelaxedAtomic(const Instruction *I);
3331
 
3332
  /// Helper function specific for intrinsics which are potentially volatile.
3333
  static bool isNoSyncIntrinsic(const Instruction *I);
3334
 
3335
  /// Helper function to determine if \p CB is an aligned (GPU) barrier. Aligned
3336
  /// barriers have to be executed by all threads. The flag \p ExecutedAligned
3337
  /// indicates if the call is executed by all threads in a (thread) block in an
3338
  /// aligned way. If that is the case, non-aligned barriers are effectively
3339
  /// aligned barriers.
3340
  static bool isAlignedBarrier(const CallBase &CB, bool ExecutedAligned);
3341
 
3342
  /// Create an abstract attribute view for the position \p IRP.
3343
  static AANoSync &createForPosition(const IRPosition &IRP, Attributor &A);
3344
 
3345
  /// See AbstractAttribute::getName()
3346
  const std::string getName() const override { return "AANoSync"; }
3347
 
3348
  /// See AbstractAttribute::getIdAddr()
3349
  const char *getIdAddr() const override { return &ID; }
3350
 
3351
  /// This function should return true if the type of the \p AA is AANoSync
3352
  static bool classof(const AbstractAttribute *AA) {
3353
    return (AA->getIdAddr() == &ID);
3354
  }
3355
 
3356
  /// Unique ID (due to the unique address)
3357
  static const char ID;
3358
};
3359
 
3360
/// An abstract interface for all nonnull attributes.
3361
struct AANonNull
3362
    : public IRAttribute<Attribute::NonNull,
3363
                         StateWrapper<BooleanState, AbstractAttribute>> {
3364
  AANonNull(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
3365
 
3366
  /// Return true if we assume that the underlying value is nonnull.
3367
  bool isAssumedNonNull() const { return getAssumed(); }
3368
 
3369
  /// Return true if we know that underlying value is nonnull.
3370
  bool isKnownNonNull() const { return getKnown(); }
3371
 
3372
  /// Create an abstract attribute view for the position \p IRP.
3373
  static AANonNull &createForPosition(const IRPosition &IRP, Attributor &A);
3374
 
3375
  /// See AbstractAttribute::getName()
3376
  const std::string getName() const override { return "AANonNull"; }
3377
 
3378
  /// See AbstractAttribute::getIdAddr()
3379
  const char *getIdAddr() const override { return &ID; }
3380
 
3381
  /// This function should return true if the type of the \p AA is AANonNull
3382
  static bool classof(const AbstractAttribute *AA) {
3383
    return (AA->getIdAddr() == &ID);
3384
  }
3385
 
3386
  /// Unique ID (due to the unique address)
3387
  static const char ID;
3388
};
3389
 
3390
/// An abstract attribute for norecurse.
3391
struct AANoRecurse
3392
    : public IRAttribute<Attribute::NoRecurse,
3393
                         StateWrapper<BooleanState, AbstractAttribute>> {
3394
  AANoRecurse(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
3395
 
3396
  /// Return true if "norecurse" is assumed.
3397
  bool isAssumedNoRecurse() const { return getAssumed(); }
3398
 
3399
  /// Return true if "norecurse" is known.
3400
  bool isKnownNoRecurse() const { return getKnown(); }
3401
 
3402
  /// Create an abstract attribute view for the position \p IRP.
3403
  static AANoRecurse &createForPosition(const IRPosition &IRP, Attributor &A);
3404
 
3405
  /// See AbstractAttribute::getName()
3406
  const std::string getName() const override { return "AANoRecurse"; }
3407
 
3408
  /// See AbstractAttribute::getIdAddr()
3409
  const char *getIdAddr() const override { return &ID; }
3410
 
3411
  /// This function should return true if the type of the \p AA is AANoRecurse
3412
  static bool classof(const AbstractAttribute *AA) {
3413
    return (AA->getIdAddr() == &ID);
3414
  }
3415
 
3416
  /// Unique ID (due to the unique address)
3417
  static const char ID;
3418
};
3419
 
3420
/// An abstract attribute for willreturn.
3421
struct AAWillReturn
3422
    : public IRAttribute<Attribute::WillReturn,
3423
                         StateWrapper<BooleanState, AbstractAttribute>> {
3424
  AAWillReturn(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
3425
 
3426
  /// Return true if "willreturn" is assumed.
3427
  bool isAssumedWillReturn() const { return getAssumed(); }
3428
 
3429
  /// Return true if "willreturn" is known.
3430
  bool isKnownWillReturn() const { return getKnown(); }
3431
 
3432
  /// Create an abstract attribute view for the position \p IRP.
3433
  static AAWillReturn &createForPosition(const IRPosition &IRP, Attributor &A);
3434
 
3435
  /// See AbstractAttribute::getName()
3436
  const std::string getName() const override { return "AAWillReturn"; }
3437
 
3438
  /// See AbstractAttribute::getIdAddr()
3439
  const char *getIdAddr() const override { return &ID; }
3440
 
3441
  /// This function should return true if the type of the \p AA is AAWillReturn
3442
  static bool classof(const AbstractAttribute *AA) {
3443
    return (AA->getIdAddr() == &ID);
3444
  }
3445
 
3446
  /// Unique ID (due to the unique address)
3447
  static const char ID;
3448
};
3449
 
3450
/// An abstract attribute for undefined behavior.
3451
struct AAUndefinedBehavior
3452
    : public StateWrapper<BooleanState, AbstractAttribute> {
3453
  using Base = StateWrapper<BooleanState, AbstractAttribute>;
3454
  AAUndefinedBehavior(const IRPosition &IRP, Attributor &A) : Base(IRP) {}
3455
 
3456
  /// Return true if "undefined behavior" is assumed.
3457
  bool isAssumedToCauseUB() const { return getAssumed(); }
3458
 
3459
  /// Return true if "undefined behavior" is assumed for a specific instruction.
3460
  virtual bool isAssumedToCauseUB(Instruction *I) const = 0;
3461
 
3462
  /// Return true if "undefined behavior" is known.
3463
  bool isKnownToCauseUB() const { return getKnown(); }
3464
 
3465
  /// Return true if "undefined behavior" is known for a specific instruction.
3466
  virtual bool isKnownToCauseUB(Instruction *I) const = 0;
3467
 
3468
  /// Create an abstract attribute view for the position \p IRP.
3469
  static AAUndefinedBehavior &createForPosition(const IRPosition &IRP,
3470
                                                Attributor &A);
3471
 
3472
  /// See AbstractAttribute::getName()
3473
  const std::string getName() const override { return "AAUndefinedBehavior"; }
3474
 
3475
  /// See AbstractAttribute::getIdAddr()
3476
  const char *getIdAddr() const override { return &ID; }
3477
 
3478
  /// This function should return true if the type of the \p AA is
3479
  /// AAUndefineBehavior
3480
  static bool classof(const AbstractAttribute *AA) {
3481
    return (AA->getIdAddr() == &ID);
3482
  }
3483
 
3484
  /// Unique ID (due to the unique address)
3485
  static const char ID;
3486
};
3487
 
3488
/// An abstract interface to determine reachability of point A to B.
3489
struct AAIntraFnReachability
3490
    : public StateWrapper<BooleanState, AbstractAttribute> {
3491
  using Base = StateWrapper<BooleanState, AbstractAttribute>;
3492
  AAIntraFnReachability(const IRPosition &IRP, Attributor &A) : Base(IRP) {}
3493
 
3494
  /// Returns true if 'From' instruction is assumed to reach, 'To' instruction.
3495
  /// Users should provide two positions they are interested in, and the class
3496
  /// determines (and caches) reachability.
3497
  virtual bool isAssumedReachable(
3498
      Attributor &A, const Instruction &From, const Instruction &To,
3499
      const AA::InstExclusionSetTy *ExclusionSet = nullptr) const = 0;
3500
 
3501
  /// Create an abstract attribute view for the position \p IRP.
3502
  static AAIntraFnReachability &createForPosition(const IRPosition &IRP,
3503
                                                  Attributor &A);
3504
 
3505
  /// See AbstractAttribute::getName()
3506
  const std::string getName() const override { return "AAIntraFnReachability"; }
3507
 
3508
  /// See AbstractAttribute::getIdAddr()
3509
  const char *getIdAddr() const override { return &ID; }
3510
 
3511
  /// This function should return true if the type of the \p AA is
3512
  /// AAIntraFnReachability
3513
  static bool classof(const AbstractAttribute *AA) {
3514
    return (AA->getIdAddr() == &ID);
3515
  }
3516
 
3517
  /// Unique ID (due to the unique address)
3518
  static const char ID;
3519
};
3520
 
3521
/// An abstract interface for all noalias attributes.
3522
struct AANoAlias
3523
    : public IRAttribute<Attribute::NoAlias,
3524
                         StateWrapper<BooleanState, AbstractAttribute>> {
3525
  AANoAlias(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
3526
 
3527
  /// Return true if we assume that the underlying value is alias.
3528
  bool isAssumedNoAlias() const { return getAssumed(); }
3529
 
3530
  /// Return true if we know that underlying value is noalias.
3531
  bool isKnownNoAlias() const { return getKnown(); }
3532
 
3533
  /// Create an abstract attribute view for the position \p IRP.
3534
  static AANoAlias &createForPosition(const IRPosition &IRP, Attributor &A);
3535
 
3536
  /// See AbstractAttribute::getName()
3537
  const std::string getName() const override { return "AANoAlias"; }
3538
 
3539
  /// See AbstractAttribute::getIdAddr()
3540
  const char *getIdAddr() const override { return &ID; }
3541
 
3542
  /// This function should return true if the type of the \p AA is AANoAlias
3543
  static bool classof(const AbstractAttribute *AA) {
3544
    return (AA->getIdAddr() == &ID);
3545
  }
3546
 
3547
  /// Unique ID (due to the unique address)
3548
  static const char ID;
3549
};
3550
 
3551
/// An AbstractAttribute for nofree.
3552
struct AANoFree
3553
    : public IRAttribute<Attribute::NoFree,
3554
                         StateWrapper<BooleanState, AbstractAttribute>> {
3555
  AANoFree(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
3556
 
3557
  /// Return true if "nofree" is assumed.
3558
  bool isAssumedNoFree() const { return getAssumed(); }
3559
 
3560
  /// Return true if "nofree" is known.
3561
  bool isKnownNoFree() const { return getKnown(); }
3562
 
3563
  /// Create an abstract attribute view for the position \p IRP.
3564
  static AANoFree &createForPosition(const IRPosition &IRP, Attributor &A);
3565
 
3566
  /// See AbstractAttribute::getName()
3567
  const std::string getName() const override { return "AANoFree"; }
3568
 
3569
  /// See AbstractAttribute::getIdAddr()
3570
  const char *getIdAddr() const override { return &ID; }
3571
 
3572
  /// This function should return true if the type of the \p AA is AANoFree
3573
  static bool classof(const AbstractAttribute *AA) {
3574
    return (AA->getIdAddr() == &ID);
3575
  }
3576
 
3577
  /// Unique ID (due to the unique address)
3578
  static const char ID;
3579
};
3580
 
3581
/// An AbstractAttribute for noreturn.
3582
struct AANoReturn
3583
    : public IRAttribute<Attribute::NoReturn,
3584
                         StateWrapper<BooleanState, AbstractAttribute>> {
3585
  AANoReturn(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
3586
 
3587
  /// Return true if the underlying object is assumed to never return.
3588
  bool isAssumedNoReturn() const { return getAssumed(); }
3589
 
3590
  /// Return true if the underlying object is known to never return.
3591
  bool isKnownNoReturn() const { return getKnown(); }
3592
 
3593
  /// Create an abstract attribute view for the position \p IRP.
3594
  static AANoReturn &createForPosition(const IRPosition &IRP, Attributor &A);
3595
 
3596
  /// See AbstractAttribute::getName()
3597
  const std::string getName() const override { return "AANoReturn"; }
3598
 
3599
  /// See AbstractAttribute::getIdAddr()
3600
  const char *getIdAddr() const override { return &ID; }
3601
 
3602
  /// This function should return true if the type of the \p AA is AANoReturn
3603
  static bool classof(const AbstractAttribute *AA) {
3604
    return (AA->getIdAddr() == &ID);
3605
  }
3606
 
3607
  /// Unique ID (due to the unique address)
3608
  static const char ID;
3609
};
3610
 
3611
/// An abstract interface for liveness abstract attribute.
3612
struct AAIsDead
3613
    : public StateWrapper<BitIntegerState<uint8_t, 3, 0>, AbstractAttribute> {
3614
  using Base = StateWrapper<BitIntegerState<uint8_t, 3, 0>, AbstractAttribute>;
3615
  AAIsDead(const IRPosition &IRP, Attributor &A) : Base(IRP) {}
3616
 
3617
  /// State encoding bits. A set bit in the state means the property holds.
3618
  enum {
3619
    HAS_NO_EFFECT = 1 << 0,
3620
    IS_REMOVABLE = 1 << 1,
3621
 
3622
    IS_DEAD = HAS_NO_EFFECT | IS_REMOVABLE,
3623
  };
3624
  static_assert(IS_DEAD == getBestState(), "Unexpected BEST_STATE value");
3625
 
3626
protected:
3627
  /// The query functions are protected such that other attributes need to go
3628
  /// through the Attributor interfaces: `Attributor::isAssumedDead(...)`
3629
 
3630
  /// Returns true if the underlying value is assumed dead.
3631
  virtual bool isAssumedDead() const = 0;
3632
 
3633
  /// Returns true if the underlying value is known dead.
3634
  virtual bool isKnownDead() const = 0;
3635
 
3636
  /// Returns true if \p BB is known dead.
3637
  virtual bool isKnownDead(const BasicBlock *BB) const = 0;
3638
 
3639
  /// Returns true if \p I is assumed dead.
3640
  virtual bool isAssumedDead(const Instruction *I) const = 0;
3641
 
3642
  /// Returns true if \p I is known dead.
3643
  virtual bool isKnownDead(const Instruction *I) const = 0;
3644
 
3645
  /// Return true if the underlying value is a store that is known to be
3646
  /// removable. This is different from dead stores as the removable store
3647
  /// can have an effect on live values, especially loads, but that effect
3648
  /// is propagated which allows us to remove the store in turn.
3649
  virtual bool isRemovableStore() const { return false; }
3650
 
3651
  /// This method is used to check if at least one instruction in a collection
3652
  /// of instructions is live.
3653
  template <typename T> bool isLiveInstSet(T begin, T end) const {
3654
    for (const auto &I : llvm::make_range(begin, end)) {
3655
      assert(I->getFunction() == getIRPosition().getAssociatedFunction() &&
3656
             "Instruction must be in the same anchor scope function.");
3657
 
3658
      if (!isAssumedDead(I))
3659
        return true;
3660
    }
3661
 
3662
    return false;
3663
  }
3664
 
3665
public:
3666
  /// Create an abstract attribute view for the position \p IRP.
3667
  static AAIsDead &createForPosition(const IRPosition &IRP, Attributor &A);
3668
 
3669
  /// Determine if \p F might catch asynchronous exceptions.
3670
  static bool mayCatchAsynchronousExceptions(const Function &F) {
3671
    return F.hasPersonalityFn() && !canSimplifyInvokeNoUnwind(&F);
3672
  }
3673
 
3674
  /// Returns true if \p BB is assumed dead.
3675
  virtual bool isAssumedDead(const BasicBlock *BB) const = 0;
3676
 
3677
  /// Return if the edge from \p From BB to \p To BB is assumed dead.
3678
  /// This is specifically useful in AAReachability.
3679
  virtual bool isEdgeDead(const BasicBlock *From, const BasicBlock *To) const {
3680
    return false;
3681
  }
3682
 
3683
  /// See AbstractAttribute::getName()
3684
  const std::string getName() const override { return "AAIsDead"; }
3685
 
3686
  /// See AbstractAttribute::getIdAddr()
3687
  const char *getIdAddr() const override { return &ID; }
3688
 
3689
  /// This function should return true if the type of the \p AA is AAIsDead
3690
  static bool classof(const AbstractAttribute *AA) {
3691
    return (AA->getIdAddr() == &ID);
3692
  }
3693
 
3694
  /// Unique ID (due to the unique address)
3695
  static const char ID;
3696
 
3697
  friend struct Attributor;
3698
};
3699
 
3700
/// State for dereferenceable attribute
3701
struct DerefState : AbstractState {
3702
 
3703
  static DerefState getBestState() { return DerefState(); }
3704
  static DerefState getBestState(const DerefState &) { return getBestState(); }
3705
 
3706
  /// Return the worst possible representable state.
3707
  static DerefState getWorstState() {
3708
    DerefState DS;
3709
    DS.indicatePessimisticFixpoint();
3710
    return DS;
3711
  }
3712
  static DerefState getWorstState(const DerefState &) {
3713
    return getWorstState();
3714
  }
3715
 
3716
  /// State representing for dereferenceable bytes.
3717
  IncIntegerState<> DerefBytesState;
3718
 
3719
  /// Map representing for accessed memory offsets and sizes.
3720
  /// A key is Offset and a value is size.
3721
  /// If there is a load/store instruction something like,
3722
  ///   p[offset] = v;
3723
  /// (offset, sizeof(v)) will be inserted to this map.
3724
  /// std::map is used because we want to iterate keys in ascending order.
3725
  std::map<int64_t, uint64_t> AccessedBytesMap;
3726
 
3727
  /// Helper function to calculate dereferenceable bytes from current known
3728
  /// bytes and accessed bytes.
3729
  ///
3730
  /// int f(int *A){
3731
  ///    *A = 0;
3732
  ///    *(A+2) = 2;
3733
  ///    *(A+1) = 1;
3734
  ///    *(A+10) = 10;
3735
  /// }
3736
  /// ```
3737
  /// In that case, AccessedBytesMap is `{0:4, 4:4, 8:4, 40:4}`.
3738
  /// AccessedBytesMap is std::map so it is iterated in accending order on
3739
  /// key(Offset). So KnownBytes will be updated like this:
3740
  ///
3741
  /// |Access | KnownBytes
3742
  /// |(0, 4)| 0 -> 4
3743
  /// |(4, 4)| 4 -> 8
3744
  /// |(8, 4)| 8 -> 12
3745
  /// |(40, 4) | 12 (break)
3746
  void computeKnownDerefBytesFromAccessedMap() {
3747
    int64_t KnownBytes = DerefBytesState.getKnown();
3748
    for (auto &Access : AccessedBytesMap) {
3749
      if (KnownBytes < Access.first)
3750
        break;
3751
      KnownBytes = std::max(KnownBytes, Access.first + (int64_t)Access.second);
3752
    }
3753
 
3754
    DerefBytesState.takeKnownMaximum(KnownBytes);
3755
  }
3756
 
3757
  /// State representing that whether the value is globaly dereferenceable.
3758
  BooleanState GlobalState;
3759
 
3760
  /// See AbstractState::isValidState()
3761
  bool isValidState() const override { return DerefBytesState.isValidState(); }
3762
 
3763
  /// See AbstractState::isAtFixpoint()
3764
  bool isAtFixpoint() const override {
3765
    return !isValidState() ||
3766
           (DerefBytesState.isAtFixpoint() && GlobalState.isAtFixpoint());
3767
  }
3768
 
3769
  /// See AbstractState::indicateOptimisticFixpoint(...)
3770
  ChangeStatus indicateOptimisticFixpoint() override {
3771
    DerefBytesState.indicateOptimisticFixpoint();
3772
    GlobalState.indicateOptimisticFixpoint();
3773
    return ChangeStatus::UNCHANGED;
3774
  }
3775
 
3776
  /// See AbstractState::indicatePessimisticFixpoint(...)
3777
  ChangeStatus indicatePessimisticFixpoint() override {
3778
    DerefBytesState.indicatePessimisticFixpoint();
3779
    GlobalState.indicatePessimisticFixpoint();
3780
    return ChangeStatus::CHANGED;
3781
  }
3782
 
3783
  /// Update known dereferenceable bytes.
3784
  void takeKnownDerefBytesMaximum(uint64_t Bytes) {
3785
    DerefBytesState.takeKnownMaximum(Bytes);
3786
 
3787
    // Known bytes might increase.
3788
    computeKnownDerefBytesFromAccessedMap();
3789
  }
3790
 
3791
  /// Update assumed dereferenceable bytes.
3792
  void takeAssumedDerefBytesMinimum(uint64_t Bytes) {
3793
    DerefBytesState.takeAssumedMinimum(Bytes);
3794
  }
3795
 
3796
  /// Add accessed bytes to the map.
3797
  void addAccessedBytes(int64_t Offset, uint64_t Size) {
3798
    uint64_t &AccessedBytes = AccessedBytesMap[Offset];
3799
    AccessedBytes = std::max(AccessedBytes, Size);
3800
 
3801
    // Known bytes might increase.
3802
    computeKnownDerefBytesFromAccessedMap();
3803
  }
3804
 
3805
  /// Equality for DerefState.
3806
  bool operator==(const DerefState &R) const {
3807
    return this->DerefBytesState == R.DerefBytesState &&
3808
           this->GlobalState == R.GlobalState;
3809
  }
3810
 
3811
  /// Inequality for DerefState.
3812
  bool operator!=(const DerefState &R) const { return !(*this == R); }
3813
 
3814
  /// See IntegerStateBase::operator^=
3815
  DerefState operator^=(const DerefState &R) {
3816
    DerefBytesState ^= R.DerefBytesState;
3817
    GlobalState ^= R.GlobalState;
3818
    return *this;
3819
  }
3820
 
3821
  /// See IntegerStateBase::operator+=
3822
  DerefState operator+=(const DerefState &R) {
3823
    DerefBytesState += R.DerefBytesState;
3824
    GlobalState += R.GlobalState;
3825
    return *this;
3826
  }
3827
 
3828
  /// See IntegerStateBase::operator&=
3829
  DerefState operator&=(const DerefState &R) {
3830
    DerefBytesState &= R.DerefBytesState;
3831
    GlobalState &= R.GlobalState;
3832
    return *this;
3833
  }
3834
 
3835
  /// See IntegerStateBase::operator|=
3836
  DerefState operator|=(const DerefState &R) {
3837
    DerefBytesState |= R.DerefBytesState;
3838
    GlobalState |= R.GlobalState;
3839
    return *this;
3840
  }
3841
 
3842
protected:
3843
  const AANonNull *NonNullAA = nullptr;
3844
};
3845
 
3846
/// An abstract interface for all dereferenceable attribute.
3847
struct AADereferenceable
3848
    : public IRAttribute<Attribute::Dereferenceable,
3849
                         StateWrapper<DerefState, AbstractAttribute>> {
3850
  AADereferenceable(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
3851
 
3852
  /// Return true if we assume that the underlying value is nonnull.
3853
  bool isAssumedNonNull() const {
3854
    return NonNullAA && NonNullAA->isAssumedNonNull();
3855
  }
3856
 
3857
  /// Return true if we know that the underlying value is nonnull.
3858
  bool isKnownNonNull() const {
3859
    return NonNullAA && NonNullAA->isKnownNonNull();
3860
  }
3861
 
3862
  /// Return true if we assume that underlying value is
3863
  /// dereferenceable(_or_null) globally.
3864
  bool isAssumedGlobal() const { return GlobalState.getAssumed(); }
3865
 
3866
  /// Return true if we know that underlying value is
3867
  /// dereferenceable(_or_null) globally.
3868
  bool isKnownGlobal() const { return GlobalState.getKnown(); }
3869
 
3870
  /// Return assumed dereferenceable bytes.
3871
  uint32_t getAssumedDereferenceableBytes() const {
3872
    return DerefBytesState.getAssumed();
3873
  }
3874
 
3875
  /// Return known dereferenceable bytes.
3876
  uint32_t getKnownDereferenceableBytes() const {
3877
    return DerefBytesState.getKnown();
3878
  }
3879
 
3880
  /// Create an abstract attribute view for the position \p IRP.
3881
  static AADereferenceable &createForPosition(const IRPosition &IRP,
3882
                                              Attributor &A);
3883
 
3884
  /// See AbstractAttribute::getName()
3885
  const std::string getName() const override { return "AADereferenceable"; }
3886
 
3887
  /// See AbstractAttribute::getIdAddr()
3888
  const char *getIdAddr() const override { return &ID; }
3889
 
3890
  /// This function should return true if the type of the \p AA is
3891
  /// AADereferenceable
3892
  static bool classof(const AbstractAttribute *AA) {
3893
    return (AA->getIdAddr() == &ID);
3894
  }
3895
 
3896
  /// Unique ID (due to the unique address)
3897
  static const char ID;
3898
};
3899
 
3900
using AAAlignmentStateType =
3901
    IncIntegerState<uint64_t, Value::MaximumAlignment, 1>;
3902
/// An abstract interface for all align attributes.
3903
struct AAAlign : public IRAttribute<
3904
                     Attribute::Alignment,
3905
                     StateWrapper<AAAlignmentStateType, AbstractAttribute>> {
3906
  AAAlign(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
3907
 
3908
  /// Return assumed alignment.
3909
  Align getAssumedAlign() const { return Align(getAssumed()); }
3910
 
3911
  /// Return known alignment.
3912
  Align getKnownAlign() const { return Align(getKnown()); }
3913
 
3914
  /// See AbstractAttribute::getName()
3915
  const std::string getName() const override { return "AAAlign"; }
3916
 
3917
  /// See AbstractAttribute::getIdAddr()
3918
  const char *getIdAddr() const override { return &ID; }
3919
 
3920
  /// This function should return true if the type of the \p AA is AAAlign
3921
  static bool classof(const AbstractAttribute *AA) {
3922
    return (AA->getIdAddr() == &ID);
3923
  }
3924
 
3925
  /// Create an abstract attribute view for the position \p IRP.
3926
  static AAAlign &createForPosition(const IRPosition &IRP, Attributor &A);
3927
 
3928
  /// Unique ID (due to the unique address)
3929
  static const char ID;
3930
};
3931
 
3932
/// An abstract interface to track if a value leaves it's defining function
3933
/// instance.
3934
/// TODO: We should make it a ternary AA tracking uniqueness, and uniqueness
3935
/// wrt. the Attributor analysis separately.
3936
struct AAInstanceInfo : public StateWrapper<BooleanState, AbstractAttribute> {
3937
  AAInstanceInfo(const IRPosition &IRP, Attributor &A)
3938
      : StateWrapper<BooleanState, AbstractAttribute>(IRP) {}
3939
 
3940
  /// Return true if we know that the underlying value is unique in its scope
3941
  /// wrt. the Attributor analysis. That means it might not be unique but we can
3942
  /// still use pointer equality without risking to represent two instances with
3943
  /// one `llvm::Value`.
3944
  bool isKnownUniqueForAnalysis() const { return isKnown(); }
3945
 
3946
  /// Return true if we assume that the underlying value is unique in its scope
3947
  /// wrt. the Attributor analysis. That means it might not be unique but we can
3948
  /// still use pointer equality without risking to represent two instances with
3949
  /// one `llvm::Value`.
3950
  bool isAssumedUniqueForAnalysis() const { return isAssumed(); }
3951
 
3952
  /// Create an abstract attribute view for the position \p IRP.
3953
  static AAInstanceInfo &createForPosition(const IRPosition &IRP,
3954
                                           Attributor &A);
3955
 
3956
  /// See AbstractAttribute::getName()
3957
  const std::string getName() const override { return "AAInstanceInfo"; }
3958
 
3959
  /// See AbstractAttribute::getIdAddr()
3960
  const char *getIdAddr() const override { return &ID; }
3961
 
3962
  /// This function should return true if the type of the \p AA is
3963
  /// AAInstanceInfo
3964
  static bool classof(const AbstractAttribute *AA) {
3965
    return (AA->getIdAddr() == &ID);
3966
  }
3967
 
3968
  /// Unique ID (due to the unique address)
3969
  static const char ID;
3970
};
3971
 
3972
/// An abstract interface for all nocapture attributes.
3973
struct AANoCapture
3974
    : public IRAttribute<
3975
          Attribute::NoCapture,
3976
          StateWrapper<BitIntegerState<uint16_t, 7, 0>, AbstractAttribute>> {
3977
  AANoCapture(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
3978
 
3979
  /// State encoding bits. A set bit in the state means the property holds.
3980
  /// NO_CAPTURE is the best possible state, 0 the worst possible state.
3981
  enum {
3982
    NOT_CAPTURED_IN_MEM = 1 << 0,
3983
    NOT_CAPTURED_IN_INT = 1 << 1,
3984
    NOT_CAPTURED_IN_RET = 1 << 2,
3985
 
3986
    /// If we do not capture the value in memory or through integers we can only
3987
    /// communicate it back as a derived pointer.
3988
    NO_CAPTURE_MAYBE_RETURNED = NOT_CAPTURED_IN_MEM | NOT_CAPTURED_IN_INT,
3989
 
3990
    /// If we do not capture the value in memory, through integers, or as a
3991
    /// derived pointer we know it is not captured.
3992
    NO_CAPTURE =
3993
        NOT_CAPTURED_IN_MEM | NOT_CAPTURED_IN_INT | NOT_CAPTURED_IN_RET,
3994
  };
3995
 
3996
  /// Return true if we know that the underlying value is not captured in its
3997
  /// respective scope.
3998
  bool isKnownNoCapture() const { return isKnown(NO_CAPTURE); }
3999
 
4000
  /// Return true if we assume that the underlying value is not captured in its
4001
  /// respective scope.
4002
  bool isAssumedNoCapture() const { return isAssumed(NO_CAPTURE); }
4003
 
4004
  /// Return true if we know that the underlying value is not captured in its
4005
  /// respective scope but we allow it to escape through a "return".
4006
  bool isKnownNoCaptureMaybeReturned() const {
4007
    return isKnown(NO_CAPTURE_MAYBE_RETURNED);
4008
  }
4009
 
4010
  /// Return true if we assume that the underlying value is not captured in its
4011
  /// respective scope but we allow it to escape through a "return".
4012
  bool isAssumedNoCaptureMaybeReturned() const {
4013
    return isAssumed(NO_CAPTURE_MAYBE_RETURNED);
4014
  }
4015
 
4016
  /// Create an abstract attribute view for the position \p IRP.
4017
  static AANoCapture &createForPosition(const IRPosition &IRP, Attributor &A);
4018
 
4019
  /// See AbstractAttribute::getName()
4020
  const std::string getName() const override { return "AANoCapture"; }
4021
 
4022
  /// See AbstractAttribute::getIdAddr()
4023
  const char *getIdAddr() const override { return &ID; }
4024
 
4025
  /// This function should return true if the type of the \p AA is AANoCapture
4026
  static bool classof(const AbstractAttribute *AA) {
4027
    return (AA->getIdAddr() == &ID);
4028
  }
4029
 
4030
  /// Unique ID (due to the unique address)
4031
  static const char ID;
4032
};
4033
 
4034
struct ValueSimplifyStateType : public AbstractState {
4035
 
4036
  ValueSimplifyStateType(Type *Ty) : Ty(Ty) {}
4037
 
4038
  static ValueSimplifyStateType getBestState(Type *Ty) {
4039
    return ValueSimplifyStateType(Ty);
4040
  }
4041
  static ValueSimplifyStateType getBestState(const ValueSimplifyStateType &VS) {
4042
    return getBestState(VS.Ty);
4043
  }
4044
 
4045
  /// Return the worst possible representable state.
4046
  static ValueSimplifyStateType getWorstState(Type *Ty) {
4047
    ValueSimplifyStateType DS(Ty);
4048
    DS.indicatePessimisticFixpoint();
4049
    return DS;
4050
  }
4051
  static ValueSimplifyStateType
4052
  getWorstState(const ValueSimplifyStateType &VS) {
4053
    return getWorstState(VS.Ty);
4054
  }
4055
 
4056
  /// See AbstractState::isValidState(...)
4057
  bool isValidState() const override { return BS.isValidState(); }
4058
 
4059
  /// See AbstractState::isAtFixpoint(...)
4060
  bool isAtFixpoint() const override { return BS.isAtFixpoint(); }
4061
 
4062
  /// Return the assumed state encoding.
4063
  ValueSimplifyStateType getAssumed() { return *this; }
4064
  const ValueSimplifyStateType &getAssumed() const { return *this; }
4065
 
4066
  /// See AbstractState::indicatePessimisticFixpoint(...)
4067
  ChangeStatus indicatePessimisticFixpoint() override {
4068
    return BS.indicatePessimisticFixpoint();
4069
  }
4070
 
4071
  /// See AbstractState::indicateOptimisticFixpoint(...)
4072
  ChangeStatus indicateOptimisticFixpoint() override {
4073
    return BS.indicateOptimisticFixpoint();
4074
  }
4075
 
4076
  /// "Clamp" this state with \p PVS.
4077
  ValueSimplifyStateType operator^=(const ValueSimplifyStateType &VS) {
4078
    BS ^= VS.BS;
4079
    unionAssumed(VS.SimplifiedAssociatedValue);
4080
    return *this;
4081
  }
4082
 
4083
  bool operator==(const ValueSimplifyStateType &RHS) const {
4084
    if (isValidState() != RHS.isValidState())
4085
      return false;
4086
    if (!isValidState() && !RHS.isValidState())
4087
      return true;
4088
    return SimplifiedAssociatedValue == RHS.SimplifiedAssociatedValue;
4089
  }
4090
 
4091
protected:
4092
  /// The type of the original value.
4093
  Type *Ty;
4094
 
4095
  /// Merge \p Other into the currently assumed simplified value
4096
  bool unionAssumed(std::optional<Value *> Other);
4097
 
4098
  /// Helper to track validity and fixpoint
4099
  BooleanState BS;
4100
 
4101
  /// An assumed simplified value. Initially, it is set to std::nullopt, which
4102
  /// means that the value is not clear under current assumption. If in the
4103
  /// pessimistic state, getAssumedSimplifiedValue doesn't return this value but
4104
  /// returns orignal associated value.
4105
  std::optional<Value *> SimplifiedAssociatedValue;
4106
};
4107
 
4108
/// An abstract interface for value simplify abstract attribute.
4109
struct AAValueSimplify
4110
    : public StateWrapper<ValueSimplifyStateType, AbstractAttribute, Type *> {
4111
  using Base = StateWrapper<ValueSimplifyStateType, AbstractAttribute, Type *>;
4112
  AAValueSimplify(const IRPosition &IRP, Attributor &A)
4113
      : Base(IRP, IRP.getAssociatedType()) {}
4114
 
4115
  /// Create an abstract attribute view for the position \p IRP.
4116
  static AAValueSimplify &createForPosition(const IRPosition &IRP,
4117
                                            Attributor &A);
4118
 
4119
  /// See AbstractAttribute::getName()
4120
  const std::string getName() const override { return "AAValueSimplify"; }
4121
 
4122
  /// See AbstractAttribute::getIdAddr()
4123
  const char *getIdAddr() const override { return &ID; }
4124
 
4125
  /// This function should return true if the type of the \p AA is
4126
  /// AAValueSimplify
4127
  static bool classof(const AbstractAttribute *AA) {
4128
    return (AA->getIdAddr() == &ID);
4129
  }
4130
 
4131
  /// Unique ID (due to the unique address)
4132
  static const char ID;
4133
 
4134
private:
4135
  /// Return an assumed simplified value if a single candidate is found. If
4136
  /// there cannot be one, return original value. If it is not clear yet, return
4137
  /// std::nullopt.
4138
  ///
4139
  /// Use `Attributor::getAssumedSimplified` for value simplification.
4140
  virtual std::optional<Value *>
4141
  getAssumedSimplifiedValue(Attributor &A) const = 0;
4142
 
4143
  friend struct Attributor;
4144
};
4145
 
4146
struct AAHeapToStack : public StateWrapper<BooleanState, AbstractAttribute> {
4147
  using Base = StateWrapper<BooleanState, AbstractAttribute>;
4148
  AAHeapToStack(const IRPosition &IRP, Attributor &A) : Base(IRP) {}
4149
 
4150
  /// Returns true if HeapToStack conversion is assumed to be possible.
4151
  virtual bool isAssumedHeapToStack(const CallBase &CB) const = 0;
4152
 
4153
  /// Returns true if HeapToStack conversion is assumed and the CB is a
4154
  /// callsite to a free operation to be removed.
4155
  virtual bool isAssumedHeapToStackRemovedFree(CallBase &CB) const = 0;
4156
 
4157
  /// Create an abstract attribute view for the position \p IRP.
4158
  static AAHeapToStack &createForPosition(const IRPosition &IRP, Attributor &A);
4159
 
4160
  /// See AbstractAttribute::getName()
4161
  const std::string getName() const override { return "AAHeapToStack"; }
4162
 
4163
  /// See AbstractAttribute::getIdAddr()
4164
  const char *getIdAddr() const override { return &ID; }
4165
 
4166
  /// This function should return true if the type of the \p AA is AAHeapToStack
4167
  static bool classof(const AbstractAttribute *AA) {
4168
    return (AA->getIdAddr() == &ID);
4169
  }
4170
 
4171
  /// Unique ID (due to the unique address)
4172
  static const char ID;
4173
};
4174
 
4175
/// An abstract interface for privatizability.
4176
///
4177
/// A pointer is privatizable if it can be replaced by a new, private one.
4178
/// Privatizing pointer reduces the use count, interaction between unrelated
4179
/// code parts.
4180
///
4181
/// In order for a pointer to be privatizable its value cannot be observed
4182
/// (=nocapture), it is (for now) not written (=readonly & noalias), we know
4183
/// what values are necessary to make the private copy look like the original
4184
/// one, and the values we need can be loaded (=dereferenceable).
4185
struct AAPrivatizablePtr
4186
    : public StateWrapper<BooleanState, AbstractAttribute> {
4187
  using Base = StateWrapper<BooleanState, AbstractAttribute>;
4188
  AAPrivatizablePtr(const IRPosition &IRP, Attributor &A) : Base(IRP) {}
4189
 
4190
  /// Returns true if pointer privatization is assumed to be possible.
4191
  bool isAssumedPrivatizablePtr() const { return getAssumed(); }
4192
 
4193
  /// Returns true if pointer privatization is known to be possible.
4194
  bool isKnownPrivatizablePtr() const { return getKnown(); }
4195
 
4196
  /// Return the type we can choose for a private copy of the underlying
4197
  /// value. std::nullopt means it is not clear yet, nullptr means there is
4198
  /// none.
4199
  virtual std::optional<Type *> getPrivatizableType() const = 0;
4200
 
4201
  /// Create an abstract attribute view for the position \p IRP.
4202
  static AAPrivatizablePtr &createForPosition(const IRPosition &IRP,
4203
                                              Attributor &A);
4204
 
4205
  /// See AbstractAttribute::getName()
4206
  const std::string getName() const override { return "AAPrivatizablePtr"; }
4207
 
4208
  /// See AbstractAttribute::getIdAddr()
4209
  const char *getIdAddr() const override { return &ID; }
4210
 
4211
  /// This function should return true if the type of the \p AA is
4212
  /// AAPricatizablePtr
4213
  static bool classof(const AbstractAttribute *AA) {
4214
    return (AA->getIdAddr() == &ID);
4215
  }
4216
 
4217
  /// Unique ID (due to the unique address)
4218
  static const char ID;
4219
};
4220
 
4221
/// An abstract interface for memory access kind related attributes
4222
/// (readnone/readonly/writeonly).
4223
struct AAMemoryBehavior
4224
    : public IRAttribute<
4225
          Attribute::ReadNone,
4226
          StateWrapper<BitIntegerState<uint8_t, 3>, AbstractAttribute>> {
4227
  AAMemoryBehavior(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
4228
 
4229
  /// State encoding bits. A set bit in the state means the property holds.
4230
  /// BEST_STATE is the best possible state, 0 the worst possible state.
4231
  enum {
4232
    NO_READS = 1 << 0,
4233
    NO_WRITES = 1 << 1,
4234
    NO_ACCESSES = NO_READS | NO_WRITES,
4235
 
4236
    BEST_STATE = NO_ACCESSES,
4237
  };
4238
  static_assert(BEST_STATE == getBestState(), "Unexpected BEST_STATE value");
4239
 
4240
  /// Return true if we know that the underlying value is not read or accessed
4241
  /// in its respective scope.
4242
  bool isKnownReadNone() const { return isKnown(NO_ACCESSES); }
4243
 
4244
  /// Return true if we assume that the underlying value is not read or accessed
4245
  /// in its respective scope.
4246
  bool isAssumedReadNone() const { return isAssumed(NO_ACCESSES); }
4247
 
4248
  /// Return true if we know that the underlying value is not accessed
4249
  /// (=written) in its respective scope.
4250
  bool isKnownReadOnly() const { return isKnown(NO_WRITES); }
4251
 
4252
  /// Return true if we assume that the underlying value is not accessed
4253
  /// (=written) in its respective scope.
4254
  bool isAssumedReadOnly() const { return isAssumed(NO_WRITES); }
4255
 
4256
  /// Return true if we know that the underlying value is not read in its
4257
  /// respective scope.
4258
  bool isKnownWriteOnly() const { return isKnown(NO_READS); }
4259
 
4260
  /// Return true if we assume that the underlying value is not read in its
4261
  /// respective scope.
4262
  bool isAssumedWriteOnly() const { return isAssumed(NO_READS); }
4263
 
4264
  /// Create an abstract attribute view for the position \p IRP.
4265
  static AAMemoryBehavior &createForPosition(const IRPosition &IRP,
4266
                                             Attributor &A);
4267
 
4268
  /// See AbstractAttribute::getName()
4269
  const std::string getName() const override { return "AAMemoryBehavior"; }
4270
 
4271
  /// See AbstractAttribute::getIdAddr()
4272
  const char *getIdAddr() const override { return &ID; }
4273
 
4274
  /// This function should return true if the type of the \p AA is
4275
  /// AAMemoryBehavior
4276
  static bool classof(const AbstractAttribute *AA) {
4277
    return (AA->getIdAddr() == &ID);
4278
  }
4279
 
4280
  /// Unique ID (due to the unique address)
4281
  static const char ID;
4282
};
4283
 
4284
/// An abstract interface for all memory location attributes
4285
/// (readnone/argmemonly/inaccessiblememonly/inaccessibleorargmemonly).
4286
struct AAMemoryLocation
4287
    : public IRAttribute<
4288
          Attribute::ReadNone,
4289
          StateWrapper<BitIntegerState<uint32_t, 511>, AbstractAttribute>> {
4290
  using MemoryLocationsKind = StateType::base_t;
4291
 
4292
  AAMemoryLocation(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
4293
 
4294
  /// Encoding of different locations that could be accessed by a memory
4295
  /// access.
4296
  enum {
4297
    ALL_LOCATIONS = 0,
4298
    NO_LOCAL_MEM = 1 << 0,
4299
    NO_CONST_MEM = 1 << 1,
4300
    NO_GLOBAL_INTERNAL_MEM = 1 << 2,
4301
    NO_GLOBAL_EXTERNAL_MEM = 1 << 3,
4302
    NO_GLOBAL_MEM = NO_GLOBAL_INTERNAL_MEM | NO_GLOBAL_EXTERNAL_MEM,
4303
    NO_ARGUMENT_MEM = 1 << 4,
4304
    NO_INACCESSIBLE_MEM = 1 << 5,
4305
    NO_MALLOCED_MEM = 1 << 6,
4306
    NO_UNKOWN_MEM = 1 << 7,
4307
    NO_LOCATIONS = NO_LOCAL_MEM | NO_CONST_MEM | NO_GLOBAL_INTERNAL_MEM |
4308
                   NO_GLOBAL_EXTERNAL_MEM | NO_ARGUMENT_MEM |
4309
                   NO_INACCESSIBLE_MEM | NO_MALLOCED_MEM | NO_UNKOWN_MEM,
4310
 
4311
    // Helper bit to track if we gave up or not.
4312
    VALID_STATE = NO_LOCATIONS + 1,
4313
 
4314
    BEST_STATE = NO_LOCATIONS | VALID_STATE,
4315
  };
4316
  static_assert(BEST_STATE == getBestState(), "Unexpected BEST_STATE value");
4317
 
4318
  /// Return true if we know that the associated functions has no observable
4319
  /// accesses.
4320
  bool isKnownReadNone() const { return isKnown(NO_LOCATIONS); }
4321
 
4322
  /// Return true if we assume that the associated functions has no observable
4323
  /// accesses.
4324
  bool isAssumedReadNone() const {
4325
    return isAssumed(NO_LOCATIONS) || isAssumedStackOnly();
4326
  }
4327
 
4328
  /// Return true if we know that the associated functions has at most
4329
  /// local/stack accesses.
4330
  bool isKnowStackOnly() const {
4331
    return isKnown(inverseLocation(NO_LOCAL_MEM, true, true));
4332
  }
4333
 
4334
  /// Return true if we assume that the associated functions has at most
4335
  /// local/stack accesses.
4336
  bool isAssumedStackOnly() const {
4337
    return isAssumed(inverseLocation(NO_LOCAL_MEM, true, true));
4338
  }
4339
 
4340
  /// Return true if we know that the underlying value will only access
4341
  /// inaccesible memory only (see Attribute::InaccessibleMemOnly).
4342
  bool isKnownInaccessibleMemOnly() const {
4343
    return isKnown(inverseLocation(NO_INACCESSIBLE_MEM, true, true));
4344
  }
4345
 
4346
  /// Return true if we assume that the underlying value will only access
4347
  /// inaccesible memory only (see Attribute::InaccessibleMemOnly).
4348
  bool isAssumedInaccessibleMemOnly() const {
4349
    return isAssumed(inverseLocation(NO_INACCESSIBLE_MEM, true, true));
4350
  }
4351
 
4352
  /// Return true if we know that the underlying value will only access
4353
  /// argument pointees (see Attribute::ArgMemOnly).
4354
  bool isKnownArgMemOnly() const {
4355
    return isKnown(inverseLocation(NO_ARGUMENT_MEM, true, true));
4356
  }
4357
 
4358
  /// Return true if we assume that the underlying value will only access
4359
  /// argument pointees (see Attribute::ArgMemOnly).
4360
  bool isAssumedArgMemOnly() const {
4361
    return isAssumed(inverseLocation(NO_ARGUMENT_MEM, true, true));
4362
  }
4363
 
4364
  /// Return true if we know that the underlying value will only access
4365
  /// inaccesible memory or argument pointees (see
4366
  /// Attribute::InaccessibleOrArgMemOnly).
4367
  bool isKnownInaccessibleOrArgMemOnly() const {
4368
    return isKnown(
4369
        inverseLocation(NO_INACCESSIBLE_MEM | NO_ARGUMENT_MEM, true, true));
4370
  }
4371
 
4372
  /// Return true if we assume that the underlying value will only access
4373
  /// inaccesible memory or argument pointees (see
4374
  /// Attribute::InaccessibleOrArgMemOnly).
4375
  bool isAssumedInaccessibleOrArgMemOnly() const {
4376
    return isAssumed(
4377
        inverseLocation(NO_INACCESSIBLE_MEM | NO_ARGUMENT_MEM, true, true));
4378
  }
4379
 
4380
  /// Return true if the underlying value may access memory through arguement
4381
  /// pointers of the associated function, if any.
4382
  bool mayAccessArgMem() const { return !isAssumed(NO_ARGUMENT_MEM); }
4383
 
4384
  /// Return true if only the memory locations specififed by \p MLK are assumed
4385
  /// to be accessed by the associated function.
4386
  bool isAssumedSpecifiedMemOnly(MemoryLocationsKind MLK) const {
4387
    return isAssumed(MLK);
4388
  }
4389
 
4390
  /// Return the locations that are assumed to be not accessed by the associated
4391
  /// function, if any.
4392
  MemoryLocationsKind getAssumedNotAccessedLocation() const {
4393
    return getAssumed();
4394
  }
4395
 
4396
  /// Return the inverse of location \p Loc, thus for NO_XXX the return
4397
  /// describes ONLY_XXX. The flags \p AndLocalMem and \p AndConstMem determine
4398
  /// if local (=stack) and constant memory are allowed as well. Most of the
4399
  /// time we do want them to be included, e.g., argmemonly allows accesses via
4400
  /// argument pointers or local or constant memory accesses.
4401
  static MemoryLocationsKind
4402
  inverseLocation(MemoryLocationsKind Loc, bool AndLocalMem, bool AndConstMem) {
4403
    return NO_LOCATIONS & ~(Loc | (AndLocalMem ? NO_LOCAL_MEM : 0) |
4404
                            (AndConstMem ? NO_CONST_MEM : 0));
4405
  };
4406
 
4407
  /// Return the locations encoded by \p MLK as a readable string.
4408
  static std::string getMemoryLocationsAsStr(MemoryLocationsKind MLK);
4409
 
4410
  /// Simple enum to distinguish read/write/read-write accesses.
4411
  enum AccessKind {
4412
    NONE = 0,
4413
    READ = 1 << 0,
4414
    WRITE = 1 << 1,
4415
    READ_WRITE = READ | WRITE,
4416
  };
4417
 
4418
  /// Check \p Pred on all accesses to the memory kinds specified by \p MLK.
4419
  ///
4420
  /// This method will evaluate \p Pred on all accesses (access instruction +
4421
  /// underlying accessed memory pointer) and it will return true if \p Pred
4422
  /// holds every time.
4423
  virtual bool checkForAllAccessesToMemoryKind(
4424
      function_ref<bool(const Instruction *, const Value *, AccessKind,
4425
                        MemoryLocationsKind)>
4426
          Pred,
4427
      MemoryLocationsKind MLK) const = 0;
4428
 
4429
  /// Create an abstract attribute view for the position \p IRP.
4430
  static AAMemoryLocation &createForPosition(const IRPosition &IRP,
4431
                                             Attributor &A);
4432
 
4433
  /// See AbstractState::getAsStr().
4434
  const std::string getAsStr() const override {
4435
    return getMemoryLocationsAsStr(getAssumedNotAccessedLocation());
4436
  }
4437
 
4438
  /// See AbstractAttribute::getName()
4439
  const std::string getName() const override { return "AAMemoryLocation"; }
4440
 
4441
  /// See AbstractAttribute::getIdAddr()
4442
  const char *getIdAddr() const override { return &ID; }
4443
 
4444
  /// This function should return true if the type of the \p AA is
4445
  /// AAMemoryLocation
4446
  static bool classof(const AbstractAttribute *AA) {
4447
    return (AA->getIdAddr() == &ID);
4448
  }
4449
 
4450
  /// Unique ID (due to the unique address)
4451
  static const char ID;
4452
};
4453
 
4454
/// An abstract interface for range value analysis.
4455
struct AAValueConstantRange
4456
    : public StateWrapper<IntegerRangeState, AbstractAttribute, uint32_t> {
4457
  using Base = StateWrapper<IntegerRangeState, AbstractAttribute, uint32_t>;
4458
  AAValueConstantRange(const IRPosition &IRP, Attributor &A)
4459
      : Base(IRP, IRP.getAssociatedType()->getIntegerBitWidth()) {}
4460
 
4461
  /// See AbstractAttribute::getState(...).
4462
  IntegerRangeState &getState() override { return *this; }
4463
  const IntegerRangeState &getState() const override { return *this; }
4464
 
4465
  /// Create an abstract attribute view for the position \p IRP.
4466
  static AAValueConstantRange &createForPosition(const IRPosition &IRP,
4467
                                                 Attributor &A);
4468
 
4469
  /// Return an assumed range for the associated value a program point \p CtxI.
4470
  /// If \p I is nullptr, simply return an assumed range.
4471
  virtual ConstantRange
4472
  getAssumedConstantRange(Attributor &A,
4473
                          const Instruction *CtxI = nullptr) const = 0;
4474
 
4475
  /// Return a known range for the associated value at a program point \p CtxI.
4476
  /// If \p I is nullptr, simply return a known range.
4477
  virtual ConstantRange
4478
  getKnownConstantRange(Attributor &A,
4479
                        const Instruction *CtxI = nullptr) const = 0;
4480
 
4481
  /// Return an assumed constant for the associated value a program point \p
4482
  /// CtxI.
4483
  std::optional<Constant *>
4484
  getAssumedConstant(Attributor &A, const Instruction *CtxI = nullptr) const {
4485
    ConstantRange RangeV = getAssumedConstantRange(A, CtxI);
4486
    if (auto *C = RangeV.getSingleElement()) {
4487
      Type *Ty = getAssociatedValue().getType();
4488
      return cast_or_null<Constant>(
4489
          AA::getWithType(*ConstantInt::get(Ty->getContext(), *C), *Ty));
4490
    }
4491
    if (RangeV.isEmptySet())
4492
      return std::nullopt;
4493
    return nullptr;
4494
  }
4495
 
4496
  /// See AbstractAttribute::getName()
4497
  const std::string getName() const override { return "AAValueConstantRange"; }
4498
 
4499
  /// See AbstractAttribute::getIdAddr()
4500
  const char *getIdAddr() const override { return &ID; }
4501
 
4502
  /// This function should return true if the type of the \p AA is
4503
  /// AAValueConstantRange
4504
  static bool classof(const AbstractAttribute *AA) {
4505
    return (AA->getIdAddr() == &ID);
4506
  }
4507
 
4508
  /// Unique ID (due to the unique address)
4509
  static const char ID;
4510
};
4511
 
4512
/// A class for a set state.
4513
/// The assumed boolean state indicates whether the corresponding set is full
4514
/// set or not. If the assumed state is false, this is the worst state. The
4515
/// worst state (invalid state) of set of potential values is when the set
4516
/// contains every possible value (i.e. we cannot in any way limit the value
4517
/// that the target position can take). That never happens naturally, we only
4518
/// force it. As for the conditions under which we force it, see
4519
/// AAPotentialConstantValues.
4520
template <typename MemberTy> struct PotentialValuesState : AbstractState {
4521
  using SetTy = SmallSetVector<MemberTy, 8>;
4522
 
4523
  PotentialValuesState() : IsValidState(true), UndefIsContained(false) {}
4524
 
4525
  PotentialValuesState(bool IsValid)
4526
      : IsValidState(IsValid), UndefIsContained(false) {}
4527
 
4528
  /// See AbstractState::isValidState(...)
4529
  bool isValidState() const override { return IsValidState.isValidState(); }
4530
 
4531
  /// See AbstractState::isAtFixpoint(...)
4532
  bool isAtFixpoint() const override { return IsValidState.isAtFixpoint(); }
4533
 
4534
  /// See AbstractState::indicatePessimisticFixpoint(...)
4535
  ChangeStatus indicatePessimisticFixpoint() override {
4536
    return IsValidState.indicatePessimisticFixpoint();
4537
  }
4538
 
4539
  /// See AbstractState::indicateOptimisticFixpoint(...)
4540
  ChangeStatus indicateOptimisticFixpoint() override {
4541
    return IsValidState.indicateOptimisticFixpoint();
4542
  }
4543
 
4544
  /// Return the assumed state
4545
  PotentialValuesState &getAssumed() { return *this; }
4546
  const PotentialValuesState &getAssumed() const { return *this; }
4547
 
4548
  /// Return this set. We should check whether this set is valid or not by
4549
  /// isValidState() before calling this function.
4550
  const SetTy &getAssumedSet() const {
4551
    assert(isValidState() && "This set shoud not be used when it is invalid!");
4552
    return Set;
4553
  }
4554
 
4555
  /// Returns whether this state contains an undef value or not.
4556
  bool undefIsContained() const {
4557
    assert(isValidState() && "This flag shoud not be used when it is invalid!");
4558
    return UndefIsContained;
4559
  }
4560
 
4561
  bool operator==(const PotentialValuesState &RHS) const {
4562
    if (isValidState() != RHS.isValidState())
4563
      return false;
4564
    if (!isValidState() && !RHS.isValidState())
4565
      return true;
4566
    if (undefIsContained() != RHS.undefIsContained())
4567
      return false;
4568
    return Set == RHS.getAssumedSet();
4569
  }
4570
 
4571
  /// Maximum number of potential values to be tracked.
4572
  /// This is set by -attributor-max-potential-values command line option
4573
  static unsigned MaxPotentialValues;
4574
 
4575
  /// Return empty set as the best state of potential values.
4576
  static PotentialValuesState getBestState() {
4577
    return PotentialValuesState(true);
4578
  }
4579
 
4580
  static PotentialValuesState getBestState(const PotentialValuesState &PVS) {
4581
    return getBestState();
4582
  }
4583
 
4584
  /// Return full set as the worst state of potential values.
4585
  static PotentialValuesState getWorstState() {
4586
    return PotentialValuesState(false);
4587
  }
4588
 
4589
  /// Union assumed set with the passed value.
4590
  void unionAssumed(const MemberTy &C) { insert(C); }
4591
 
4592
  /// Union assumed set with assumed set of the passed state \p PVS.
4593
  void unionAssumed(const PotentialValuesState &PVS) { unionWith(PVS); }
4594
 
4595
  /// Union assumed set with an undef value.
4596
  void unionAssumedWithUndef() { unionWithUndef(); }
4597
 
4598
  /// "Clamp" this state with \p PVS.
4599
  PotentialValuesState operator^=(const PotentialValuesState &PVS) {
4600
    IsValidState ^= PVS.IsValidState;
4601
    unionAssumed(PVS);
4602
    return *this;
4603
  }
4604
 
4605
  PotentialValuesState operator&=(const PotentialValuesState &PVS) {
4606
    IsValidState &= PVS.IsValidState;
4607
    unionAssumed(PVS);
4608
    return *this;
4609
  }
4610
 
4611
  bool contains(const MemberTy &V) const {
4612
    return !isValidState() ? true : Set.contains(V);
4613
  }
4614
 
4615
protected:
4616
  SetTy &getAssumedSet() {
4617
    assert(isValidState() && "This set shoud not be used when it is invalid!");
4618
    return Set;
4619
  }
4620
 
4621
private:
4622
  /// Check the size of this set, and invalidate when the size is no
4623
  /// less than \p MaxPotentialValues threshold.
4624
  void checkAndInvalidate() {
4625
    if (Set.size() >= MaxPotentialValues)
4626
      indicatePessimisticFixpoint();
4627
    else
4628
      reduceUndefValue();
4629
  }
4630
 
4631
  /// If this state contains both undef and not undef, we can reduce
4632
  /// undef to the not undef value.
4633
  void reduceUndefValue() { UndefIsContained = UndefIsContained & Set.empty(); }
4634
 
4635
  /// Insert an element into this set.
4636
  void insert(const MemberTy &C) {
4637
    if (!isValidState())
4638
      return;
4639
    Set.insert(C);
4640
    checkAndInvalidate();
4641
  }
4642
 
4643
  /// Take union with R.
4644
  void unionWith(const PotentialValuesState &R) {
4645
    /// If this is a full set, do nothing.
4646
    if (!isValidState())
4647
      return;
4648
    /// If R is full set, change L to a full set.
4649
    if (!R.isValidState()) {
4650
      indicatePessimisticFixpoint();
4651
      return;
4652
    }
4653
    for (const MemberTy &C : R.Set)
4654
      Set.insert(C);
4655
    UndefIsContained |= R.undefIsContained();
4656
    checkAndInvalidate();
4657
  }
4658
 
4659
  /// Take union with an undef value.
4660
  void unionWithUndef() {
4661
    UndefIsContained = true;
4662
    reduceUndefValue();
4663
  }
4664
 
4665
  /// Take intersection with R.
4666
  void intersectWith(const PotentialValuesState &R) {
4667
    /// If R is a full set, do nothing.
4668
    if (!R.isValidState())
4669
      return;
4670
    /// If this is a full set, change this to R.
4671
    if (!isValidState()) {
4672
      *this = R;
4673
      return;
4674
    }
4675
    SetTy IntersectSet;
4676
    for (const MemberTy &C : Set) {
4677
      if (R.Set.count(C))
4678
        IntersectSet.insert(C);
4679
    }
4680
    Set = IntersectSet;
4681
    UndefIsContained &= R.undefIsContained();
4682
    reduceUndefValue();
4683
  }
4684
 
4685
  /// A helper state which indicate whether this state is valid or not.
4686
  BooleanState IsValidState;
4687
 
4688
  /// Container for potential values
4689
  SetTy Set;
4690
 
4691
  /// Flag for undef value
4692
  bool UndefIsContained;
4693
};
4694
 
4695
using PotentialConstantIntValuesState = PotentialValuesState<APInt>;
4696
using PotentialLLVMValuesState =
4697
    PotentialValuesState<std::pair<AA::ValueAndContext, AA::ValueScope>>;
4698
 
4699
raw_ostream &operator<<(raw_ostream &OS,
4700
                        const PotentialConstantIntValuesState &R);
4701
raw_ostream &operator<<(raw_ostream &OS, const PotentialLLVMValuesState &R);
4702
 
4703
/// An abstract interface for potential values analysis.
4704
///
4705
/// This AA collects potential values for each IR position.
4706
/// An assumed set of potential values is initialized with the empty set (the
4707
/// best state) and it will grow monotonically as we find more potential values
4708
/// for this position.
4709
/// The set might be forced to the worst state, that is, to contain every
4710
/// possible value for this position in 2 cases.
4711
///   1. We surpassed the \p MaxPotentialValues threshold. This includes the
4712
///      case that this position is affected (e.g. because of an operation) by a
4713
///      Value that is in the worst state.
4714
///   2. We tried to initialize on a Value that we cannot handle (e.g. an
4715
///      operator we do not currently handle).
4716
///
4717
/// For non constant integers see AAPotentialValues.
4718
struct AAPotentialConstantValues
4719
    : public StateWrapper<PotentialConstantIntValuesState, AbstractAttribute> {
4720
  using Base = StateWrapper<PotentialConstantIntValuesState, AbstractAttribute>;
4721
  AAPotentialConstantValues(const IRPosition &IRP, Attributor &A) : Base(IRP) {}
4722
 
4723
  /// See AbstractAttribute::getState(...).
4724
  PotentialConstantIntValuesState &getState() override { return *this; }
4725
  const PotentialConstantIntValuesState &getState() const override {
4726
    return *this;
4727
  }
4728
 
4729
  /// Create an abstract attribute view for the position \p IRP.
4730
  static AAPotentialConstantValues &createForPosition(const IRPosition &IRP,
4731
                                                      Attributor &A);
4732
 
4733
  /// Return assumed constant for the associated value
4734
  std::optional<Constant *>
4735
  getAssumedConstant(Attributor &A, const Instruction *CtxI = nullptr) const {
4736
    if (!isValidState())
4737
      return nullptr;
4738
    if (getAssumedSet().size() == 1) {
4739
      Type *Ty = getAssociatedValue().getType();
4740
      return cast_or_null<Constant>(AA::getWithType(
4741
          *ConstantInt::get(Ty->getContext(), *(getAssumedSet().begin())),
4742
          *Ty));
4743
    }
4744
    if (getAssumedSet().size() == 0) {
4745
      if (undefIsContained())
4746
        return UndefValue::get(getAssociatedValue().getType());
4747
      return std::nullopt;
4748
    }
4749
 
4750
    return nullptr;
4751
  }
4752
 
4753
  /// See AbstractAttribute::getName()
4754
  const std::string getName() const override {
4755
    return "AAPotentialConstantValues";
4756
  }
4757
 
4758
  /// See AbstractAttribute::getIdAddr()
4759
  const char *getIdAddr() const override { return &ID; }
4760
 
4761
  /// This function should return true if the type of the \p AA is
4762
  /// AAPotentialConstantValues
4763
  static bool classof(const AbstractAttribute *AA) {
4764
    return (AA->getIdAddr() == &ID);
4765
  }
4766
 
4767
  /// Unique ID (due to the unique address)
4768
  static const char ID;
4769
};
4770
 
4771
struct AAPotentialValues
4772
    : public StateWrapper<PotentialLLVMValuesState, AbstractAttribute> {
4773
  using Base = StateWrapper<PotentialLLVMValuesState, AbstractAttribute>;
4774
  AAPotentialValues(const IRPosition &IRP, Attributor &A) : Base(IRP) {}
4775
 
4776
  /// See AbstractAttribute::getState(...).
4777
  PotentialLLVMValuesState &getState() override { return *this; }
4778
  const PotentialLLVMValuesState &getState() const override { return *this; }
4779
 
4780
  /// Create an abstract attribute view for the position \p IRP.
4781
  static AAPotentialValues &createForPosition(const IRPosition &IRP,
4782
                                              Attributor &A);
4783
 
4784
  /// Extract the single value in \p Values if any.
4785
  static Value *getSingleValue(Attributor &A, const AbstractAttribute &AA,
4786
                               const IRPosition &IRP,
4787
                               SmallVectorImpl<AA::ValueAndContext> &Values);
4788
 
4789
  /// See AbstractAttribute::getName()
4790
  const std::string getName() const override { return "AAPotentialValues"; }
4791
 
4792
  /// See AbstractAttribute::getIdAddr()
4793
  const char *getIdAddr() const override { return &ID; }
4794
 
4795
  /// This function should return true if the type of the \p AA is
4796
  /// AAPotentialValues
4797
  static bool classof(const AbstractAttribute *AA) {
4798
    return (AA->getIdAddr() == &ID);
4799
  }
4800
 
4801
  /// Unique ID (due to the unique address)
4802
  static const char ID;
4803
 
4804
private:
4805
  virtual bool
4806
  getAssumedSimplifiedValues(Attributor &A,
4807
                             SmallVectorImpl<AA::ValueAndContext> &Values,
4808
                             AA::ValueScope) const = 0;
4809
 
4810
  friend struct Attributor;
4811
};
4812
 
4813
/// An abstract interface for all noundef attributes.
4814
struct AANoUndef
4815
    : public IRAttribute<Attribute::NoUndef,
4816
                         StateWrapper<BooleanState, AbstractAttribute>> {
4817
  AANoUndef(const IRPosition &IRP, Attributor &A) : IRAttribute(IRP) {}
4818
 
4819
  /// Return true if we assume that the underlying value is noundef.
4820
  bool isAssumedNoUndef() const { return getAssumed(); }
4821
 
4822
  /// Return true if we know that underlying value is noundef.
4823
  bool isKnownNoUndef() const { return getKnown(); }
4824
 
4825
  /// Create an abstract attribute view for the position \p IRP.
4826
  static AANoUndef &createForPosition(const IRPosition &IRP, Attributor &A);
4827
 
4828
  /// See AbstractAttribute::getName()
4829
  const std::string getName() const override { return "AANoUndef"; }
4830
 
4831
  /// See AbstractAttribute::getIdAddr()
4832
  const char *getIdAddr() const override { return &ID; }
4833
 
4834
  /// This function should return true if the type of the \p AA is AANoUndef
4835
  static bool classof(const AbstractAttribute *AA) {
4836
    return (AA->getIdAddr() == &ID);
4837
  }
4838
 
4839
  /// Unique ID (due to the unique address)
4840
  static const char ID;
4841
};
4842
 
4843
struct AACallGraphNode;
4844
struct AACallEdges;
4845
 
4846
/// An Iterator for call edges, creates AACallEdges attributes in a lazy way.
4847
/// This iterator becomes invalid if the underlying edge list changes.
4848
/// So This shouldn't outlive a iteration of Attributor.
4849
class AACallEdgeIterator
4850
    : public iterator_adaptor_base<AACallEdgeIterator,
4851
                                   SetVector<Function *>::iterator> {
4852
  AACallEdgeIterator(Attributor &A, SetVector<Function *>::iterator Begin)
4853
      : iterator_adaptor_base(Begin), A(A) {}
4854
 
4855
public:
4856
  AACallGraphNode *operator*() const;
4857
 
4858
private:
4859
  Attributor &A;
4860
  friend AACallEdges;
4861
  friend AttributorCallGraph;
4862
};
4863
 
4864
struct AACallGraphNode {
4865
  AACallGraphNode(Attributor &A) : A(A) {}
4866
  virtual ~AACallGraphNode() = default;
4867
 
4868
  virtual AACallEdgeIterator optimisticEdgesBegin() const = 0;
4869
  virtual AACallEdgeIterator optimisticEdgesEnd() const = 0;
4870
 
4871
  /// Iterator range for exploring the call graph.
4872
  iterator_range<AACallEdgeIterator> optimisticEdgesRange() const {
4873
    return iterator_range<AACallEdgeIterator>(optimisticEdgesBegin(),
4874
                                              optimisticEdgesEnd());
4875
  }
4876
 
4877
protected:
4878
  /// Reference to Attributor needed for GraphTraits implementation.
4879
  Attributor &A;
4880
};
4881
 
4882
/// An abstract state for querying live call edges.
4883
/// This interface uses the Attributor's optimistic liveness
4884
/// information to compute the edges that are alive.
4885
struct AACallEdges : public StateWrapper<BooleanState, AbstractAttribute>,
4886
                     AACallGraphNode {
4887
  using Base = StateWrapper<BooleanState, AbstractAttribute>;
4888
 
4889
  AACallEdges(const IRPosition &IRP, Attributor &A)
4890
      : Base(IRP), AACallGraphNode(A) {}
4891
 
4892
  /// Get the optimistic edges.
4893
  virtual const SetVector<Function *> &getOptimisticEdges() const = 0;
4894
 
4895
  /// Is there any call with a unknown callee.
4896
  virtual bool hasUnknownCallee() const = 0;
4897
 
4898
  /// Is there any call with a unknown callee, excluding any inline asm.
4899
  virtual bool hasNonAsmUnknownCallee() const = 0;
4900
 
4901
  /// Iterator for exploring the call graph.
4902
  AACallEdgeIterator optimisticEdgesBegin() const override {
4903
    return AACallEdgeIterator(A, getOptimisticEdges().begin());
4904
  }
4905
 
4906
  /// Iterator for exploring the call graph.
4907
  AACallEdgeIterator optimisticEdgesEnd() const override {
4908
    return AACallEdgeIterator(A, getOptimisticEdges().end());
4909
  }
4910
 
4911
  /// Create an abstract attribute view for the position \p IRP.
4912
  static AACallEdges &createForPosition(const IRPosition &IRP, Attributor &A);
4913
 
4914
  /// See AbstractAttribute::getName()
4915
  const std::string getName() const override { return "AACallEdges"; }
4916
 
4917
  /// See AbstractAttribute::getIdAddr()
4918
  const char *getIdAddr() const override { return &ID; }
4919
 
4920
  /// This function should return true if the type of the \p AA is AACallEdges.
4921
  static bool classof(const AbstractAttribute *AA) {
4922
    return (AA->getIdAddr() == &ID);
4923
  }
4924
 
4925
  /// Unique ID (due to the unique address)
4926
  static const char ID;
4927
};
4928
 
4929
// Synthetic root node for the Attributor's internal call graph.
4930
struct AttributorCallGraph : public AACallGraphNode {
4931
  AttributorCallGraph(Attributor &A) : AACallGraphNode(A) {}
4932
  virtual ~AttributorCallGraph() = default;
4933
 
4934
  AACallEdgeIterator optimisticEdgesBegin() const override {
4935
    return AACallEdgeIterator(A, A.Functions.begin());
4936
  }
4937
 
4938
  AACallEdgeIterator optimisticEdgesEnd() const override {
4939
    return AACallEdgeIterator(A, A.Functions.end());
4940
  }
4941
 
4942
  /// Force populate the entire call graph.
4943
  void populateAll() const {
4944
    for (const AACallGraphNode *AA : optimisticEdgesRange()) {
4945
      // Nothing else to do here.
4946
      (void)AA;
4947
    }
4948
  }
4949
 
4950
  void print();
4951
};
4952
 
4953
template <> struct GraphTraits<AACallGraphNode *> {
4954
  using NodeRef = AACallGraphNode *;
4955
  using ChildIteratorType = AACallEdgeIterator;
4956
 
4957
  static AACallEdgeIterator child_begin(AACallGraphNode *Node) {
4958
    return Node->optimisticEdgesBegin();
4959
  }
4960
 
4961
  static AACallEdgeIterator child_end(AACallGraphNode *Node) {
4962
    return Node->optimisticEdgesEnd();
4963
  }
4964
};
4965
 
4966
template <>
4967
struct GraphTraits<AttributorCallGraph *>
4968
    : public GraphTraits<AACallGraphNode *> {
4969
  using nodes_iterator = AACallEdgeIterator;
4970
 
4971
  static AACallGraphNode *getEntryNode(AttributorCallGraph *G) {
4972
    return static_cast<AACallGraphNode *>(G);
4973
  }
4974
 
4975
  static AACallEdgeIterator nodes_begin(const AttributorCallGraph *G) {
4976
    return G->optimisticEdgesBegin();
4977
  }
4978
 
4979
  static AACallEdgeIterator nodes_end(const AttributorCallGraph *G) {
4980
    return G->optimisticEdgesEnd();
4981
  }
4982
};
4983
 
4984
template <>
4985
struct DOTGraphTraits<AttributorCallGraph *> : public DefaultDOTGraphTraits {
4986
  DOTGraphTraits(bool Simple = false) : DefaultDOTGraphTraits(Simple) {}
4987
 
4988
  std::string getNodeLabel(const AACallGraphNode *Node,
4989
                           const AttributorCallGraph *Graph) {
4990
    const AACallEdges *AACE = static_cast<const AACallEdges *>(Node);
4991
    return AACE->getAssociatedFunction()->getName().str();
4992
  }
4993
 
4994
  static bool isNodeHidden(const AACallGraphNode *Node,
4995
                           const AttributorCallGraph *Graph) {
4996
    // Hide the synth root.
4997
    return static_cast<const AACallGraphNode *>(Graph) == Node;
4998
  }
4999
};
5000
 
5001
struct AAExecutionDomain
5002
    : public StateWrapper<BooleanState, AbstractAttribute> {
5003
  using Base = StateWrapper<BooleanState, AbstractAttribute>;
5004
  AAExecutionDomain(const IRPosition &IRP, Attributor &A) : Base(IRP) {}
5005
 
5006
  /// Summary about the execution domain of a block or instruction.
5007
  struct ExecutionDomainTy {
5008
    using BarriersSetTy = SmallPtrSet<CallBase *, 2>;
5009
    using AssumesSetTy = SmallPtrSet<AssumeInst *, 4>;
5010
 
5011
    void addAssumeInst(Attributor &A, AssumeInst &AI) {
5012
      EncounteredAssumes.insert(&AI);
5013
    }
5014
 
5015
    void addAlignedBarrier(Attributor &A, CallBase &CB) {
5016
      AlignedBarriers.insert(&CB);
5017
    }
5018
 
5019
    void clearAssumeInstAndAlignedBarriers() {
5020
      EncounteredAssumes.clear();
5021
      AlignedBarriers.clear();
5022
    }
5023
 
5024
    bool IsExecutedByInitialThreadOnly = true;
5025
    bool IsReachedFromAlignedBarrierOnly = true;
5026
    bool IsReachingAlignedBarrierOnly = true;
5027
    bool EncounteredNonLocalSideEffect = false;
5028
    BarriersSetTy AlignedBarriers;
5029
    AssumesSetTy EncounteredAssumes;
5030
  };
5031
 
5032
  /// Create an abstract attribute view for the position \p IRP.
5033
  static AAExecutionDomain &createForPosition(const IRPosition &IRP,
5034
                                              Attributor &A);
5035
 
5036
  /// See AbstractAttribute::getName().
5037
  const std::string getName() const override { return "AAExecutionDomain"; }
5038
 
5039
  /// See AbstractAttribute::getIdAddr().
5040
  const char *getIdAddr() const override { return &ID; }
5041
 
5042
  /// Check if an instruction is executed only by the initial thread.
5043
  bool isExecutedByInitialThreadOnly(const Instruction &I) const {
5044
    return isExecutedByInitialThreadOnly(*I.getParent());
5045
  }
5046
 
5047
  /// Check if a basic block is executed only by the initial thread.
5048
  virtual bool isExecutedByInitialThreadOnly(const BasicBlock &) const = 0;
5049
 
5050
  /// Check if the instruction \p I is executed in an aligned region, that is,
5051
  /// the synchronizing effects before and after \p I are both aligned barriers.
5052
  /// This effectively means all threads execute \p I together.
5053
  virtual bool isExecutedInAlignedRegion(Attributor &A,
5054
                                         const Instruction &I) const = 0;
5055
 
5056
  virtual ExecutionDomainTy getExecutionDomain(const BasicBlock &) const = 0;
5057
  virtual ExecutionDomainTy getExecutionDomain(const CallBase &) const = 0;
5058
  virtual ExecutionDomainTy getFunctionExecutionDomain() const = 0;
5059
 
5060
  /// This function should return true if the type of the \p AA is
5061
  /// AAExecutionDomain.
5062
  static bool classof(const AbstractAttribute *AA) {
5063
    return (AA->getIdAddr() == &ID);
5064
  }
5065
 
5066
  /// Unique ID (due to the unique address)
5067
  static const char ID;
5068
};
5069
 
5070
/// An abstract Attribute for computing reachability between functions.
5071
struct AAInterFnReachability
5072
    : public StateWrapper<BooleanState, AbstractAttribute> {
5073
  using Base = StateWrapper<BooleanState, AbstractAttribute>;
5074
 
5075
  AAInterFnReachability(const IRPosition &IRP, Attributor &A) : Base(IRP) {}
5076
 
5077
  /// If the function represented by this possition can reach \p Fn.
5078
  bool canReach(Attributor &A, const Function &Fn) const {
5079
    Function *Scope = getAnchorScope();
5080
    if (!Scope || Scope->isDeclaration())
5081
      return true;
5082
    return instructionCanReach(A, Scope->getEntryBlock().front(), Fn);
5083
  }
5084
 
5085
  /// Can  \p Inst reach \p Fn.
5086
  /// See also AA::isPotentiallyReachable.
5087
  virtual bool instructionCanReach(
5088
      Attributor &A, const Instruction &Inst, const Function &Fn,
5089
      const AA::InstExclusionSetTy *ExclusionSet = nullptr,
5090
      SmallPtrSet<const Function *, 16> *Visited = nullptr) const = 0;
5091
 
5092
  /// Create an abstract attribute view for the position \p IRP.
5093
  static AAInterFnReachability &createForPosition(const IRPosition &IRP,
5094
                                                  Attributor &A);
5095
 
5096
  /// See AbstractAttribute::getName()
5097
  const std::string getName() const override { return "AAInterFnReachability"; }
5098
 
5099
  /// See AbstractAttribute::getIdAddr()
5100
  const char *getIdAddr() const override { return &ID; }
5101
 
5102
  /// This function should return true if the type of the \p AA is AACallEdges.
5103
  static bool classof(const AbstractAttribute *AA) {
5104
    return (AA->getIdAddr() == &ID);
5105
  }
5106
 
5107
  /// Unique ID (due to the unique address)
5108
  static const char ID;
5109
};
5110
 
5111
/// An abstract interface for struct information.
5112
struct AAPointerInfo : public AbstractAttribute {
5113
  AAPointerInfo(const IRPosition &IRP) : AbstractAttribute(IRP) {}
5114
 
5115
  enum AccessKind {
5116
    // First two bits to distinguish may and must accesses.
5117
    AK_MUST = 1 << 0,
5118
    AK_MAY = 1 << 1,
5119
 
5120
    // Then two bits for read and write. These are not exclusive.
5121
    AK_R = 1 << 2,
5122
    AK_W = 1 << 3,
5123
    AK_RW = AK_R | AK_W,
5124
 
5125
    // One special case for assumptions about memory content. These
5126
    // are neither reads nor writes. They are however always modeled
5127
    // as read to avoid using them for write removal.
5128
    AK_ASSUMPTION = (1 << 4) | AK_MUST,
5129
 
5130
    // Helper for easy access.
5131
    AK_MAY_READ = AK_MAY | AK_R,
5132
    AK_MAY_WRITE = AK_MAY | AK_W,
5133
    AK_MAY_READ_WRITE = AK_MAY | AK_R | AK_W,
5134
    AK_MUST_READ = AK_MUST | AK_R,
5135
    AK_MUST_WRITE = AK_MUST | AK_W,
5136
    AK_MUST_READ_WRITE = AK_MUST | AK_R | AK_W,
5137
  };
5138
 
5139
  /// A container for a list of ranges.
5140
  struct RangeList {
5141
    // The set of ranges rarely contains more than one element, and is unlikely
5142
    // to contain more than say four elements. So we find the middle-ground with
5143
    // a sorted vector. This avoids hard-coding a rarely used number like "four"
5144
    // into every instance of a SmallSet.
5145
    using RangeTy = AA::RangeTy;
5146
    using VecTy = SmallVector<RangeTy>;
5147
    using iterator = VecTy::iterator;
5148
    using const_iterator = VecTy::const_iterator;
5149
    VecTy Ranges;
5150
 
5151
    RangeList(const RangeTy &R) { Ranges.push_back(R); }
5152
    RangeList(ArrayRef<int64_t> Offsets, int64_t Size) {
5153
      Ranges.reserve(Offsets.size());
5154
      for (unsigned i = 0, e = Offsets.size(); i != e; ++i) {
5155
        assert(((i + 1 == e) || Offsets[i] < Offsets[i + 1]) &&
5156
               "Expected strictly ascending offsets.");
5157
        Ranges.emplace_back(Offsets[i], Size);
5158
      }
5159
    }
5160
    RangeList() = default;
5161
 
5162
    iterator begin() { return Ranges.begin(); }
5163
    iterator end() { return Ranges.end(); }
5164
    const_iterator begin() const { return Ranges.begin(); }
5165
    const_iterator end() const { return Ranges.end(); }
5166
 
5167
    // Helpers required for std::set_difference
5168
    using value_type = RangeTy;
5169
    void push_back(const RangeTy &R) {
5170
      assert((Ranges.empty() || RangeTy::OffsetLessThan(Ranges.back(), R)) &&
5171
             "Ensure the last element is the greatest.");
5172
      Ranges.push_back(R);
5173
    }
5174
 
5175
    /// Copy ranges from \p L that are not in \p R, into \p D.
5176
    static void set_difference(const RangeList &L, const RangeList &R,
5177
                               RangeList &D) {
5178
      std::set_difference(L.begin(), L.end(), R.begin(), R.end(),
5179
                          std::back_inserter(D), RangeTy::OffsetLessThan);
5180
    }
5181
 
5182
    unsigned size() const { return Ranges.size(); }
5183
 
5184
    bool operator==(const RangeList &OI) const { return Ranges == OI.Ranges; }
5185
 
5186
    /// Merge the ranges in \p RHS into the current ranges.
5187
    /// - Merging a list of  unknown ranges makes the current list unknown.
5188
    /// - Ranges with the same offset are merged according to RangeTy::operator&
5189
    /// \return true if the current RangeList changed.
5190
    bool merge(const RangeList &RHS) {
5191
      if (isUnknown())
5192
        return false;
5193
      if (RHS.isUnknown()) {
5194
        setUnknown();
5195
        return true;
5196
      }
5197
 
5198
      if (Ranges.empty()) {
5199
        Ranges = RHS.Ranges;
5200
        return true;
5201
      }
5202
 
5203
      bool Changed = false;
5204
      auto LPos = Ranges.begin();
5205
      for (auto &R : RHS.Ranges) {
5206
        auto Result = insert(LPos, R);
5207
        if (isUnknown())
5208
          return true;
5209
        LPos = Result.first;
5210
        Changed |= Result.second;
5211
      }
5212
      return Changed;
5213
    }
5214
 
5215
    /// Insert \p R at the given iterator \p Pos, and merge if necessary.
5216
    ///
5217
    /// This assumes that all ranges before \p Pos are OffsetLessThan \p R, and
5218
    /// then maintains the sorted order for the suffix list.
5219
    ///
5220
    /// \return The place of insertion and true iff anything changed.
5221
    std::pair<iterator, bool> insert(iterator Pos, const RangeTy &R) {
5222
      if (isUnknown())
5223
        return std::make_pair(Ranges.begin(), false);
5224
      if (R.offsetOrSizeAreUnknown()) {
5225
        return std::make_pair(setUnknown(), true);
5226
      }
5227
 
5228
      // Maintain this as a sorted vector of unique entries.
5229
      auto LB = std::lower_bound(Pos, Ranges.end(), R, RangeTy::OffsetLessThan);
5230
      if (LB == Ranges.end() || LB->Offset != R.Offset)
5231
        return std::make_pair(Ranges.insert(LB, R), true);
5232
      bool Changed = *LB != R;
5233
      *LB &= R;
5234
      if (LB->offsetOrSizeAreUnknown())
5235
        return std::make_pair(setUnknown(), true);
5236
      return std::make_pair(LB, Changed);
5237
    }
5238
 
5239
    /// Insert the given range \p R, maintaining sorted order.
5240
    ///
5241
    /// \return The place of insertion and true iff anything changed.
5242
    std::pair<iterator, bool> insert(const RangeTy &R) {
5243
      return insert(Ranges.begin(), R);
5244
    }
5245
 
5246
    /// Add the increment \p Inc to the offset of every range.
5247
    void addToAllOffsets(int64_t Inc) {
5248
      assert(!isUnassigned() &&
5249
             "Cannot increment if the offset is not yet computed!");
5250
      if (isUnknown())
5251
        return;
5252
      for (auto &R : Ranges) {
5253
        R.Offset += Inc;
5254
      }
5255
    }
5256
 
5257
    /// Return true iff there is exactly one range and it is known.
5258
    bool isUnique() const {
5259
      return Ranges.size() == 1 && !Ranges.front().offsetOrSizeAreUnknown();
5260
    }
5261
 
5262
    /// Return the unique range, assuming it exists.
5263
    const RangeTy &getUnique() const {
5264
      assert(isUnique() && "No unique range to return!");
5265
      return Ranges.front();
5266
    }
5267
 
5268
    /// Return true iff the list contains an unknown range.
5269
    bool isUnknown() const {
5270
      if (isUnassigned())
5271
        return false;
5272
      if (Ranges.front().offsetOrSizeAreUnknown()) {
5273
        assert(Ranges.size() == 1 && "Unknown is a singleton range.");
5274
        return true;
5275
      }
5276
      return false;
5277
    }
5278
 
5279
    /// Discard all ranges and insert a single unknown range.
5280
    iterator setUnknown() {
5281
      Ranges.clear();
5282
      Ranges.push_back(RangeTy::getUnknown());
5283
      return Ranges.begin();
5284
    }
5285
 
5286
    /// Return true if no ranges have been inserted.
5287
    bool isUnassigned() const { return Ranges.size() == 0; }
5288
  };
5289
 
5290
  /// An access description.
5291
  struct Access {
5292
    Access(Instruction *I, int64_t Offset, int64_t Size,
5293
           std::optional<Value *> Content, AccessKind Kind, Type *Ty)
5294
        : LocalI(I), RemoteI(I), Content(Content), Ranges(Offset, Size),
5295
          Kind(Kind), Ty(Ty) {
5296
      verify();
5297
    }
5298
    Access(Instruction *LocalI, Instruction *RemoteI, const RangeList &Ranges,
5299
           std::optional<Value *> Content, AccessKind K, Type *Ty)
5300
        : LocalI(LocalI), RemoteI(RemoteI), Content(Content), Ranges(Ranges),
5301
          Kind(K), Ty(Ty) {
5302
      if (Ranges.size() > 1) {
5303
        Kind = AccessKind(Kind | AK_MAY);
5304
        Kind = AccessKind(Kind & ~AK_MUST);
5305
      }
5306
      verify();
5307
    }
5308
    Access(Instruction *LocalI, Instruction *RemoteI, int64_t Offset,
5309
           int64_t Size, std::optional<Value *> Content, AccessKind Kind,
5310
           Type *Ty)
5311
        : LocalI(LocalI), RemoteI(RemoteI), Content(Content),
5312
          Ranges(Offset, Size), Kind(Kind), Ty(Ty) {
5313
      verify();
5314
    }
5315
    Access(const Access &Other) = default;
5316
 
5317
    Access &operator=(const Access &Other) = default;
5318
    bool operator==(const Access &R) const {
5319
      return LocalI == R.LocalI && RemoteI == R.RemoteI && Ranges == R.Ranges &&
5320
             Content == R.Content && Kind == R.Kind;
5321
    }
5322
    bool operator!=(const Access &R) const { return !(*this == R); }
5323
 
5324
    Access &operator&=(const Access &R) {
5325
      assert(RemoteI == R.RemoteI && "Expected same instruction!");
5326
      assert(LocalI == R.LocalI && "Expected same instruction!");
5327
 
5328
      // Note that every Access object corresponds to a unique Value, and only
5329
      // accesses to the same Value are merged. Hence we assume that all ranges
5330
      // are the same size. If ranges can be different size, then the contents
5331
      // must be dropped.
5332
      Ranges.merge(R.Ranges);
5333
      Content =
5334
          AA::combineOptionalValuesInAAValueLatice(Content, R.Content, Ty);
5335
 
5336
      // Combine the access kind, which results in a bitwise union.
5337
      // If there is more than one range, then this must be a MAY.
5338
      // If we combine a may and a must access we clear the must bit.
5339
      Kind = AccessKind(Kind | R.Kind);
5340
      if ((Kind & AK_MAY) || Ranges.size() > 1) {
5341
        Kind = AccessKind(Kind | AK_MAY);
5342
        Kind = AccessKind(Kind & ~AK_MUST);
5343
      }
5344
      verify();
5345
      return *this;
5346
    }
5347
 
5348
    void verify() {
5349
      assert(isMustAccess() + isMayAccess() == 1 &&
5350
             "Expect must or may access, not both.");
5351
      assert(isAssumption() + isWrite() <= 1 &&
5352
             "Expect assumption access or write access, never both.");
5353
      assert((isMayAccess() || Ranges.size() == 1) &&
5354
             "Cannot be a must access if there are multiple ranges.");
5355
    }
5356
 
5357
    /// Return the access kind.
5358
    AccessKind getKind() const { return Kind; }
5359
 
5360
    /// Return true if this is a read access.
5361
    bool isRead() const { return Kind & AK_R; }
5362
 
5363
    /// Return true if this is a write access.
5364
    bool isWrite() const { return Kind & AK_W; }
5365
 
5366
    /// Return true if this is a write access.
5367
    bool isWriteOrAssumption() const { return isWrite() || isAssumption(); }
5368
 
5369
    /// Return true if this is an assumption access.
5370
    bool isAssumption() const { return Kind == AK_ASSUMPTION; }
5371
 
5372
    bool isMustAccess() const {
5373
      bool MustAccess = Kind & AK_MUST;
5374
      assert((!MustAccess || Ranges.size() < 2) &&
5375
             "Cannot be a must access if there are multiple ranges.");
5376
      return MustAccess;
5377
    }
5378
 
5379
    bool isMayAccess() const {
5380
      bool MayAccess = Kind & AK_MAY;
5381
      assert((MayAccess || Ranges.size() < 2) &&
5382
             "Cannot be a must access if there are multiple ranges.");
5383
      return MayAccess;
5384
    }
5385
 
5386
    /// Return the instruction that causes the access with respect to the local
5387
    /// scope of the associated attribute.
5388
    Instruction *getLocalInst() const { return LocalI; }
5389
 
5390
    /// Return the actual instruction that causes the access.
5391
    Instruction *getRemoteInst() const { return RemoteI; }
5392
 
5393
    /// Return true if the value written is not known yet.
5394
    bool isWrittenValueYetUndetermined() const { return !Content; }
5395
 
5396
    /// Return true if the value written cannot be determined at all.
5397
    bool isWrittenValueUnknown() const {
5398
      return Content.has_value() && !*Content;
5399
    }
5400
 
5401
    /// Set the value written to nullptr, i.e., unknown.
5402
    void setWrittenValueUnknown() { Content = nullptr; }
5403
 
5404
    /// Return the type associated with the access, if known.
5405
    Type *getType() const { return Ty; }
5406
 
5407
    /// Return the value writen, if any.
5408
    Value *getWrittenValue() const {
5409
      assert(!isWrittenValueYetUndetermined() &&
5410
             "Value needs to be determined before accessing it.");
5411
      return *Content;
5412
    }
5413
 
5414
    /// Return the written value which can be `llvm::null` if it is not yet
5415
    /// determined.
5416
    std::optional<Value *> getContent() const { return Content; }
5417
 
5418
    bool hasUniqueRange() const { return Ranges.isUnique(); }
5419
    const AA::RangeTy &getUniqueRange() const { return Ranges.getUnique(); }
5420
 
5421
    /// Add a range accessed by this Access.
5422
    ///
5423
    /// If there are multiple ranges, then this is a "may access".
5424
    void addRange(int64_t Offset, int64_t Size) {
5425
      Ranges.insert({Offset, Size});
5426
      if (!hasUniqueRange()) {
5427
        Kind = AccessKind(Kind | AK_MAY);
5428
        Kind = AccessKind(Kind & ~AK_MUST);
5429
      }
5430
    }
5431
 
5432
    const RangeList &getRanges() const { return Ranges; }
5433
 
5434
    using const_iterator = RangeList::const_iterator;
5435
    const_iterator begin() const { return Ranges.begin(); }
5436
    const_iterator end() const { return Ranges.end(); }
5437
 
5438
  private:
5439
    /// The instruction responsible for the access with respect to the local
5440
    /// scope of the associated attribute.
5441
    Instruction *LocalI;
5442
 
5443
    /// The instruction responsible for the access.
5444
    Instruction *RemoteI;
5445
 
5446
    /// The value written, if any. `llvm::none` means "not known yet", `nullptr`
5447
    /// cannot be determined.
5448
    std::optional<Value *> Content;
5449
 
5450
    /// Set of potential ranges accessed from the base pointer.
5451
    RangeList Ranges;
5452
 
5453
    /// The access kind, e.g., READ, as bitset (could be more than one).
5454
    AccessKind Kind;
5455
 
5456
    /// The type of the content, thus the type read/written, can be null if not
5457
    /// available.
5458
    Type *Ty;
5459
  };
5460
 
5461
  /// Create an abstract attribute view for the position \p IRP.
5462
  static AAPointerInfo &createForPosition(const IRPosition &IRP, Attributor &A);
5463
 
5464
  /// See AbstractAttribute::getName()
5465
  const std::string getName() const override { return "AAPointerInfo"; }
5466
 
5467
  /// See AbstractAttribute::getIdAddr()
5468
  const char *getIdAddr() const override { return &ID; }
5469
 
5470
  /// Call \p CB on all accesses that might interfere with \p Range and return
5471
  /// true if all such accesses were known and the callback returned true for
5472
  /// all of them, false otherwise. An access interferes with an offset-size
5473
  /// pair if it might read or write that memory region.
5474
  virtual bool forallInterferingAccesses(
5475
      AA::RangeTy Range, function_ref<bool(const Access &, bool)> CB) const = 0;
5476
 
5477
  /// Call \p CB on all accesses that might interfere with \p I and
5478
  /// return true if all such accesses were known and the callback returned true
5479
  /// for all of them, false otherwise. In contrast to forallInterferingAccesses
5480
  /// this function will perform reasoning to exclude write accesses that cannot
5481
  /// affect the load even if they on the surface look as if they would. The
5482
  /// flag \p HasBeenWrittenTo will be set to true if we know that \p I does not
5483
  /// read the intial value of the underlying memory.
5484
  virtual bool forallInterferingAccesses(
5485
      Attributor &A, const AbstractAttribute &QueryingAA, Instruction &I,
5486
      function_ref<bool(const Access &, bool)> CB, bool &HasBeenWrittenTo,
5487
      AA::RangeTy &Range) const = 0;
5488
 
5489
  /// This function should return true if the type of the \p AA is AAPointerInfo
5490
  static bool classof(const AbstractAttribute *AA) {
5491
    return (AA->getIdAddr() == &ID);
5492
  }
5493
 
5494
  /// Unique ID (due to the unique address)
5495
  static const char ID;
5496
};
5497
 
5498
/// An abstract attribute for getting assumption information.
5499
struct AAAssumptionInfo
5500
    : public StateWrapper<SetState<StringRef>, AbstractAttribute,
5501
                          DenseSet<StringRef>> {
5502
  using Base =
5503
      StateWrapper<SetState<StringRef>, AbstractAttribute, DenseSet<StringRef>>;
5504
 
5505
  AAAssumptionInfo(const IRPosition &IRP, Attributor &A,
5506
                   const DenseSet<StringRef> &Known)
5507
      : Base(IRP, Known) {}
5508
 
5509
  /// Returns true if the assumption set contains the assumption \p Assumption.
5510
  virtual bool hasAssumption(const StringRef Assumption) const = 0;
5511
 
5512
  /// Create an abstract attribute view for the position \p IRP.
5513
  static AAAssumptionInfo &createForPosition(const IRPosition &IRP,
5514
                                             Attributor &A);
5515
 
5516
  /// See AbstractAttribute::getName()
5517
  const std::string getName() const override { return "AAAssumptionInfo"; }
5518
 
5519
  /// See AbstractAttribute::getIdAddr()
5520
  const char *getIdAddr() const override { return &ID; }
5521
 
5522
  /// This function should return true if the type of the \p AA is
5523
  /// AAAssumptionInfo
5524
  static bool classof(const AbstractAttribute *AA) {
5525
    return (AA->getIdAddr() == &ID);
5526
  }
5527
 
5528
  /// Unique ID (due to the unique address)
5529
  static const char ID;
5530
};
5531
 
5532
/// An abstract attribute for getting all assumption underlying objects.
5533
struct AAUnderlyingObjects : AbstractAttribute {
5534
  AAUnderlyingObjects(const IRPosition &IRP) : AbstractAttribute(IRP) {}
5535
 
5536
  /// Create an abstract attribute biew for the position \p IRP.
5537
  static AAUnderlyingObjects &createForPosition(const IRPosition &IRP,
5538
                                                Attributor &A);
5539
 
5540
  /// See AbstractAttribute::getName()
5541
  const std::string getName() const override { return "AAUnderlyingObjects"; }
5542
 
5543
  /// See AbstractAttribute::getIdAddr()
5544
  const char *getIdAddr() const override { return &ID; }
5545
 
5546
  /// This function should return true if the type of the \p AA is
5547
  /// AAUnderlyingObjects.
5548
  static bool classof(const AbstractAttribute *AA) {
5549
    return (AA->getIdAddr() == &ID);
5550
  }
5551
 
5552
  /// Unique ID (due to the unique address)
5553
  static const char ID;
5554
 
5555
  /// Check \p Pred on all underlying objects in \p Scope collected so far.
5556
  ///
5557
  /// This method will evaluate \p Pred on all underlying objects in \p Scope
5558
  /// collected so far and return true if \p Pred holds on all of them.
5559
  virtual bool
5560
  forallUnderlyingObjects(function_ref<bool(Value &)> Pred,
5561
                          AA::ValueScope Scope = AA::Interprocedural) const = 0;
5562
};
5563
 
5564
raw_ostream &operator<<(raw_ostream &, const AAPointerInfo::Access &);
5565
 
5566
/// Run options, used by the pass manager.
5567
enum AttributorRunOption {
5568
  NONE = 0,
5569
  MODULE = 1 << 0,
5570
  CGSCC = 1 << 1,
5571
  ALL = MODULE | CGSCC
5572
};
5573
 
5574
} // end namespace llvm
5575
 
5576
#endif // LLVM_TRANSFORMS_IPO_ATTRIBUTOR_H