14 // The programming model is global variables, or equivalently function local
15 // static variables, accessible from kernels or other functions. For uses from
17 // memory required by all the variables combined, allocate them within that.
31 //   do not alias, which permits only allocating variables they use
35 // struct instance. This duplicates the variables, accurately modelling their
48 // LDS variables with constant annotation or non-undef initializer are passed
52 // LDS variables that are always allocated at the same address can be found
55 // The simplest strategy possible is to group all LDS variables in a single
57 // variables are always at the same address. LDS is however a limited resource
69 // for variables that are known reachable from a single kernel. "Hybrid" picks
73 // The "module" lowering implemented here finds LDS variables which are used by
75 // LDS variables. Variables that are only used from kernels are excluded.
81 // Second, each kernel allocates LDS variables independent of other kernels and
89 // "Kernel" lowering is only applicable for variables that are unambiguously
99 // variable. Any futher variables that can be lowered with the module strategy
109 // - Variables which are used by lots of kernels, e.g. those injected by a
118 // graph. This relies on the specified behaviour that all dynamic LDS variables
120 // kernel. Therefore this pass creates new dynamic LDS variables for each kernel
129 // said variables. This pass essentially specialises said variables per kernel.
131 // all ConstantExpr uses of LDS variables are expanded to instructions. This
135 // where given variables should be allocated and marks them with metadata,
136 // MD_absolute_symbol. The backend places the variables in coincidentally the
143 // The backend lowers LDS variables in the order of traversal of the function.
145 // allocate the fixed-address variables immediately upon starting the function
147 // the function to the variables that it allocates. For the module scope lds,
170 // the LDS variables it creates in the same struct created here. That avoids the
173 // that no LDS variables exist outside of the structs managed here, and an
234             "Lower variables reachable from one kernel, otherwise abort"),
252     // so remove the variables from these lists before replaceAllUsesWith  in removeLocalVarsFromUsedLists()
284     // the variables are allocated in the anticipated order.  in markUsedByKernel()
312       LLVMContext &Ctx, ArrayRef<GlobalVariable *> Variables,  in getAddressesOfVariablesInKernel()  argument
321     ArrayType *KernelOffsetsType = ArrayType::get(I32, Variables.size());  in getAddressesOfVariablesInKernel()
324     for (GlobalVariable *GV : Variables) {  in getAddressesOfVariablesInKernel()
337       Module &M, ArrayRef<GlobalVariable *> Variables,  in buildLookupTable()  argument
340     if (Variables.empty()) {  in buildLookupTable()
345     const size_t NumberVariables = Variables.size();  in buildLookupTable()
362                     Ctx, Variables, Replacement->second.LDSVarsToConstantGEP);  in buildLookupTable()
660     // All LDS variables accessed indirectly have now been partitioned into  in partitionVariablesIntoIndirectStrategies()
671     // Replace all uses of those variables from non-kernel functions with the  in lowerModuleScopeStructVariables()
742     // Create a struct for each kernel for the non-module-scope variables.  in lowerKernelScopeStructVariables()
750       // Allocating variables that are used directly in this struct to get  in lowerKernelScopeStructVariables()
758       // Allocating variables that are accessed indirectly so that a lookup of  in lowerKernelScopeStructVariables()
766       // Variables allocated in module lds must all resolve to that struct,  in lowerKernelScopeStructVariables()
787         report_fatal_error("Anonymous kernels cannot use LDS variables");  in lowerKernelScopeStructVariables()
826     // reachable by that kernel. All dynamic LDS variables are allocated at the  in buildRepresentativeDynamicLDSInstance()
883             report_fatal_error("Anonymous kernels cannot use LDS variables");  in lowerDynamicLDSVariables()
933     // For each kernel, what variables does it access directly or through  in runOnModule()
947     // Partition variables accessed indirectly into the different strategies  in runOnModule()
1044         //  dynamic lds variables  in runOnModule()
1089           // Instead of explicitly marking kernels that access dynamic variables  in runOnModule()
1092           // LDS variables. That is the right condition to prevent allocating  in runOnModule()
1093           // more variables which would collide with the addresses assigned to  in runOnModule()
1094           // dynamic variables.  in runOnModule()
1126         // Only changing alignment of LDS variables  in superAlignLDSGlobals()
1163     // variables to ConstantExprGEP  in createLDSVariableReplacement()
1164     // Variables may be introduced to meet alignment requirements. No aliasing  in createLDSVariableReplacement()
1175       // variables in the argument which varies when changing how they  in createLDSVariableReplacement()
1405                       "Lower uses of LDS variables from non-kernel functions",
1409                     "Lower uses of LDS variables from non-kernel functions",  in INITIALIZE_PASS_DEPENDENCY()