Lines Matching full:we
60 * where we indicate the status after synchronizing. in pt_blk_status()
67 * Postpone it as long as we're still processing events, though. in pt_blk_status()
127 /* Set the flags we need for the query decoder we use. */ in pt_blk_decoder_init()
188 * If we're not already processing events, check the current time against the
205 /* We're not generating tick events if tracing is disabled. */ in pt_blk_tick()
213 * In rare cases where we need to proceed to an event location using in pt_blk_tick()
225 /* If we don't have wall-clock time, we use relative time. */ in pt_blk_tick()
232 /* We're done if time has not changed since the last event. */ in pt_blk_tick()
236 /* Time has changed so we create a new tick event. */ in pt_blk_tick()
241 /* Indicate if we have wall-clock time or only relative time. */ in pt_blk_tick()
248 /* We now have an event to process. */ in pt_blk_tick()
318 /* We will always have an event. in pt_blk_start()
320 * If we synchronized onto an empty PSB+, tracing is disabled and we'll in pt_blk_start()
324 * which we're going to forward to the user. in pt_blk_start()
557 /* We handle non-taken conditional branches, and compressed returns in pt_blk_next_ip()
636 * We failed to proceed without trace. This ends the current block. Now use
659 /* We do need an IP in order to proceed. */ in pt_blk_proceed_with_trace()
682 /* We know that @ip is contained in @section. in pt_blk_decode_in_section()
684 * Note that we need to translate @ip into a section offset. in pt_blk_decode_in_section()
690 /* We initialize @insn->size to the maximal possible size. It will be in pt_blk_decode_in_section()
770 /* We do not switch sections inside a block. */ in pt_blk_proceed_one_insn()
778 /* If we couldn't read @insn's memory in one chunk from @insn.isid, we in pt_blk_proceed_one_insn()
792 /* We have a new instruction. */ in pt_blk_proceed_one_insn()
806 * Proceed until we reach an instruction for which @predicate returns a positive
812 * - we would need trace: return -pte_bad_query
816 * Update @decoder->ip to point to the last IP that was reached. If we fail due
817 * to lack of trace or if we reach a desired instruction, this is @insn->ip;
841 /* We're done if this instruction matches the spec (positive in pt_blk_proceed_to_insn()
842 * status) or we run into an error (negative status). in pt_blk_proceed_to_insn()
848 /* Let's see if we can proceed to the next IP without trace. */ in pt_blk_proceed_to_insn()
855 * We only need to take care about direct near branches. in pt_blk_proceed_to_insn()
869 * Proceed until we reach @ip or until:
873 * - we would need trace: return -pte_bad_query
875 * Provide the last instruction that was reached in @insn and @iext. If we
878 * Update @decoder->ip to point to the last IP that was reached. If we fail due
896 /* We're done when we reach @ip. We may not even have to decode in pt_blk_proceed_to_ip()
906 /* Let's see if we can proceed to the next IP without trace. */ in pt_blk_proceed_to_ip()
913 * We only need to take care about direct near branches. in pt_blk_proceed_to_ip()
931 * Proceed until we reach @ip or until:
935 * - we need trace: return zero
955 * We're also OK if @block overflowed or we switched sections and we in pt_blk_proceed_to_ip_with_trace()
962 /* Needing trace is not an error. We use trace to determine the next in pt_blk_proceed_to_ip_with_trace()
990 * We have a (synchronous) disabled event pending. Proceed to the event
991 * location and indicate whether we were able to reach it.
1023 /* The erratum doesn't apply if we can bind the event to a in pt_blk_proceed_skl014()
1036 /* We need to flip the indication. in pt_blk_proceed_skl014()
1038 * We reached the event location when @ip lies inside a in pt_blk_proceed_skl014()
1054 * We only need to take care about direct near branches. in pt_blk_proceed_skl014()
1070 * We have a (synchronous) disabled event pending. Proceed to the event
1071 * location and indicate whether we were able to reach it.
1092 * If we don't have a filter configuration we assume that no in pt_blk_proceed_to_disabled()
1095 * We might otherwise disable tracing too early. in pt_blk_proceed_to_disabled()
1118 * which the event bound. That's not where we expect tracing to resume.
1121 * called function. For other types of instructions, we simply don't know.
1147 * We have an async paging event pending. Proceed to the event location and
1148 * indicate whether we were able to reach it. Needing trace in order to proceed
1164 /* Apply the event immediately if we don't have an IP. */ in pt_blk_proceed_to_async_paging()
1173 /* We may have reached the IP. */ in pt_blk_proceed_to_async_paging()
1179 * We have an async vmcs event pending. Proceed to the event location and
1180 * indicate whether we were able to reach it. Needing trace in order to proceed
1196 /* Apply the event immediately if we don't have an IP. */ in pt_blk_proceed_to_async_vmcs()
1205 /* We may have reached the IP. */ in pt_blk_proceed_to_async_vmcs()
1211 * We have an exec mode event pending. Proceed to the event location and
1212 * indicate whether we were able to reach it. Needing trace in order to proceed
1228 /* Apply the event immediately if we don't have an IP. */ in pt_blk_proceed_to_exec_mode()
1237 /* We may have reached the IP. */ in pt_blk_proceed_to_exec_mode()
1243 * We have a ptwrite event pending. Proceed to the event location and indicate
1244 * whether we were able to reach it.
1246 * In case of the event binding to a ptwrite instruction, we pass beyond that
1249 * In the case of the event binding to an IP provided in the event, we move
1267 /* If we don't have an IP, the event binds to the next PTWRITE in pt_blk_proceed_to_ptwrite()
1270 * If we have an IP it still binds to the next PTWRITE instruction but in pt_blk_proceed_to_ptwrite()
1272 * when tracing is disabled and we don't have any other means of finding in pt_blk_proceed_to_ptwrite()
1273 * the PTWRITE instruction. We nevertheless distinguish the two cases, in pt_blk_proceed_to_ptwrite()
1276 * In both cases, we move beyond the PTWRITE instruction, so it will be in pt_blk_proceed_to_ptwrite()
1286 /* We now know the IP of the PTWRITE instruction corresponding in pt_blk_proceed_to_ptwrite()
1298 /* We reached the PTWRITE instruction and @decoder->ip points to in pt_blk_proceed_to_ptwrite()
1301 * Proceed beyond the PTWRITE to account for it. Note that we in pt_blk_proceed_to_ptwrite()
1315 * If we get an asynchronous disable on VMLAUNCH or VMRESUME, the FUP that
1346 /* The erratum may apply. We can't be sure without a lot more in pt_blk_handle_erratum_skd022()
1349 * We turn the async disable into a sync disable. Our caller in pt_blk_handle_erratum_skd022()
1362 * @insn/@iext until we processed all events that bind to it.
1415 /* There's nothing to do if we have no postponed instruction. */ in pt_blk_proceed_postponed_insn()
1439 * We have an event pending. Proceed to the event location and indicate the
1442 * On our way to the event location we may also be forced to postpone the event
1443 * to the next block, e.g. if we overflow the number of instructions in the
1444 * block or if we need trace in order to reach the event location.
1446 * If we're not able to reach the event location, we return zero. This is what
1449 * - we suppress pts_eos as long as we're processing events
1450 * - we do not set pts_ip_suppressed since tracing must be enabled
1501 /* If the erratum hits, we modify the event. in pt_blk_proceed_event()
1527 /* We bound a paging event. Make sure we do not bind further in pt_blk_proceed_event()
1550 /* We bound a vmcs event. Make sure we do not bind further vmcs in pt_blk_proceed_event()
1623 /* We bound a ptwrite event. Make sure we do not bind further in pt_blk_proceed_event()
1641 * Tracing is enabled and we don't have an event pending. Proceed as far as
1642 * we get without trace. Stop when we either:
1647 * We actually proceed one instruction further to get the start IP for the next
1662 /* This is overly conservative, really. We shouldn't get a bad-query in pt_blk_proceed_no_event_uncached()
1663 * status unless we decoded at least one instruction successfully. in pt_blk_proceed_no_event_uncached()
1668 /* Proceed as far as we get without trace. */ in pt_blk_proceed_no_event_uncached()
1722 /* If we can't reach @nip without overflowing the displacement field, we in pt_blk_add_trampoline()
1762 * Tracing is enabled and we don't have an event pending. The current IP is not
1793 * Note that we also return on a status of zero that indicates that the in pt_blk_proceed_no_event_fill_cache()
1802 /* Let's see if we can proceed to the next IP without trace. in pt_blk_proceed_no_event_fill_cache()
1804 * If we can't, this is certainly a decision point. in pt_blk_proceed_no_event_fill_cache()
1857 /* If the block was truncated, we have to decode its last in pt_blk_proceed_no_event_fill_cache()
1860 * We could have skipped the above switch and size assignment in in pt_blk_proceed_no_event_fill_cache()
1878 /* Even if we were able to proceed without trace, we might have to stop in pt_blk_proceed_no_event_fill_cache()
1883 * We are forced to re-decode @insn to get the branch displacement. in pt_blk_proceed_no_event_fill_cache()
1885 * Even though it is constant, we don't cache it to avoid increasing in pt_blk_proceed_no_event_fill_cache()
1887 * zero for this entry and we might be tempted to use it - but other in pt_blk_proceed_no_event_fill_cache()
1891 * We could proceed after a near direct call but we migh as well in pt_blk_proceed_no_event_fill_cache()
1897 * In case the current section is split underneath us, we must take in pt_blk_proceed_no_event_fill_cache()
1909 * Instead of the decode stop at the jump instruction we're using we in pt_blk_proceed_no_event_fill_cache()
1914 * - if we switched sections in pt_blk_proceed_no_event_fill_cache()
1918 * We need to re-decode @insn in order to determine the start IP of in pt_blk_proceed_no_event_fill_cache()
1923 * We need to read the last instruction's memory from multiple in pt_blk_proceed_no_event_fill_cache()
1926 * We could still use the block cache but then we'd have to handle in pt_blk_proceed_no_event_fill_cache()
1930 * route it through the decode flow, where we already have in pt_blk_proceed_no_event_fill_cache()
1956 /* We proceeded one instruction. Let's see if we have a cache entry for in pt_blk_proceed_no_event_fill_cache()
1963 /* If we don't have a valid cache entry, yet, fill the cache some more. in pt_blk_proceed_no_event_fill_cache()
1965 * On our way back, we add a cache entry for this instruction based on in pt_blk_proceed_no_event_fill_cache()
1969 /* If we exceeded the maximum number of allowed steps, we insert in pt_blk_proceed_no_event_fill_cache()
1972 * The next time we encounter the same code, we will use the in pt_blk_proceed_no_event_fill_cache()
1973 * trampoline to jump directly to where we left off this time in pt_blk_proceed_no_event_fill_cache()
1987 /* Let's see if we have more luck this time. */ in pt_blk_proceed_no_event_fill_cache()
1992 /* If we still don't have a valid cache entry, we're done. Most in pt_blk_proceed_no_event_fill_cache()
1993 * likely, @block overflowed and we couldn't proceed past the in pt_blk_proceed_no_event_fill_cache()
2000 /* We must not have switched execution modes. in pt_blk_proceed_no_event_fill_cache()
2002 * This would require an event and we're on the no-event flow. in pt_blk_proceed_no_event_fill_cache()
2011 /* We may have switched sections if the section was split. See in pt_blk_proceed_no_event_fill_cache()
2014 * We're not adding a block cache entry since this won't apply to the in pt_blk_proceed_no_event_fill_cache()
2017 * We will instead take the slow path until the end of the section. in pt_blk_proceed_no_event_fill_cache()
2030 /* If none of them overflowed, we're done. in pt_blk_proceed_no_event_fill_cache()
2032 * If one or both overflowed, let's try to insert a trampoline, i.e. we in pt_blk_proceed_no_event_fill_cache()
2038 /* We're done. Add the cache entry. in pt_blk_proceed_no_event_fill_cache()
2041 * meantime. They should have come to the same conclusion as we, in pt_blk_proceed_no_event_fill_cache()
2045 * identical, we wouldn't care because they are both correct. in pt_blk_proceed_no_event_fill_cache()
2052 * We were not able to decode the instruction at @decoder->ip in @decoder's
2056 * we expect to end up with an instruction that was truncated in the section it
2057 * started. We provide the full instruction in this case and end the block.
2081 /* We shouldn't use this function if the instruction isn't truncated. */ in pt_blk_proceed_truncated()
2096 /* Let's see if we can proceed to the next IP without trace. in pt_blk_proceed_truncated()
2098 * The truncated instruction ends the block but we still need to get the in pt_blk_proceed_truncated()
2114 * Tracing is enabled and we don't have an event pending. We already set
2115 * @block's isid. All reads are done within @msec as we're not switching
2118 * Proceed as far as we get without trace. Stop when we either:
2123 * We actually proceed one instruction further to get the start IP for the next
2146 /* If we don't find a valid cache entry, fill the cache. */ in pt_blk_proceed_no_event_cached()
2152 /* If we switched sections, the origianl section must have been split in pt_blk_proceed_no_event_cached()
2156 * Crossing sections requires ending the block so we can indicate the in pt_blk_proceed_no_event_cached()
2162 * This check will also cover changes to a linear sequence of code we in pt_blk_proceed_no_event_cached()
2166 * Since we stop on every (backwards) branch (through an artificial stop in pt_blk_proceed_no_event_cached()
2167 * in the case of a near direct backward branch) we will detect all in pt_blk_proceed_no_event_cached()
2170 * Switch to the slow path until we reach the end of this section. in pt_blk_proceed_no_event_cached()
2176 /* We have a valid cache entry. Let's first check if the way to the in pt_blk_proceed_no_event_cached()
2179 * If it doesn't, we end the block without filling it as much as we in pt_blk_proceed_no_event_cached()
2182 * On the next iteration, we will start with an empty block, which is in pt_blk_proceed_no_event_cached()
2192 * We're not switching execution modes so even if @block already has an in pt_blk_proceed_no_event_cached()
2193 * execution mode, it will be the one we're going to set. in pt_blk_proceed_no_event_cached()
2197 /* We don't know the instruction class so we should be setting it to in pt_blk_proceed_no_event_cached()
2198 * ptic_error. Since we will be able to fill it back in later in most in pt_blk_proceed_no_event_cached()
2199 * cases, we move the clearing to the switch cases that don't. in pt_blk_proceed_no_event_cached()
2208 /* We're not able to reach the actual decision point due to in pt_blk_proceed_no_event_cached()
2209 * overflows so we inserted a trampoline. in pt_blk_proceed_no_event_cached()
2211 * We don't know the instruction and it is not guaranteed that in pt_blk_proceed_no_event_cached()
2212 * we will proceed further (e.g. if @block overflowed). Let's in pt_blk_proceed_no_event_cached()
2214 * become invalid when we updated @block->ninsn. in pt_blk_proceed_no_event_cached()
2222 /* We're at a conditional branch. */ in pt_blk_proceed_no_event_cached()
2225 /* Let's first check whether we know the size of the in pt_blk_proceed_no_event_cached()
2226 * instruction. If we do, we might get away without decoding in pt_blk_proceed_no_event_cached()
2229 * If we don't know the size we might as well do the full decode in pt_blk_proceed_no_event_cached()
2230 * and proceed-with-trace flow we do for ptbq_decode. in pt_blk_proceed_no_event_cached()
2236 /* If the branch is not taken, we don't need to decode in pt_blk_proceed_no_event_cached()
2239 * If it is taken, we have to implement everything here. in pt_blk_proceed_no_event_cached()
2240 * We can't use the normal decode and proceed-with-trace in pt_blk_proceed_no_event_cached()
2241 * flow since we already consumed the TNT bit. in pt_blk_proceed_no_event_cached()
2281 /* We need to decode the instruction at @decoder->ip and decide in pt_blk_proceed_no_event_cached()
2284 * We already accounted for the instruction so we can't just in pt_blk_proceed_no_event_cached()
2302 /* We just decoded @insn so we know the instruction class. */ in pt_blk_proceed_no_event_cached()
2310 /* Let's see if we can proceed to the next IP without trace. in pt_blk_proceed_no_event_cached()
2312 * Note that we also stop due to displacement overflows or to in pt_blk_proceed_no_event_cached()
2320 /* We can't, so let's proceed with trace, which in pt_blk_proceed_no_event_cached()
2328 * We only need to take care about direct near branches. in pt_blk_proceed_no_event_cached()
2338 /* If we can proceed without trace and we stay in @msec we may in pt_blk_proceed_no_event_cached()
2341 * We're done if we switch sections, though. in pt_blk_proceed_no_event_cached()
2353 /* We're at a near indirect call. */ in pt_blk_proceed_no_event_cached()
2356 /* We need to update the return-address stack and query the in pt_blk_proceed_no_event_cached()
2361 /* If we already know the size of the instruction, we don't need in pt_blk_proceed_no_event_cached()
2401 /* We're at a near return. */ in pt_blk_proceed_no_event_cached()
2410 /* The return is not compressed. We need another query in pt_blk_proceed_no_event_cached()
2439 /* We're at an indirect jump or far transfer. in pt_blk_proceed_no_event_cached()
2441 * We don't know the exact instruction class and there's no in pt_blk_proceed_no_event_cached()
2444 * Indicate that we don't know the instruction class and leave in pt_blk_proceed_no_event_cached()
2449 /* This is neither a near call nor return so we don't need to in pt_blk_proceed_no_event_cached()
2518 * Tracing is enabled and we don't have an event pending. Proceed as far as
2519 * we get without trace. Stop when we either:
2524 * We actually proceed one instruction further to get the start IP for the next
2545 /* Even if there is no such section in the image, we may still in pt_blk_proceed_no_event()
2551 /* We do not switch sections inside a block. */ in pt_blk_proceed_no_event()
2588 /* If tracing is disabled we should either be out of trace or we should in pt_blk_proceed()
2614 * If we got a transaction abort immediately following a branch that produced
2654 /* Let's check if we can reach the event location from here. in pt_blk_handle_erratum_bdm64()
2656 * If we can, let's assume the erratum did not hit. We might still be in pt_blk_handle_erratum_bdm64()
2657 * wrong but we're not able to tell. in pt_blk_handle_erratum_bdm64()
2665 /* We can't reach the event location. This could either mean that we in pt_blk_handle_erratum_bdm64()
2668 * We assume the latter and pretend that the previous branch brought us in pt_blk_handle_erratum_bdm64()
2712 * - we just synchronized onto the trace stream
2713 * - we ended a block and proceeded to the next IP
2714 * - we processed an event that was indicated by this function
2761 * trace, we can apply the disabled event. in pt_blk_proceed_trailing_event()
2777 /* We proceeded past the current instruction. */ in pt_blk_proceed_trailing_event()
2815 * call. We're done. in pt_blk_proceed_trailing_event()
2835 /* We apply the event immediately if we're not tracing. */ in pt_blk_proceed_trailing_event()
2843 * We bind at most one paging event to an instruction, though. in pt_blk_proceed_trailing_event()
2848 /* We're done if we're not binding to the currently postponed in pt_blk_proceed_trailing_event()
2849 * instruction. We will process the event on the normal event in pt_blk_proceed_trailing_event()
2855 /* We bound a paging event. Make sure we do not bind further in pt_blk_proceed_trailing_event()
2875 /* We apply the event immediately if we're not tracing. */ in pt_blk_proceed_trailing_event()
2883 * We bind at most one vmcs event to an instruction, though. in pt_blk_proceed_trailing_event()
2888 /* We're done if we're not binding to the currently postponed in pt_blk_proceed_trailing_event()
2889 * instruction. We will process the event on the normal event in pt_blk_proceed_trailing_event()
2895 /* We bound a vmcs event. Make sure we do not bind further vmcs in pt_blk_proceed_trailing_event()
2992 /* We apply the event immediately if we're not tracing. */ in pt_blk_proceed_trailing_event()
2999 * We bind at most one ptwrite event to an instruction, though. in pt_blk_proceed_trailing_event()
3004 /* We're done if we're not binding to the currently postponed in pt_blk_proceed_trailing_event()
3005 * instruction. We will process the event on the normal event in pt_blk_proceed_trailing_event()
3012 /* We bound a ptwrite event. Make sure we do not bind further in pt_blk_proceed_trailing_event()
3055 * call. Note that, unless we stop with tracing disabled, we proceed in pt_blk_next()
3058 * Some of the state may later be overwritten as we process events. in pt_blk_next()
3089 /* We must have an IP in order to start decoding. */ in pt_blk_process_enabled()
3093 /* We must currently be disabled. */ in pt_blk_process_enabled()
3118 /* We must currently be enabled. */ in pt_blk_process_disabled()
3122 /* We preserve @decoder->ip. This is where we expect tracing to resume in pt_blk_process_disabled()
3123 * and we'll indicate that on the subsequent enabled event if tracing in pt_blk_process_disabled()
3146 /* We must currently be enabled. */ in pt_blk_process_async_branch()
3150 /* Jump to the branch destination. We will continue from there in the in pt_blk_process_async_branch()
3233 * had been disabled before, as well, we might have missed the in pt_blk_process_overflow()
3239 /* Tracing is enabled and we're at the IP at which the overflow in pt_blk_process_overflow()
3246 /* We don't know the TSX state. Let's assume we execute normally. in pt_blk_process_overflow()
3248 * We also don't know the execution mode. Let's keep what we have in pt_blk_process_overflow()
3249 * in case we don't get an update before we have to decode the next in pt_blk_process_overflow()
3330 /* We must currently process an event. */ in pt_blk_event()
3460 /* Copy the event to the user. Make sure we're not writing beyond the in pt_blk_event()
3463 * We might truncate details of an event but only for those events the in pt_blk_event()