- Dec 25, 2020
-
-
Chris Wilson authored
Pull the individual strands of creating a custom heartbeat requests into a pair of common functions. This will reduce the number of changes we will need to make in future. Signed-off-by:
Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by:
Mika Kuoppala <mika.kuoppala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20201224160213.29521-1-chris@chris-wilson.co.uk
-
- Dec 24, 2020
-
-
Matthew Auld authored
The reloc batch is short lived but can exist in the user visible ppGTT, and since it's backed by an internal object, which lacks page clearing, we should take care to clear it upfront. Signed-off-by:
Matthew Auld <matthew.auld@intel.com> Reviewed-by:
-
Matthew Auld authored
The shadow batch is an internal object, which doesn't have any page clearing, and since the batch_len can be smaller than the object, we should take care to clear it. Testcase: igt/gen9_exec_parse/shadow-peek Fixes: 4f7af194 ("drm/i915: Support ro ppgtt mapped cmdparser shadow buffers") Signed-off-by:
-
Chris Wilson authored
Since schedule-in and schedule-out are now both always under the tasklet bitlock, we can reduce the individual atomic operations to simple instructions and worry less. This notably eliminates the race observed with intel_context_inflight in __engine_unpark(). Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/2583 Signed-off-by:
Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20201224135544.1713-9-chris@chris-wilson.co.uk
-
Chris Wilson authored
Now that the tasklet completely controls scheduling of the requests, and we postpone scheduling out the old requests, we can keep a hanging virtual request bound to the engine on which it hung, and remove it from te queue. On release, it will be returned to the same engine and remain in its queue until it is scheduled; after which point it will become eligible for transfer to a sibling. Instead, we could opt to resubmit the request along the virtual engine on unhold, making it eligible for load balancing immediately -- but that seems like a pointless optimisation for a hanging context. Signed-off-by:
-
Chris Wilson authored
Having recognised that we do not change the sibling until we schedule out, we can then defer the decision to resubmit the virtual engine from the unwind of the active queue to scheduling out of the virtual context. This improves our resilence in virtual engine scheduling, and should eliminate the rare cases of gem_exec_balance failing. By keeping the unwind order intact on the local engine, we can preserve data dependency ordering while doing a preempt-to-busy pass until we have determined the new ELSP. This means that if we try to timeslice between a virtual engine and a data-dependent ordinary request, the pair will maintain their relative ordering and we will avoid the resubmission, cancelling the timeslicing until further change. The dilemma though is that we then may end up in a situation where the 'demotion' of the virtual request to an ordinary request in the engine queue results in filling the ELSP[] with virtual requests instead of spreading the load across the engines. To compensate for this, we mark each virtual request and refuse to resubmit a virtual request in the secondary ELSP slots, thus forcing subsequent virtual requests to be scheduled out after timeslicing. By delaying the decision until we schedule out, we will avoid unnecessary resubmission. Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/2079 Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/2098 Signed-off-by:
-
Chris Wilson authored
Let's only wait for the list iterator when decoupling the virtual breadcrumb, as the signaling of all the requests may take a long time, during which we do not want to keep the tasklet spinning. Signed-off-by:
Matthew Brost <matthew.brost@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20201224135544.1713-6-chris@chris-wilson.co.uk
-
Chris Wilson authored
The issue with stale virtual breadcrumbs remain. Now we have the problem that if the irq-signaler is still referencing the stale breadcrumb as we transfer it to a new sibling, the list becomes spaghetti. This is a very small window, but that doesn't stop it being hit infrequently. To prevent the lists being tangled (the iterator starting on one engine's b->signalers but walking onto another list), always decouple the virtual breadcrumb on schedule-out and make sure that the walker has stepped out of the lists. Signed-off-by:
-
Chris Wilson authored
Inside schedule_out, we do extra work upon idling the context, such as updating the runtime, kicking off retires, kicking virtual engines. However, if we are in a series of processing single requests per contexts, we may find ourselves scheduling out the context, only to immediately schedule it back in during dequeue. This is just extra work that we can avoid if we keep the context marked as inflight across the dequeue. This becomes more significant later on for minimising virtual engine misses. Signed-off-by:
-
Chris Wilson authored
Once a virtual engine has been bound to a sibling, it will remain bound until we finally schedule out the last active request. We can not rebind the context to a new sibling while it is inflight as the context save will conflict, hence we wait. As we cannot then use any other sibliing while the context is inflight, only kick the bound sibling while it inflight and upon scheduling out the kick the rest (so that we can swap engines on timeslicing if the previously bound engine becomes oversubscribed). Signed-off-by:
-
Chris Wilson authored
Rather than going back and forth between the rb_node entry and the virtual_engine type, store the ve local and reuse it. As the container_of conversion from rb_node to virtual_engine requires a variable offset, performing that conversion just once shaves off a bit of code. v2: Keep a single virtual engine lookup, for typical use. Signed-off-by:
-
Chris Wilson authored
Rather than having special case code for opportunistically calling process_csb() and performing a direct submit while holding the engine spinlock for submitting the request, simply call the tasklet directly. This allows us to retain the direct submission path, including the CS draining to allow fast/immediate submissions, without requiring any duplicated code paths, and most importantly greatly simplifying the control flow by removing reentrancy. This will enable us to close a few races in the virtual engines in the next few patches. The trickiest part here is to ensure that paired operations (such as schedule_in/schedule_out) remain under consistent locking domains, e.g. when pulled outside of the engine->active.lock v2: Use bh kicking, see commit 3c53776e ("Mark HI and TASKLET softirq synchronous"). v3: Update engine-reset to be tasklet aware Signed-off-by:
-
Chris Wilson authored
As we shrink an object, also see if we can prune the dma-resv of idle fences it is maintaining a reference to. Signed-off-by:
-
Chris Wilson authored
If we want to reuse a fence that is in active use by the GPU, we have to wait an uncertain amount of time, but if we reuse an inactive fence, we can change it right away. Loop through the list of available fences twice, ignoring any active fences on the first pass. Signed-off-by:
-
Chris Wilson authored
Pull the GT clock information [used to derive CS timestamps and PM interval] under the GT so that is it local to the users. In doing so, we consolidate the two references for the same information, of which the runtime-info took note of a potential clock source override and scaling factors. Signed-off-by:
-
Chris Wilson authored
We assume that both timestamps are driven off the same clock [reported to userspace as I915_PARAM_CS_TIMESTAMP_FREQUENCY]. Verify that this is so by reading the timestamp registers around a busywait (on an otherwise idle engine so there should be no preemptions). v2: Icelake (not ehl, nor tgl) seems to be using a fixed 80ns interval for, and only for, CTX_TIMESTAMP -- or it may be GPU frequency and the test is always running at maximum frequency?. As far as I can tell, this isolated change in behaviour is undocumented. Signed-off-by:
-
Chris Wilson authored
We just need the context image from the logical state to force eviction of many contexts, so simplify by avoiding the GEM context container. Signed-off-by:
-
- Dec 23, 2020
-
-
Chris Wilson authored
The caller determines if the failure is an error or not, so avoid warning when we will try again and succeed. For example, <7> [111.319321] [drm:intel_guc_fw_upload [i915]] GuC status 0x20 <3> [111.319340] i915 0000:00:02.0: [drm] *ERROR* GuC load failed: status = 0x00000020 <3> [111.319606] i915 0000:00:02.0: [drm] *ERROR* GuC load failed: status: Reset = 0, BootROM = 0x10, UKernel = 0x00, MIA = 0x00, Auth = 0x00 <7> [111.320045] [drm:__uc_init_hw [i915]] GuC fw load failed: -110; will reset and retry 2 more time(s) <7> [111.322978] [drm:intel_guc_fw_upload [i915]] GuC status 0x8002f0ec should not have been reported as a _test_ failure, as the GuC was successfully loaded on the second attempt and the system remained operational. Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/2797 Signed-off-by:
-
Chris Wilson authored
By using the double wide cmpxchg64 on 32bit, we can use the same algorithm on both 32/64b systems. Signed-off-by:
-
- Dec 22, 2020
-
-
Chris Wilson authored
When waiting for the submit, before checking the status of the request, kick the tasklet to make sure we are processing the submission. This speeds up submission if we are using any tasklet suppression for secondary requests. Signed-off-by:
-
Chris Wilson authored
Make sure that the request has been submitted to HW before we begin our wait. This reduces our reliance on the semaphore yield interrupt driving the preemption request. Signed-off-by:
-
Chris Wilson authored
Keep on kicking the timeslice in case on the first retirement, it did not stay idle. This may happen when using semaphore yields. Signed-off-by:
-
Chris Wilson authored
We assume that the contents of the HWSP are lost across suspend, and so upon resume we must restore critical values such as the timeline seqno. Keep track of every timeline allocated that uses the HWSP as its storage and so we can then reset all seqno values by walking that list. Signed-off-by:
-
- Dec 21, 2020
-
-
Chris Wilson authored
Primarily used by selftests, but also by runtime debugging of engine w/a, is a routine to create a temporarily bound buffer for readback. Almagamate the duplicated routines into one. Suggested-by:
Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com> Signed-off-by:
-
Chris Wilson authored
Split the definition, construction and updating of the Logical Ring Context from the execlist submission interface. The LRC is used by the HW, irrespective of our different submission backends. Signed-off-by:
-
Chris Wilson authored
tasklet_kill() ensures that we _yield_ the processor until a remote tasklet is completed. However, this leads to a starvation condition as being at the bottom of the scheduler's runqueue means that anything else is able to run, including all hogs keeping the tasklet occupied. Signed-off-by:
-
- Dec 19, 2020
-
-
Chris Wilson authored
Since we allow removing the timeline map at runtime, there is a risk that rq->hwsp points into a stale page. To control that risk, we hold the RCU read lock while reading *rq->hwsp, but we missed a couple of important barriers. First, the unpinning / removal of the timeline map must be after all RCU readers into that map are complete, i.e. after an rcu barrier (in this case courtesy of call_rcu()). Secondly, we must make sure that the rq->hwsp we are about to dereference under the RCU lock is valid. In this case, we make the rq->hwsp pointer safe during i915_request_retire() and so we know that rq->hwsp may become invalid only after the request has been signaled. Therefore is the request is not yet signaled when we acquire rq->hwsp under the RCU, we know that rq->hwsp will remain valid for the duration of the RCU read lock. This is a very small window that may lead to either considering the request not completed (causing a delay until the request is checked again, any wait for the request is not affected) or dereferencing an invalid pointer. Fixes: 3adac468 ("drm/i915: Introduce concept of per-timeline (context) HWSP") Signed-off-by:
-
- Dec 18, 2020
-
-
Aditya Swarup authored
Add bound checks for TGL REV ID array. Since, there might be a possibility of using older kernels on latest platform revisions, resulting in out of bounds access for rev ID array. In this scenario, use the latest rev ID available and apply those WAs. Also, modify GT macros for TGL rev ID to reuse tgl_revids_get(). Cc: José Roberto de Souza <jose.souza@intel.com> Cc: Matt Roper <matthew.d.roper@intel.com> Cc: Lucas De Marchi <lucas.demarchi@intel.com> Cc: Jani Nikula <jani.nikula@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Signed-off-by:
Aditya Swarup <aditya.swarup@intel.com> Signed-off-by:
Lucas De Marchi <lucas.demarchi@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20201203072359.156682-2-aditya.swarup@intel.com
-
Aditya Swarup authored
Fix TGL REVID macros to fetch correct display/gt stepping based on SOC rev id from INTEL_REVID() macro. Previously, we were just returning the first element of the revid array instead of using the correct index based on SOC rev id. Fixes: c33298cb ("drm/i915/tgl: Fix stepping WA matching") Cc: José Roberto de Souza <jose.souza@intel.com> Cc: Matt Roper <matthew.d.roper@intel.com> Cc: Lucas De Marchi <lucas.demarchi@intel.com> Cc: Jani Nikula <jani.nikula@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Signed-off-by:
-
Chris Wilson authored
Since we wake the GT up before executing a request, and go to sleep as soon as it is retired, the GT wake time not only represents how long the device is powered up, but also provides a summary, albeit an overestimate, of the device runtime (i.e. the rc0 time to compare against rc6 time). v2: s/busy/awake/ v3: software-gt-awake-time and I915_PMU_SOFTWARE_GT_AWAKE_TIME Signed-off-by:
kernel test robot <oliver.sang@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20201215154456.13954-1-chris@chris-wilson.co.uk
-
Chris Wilson authored
Matthew Brost pointed out that the while-loop on a shared breadcrumb was inherently fraught with danger as it competed with the other users of the breadcrumbs. However, in order to completely drain the re-arming irq worker, the while-loop is a necessity, despite my optimism that we could force cancellation with a couple of irq_work invocations. Given that we can't merely drop the while-loop, use an activity counter on the breadcrumbs to detect when we are parking the breadcrumbs for the last time. Based on a patch by Matthew Brost. Reported-by:
-
- Dec 17, 2020
-
-
Chris Wilson authored
Use the wait_queue_entry.flags to denote the special fence behaviour (flattening continuations along fence chains, and for propagating errors) rather than trying to detect ordinary waiters by their functions. Signed-off-by:
-
- Dec 16, 2020
-
-
Chris Wilson authored
Reduce the pollution of intel_engine.h by moving gen8_emit_pipe_control and friends to gen8_engine_cs.h Signed-off-by:
-
Chris Wilson authored
The free_list and worker was introduced in commit 5f09a9c8 ("drm/i915: Allow contexts to be unreferenced locklessly"), but subsequently made redundant by the removal of the last sleeping lock in commit 2935ed53 ("drm/i915: Remove logical HW ID"). As we can now free the GEM context immediately from any context, remove the deferral of the free_list v2: Lift removing the context from the global list into close(). Suggested-by:
-
Chris Wilson authored
When inserting a VMA, we restrict the placement to the low 4G unless the caller opts into using the full range. This was done to allow usersapce the opportunity to transition slowly from a 32b address space, and to avoid breaking inherent 32b assumptions of some commands. However, for insert we limited ourselves to 4G-4K, but on verification we allowed the full 4G. This causes some attempts to bind a new buffer to sporadically fail with -ENOSPC, but at other times be bound successfully. commit 48ea1e32 ("drm/i915/gen9: Set PIN_ZONE_4G end to 4GB - 1 page") suggests that there is a genuine problem with stateless addressing that cannot utilize the last page in 4G and so we purposefully excluded it. This means that the quick pin pass may cause us to utilize a buggy placement. Reported-by:
CQ Tang <cq.tang@intel.com> Testcase: igt/gem_exec_params/larger-than-life-batch Fixes: 48ea1e32 ("drm/i915/gen9: Set PIN_ZONE_4G end to 4GB - 1 page") Signed-off-by:
-
- Dec 15, 2020
-
-
José Roberto de Souza authored
Commit 70a2b431 ("drm/i915/gt: Rename lrc.c to execlists_submission.c") renamed intel_lrc.c to intel_execlists_submission.c but forgot to update i915.rst. Fixes: 70a2b431 ("drm/i915/gt: Rename lrc.c to execlists_submission.c") Cc: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by:
José Roberto de Souza <jose.souza@intel.com> Reviewed-by:
-
- Dec 14, 2020
-
-
Tvrtko Ursulin authored
Chris spotted that since 16ffe73c ("drm/i915/pmu: Use GT parked for estimating RC6 while asleep") we don't rely on runtime pm internals when estimating RC6 while asleep. We can remove the ifdef code to simplify and at the same time wake up the device less when querying RC6 if CONFIG_PM is not compiled in. Signed-off-by:
-
Tvrtko Ursulin authored
RC6 is a hardware counter and as such estimating it using the raw clock during runtime suspend is more appropriate. Signed-off-by:
-
Tvrtko Ursulin authored
Chris found a CI report which points out calling intel_runtime_pm_get from inside i915_pmu_enable hook is not allowed since it can be invoked from hard irq context. This is something we knew but forgot, so lets fix it once again. We do this by syncing the internal book keeping with hardware rc6 counter on driver load. v2: * Always sync on parking and fully sync on init. Signed-off-by:
-
- Dec 11, 2020
-
-
Chris Wilson authored
The workarounds are tied to the GT and we should derive the tests local to the GT. Signed-off-by:
-
