1. Mar 17, 2008
  2. Mar 16, 2008
  3. Mar 15, 2008
    • Ingo Molnar's avatar
      sched: simplify sched_slice() · 6a6029b8
      Ingo Molnar authored
      
      
      Use the existing calc_delta_mine() calculation for sched_slice(). This
      saves a divide and simplifies the code because we share it with the
      other /cfs_rq->load users.
      
      It also improves code size:
      
            text    data     bss     dec     hex filename
           42659    2740     144   45543    b1e7 sched.o.before
           42093    2740     144   44977    afb1 sched.o.after
      
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      Signed-off-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
      6a6029b8
    • Ingo Molnar's avatar
      sched: fix fair sleepers · e22ecef1
      Ingo Molnar authored
      
      
      Fair sleepers need to scale their latency target down by runqueue
      weight. Otherwise busy systems will gain ever larger sleep bonus.
      
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      Signed-off-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
      e22ecef1
    • Peter Zijlstra's avatar
      sched: fix overload performance: buddy wakeups · aa2ac252
      Peter Zijlstra authored
      
      
      Currently we schedule to the leftmost task in the runqueue. When the
      runtimes are very short because of some server/client ping-pong,
      especially in over-saturated workloads, this will cycle through all
      tasks trashing the cache.
      
      Reduce cache trashing by keeping dependent tasks together by running
      newly woken tasks first. However, by not running the leftmost task first
      we could starve tasks because the wakee can gain unlimited runtime.
      
      Therefore we only run the wakee if its within a small
      (wakeup_granularity) window of the leftmost task. This preserves
      fairness, but does alternate server/client task groups.
      
      Signed-off-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      aa2ac252
    • Ingo Molnar's avatar
      sched: fix calc_delta_mine() · 27d11726
      Ingo Molnar authored
      
      
      lw->weight can be 0 for a short time during bootup.
      
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      Signed-off-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
      27d11726
    • Ingo Molnar's avatar
      sched: fix update_load_add()/sub() · e89996ae
      Ingo Molnar authored
      
      
      Clear the cached inverse value when updating load. This is needed for
      calc_delta_mine() to work correctly when using the rq load.
      
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      Signed-off-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
      e89996ae
    • Peter Zijlstra's avatar
      sched: min_vruntime fix · 3fe69747
      Peter Zijlstra authored
      
      
      Current min_vruntime tracking is incorrect and will cause serious
      problems when we don't run the leftmost task for some reason.
      
      min_vruntime does two things; 1) it's used to determine a forward
      direction when the u64 vruntime wraps, 2) it's used to track the
      leftmost vruntime to position newly enqueued tasks from.
      
      The current logic advances min_vruntime whenever the current task's
      vruntime advance. Because the current task may pass the leftmost task
      still waiting we're failing the second goal. This causes new tasks to be
      placed too far ahead and thus penalizes their runtime.
      
      Fix this by making min_vruntime the min_vruntime of the waiting tasks by
      tracking it in enqueue/dequeue, and compare against current's vruntime
      to obtain the absolute minimum when placing new tasks.
      
      Signed-off-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      3fe69747