1. Oct 05, 2018
    • Andy Lutomirski's avatar
      x86/vdso: Document vgtod_ts better · bcc4a62a
      Andy Lutomirski authored
      
      
      After reading do_hres() and do_course() and scratching my head a
      bit, I figured out why the arithmetic is strange.  Document it.
      
      Signed-off-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Link: http://lkml.kernel.org/r/f66f53d81150bbad47d7b282c9207a71a3ce1c16.1538689401.git.luto@kernel.org
      
      
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      bcc4a62a
    • Andy Lutomirski's avatar
      x86/vdso: Remove "memory" clobbers in the vDSO syscall fallbacks · 89fe0a1f
      Andy Lutomirski authored
      
      
      When a vDSO clock function falls back to the syscall, no special
      barriers or ordering is needed, and the syscall fallbacks don't
      clobber any memory that is not explicitly listed in the asm
      constraints.  Remove the "memory" clobber.
      
      This causes minor changes to the generated code, but otherwise has
      no obvious performance impact.  I think it's nice to have, though,
      since it may help the optimizer in the future.
      
      Signed-off-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Link: http://lkml.kernel.org/r/3a7438f5fb2422ed881683d2ccffd7f987b2dc44.1538689401.git.luto@kernel.org
      
      
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      89fe0a1f
    • Thomas Gleixner's avatar
      x66/vdso: Add CLOCK_TAI support · 315f28fa
      Thomas Gleixner authored
      
      
      With the storage array in place it's now trivial to support CLOCK_TAI in
      the vdso. Extend the base time storage array and add the update code.
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Tested-by: default avatarMatt Rickard <matt@softrans.com.au>
      Acked-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Stephen Boyd <sboyd@kernel.org>
      Cc: John Stultz <john.stultz@linaro.org>
      Cc: Florian Weimer <fweimer@redhat.com>
      Cc: "K. Y. Srinivasan" <kys@microsoft.com>
      Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
      Cc: devel@linuxdriverproject.org
      Cc: virtualization@lists.linux-foundation.org
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Juergen Gross <jgross@suse.com>
      Link: https://lkml.kernel.org/r/20180917130707.823878601@linutronix.de
      315f28fa
    • Thomas Gleixner's avatar
      x86/vdso: Move cycle_last handling into the caller · 3e89bf35
      Thomas Gleixner authored
      
      
      Dereferencing gtod->cycle_last all over the place and foing the cycles <
      last comparison in the vclock read functions generates horrible code. Doing
      it at the call site is much better and gains a few cycles both for TSC and
      pvclock.
      
      Caveat: This adds the comparison to the hyperv vclock as well, but I have
      no way to test that.
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Acked-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Matt Rickard <matt@softrans.com.au>
      Cc: Stephen Boyd <sboyd@kernel.org>
      Cc: John Stultz <john.stultz@linaro.org>
      Cc: Florian Weimer <fweimer@redhat.com>
      Cc: "K. Y. Srinivasan" <kys@microsoft.com>
      Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
      Cc: devel@linuxdriverproject.org
      Cc: virtualization@lists.linux-foundation.org
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Juergen Gross <jgross@suse.com>
      Link: https://lkml.kernel.org/r/20180917130707.741440803@linutronix.de
      3e89bf35
    • Thomas Gleixner's avatar
      x86/vdso: Simplify the invalid vclock case · 4f72adc5
      Thomas Gleixner authored
      
      
      The code flow for the vclocks is convoluted as it requires the vclocks
      which can be invalidated separately from the vsyscall_gtod_data sequence to
      store the fact in a separate variable. That's inefficient.
      
      Restructure the code so the vclock readout returns cycles and the
      conversion to nanoseconds is handled at the call site.
      
      If the clock gets invalidated or vclock is already VCLOCK_NONE, return
      U64_MAX as the cycle value, which is invalid for all clocks and leave the
      sequence loop immediately in that case by calling the fallback function
      directly.
      
      This allows to remove the gettimeofday fallback as it now uses the
      clock_gettime() fallback and does the nanoseconds to microseconds
      conversion in the same way as it does when the vclock is functional. It
      does not make a difference whether the division by 1000 happens in the
      kernel fallback or in userspace.
      
      Generates way better code and gains a few cycles back.
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Acked-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Matt Rickard <matt@softrans.com.au>
      Cc: Stephen Boyd <sboyd@kernel.org>
      Cc: John Stultz <john.stultz@linaro.org>
      Cc: Florian Weimer <fweimer@redhat.com>
      Cc: "K. Y. Srinivasan" <kys@microsoft.com>
      Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
      Cc: devel@linuxdriverproject.org
      Cc: virtualization@lists.linux-foundation.org
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Juergen Gross <jgross@suse.com>
      Link: https://lkml.kernel.org/r/20180917130707.657928937@linutronix.de
      4f72adc5
    • Thomas Gleixner's avatar
      x86/vdso: Replace the clockid switch case · f3e83938
      Thomas Gleixner authored
      
      
      Now that the time getter functions use the clockid as index into the
      storage array for the base time access, the switch case can be replaced.
      
      - Check for clockid >= MAX_CLOCKS and for negative clockid (CPU/FD) first
        and call the fallback function right away.
      
      - After establishing that clockid is < MAX_CLOCKS, convert the clockid to a
        bitmask
      
      - Check for the supported high resolution and coarse functions by anding
        the bitmask of supported clocks and check whether a bit is set.
      
      This completely avoids jump tables, reduces the number of conditionals and
      makes the VDSO extensible for other clock ids.
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Acked-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Matt Rickard <matt@softrans.com.au>
      Cc: Stephen Boyd <sboyd@kernel.org>
      Cc: John Stultz <john.stultz@linaro.org>
      Cc: Florian Weimer <fweimer@redhat.com>
      Cc: "K. Y. Srinivasan" <kys@microsoft.com>
      Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
      Cc: devel@linuxdriverproject.org
      Cc: virtualization@lists.linux-foundation.org
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Juergen Gross <jgross@suse.com>
      Link: https://lkml.kernel.org/r/20180917130707.574315796@linutronix.de
      f3e83938
    • Thomas Gleixner's avatar
      x86/vdso: Collapse coarse functions · 6deec5bd
      Thomas Gleixner authored
      
      
      do_realtime_coarse() and do_monotonic_coarse() are now the same except for
      the storage array index. Hand the index in as an argument and collapse the
      functions.
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Acked-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Matt Rickard <matt@softrans.com.au>
      Cc: Stephen Boyd <sboyd@kernel.org>
      Cc: John Stultz <john.stultz@linaro.org>
      Cc: Florian Weimer <fweimer@redhat.com>
      Cc: "K. Y. Srinivasan" <kys@microsoft.com>
      Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
      Cc: devel@linuxdriverproject.org
      Cc: virtualization@lists.linux-foundation.org
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Juergen Gross <jgross@suse.com>
      Link: https://lkml.kernel.org/r/20180917130707.490733779@linutronix.de
      6deec5bd
    • Thomas Gleixner's avatar
      x86/vdso: Collapse high resolution functions · e9a62f76
      Thomas Gleixner authored
      
      
      do_realtime() and do_monotonic() are now the same except for the storage
      array index. Hand the index in as an argument and collapse the functions.
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Acked-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Matt Rickard <matt@softrans.com.au>
      Cc: Stephen Boyd <sboyd@kernel.org>
      Cc: John Stultz <john.stultz@linaro.org>
      Cc: Florian Weimer <fweimer@redhat.com>
      Cc: "K. Y. Srinivasan" <kys@microsoft.com>
      Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
      Cc: devel@linuxdriverproject.org
      Cc: virtualization@lists.linux-foundation.org
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Juergen Gross <jgross@suse.com>
      Link: https://lkml.kernel.org/r/20180917130707.407955860@linutronix.de
      e9a62f76
    • Thomas Gleixner's avatar
      x86/vdso: Introduce and use vgtod_ts · 49116f20
      Thomas Gleixner authored
      
      
      It's desired to support more clocks in the VDSO, e.g. CLOCK_TAI. This
      results either in indirect calls due to the larger switch case, which then
      requires retpolines or when the compiler is forced to avoid jump tables it
      results in even more conditionals.
      
      To avoid both variants which are bad for performance the high resolution
      functions and the coarse grained functions will be collapsed into one for
      each. That requires to store the clock specific base time in an array.
      
      Introcude struct vgtod_ts for storage and convert the data store, the
      update function and the individual clock functions over to use it.
      
      The new storage does not longer use gtod_long_t for seconds depending on 32
      or 64 bit compile because this needs to be the full 64bit value even for
      32bit when a Y2038 function is added. No point in keeping the distinction
      alive in the internal representation.
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Acked-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Matt Rickard <matt@softrans.com.au>
      Cc: Stephen Boyd <sboyd@kernel.org>
      Cc: John Stultz <john.stultz@linaro.org>
      Cc: Florian Weimer <fweimer@redhat.com>
      Cc: "K. Y. Srinivasan" <kys@microsoft.com>
      Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
      Cc: devel@linuxdriverproject.org
      Cc: virtualization@lists.linux-foundation.org
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Juergen Gross <jgross@suse.com>
      Link: https://lkml.kernel.org/r/20180917130707.324679401@linutronix.de
      49116f20
    • Thomas Gleixner's avatar
      x86/vdso: Use unsigned int consistently for vsyscall_gtod_data:: Seq · 77e9c678
      Thomas Gleixner authored
      
      
      The sequence count in vgtod_data is unsigned int, but the call sites use
      unsigned long, which is a pointless exercise. Fix the call sites and
      replace 'unsigned' with unsinged 'int' while at it.
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Acked-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Matt Rickard <matt@softrans.com.au>
      Cc: Stephen Boyd <sboyd@kernel.org>
      Cc: John Stultz <john.stultz@linaro.org>
      Cc: Florian Weimer <fweimer@redhat.com>
      Cc: "K. Y. Srinivasan" <kys@microsoft.com>
      Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
      Cc: devel@linuxdriverproject.org
      Cc: virtualization@lists.linux-foundation.org
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Juergen Gross <jgross@suse.com>
      Link: https://lkml.kernel.org/r/20180917130707.236250416@linutronix.de
      77e9c678
    • Thomas Gleixner's avatar
      x86/vdso: Enforce 64bit clocksource · a51e996d
      Thomas Gleixner authored
      
      
      All VDSO clock sources are TSC based and use CLOCKSOURCE_MASK(64). There is
      no point in masking with all FF. Get rid of it and enforce the mask in the
      sanity checker.
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Acked-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Matt Rickard <matt@softrans.com.au>
      Cc: Stephen Boyd <sboyd@kernel.org>
      Cc: John Stultz <john.stultz@linaro.org>
      Cc: Florian Weimer <fweimer@redhat.com>
      Cc: "K. Y. Srinivasan" <kys@microsoft.com>
      Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
      Cc: devel@linuxdriverproject.org
      Cc: virtualization@lists.linux-foundation.org
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Juergen Gross <jgross@suse.com>
      Link: https://lkml.kernel.org/r/20180917130707.151963007@linutronix.de
      a51e996d
    • Thomas Gleixner's avatar
      x86/time: Implement clocksource_arch_init() · 2a21ad57
      Thomas Gleixner authored
      
      
      Runtime validate the VCLOCK_MODE in clocksource::archdata and disable
      VCLOCK if invalid, which disables the VDSO but keeps the system running.
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Acked-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Matt Rickard <matt@softrans.com.au>
      Cc: Stephen Boyd <sboyd@kernel.org>
      Cc: John Stultz <john.stultz@linaro.org>
      Cc: Florian Weimer <fweimer@redhat.com>
      Cc: "K. Y. Srinivasan" <kys@microsoft.com>
      Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
      Cc: devel@linuxdriverproject.org
      Cc: virtualization@lists.linux-foundation.org
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Juergen Gross <jgross@suse.com>
      Link: https://lkml.kernel.org/r/20180917130707.069167446@linutronix.de
      2a21ad57
    • Thomas Gleixner's avatar
      clocksource: Provide clocksource_arch_init() · d67f34c1
      Thomas Gleixner authored
      
      
      Architectures have extra archdata in the clocksource, e.g. for VDSO
      support. There are no sanity checks or general initializations for this
      available. Add support for that.
      
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Acked-by: default avatarAndy Lutomirski <luto@kernel.org>
      Acked-by: default avatarJohn Stultz <john.stultz@linaro.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Matt Rickard <matt@softrans.com.au>
      Cc: Stephen Boyd <sboyd@kernel.org>
      Cc: Florian Weimer <fweimer@redhat.com>
      Cc: "K. Y. Srinivasan" <kys@microsoft.com>
      Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
      Cc: devel@linuxdriverproject.org
      Cc: virtualization@lists.linux-foundation.org
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Juergen Gross <jgross@suse.com>
      Link: https://lkml.kernel.org/r/20180917130706.973042587@linutronix.de
      d67f34c1
  2. Oct 04, 2018
  3. Oct 03, 2018
  4. Oct 02, 2018
  5. Sep 30, 2018
  6. Sep 29, 2018