1. Jul 03, 2023
    • Marc Durdin's avatar
      fix(developer): strip .kvk extension · 3d86e608
      Marc Durdin authored
      Fixes #9152.
      3d86e608
    • Marc Durdin's avatar
      fix(developer): improve warnings around keyboard version matching · 92c8c649
      Marc Durdin authored
      Fixes #9145.
      
      * `WARN_KeyboardVersionsDoNotMatch` is now only raised when
        `FollowKeyboardVersion` is set.
      * `WARN_KeyboardVersionsDoNotMatchPackageVersion` has been removed,
        because it did not really make sense; if 'FollowKeyboardVersion' is
        set, it could not be raised, and otherwise, the author may wish to
        have separate keyboard + package versions anyway.
      * Note that the 0x0013 compiler message allocation for
        `WARN_KeyboardVersionsDoNotMatchPackageVersion` has been left alone;
        as this was only used in pre-release, we can probably re-task it in
        the future.
      92c8c649
    • Marc Durdin's avatar
      fix(developer): split keyboard and model language metadata tests · 9a1b8782
      Marc Durdin authored
      Fixes #9146.
      
      For model packages, 304016 (ERROR_ModelMustHaveAtLeastOneLanguage)
      remains as an error (this is a new error for kmc, kmcomp did not
      validate model metadata). For keyboard packages, a new warning is
      introduced to match the existing kmcomp behaviour, 20401B
      (WARN_KeyboardShouldHaveAtLeastOneLanguage).
      9a1b8782
    • Marc Durdin's avatar
      fix(developer): allow kmcmplib memory usage to grow · be0b2ae5
      Marc Durdin authored
      Fixes #9112.
      
      Note that this does not address any performance concerns which are a
      much bigger issue; it just avoids the crash.
      be0b2ae5
    • Marc Durdin's avatar
      fix(developer): compiler needs to support loading .kvk files · 7aede787
      Marc Durdin authored
      Fixes #9150.
      
      Legacy .kmn keyboards can refer to a binary .kvk file. kmc needs to be
      able to load these as well.
      
      Note: there was a short period of time in which .kvk files were either
      source or binary format. We moved to .kvk always being binary, and .kvks
      always being source (xml), and so IMO we don't need to include support
      for mismatched file formats.
      7aede787
    • Marc Durdin's avatar
      fix(common): legacy .kpj schema · 33affc66
      Marc Durdin authored
      Fixes #9140.
      Fixes #9148.
      
      Keyman Developer 9.0 .kpj files included a lot of additional state
      metadata. We need a schema which validates these files, as they are
      otherwise valid to load (we'll never save them any more). Rather than
      add all the extra metadata to what is otherwise a fairly clean schema,
      we'll provide a legacy .schema.json.
      
      In the future, we may be able to merge these schemas, as we move towards
      the .kpj 2.0 format which doesn't list files. Ideally, all three formats
      (Keyman Developer 9.0 schema, call it legacy, 1.0 schema for Keyman
      Developer 10.0+ which has Options and Files listed, 2.0 schema for
      capturing just project settings for a folder) will be supported by a
      single schema file.
      33affc66
  2. Jun 30, 2023
    • Marc Durdin's avatar
      fix(developer): kmc needs to support .js-only packages · f85478ac
      Marc Durdin authored
      Fixes #9111.
      
      We have existing packages which have only .js in them, for touch-only
      keyboards (mostly legacy but still...), so we need to support the freaky
      Javascript regex search which we did in the past for extracting
      metadata. While this is not necessarily going to work with hand-crafted
      Javascript keyboards, it should work with all kmc- and kmcomp-generated
      keyboards, so it will suffice to support these legacy packages.
      
      In the future, we will be giving a hint when a package includes a .js
      but not a .kmx, gradually upgrading this to a warning and finally an
      error as we attempt to phase out .js-based keyboards in preference for
      .kmx keyboards.
      f85478ac
    • Marc Durdin's avatar
      fix(common): give friendly error when .kvks is in binary format · d021526e
      Marc Durdin authored
      Fixes #9124.
      d021526e
    • Marc Durdin's avatar
      fix(developer): kmcmplib was clobbering previous output · f2a7efa2
      Marc Durdin authored
      Running kmc on a set of keyboards would sporadically fail after some
      time. It turns out this was related to memory allocation, specifically,
      resizing the WASM module's memory buffer caused `TypedArray` views into
      the memory to be reset!
      
      By default, when a `Uint8Array` is created from an `ArrayBuffer` (e.g.
      `Module.HEAP8.buffer`), it is a dynamic view into that buffer. This
      module buffer can be dynamically reallocated at any time, which can
      happen when allocating memory in WASM code (so the change will look
      _really_ weird in a stack trace). Thus, to ensure we don't trip over
      ourselves, we need to copy the buffer. Fortunately, creating a
      `Uint8Array` from a `Uint8Array` copies the data, and is pretty quick.
      f2a7efa2
    • Marc Durdin's avatar
      fix(common): specify alignment for COMP_ structs · 56da6479
      Marc Durdin authored
      When we read .kmx files, they have no alignment guarantees, so we need
      to tell the compiler to generate unaligned-safe code for accesses to
      `COMP_` structure members, because typically we point into the buffer
      and read at any offset.
      
      A .kmx file will generally be 2-byte aligned, as there are no structures
      or data types with smaller than 2 byte widths. There is no requirement
      that this be the case per the .kmx spec, though. So we use a 1 byte
      alignment attribute for the compiler, so it will generate safe code when
      reading these structs.
      
      Note that we are assuming that `COMP_KEYBOARD` is aligned because it is
      always the start of the file, so will be at the start of any buffer
      which will automatically be aligned correctly.
      
      We should probably review some of our design decisions and structs for
      KMXPlus given this.
      56da6479
    • Marc Durdin's avatar
      fix(developer): GetDelimitedString was using wrong strchr · 562ced52
      Marc Durdin authored
      Picked this up while running `-fsanitize=address` for trying to track
      down another bug.
      
      This fixes a buffer overrun (normally invisible) in kmcmplib, where
      `xstrchr` was being called, which takes a string as its second
      parameter, but being passed a single character. This was incorrect
      behaviour for two reasons:
      
      1. Passing a single character doesn't have null termination (big bug!)
      2. We were not wanting xstring semantics on the token being parsed here
      
      Replaced with `u16chr`, which does do what we want. Note the
      casting because `u16chr` only has a `const` version of its input/output
      at present.
      
      Removed `xstrchr` because it is not used anywhere else in kmcmplib.
      562ced52
    • Marc Durdin's avatar
      fix(developer): ensure `delete` uses array semantics where appropriate · 3b735cdc
      Marc Durdin authored
      The kmcmplib compiler source used `delete` instead of `delete[]` in a
      number of locations where the corresponding allocation was an array.
      
      This doesn't appear to matter on arrays of primitives on most platforms
      that we are targeting, but it _is_ undefined behaviour so we should
      correct it.
      
      https://stackoverflow.com/a/2425749/1836776
      3b735cdc
  3. Jun 28, 2023
  4. Jun 27, 2023
  5. Jun 26, 2023
  6. Jun 24, 2023