1. Mar 27, 2018
    • Pirama Arumuga Nainar's avatar
      [CodeGen] Mark fma as const for Android · fbfba29d
      Pirama Arumuga Nainar authored
      Summary:
      r318093 sets fma, fmaf, fmal as const for Gnu and MSVC.  Android also
      does not set errno for these functions.  So mark these const for
      Android.
      
      Reviewers: spatel, efriedma, srhines, chh, enh
      
      Subscribers: cfe-commits, llvm-commits
      
      Differential Revision: https://reviews.llvm.org/D44852
      
      llvm-svn: 328552
      fbfba29d
    • Simon Pilgrim's avatar
      [X86][Btver2] Add CVTSI2SD/CVTSI2SS scheduler costs · 86ea5312
      Simon Pilgrim authored
      We still need to account for how Jaguar passes data from GPR -> XMM, which isn't as clean as XMM -> GPR.....
      
      llvm-svn: 328551
      86ea5312
    • Krzysztof Parzyszek's avatar
      [Pipeliner] Check for affine expression in isLoopCarriedOrder · 8c07d0c4
      Krzysztof Parzyszek authored
      The pipeliner must add a loop carried dependence between two memory
      operations if the base register is not an affine (linear) exression.
      The current implementation doesn't check how the base register is
      defined, which allows non-affine expressions, and then the pipeliner
      does not add a loop carried dependence when one is needed.
      
      This patch adds code to isLoopCarriedOrder that checks if the base
      register of the memory operations is defined by a phi, and the loop
      definition for the phi is a constant increment value.  This is a very
      simple check for a linear expression.
      
      Patch by Brendon Cahoon.
      
      llvm-svn: 328550
      8c07d0c4
    • David Blaikie's avatar
    • David Blaikie's avatar
      a1b2bf4c
    • Krzysztof Parzyszek's avatar
      [Pipeliner] Add missing loop carried dependences · 9f041b18
      Krzysztof Parzyszek authored
      The pipeliner is not adding a dependence edge for a loop carried
      dependence, and ends up scheduling a load from iteration n prior
      to an aliased store in iteration n-1.
      
      The code that adds the loop carried dependences in the pipeliner
      doesn't check if the memory objects for loads and stores are
      "identified" (i.e., distinct) objects. If they are not, then the
      code that adds the dependences needs to be conservative. The
      objects can be used to check dependences only when they are
      distinct objects.
      
      The code that checks for loop carried dependences has been updated
      to classify loads and stores that are not identified as "unknown"
      values. A store with an "unknown" value can potentially create
      a loop carried dependence with any pending load.
      
      Patch by Brendon Cahoon.
      
      llvm-svn: 328547
      9f041b18
    • Haicheng Wu's avatar
      [SLP] Add a test case. NFC. · 0ec1dbe4
      Haicheng Wu authored
      llvm-svn: 328546
      0ec1dbe4
    • Krzysztof Parzyszek's avatar
      [Pipeliner] Fix renaming in pipeliner when eliminating phis · 16e66f59
      Krzysztof Parzyszek authored
      The phi renaming code in the pipeliner uses the wrong value when
      rewriting phi uses, which results in an undefined value. In this
      case, the original phi is no longer needed due to the order of
      instruction in the pipelined loop. The pipeliner was assuming, in
      this case, the the phi loop definition should be used to
      rewrite the uses. However, the pipeliner needs to check to make
      sure that the loop definition has already been scheduled. If not,
      then the phi initial value needs to be used instead.
      
      Patch by Brendon Cahoon.
      
      llvm-svn: 328545
      16e66f59
    • Alexey Bataev's avatar
      [OPENMP] Codegen for declare target with link clause. · 92327c50
      Alexey Bataev authored
      If the link clause is used on the declare target directive, the object
      should be linked on target or target data directives, not during the
      codegen. Patch adds support for this clause.
      
      llvm-svn: 328544
      92327c50
    • Krzysztof Parzyszek's avatar
      [Pipeliner] Fix number of phis to generate in the epilog · 3f72a6b7
      Krzysztof Parzyszek authored
      The pipeliner was generating too many phis in the epilog blocks, which
      caused incorrect code generation when rewriting an instruction that uses
      the phi.
      
      In this case, there 3 prolog and epilog stages. An existing phi was
      scheduled at stage 1. When generating the code for the 2nd epilog an
      extra new phi was generated.
      
      To fix this, we need to update the code that calculates the maximum
      number of phis that can be generated, which is based upon the current
      prolog stage and the stage of the original phi. In this case, when the
      prolog stage is 1 and the original phi stage is 1, the maximum number
      of phis to generate is 2.
      
      Patch by Brendon Cahoon.
      
      llvm-svn: 328543
      3f72a6b7
    • Krzysztof Parzyszek's avatar
      [Pipeliner] Use latency to compute RecMII · a2122044
      Krzysztof Parzyszek authored
      The patch contains severals changes needed to pipeline an example
      that was transformed so that a Phi with a subreg is converted to
      copies.
      
      The pipeliner wasn't working for a couple of reasons.
      - The RecMII was 3 instead of 2 due to the extra copies.
      - Copy instructions contained a latency of 1.
      - The node order algorithm was not choosing the best "bottom"
      node, which caused an instruction to be scheduled that had a 
      predecessor and successor already scheduled.
      - Updated the Hexagon Machine Scheduler to check if the node is
      latency bound when adding the cost for a 0-latency dependence.
      
      The RecMII was 3 because the computation looks at the number of
      nodes in the recurrence. The extra copy is an extra node but
      it shouldn't increase the latency. The new RecMII computation
      looks at the latency of the instructions in the recurrence. We
      changed the latency of the dependence of a copy to 0. The latency
      computation for the copy also checks the use of the copy (similar
      to a reg_sequence).
      
      The node order algorithm was not choosing the last instruction
      in the recurrence for a bottom up traversal. This was when the
      last instruction is a copy. A check was added when choosing the
      instruction to check for NodeNum if the maxASAP is the same. This
      means that the scheduler will not end up with another node in
      the recurrence that has both a predecessor and successor already
      scheduled.
      
      The cost computation in Hexagon Machine Scheduler adds cost when
      an instruction can be packetized with a zero-latency instruction.
      We should only do this if the schedule is latency bound. 
      
      Patch by Brendon Cahoon.
      
      llvm-svn: 328542
      a2122044
    • Simon Pilgrim's avatar
      [X86][Btver2] Add CVTSD2SS/CVTSS2SD scheduler costs · 8815105c
      Simon Pilgrim authored
      llvm-svn: 328541
      8815105c
    • Krzysztof Parzyszek's avatar
      [Pipeliner] Fix assert caused by pipeliner serialization · f13bbf1d
      Krzysztof Parzyszek authored
      The pipeliner is asserting because the serialization step that 
      occurs at the end is deleting an instruction.  The assert
      occurs later on because there is a use without a definition.  
      
      The problem occurs when an instruction defines a value used 
      by a REQ_SEQUENCE and that value is used by a COPY instruction.
      The latencies between these instructions are zero, so they are
      put in to the same packet.  The serialization code is unable to
      handle this correctly, and ends up putting the REG_SEQUENCE
      before its definition.
      
      There is special code in the serialization step that attempts
      to handle zero-cost instructions (phis, copy, reg_sequence)
      differently than regular instructions. Unfortunately, this means
      the order does not come out correct.
      
      This patch simplifies the code by changing the seperate steps for
      handling zero-cost and regular instructions. Only phis are
      handled separate now, since they should occurs first. Then, this
      patch adds checks to make use the MoveUse is set to the smallest
      value if there are multiple uses in a cycle.
      
      Patch by Brendon Cahoon.
      
      llvm-svn: 328540
      f13bbf1d
    • Sebastian Pop's avatar
      [InstCombine] reassociate loop invariant GEP chains to enable LICM · d870aea0
      Sebastian Pop authored
      This change brings performance of zlib up by 10%. The example below is from a
      hot loop in longest_match() from zlib.
      
      do.body:
        %cur_match.addr.0 = phi i32 [ %cur_match, %entry ], [ %2, %do.cond ]
        %idx.ext = zext i32 %cur_match.addr.0 to i64
        %add.ptr = getelementptr inbounds i8, i8* %win, i64 %idx.ext
        %add.ptr2 = getelementptr inbounds i8, i8* %add.ptr, i64 %idx.ext1
        %add.ptr3 = getelementptr inbounds i8, i8* %add.ptr2, i64 -1
      
      In this example %idx.ext1 is a loop invariant. It will be moved above the use of
      loop induction variable %idx.ext such that it can be hoisted out of the loop by
      LICM. The operands that have dependences carried by the loop will be sinked down
      in the GEP chain. This patch will produce the following output:
      
      do.body:
        %cur_match.addr.0 = phi i32 [ %cur_match, %entry ], [ %2, %do.cond ]
        %idx.ext = zext i32 %cur_match.addr.0 to i64
        %add.ptr = getelementptr inbounds i8, i8* %win, i64 %idx.ext1
        %add.ptr2 = getelementptr inbounds i8, i8* %add.ptr, i64 -1
        %add.ptr3 = getelementptr inbounds i8, i8* %add.ptr2, i64 %idx.ext
      
      llvm-svn: 328539
      d870aea0
    • Krzysztof Parzyszek's avatar
      [Pipeliner] Enable more base+offset dependence changes in pipeliner · 40df8a2b
      Krzysztof Parzyszek authored
      The pipeliner changes dependences between base+offset instructions
      (loads and stores) so that the instructions have more flexibility
      to be scheduled with respect to each other. This occurs when the
      pipeliner is able to compute that the instructions will not alias
      if their order is changed. The prevous code enforced the alias
      property by checking if the base register is the same, and that the
      offset values are either both positive or negative.
      
      This patch improves the alias check by using the API
      areMemAccessesTriviallyDisjoint instead. This enables more cases,
      especially if the offset is a negative value. The pipeliner uses
      the function by creating a new instruction with the offset used
      in the next iteration.
      
      Patch by Brendon Cahoon.
      
      llvm-svn: 328538
      40df8a2b
    • Krzysztof Parzyszek's avatar
      [Pipeliner] Fix calculation when reusing phis · 55cb4986
      Krzysztof Parzyszek authored
      A schedule may require that a phi from the original loop is used in
      multiple iterations in the scheduled loop. When this occurs, we generate
      multiple phis in the pipelined loop to save the value across iterations.
      
      When we generate the new phis and update the register names in the
      pipelined loop, the pipeliner attempts to reuse a previously generated
      phi, when possible. The calculation for the name of the new phi needs
      to account for the version/iteration of the original phi. Also, in the
      epilog, the code only needs to check backwards for a previous iteration
      until reaching the first prolog block.
      
      Patch by Brendon Cahoon.
      
      llvm-svn: 328537
      55cb4986
    • Simon Pilgrim's avatar
    • Krzysztof Parzyszek's avatar
      [Pipeliner] Fix check for order dependences when finalizing instructions · 8e1363df
      Krzysztof Parzyszek authored
      The code in orderDepdences that looks at the order dependences between
      instructions was processing all the successor and predecessor order
      dependences. However, we really only want to check for an order dependence
      for instructions scheduled in the same cycle.
      
      Also, fixed how the pipeliner handles output dependences. An output
      dependence is also a potential loop carried dependence. The pipeliner
      didn't handle this case properly so an invalid schedule could be created
      that allowed an output dependence to be scheduled in the next iteration
      at the same cycle.
      
      Patch by Brendon Cahoon.
      
      llvm-svn: 328516
      8e1363df
  2. Mar 26, 2018