Commits

Branch selector

 main

User selector

lucidrains

All time

Commit History

Commits on Jan 14, 2024

• readme

  lucidrains

  committedJan 14, 2024
  6b02ee3

Commits on Apr 5, 2023

• fix a bug with the final norm in palm, thanks to @conceptofmind and @dmahan93

  lucidrains

  committedApr 5, 2023
  e624afd

Commits on Mar 23, 2023

• address #41 , be faithful to the paper

  lucidrains

  committedMar 23, 2023
  1487fd4

Commits on Mar 22, 2023

• old action log probs should be the true distribution in the kl div loss, addressing #43

  lucidrains

  committedMar 22, 2023
  d4faf48

Commits on Mar 17, 2023

• comment

  lucidrains

  committedMar 17, 2023
  ad001d8

• flash attention sdp context config only needs to be done once

  lucidrains

  committedMar 17, 2023
  6c8028a

• draw attention

  lucidrains

  committedMar 17, 2023
  734afb3

• fix assert

  lucidrains

  committedMar 17, 2023
  570ebb7

• project management

  lucidrains

  committedMar 17, 2023
  4ae1f6d

• add ability to use flash attention if using pytorch 2.0, thanks to @conceptofmind for the initial PR!

  lucidrains

  committedMar 17, 2023
  0a18450

• appreciation

  lucidrains

  committedMar 17, 2023
  caf4755

• Merge pull request #40 from conceptofmind/main

  lucidrains

  committedMar 17, 2023
  77c1016

Commits on Mar 13, 2023

• 0.0.68

  lucidrains

  committedMar 13, 2023
  d61b698

• fix mask logic in rlhf trainer when eos_token is designated

  lucidrains

  committedMar 13, 2023
  1a23215

• appreciation

  lucidrains

  committedMar 13, 2023
  6e3a60d

Commits on Mar 12, 2023

• fix silly error in masked kl div loss, thanks to @taynoel84

  lucidrains

  committedMar 12, 2023
  2bb47d4

• allow for setting critic palm from rlhftrainer

  lucidrains

  committedMar 12, 2023
  82fa3d0

Commits on Mar 1, 2023

• seems promising

  lucidrains

  committedMar 1, 2023
  12f3467

Commits on Feb 22, 2023

• fix an error with the way action log prob is collected during the episode rollouts, addressing #31 and thanks to @kisseternity

  lucidrains

  committedFeb 22, 2023
  a159310

Commits on Feb 15, 2023

• default qk rmsnorm to off and add lion

  lucidrains

  committedFeb 15, 2023
  bfcffe7

Commits on Feb 13, 2023

• bet on a new technique circulating within Brain for stabilizing attention

  lucidrains

  committedFeb 13, 2023
  792d82f

• there may be an off-by-one issue, as the action logits are taken starting from the last prompt token

  lucidrains

  committedFeb 13, 2023
  6f9dedd

Commits on Jan 26, 2023

• remove encoder / decoder, add prompt lengths

  lucidrains

  committedJan 26, 2023
  9b3caf5

Commits on Jan 10, 2023

• some reorganization, to ready for encoder / decoder

  lucidrains

  committedJan 10, 2023
  cd714f7

Commits on Jan 8, 2023

• cleanup

  lucidrains

  committedJan 8, 2023
  126dbc7

• add diagram

  lucidrains

  committedJan 8, 2023
  4d9d1b3

• fix masking logic when using palm as encoder

  lucidrains

  committedJan 8, 2023
  6e1baf6

Commits on Jan 7, 2023

• make sure key padding mask is in effect if training reward model as encoder

  lucidrains

  committedJan 7, 2023
  4cac0e6

• turn off xpos if using palm as encoder

  lucidrains

  committedJan 7, 2023
  4fd9455

Commits on Jan 3, 2023

• fix pooled critic values during generation, thanks to @Nightbringers

  lucidrains

  committedJan 3, 2023
  a0b9774

Commits on Dec 31, 2022

• note

  lucidrains

  committedDec 31, 2022
  c640d1e

Commits on Dec 30, 2022

• dry a little

  lucidrains

  committedDec 30, 2022
  f687339

• todo

  lucidrains

  committedDec 30, 2022
  44acd46

Commits on Dec 29, 2022

• able to override lora R value when adding a new finetuning scope

  lucidrains

  committedDec 29, 2022
  0e45bac

• make sure xpos scale base value is customizable from palm init

  lucidrains

  committedDec 29, 2022
  6126cde

Pagination

PreviousNext