Commits

Branch selector

 main

User selector

All users

All time

Commit History

Commits on Jan 14, 2024

• readme

  lucidrains

  committedJan 14, 2024
  6b02ee3

Commits on Apr 5, 2023

• fix a bug with the final norm in palm, thanks to @conceptofmind and @dmahan93

  lucidrains

  committedApr 5, 2023
  e624afd

Commits on Mar 23, 2023

• address #41 , be faithful to the paper

  lucidrains

  committedMar 23, 2023
  1487fd4

Commits on Mar 22, 2023

• old action log probs should be the true distribution in the kl div loss, addressing #43

  lucidrains

  committedMar 22, 2023
  d4faf48

Commits on Mar 17, 2023

• comment

  lucidrains

  committedMar 17, 2023
  ad001d8

• flash attention sdp context config only needs to be done once

  lucidrains

  committedMar 17, 2023
  6c8028a

• draw attention

  lucidrains

  committedMar 17, 2023
  734afb3

• fix assert

  lucidrains

  committedMar 17, 2023
  570ebb7

• project management

  lucidrains

  committedMar 17, 2023
  4ae1f6d

• add ability to use flash attention if using pytorch 2.0, thanks to @conceptofmind for the initial PR!

  lucidrains

  committedMar 17, 2023
  0a18450

• appreciation

  lucidrains

  committedMar 17, 2023
  caf4755

• Merge pull request #40 from conceptofmind/main

  lucidrains

  committedMar 17, 2023
  77c1016

Commits on Mar 16, 2023

• Use standard pytorch instead of einops

  conceptofmind

  committedMar 16, 2023
  81178c3

• Add extra logic. Mask as recommended by Lucid

  conceptofmind

  committedMar 16, 2023
  1b68111

• Add logic for PyTorch version and GPU. Add expand mask

  conceptofmind

  committedMar 16, 2023
  aea58d7

• Add Flash Attention citation

  conceptofmind

  committedMar 16, 2023
  808172d

• Update to use correct syntax

  conceptofmind

  committedMar 16, 2023
  9af089b

• Merge branch 'main' of https://github.com/conceptofmind/PaLM-rlhf-pytorch into main

  conceptofmind

  committedMar 16, 2023
  2344108

• update to ensure only Flash Attention is active

  conceptofmind

  committedMar 16, 2023
  0370450

Commits on Mar 15, 2023

• Delete flash_attn_triton.py

  conceptofmind

  committedMar 15, 2023
  efca8bf

• Add pytorch 2.0 flash attention

  conceptofmind

  committedMar 15, 2023
  3d3cd12

Commits on Mar 13, 2023

• Add Tri Dao Flash Attn Triton

  conceptofmind

  committedMar 13, 2023
  09e9a4f

• 0.0.68

  lucidrains

  committedMar 13, 2023
  d61b698

• fix mask logic in rlhf trainer when eos_token is designated

  lucidrains

  committedMar 13, 2023
  1a23215

• appreciation

  lucidrains

  committedMar 13, 2023
  6e3a60d

Commits on Mar 12, 2023

• fix silly error in masked kl div loss, thanks to @taynoel84

  lucidrains

  committedMar 12, 2023
  2bb47d4

• allow for setting critic palm from rlhftrainer

  lucidrains

  committedMar 12, 2023
  82fa3d0

Commits on Mar 1, 2023

• seems promising

  lucidrains

  committedMar 1, 2023
  12f3467

Commits on Feb 22, 2023

• fix an error with the way action log prob is collected during the episode rollouts, addressing #31 and thanks to @kisseternity

  lucidrains

  committedFeb 22, 2023
  a159310

Commits on Feb 15, 2023

• default qk rmsnorm to off and add lion

  lucidrains

  committedFeb 15, 2023
  bfcffe7

Commits on Feb 13, 2023

• bet on a new technique circulating within Brain for stabilizing attention

  lucidrains

  committedFeb 13, 2023
  792d82f

• there may be an off-by-one issue, as the action logits are taken starting from the last prompt token

  lucidrains

  committedFeb 13, 2023
  6f9dedd

Commits on Jan 26, 2023

• remove encoder / decoder, add prompt lengths

  lucidrains

  committedJan 26, 2023
  9b3caf5

Commits on Jan 10, 2023

• some reorganization, to ready for encoder / decoder

  lucidrains

  committedJan 10, 2023
  cd714f7

Commits on Jan 8, 2023

• cleanup

  lucidrains

  committedJan 8, 2023
  126dbc7

Pagination

PreviousNext