Content-Length: 253901 | pFad | http://github.com/Andras7/gpt2-pytorch

6D GitHub - Andras7/gpt2-pytorch: Extremely simple and understandable GPT2 implementation with minor tweaks
Skip to content

Extremely simple and understandable GPT2 implementation with minor tweaks

Notifications You must be signed in to change notification settings

Andras7/gpt2-pytorch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GPT2 Pytorch

Extremely simple and understandable GPT2 implementation with minor tweaks.

Advantages

  • You can train even the subword tokenizer, good for non-English languages.
  • Fast optimized code, enough a single GTX 2080ti card
  • Easy to understand, solid code
  • Easy to extend for new experiments

Supported extra features

  • Lamb optimizer
  • Mixed precision training, the important layers still remained in fp32.
  • sin, cos positional encoding

About

Extremely simple and understandable GPT2 implementation with minor tweaks

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages









ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: http://github.com/Andras7/gpt2-pytorch

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy