Initial commit: SheepOp LLM - Transformer-based language model implementation

Browse Source

- Complete transformer implementation from scratch
- Training pipeline with gradient accumulation and mixed precision
- Optimized inference with KV caching
- Multi-format data processing (PDFs, images, code, text)
- Comprehensive documentation
- Apache 2.0 license
- Example training plots included in docs/images/

...

This commit is contained in:

Carlos Gutierrez

2025-11-06 22:07:41 -05:00

commit 3d2da94ce2

60 changed files with 25153 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

BIN

docs/images/loss_by_epoch.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 26 KiB

BIN

docs/images/training_curve.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 73 KiB