# OpenAI Grok Curve Experiments

## Paper

This is the code for the paper [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://arxiv.org/abs/2201.02177) by Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra

## Installation and Training

```bash
pip install -e .
./scripts/train.py
```


## grok-1

特斯拉24年最新开源 grok 模型，上面是 openai 21年开源的：

```
git clone https://github.com/xai-org/grok-1
pip install -r requirements.txt
python run.py


git clone https://github.com/xai-org/grok-1.git && cd grok-1
pip install huggingface_hub[hf_transfer]
huggingface-cli download xai-org/grok-1 --repo-type model --include ckpt-0/* --local-dir checkpoints --local-dir-use-symlinks False

```
使用 huggingface 下载模型权重，包含非常多的权重文件 https://huggingface.co/xai-org/grok-1/tree/main