Llama 3.1 8B Instruct

ChatInput: TextReleased Jul 23, 2024

Built by Meta · United States · llama.com

Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 8B instruct-tuned version is fast and efficient. It has demonstrated strong performance compared to...

How to use

model: "meta-llama/llama-3.1-8b-instruct"

Cap quality here and save

Hard questions go to this model, easy ones drop to something cheaper. Set a savings target (0–0.8) and the router tunes itself until realized savings converge on it.

{
  "model": "celon/auto",
  "celon": {
    "anchor": "meta-llama/llama-3.1-8b-instruct",
    "savings_target": 0.4
  }
}

Naming a model directly gets you that model — savings controls don't apply.

Context

16K

Max output

131K

Input $/1M

$0.02

Output $/1M

$0.05

Features

—

Serving providers

NovitaW&B Inference

Providers

Price, latency, and uptime per provider serving this model. Celon routes each request to the fastest and cheapest of them. Click a row for regions and data policies.

Loading provider metrics…

Performance

Benchmark quality scores, and measured speed per provider.

Quality

Epoch Capabilities IndexComposite over 50+ benchmarks

115.5

GPQA DiamondPhD-level science questions

25.9%

AIMECompetition mathematics

2.5%

SciCodeScientific code generation

13.2%

At each model's best reasoning effort. Benchmarks it wasn't run on are omitted.

Speed

Loading provider metrics…