Disclosure: I'm 于侃 (Yu Kan), the developer behind NovAI, an API gateway for Chinese AI models. Every number in this article was measured by running the test script below. I tested DeepSeek's official API separately to verify the gateway adds zero overhead. The test prompts and full results are open-sourced — links at the bottom.
Everyone publishing API pricing comparisons is either cherry-picking prompts or quoting the official pricing page without testing. I got tired of it, so I ran the same 50 real-world prompts through 8 different AI APIs and tracked every single token and every cent.
Here's what I actually found — including a few results that surprised me.
TL;DR
| Rank | API | Cost (50 prompts) | Quality (1-5) |
|---|---|---|---|
| 1 | NovAI / Qwen-Plus | $0.011 | 3.9 |
| 2 | NovAI / DeepSeek V4 Flash | $0.019 | 4.1 |
| 3 | DeepSeek (official) V4 Flash | $0.019 | 4.1 |
| 4 | Google Gemini 3.1 Flash | $0.027 | 3.8 |
| 5 | NovAI / Doubao Seed 2.0 Lite | $0.034 | 3.7 |
| 6 | OpenAI GPT-5 | $0.142 | 4.6 |
| 7 | OpenAI GPT-4o | $0.198 | 4.3 |
| 8 | Anthropic Claude Sonnet 4.6 | $0.287 | 4.7 |
The cheapest option was 26× cheaper than the most expensive one. The quality gap between #1 and #8 was 0.8 points on a 5-point scale.
What I tested
I used 50 prompts across 5 categories:
- 10 coding tasks (debug Python, write SQL, explain Rust errors)
- 10 reasoning tasks (math word problems, logic puzzles)
- 10 writing tasks (blog intros, email drafts, documentation)
- 10 multilingual tasks (English↔Chinese↔Japanese translation)
- 10 long-context tasks (summarize 8K-token documents)
For each prompt, I measured:
- Input tokens (exact count from API response)
- Output tokens (exact count)
- Wall-clock latency (time to first token + total time)
- Quality (manual review on a 1-5 scale)
- Cost (calculated from official pricing, not marketing claims)
The 8 APIs I tested
Pricing verified July 2026 from official pricing pages. Prices change — always check the source.
| API | Model used | Input $/1M | Output $/1M | Context |
|---|---|---|---|---|
| OpenAI | GPT-5 | $1.25 | $10.00 | 128K |
| OpenAI | GPT-4o | $2.50 | $10.00 | 128K |
| Anthropic | Claude Sonnet 4.6 | $3.00 | $15.00 | 200K |
| Gemini 3.1 Flash | $0.15 | $0.60 | 1M | |
| DeepSeek (official) | DeepSeek V4 Flash | $0.14 | $0.28 | 128K |
| NovAI | DeepSeek V4 Flash | $0.14 | $0.28 | 128K |
| NovAI | Qwen-Plus | $0.08 | $0.32 | 128K |
| NovAI | Doubao Seed 2.0 Lite | $0.25 | $0.80 | 256K |
I tested both DeepSeek's official API and DeepSeek via NovAI to check whether the gateway adds overhead (spoiler: it doesn't — identical token counts, <50ms latency difference).
The results: total cost for all 50 prompts
| Rank | API | Total cost (50 prompts) | Avg quality (1-5) |
|---|---|---|---|
| 1 | NovAI / Qwen-Plus | $0.011 | 3.9 |
| 2 | NovAI / DeepSeek V4 Flash | $0.019 | 4.1 |
| 3 | DeepSeek (official) V4 Flash | $0.019 | 4.1 |
| 4 | Google Gemini 3.1 Flash | $0.027 | 3.8 |
| 5 | NovAI / Doubao Seed 2.0 Lite | $0.034 | 3.7 |
| 6 | OpenAI GPT-5 | $0.142 | 4.6 |
| 7 | OpenAI GPT-4o | $0.198 | 4.3 |
| 8 | Anthropic Claude Sonnet 4.6 | $0.287 | 4.7 |
The cheapest option was 26× cheaper than the most expensive one. And the quality difference between #1 (Qwen-Plus at $0.011) and #8 (Claude at $0.287) was 0.8 points on a 5-point scale.
What surprised me
1. Qwen-Plus is absurdly cheap and genuinely good.
At $0.08/1M input and $0.32/1M output, Qwen-Plus was the cheapest model I tested — and it scored 3.9/5 on quality, beating Gemini 3.1 Flash (3.8/5) which costs 2.4× more. For coding tasks specifically, it scored 4.2/5, matching DeepSeek V4 Flash.
2. DeepSeek via NovAI is identical to DeepSeek's official API.
I was skeptical that a gateway would add latency or modify token counts. I tested the exact same prompts through both. Result:
- Token counts: identical (0% difference)
- Latency: NovAI averaged 42ms slower (640ms vs 598ms for first token) — negligible
- Cost: identical (same per-token pricing)
The gateway adds zero overhead. This matters because NovAI gives you access to 40+ Chinese models through one API key, while DeepSeek's official API only gives you DeepSeek.
3. The "quality gap" is smaller than pricing pages suggest.
Look at pricing tables and you'd think there's a massive quality difference between GPT-5 ($1.25/$10) and DeepSeek V4 Flash ($0.14/$0.28). In practice, for the 50 prompts I tested:
- GPT-5 scored 4.6/5
- DeepSeek V4 Flash scored 4.1/5
- The gap is 0.5 points — noticeable but not "10× better"
- The price gap is 10× on input, 36× on output
For 90% of production workloads (chatbots, content generation, code completion, document processing), the 0.5-point quality difference is invisible to end users. The 10-36× price difference is very visible to your CFO.
4. Claude is the best, but only marginally — and at a huge premium.
Claude Sonnet 4.6 scored 4.7/5 — the highest quality. But it cost $0.287 for 50 prompts vs $0.019 for DeepSeek V4 Flash (which scored 4.1/5). That's 15× more expensive for 0.6 points of quality. Claude makes sense for legal documents and medical summaries. For everything else, it's overkill.
Where each API actually wins
Use OpenAI GPT-5 when:
- You need the absolute best reasoning quality
- You're building a customer-facing product where quality is the differentiator
- Your margins can absorb $1.25/$10 per million tokens
Use DeepSeek V4 Flash when:
- You need GPT-4-level quality at 1/10th the price
- You're doing coding tasks, reasoning, or general chat
- You want the best price-to-quality ratio in 2026
Use Qwen-Plus when:
- Cost is the #1 priority
- You're doing multilingual work (especially Chinese/English)
- You can tolerate slightly lower quality on complex reasoning
Use Claude Sonnet 4.6 when:
- Quality is non-negotiable (legal, medical, compliance)
- You're processing long documents (200K context)
- Budget is not a constraint
The code I used to test
Here's the Python script I used. It's simple, reproducible, and you can run it yourself.
from openai import OpenAI
import time, json
# Test with NovAI (works identically with any OpenAI-compatible API)
client = OpenAI(
base_url="https://aiapi-pro.com/v1",
api_key="nvai-your-key-here"
)
prompts = [
"Write a Python function to merge two sorted linked lists",
"Explain the difference between mutex and semaphore",
"Translate to Chinese: 'The quick brown fox jumps over the lazy dog'",
# ... 47 more prompts (full list in the GitHub repo)
]
results = []
for prompt in prompts:
start = time.time()
response = client.chat.completions.create(
model="deepseek-v4-flash", # or qwen-plus, doubao-seed-2.0-lite
messages=[{"role": "user", "content": prompt}],
)
latency = time.time() - start
results.append({
"prompt": prompt,
"input_tokens": response.usage.prompt_tokens,
"output_tokens": response.usage.completion_tokens,
"latency_ms": round(latency * 1000),
"response": response.choices[0].message.content,
})
# Calculate cost
INPUT_PRICE = 0.14 # $/1M tokens for deepseek-v4-flash
OUTPUT_PRICE = 0.28
total_input = sum(r["input_tokens"] for r in results)
total_output = sum(r["output_tokens"] for r in results)
total_cost = (total_input * INPUT_PRICE + total_output * OUTPUT_PRICE) / 1_000_000
print(f"Total cost for {len(prompts)} prompts: ${total_cost:.4f}")
print(f"Avg latency: {sum(r['latency_ms'] for r in results)/len(results):.0f}ms")
Enter fullscreen mode Exit fullscreen mode
You can swap the base_url and api_key to test any provider. The OpenAI Python SDK works with any OpenAI-compatible API — that's the beauty of the standard.
How to reproduce this yourself
If you want to verify these numbers (and you should — don't trust random blog posts), here's how:
Get API keys from each provider. OpenAI, Anthropic, Google, DeepSeek all offer free credits. For NovAI, you can register at aiapi-pro.com/register and get free credits — no credit card needed.
Run the same 50 prompts. I've open-sourced my test prompts and script at github.com/vvvvking/ai-api-cost-comparison.
-
Calculate costs using official pricing pages. Don't use my numbers — use the official pricing pages:
The honest caveats
50 prompts is a small sample. Your workload will differ. Run the test on your own prompts.
Quality scoring is subjective. I scored manually. You might disagree with my 4.1 vs 4.6 ratings. The point is the relative difference, not the absolute numbers.
I didn't test image/video generation. This is text-only. Image and video generation has completely different economics — and Chinese models have a mandatory
AI生成watermark that you should know about before using them. (Disclosure: I work on NovAI, which is a gateway for these models. The watermark is a legal requirement in China and cannot be removed. See aiapi-pro.com/watermark-notice.)Latency varies by region. I tested from Southeast Asia. If you're in the US or Europe, add 50-150ms to all latency numbers. DeepSeek and NovAI have servers in Asia, so they're naturally faster for Asian users.
I work on NovAI. I've been transparent about this throughout the article. But the numbers are real and reproducible — I tested DeepSeek's official API separately and got identical results. The gateway overhead is negligible. Run the test yourself if you're skeptical.
The bottom line
If you're paying OpenAI prices for production workloads where GPT-4-level quality is sufficient, you're burning money. The math is simple:
- GPT-5: $1.25/$10 per million tokens
- DeepSeek V4 Flash: $0.14/$0.28 per million tokens
- Same quality tier for 10-36× less money
For a typical SaaS app doing 1M input + 500K output tokens per day:
- GPT-5: $6.25/day = $2,281/year
- DeepSeek V4 Flash: $0.28/day = $102/year
That's $2,179/year in savings, for a model that scores 4.1/5 vs 4.6/5 on quality. For most apps, users won't notice the difference. Your CFO will notice the $2,179.
The code to switch is one line:
client = OpenAI(
base_url="https://aiapi-pro.com/v1", # was: https://api.openai.com/v1
api_key="nvai-your-key-here" # was: sk-your-openai-key
)
Enter fullscreen mode Exit fullscreen mode
Test it yourself. The free credits are enough to run 50 prompts and verify every number in this article.
Disclosure: I'm 于侃 (Yu Kan), the developer behind NovAI (aiapi-pro.com). I wrote this article because I was frustrated by the lack of honest, reproducible API cost comparisons. Every number in this article was measured by running the test script above. The test prompts and full results are open-sourced at github.com/vvvvking/ai-api-cost-comparison. If you find an error, open an issue.
0 Comments
Log in to join the conversation.No comments yet. Be the first to share your thoughts.