World Programming Society
Write Log in
Home Events

Channels

General Discussion Web Development Backend DevOps AI Career & Hiring TypeScript Rust Python Go Cloud AWS Azure GCP C/C++ FPGA Assembly

World Programming Society · 2026 Copyright

AI

评估过程和获奖者选择中存在不一致的证据

设计高质量的基准测试，超越单纯的记忆能力评估，以真正衡量前沿模型的推理、行动和判断能力。

Hacker News Best Hacker News Best · Jul 17, 2026
0
Evidence of inconsistencies in evaluation process and selection of winners
Read the original source
#Performance
Hacker News Best

Publisher

Originally by twerkmeister

0 Comments

Log in to join the conversation.

No comments yet. Be the first to share your thoughts.

More in AI

Platform engineering’s new job: serving environments at agent speed

Platform engineering’s new job: serving environments at agent speed

Jul 18 · 8m read

Pinecone Introduces Nexus Engine for Compiling Business Context into Structured Data for AI Agents

Pinecone Introduces Nexus Engine for Compiling Business Context into Structured Data for AI Agents

Jul 18 · 3m read

I trust Claude for everything. This test made me rethink that.

I trust Claude for everything. This test made me rethink that.

Jul 18 · 7m read

Controlling Reasoning Effort in LLMs

Controlling Reasoning Effort in LLMs

Jul 18 · 31m read

Why do AI company logos look like buttholes?

为什么 AI 公司的 Logo 看起来像屁眼？

Jul 18 · 7m read

Musk open-sourced Grok Build to fight Anthropic. Anthropic pays him $1.25 billion a month.

Musk open-sourced Grok Build to fight Anthropic. Anthropic pays him $1.25 billion a month.

Jul 18 · 5m read

How does Pangram work?

How does Pangram work?

Jul 18 · 7m read

A

Claude make Fable 5 permanent

Jul 18 · 1m read

Kaiser nurses say AI, workplace surveillance are making their jobs, care worse

Kaiser nurses say AI, workplace surveillance are making their jobs, care worse

Jul 17 · 17m read

In-House LLM Serving at Netflix

In-House LLM Serving at Netflix

Jul 17 · 12m read

Faulty Towers, vibe sickness, and the vibe bobsled

故障塔、氛围病与氛围雪橇

Jul 17 · 13m read

Tech builds on AI. Finance protects the margin.

Tech builds on AI. Finance protects the margin.

Jul 17 · 8m read

The Protocol Wars in Factories, Nvidia's Thor Modules for Robotics and Edge AI, 'Mind of the Engineer' survey: Embedded Week Insights

The Protocol Wars in Factories, Nvidia's Thor Modules for Robotics and Edge AI, 'Mind of the Engineer' survey: Embedded Week Insights

Jul 17 · 1m read

OpenAI is being sued for stealing, again…

OpenAI is being sued for stealing, again…

Jul 17 · 1m read

Following the questions where they lead

Following the questions where they lead

Jul 17 · 7m read

New NadMesh Botnet Hunts Exposed AI Services for Cloud Keys and Kubernetes Tokens

New NadMesh Botnet Hunts Exposed AI Services for Cloud Keys and Kubernetes Tokens

Jul 17 · 6m read

Building a soccer coaching app on Databricks

在 Databricks 上构建足球教练应用

Jul 17 · 9m read

The cost of saying yes has changed

说“是”的成本已经改变

Jul 17 · 6m read

Kimi K3 tops Arena’s coding leaderboard — and it’s open-weight

Kimi K3 登顶 Arena 编程排行榜——且为开源权重模型

Jul 17 · 4m read

1Password’s new browser integration for Claude changes how AI uses your credentials

1Password 的全新 Claude 浏览器集成改变了 AI 使用凭证的方式

Jul 17 · 4m read

What a $20 coding subscription actually buys

What a $20 coding subscription actually buys

Jul 17 · 9m read

13 hands-on demos to build on Gemini Enterprise Agent Platform

13 hands-on demos to build on Gemini Enterprise Agent Platform

Jul 17 · 7m read

Fine-tune video and image models at scale with NVIDIA NeMo Automodel and 🤗 Diffusers

Fine-tune video and image models at scale with NVIDIA NeMo Automodel and 🤗 Diffusers

Jul 17 · 9m read

“They’re dead if they don’t offer this”: DoorDash’s CLI for agents may be out of necessity

“如果不提供这个，他们就死定了”：DoorDash 为代理推出的 CLI 可能是出于必要

Jul 17 · 4m read

A

Mozilla: The state of open source AI

Jul 17 · 17m read

Quoting Kimi K3

引用 Kimi K3

Jul 17 · 1m read

How Far Behind the Frontier are Leading Open Weight Models on Cyber?

How Far Behind the Frontier are Leading Open Weight Models on Cyber?

Jul 17 · 8m read

Presentation: From OTEL to SLMs: Distilling Frontier Model Behaviour from Production Telemetry

Presentation: From OTEL to SLMs: Distilling Frontier Model Behaviour from Production Telemetry

Jul 17 · 30m read

Meta’s Spark Muse 1.1 is now available on Databricks, fully governed by Unity AI Gateway

Meta’s Spark Muse 1.1 is now available on Databricks, fully governed by Unity AI Gateway

Jul 17 · 7m read

Why every AI agent decision needs a receipt

Why every AI agent decision needs a receipt

Jul 17 · 6m read

We are Changing our Developer Productivity Experiment Design

We are Changing our Developer Productivity Experiment Design

Jul 17 · 8m read

Bitcoin faces fresh headwinds as China’s Kimi beats Claude, GPT in coding benchmark

比特币面临新压力：中国 Kimi 在编码基准测试中击败 Claude 和 GPT

Jul 17 · 4m read

A

LLM cliché highlighter

Jul 17 · 1m read

Agentic AI Security: Defending Against Prompt Injection and Tool Misuse

Agentic AI 安全：防御提示注入与工具滥用

Jul 17 · 5m read

Arm and Google offer a smarter option to run agentic AI workloads

Arm and Google offer a smarter option to run agentic AI workloads

Jul 17 · 5m read

Cloud Native Infrastructure Emerges as the Foundation for Trustworthy Agentic AI

Cloud Native Infrastructure Emerges as the Foundation for Trustworthy Agentic AI

Jul 17 · 3m read

AI frenzy losing steam leaves bitcoin less volatile than South Korean stocks

AI frenzy losing steam leaves bitcoin less volatile than South Korean stocks

Jul 17 · 4m read

E.U. Orders Google to Open Android Mic, Camera and Screen to Rival AI Assistants

欧盟命令谷歌向竞争对手 AI 助手开放 Android 麦克风、摄像头和屏幕访问权限

Jul 17 · 9m read

The Race to Field Military Autonomy Is On, Can Trusted Information Infrastructure Keep Pace?

The Race to Field Military Autonomy Is On, Can Trusted Information Infrastructure Keep Pace?

Jul 17 · 5m read

PhpStorm 2026.2 is Now Out

PhpStorm 2026.2 现已发布

Jul 17 · 10m read

Guide to AI Tokenomics: Eleven Principles for Token Efficient Software Engineering

Guide to AI Tokenomics: Eleven Principles for Token Efficient Software Engineering

Jul 17 · 4m read

QCon AI Boston: Production AI Moves Beyond Prompts to Platforms, Harnesses, and Evals

QCon AI Boston：生产级 AI 从提示转向平台、Harness 与评估

Jul 17 · 4m read

The human-in-the-loop is tired

The human-in-the-loop is tired

Jul 17 · 11m read

GLM 5.2 is 35% off via Novita on AI Gateway

GLM 5.2 is 35% off via Novita on AI Gateway

Jul 17 · 1m read

Chat SDK adds native Slack agent support

Chat SDK adds native Slack agent support

Jul 17 · 2m read

Vocalizer: Local Text-to-Speech for PHP

Vocalizer: Local Text-to-Speech for PHP

Jul 16 · 4m read

Your AI is ready. Your data foundation probably isn’t

Your AI is ready. Your data foundation probably isn’t

Jul 16 · 7m read

From experiment to insight: how Dotmatics Luma and Databricks make AI-ready science a reality

From experiment to insight: how Dotmatics Luma and Databricks make AI-ready science a reality

Jul 16 · 7m read

Mathematics of Data Science

Mathematics of Data Science

Jul 16 · 1m read

Kimi K3, and what we can still learn from the pelican benchmark

Kimi K3, and what we can still learn from the pelican benchmark

Jul 16 · 6m read

LM Studio Bionic: the AI agent for open models

LM Studio Bionic: the AI agent for open models

Jul 16 · 4m read

$100 AI Music Video: Claude Fable 5 vs. GPT-5.6 Sol

$100 AI Music Video: Claude Fable 5 vs. GPT-5.6 Sol

Jul 16 · 7m read

Using AI to build your own software

Using AI to build your own software

Jul 16 · 2m read

Google is a Leader and positioned furthest in Vision and highest in Execution in the 2026 Gartner® Magic Quadrant™ for Conversational AI Platforms

Google is a Leader and positioned furthest in Vision and highest in Execution in the 2026 Gartner® Magic Quadrant™ for Conversational AI Platforms

Jul 16 · 6m read

We're Going to Make Out Like Bandits

We're Going to Make Out Like Bandits

Jul 16 · 4m read

Unified context: The missing layer for enterprise AI coworkers

Unified context: The missing layer for enterprise AI coworkers

Jul 16 · 7m read

What happens in the milliseconds after you tap pay

What happens in the milliseconds after you tap pay

Jul 16 · 12m read

The skills gap behind agentic AI — and how Databricks is closing it with a new context engineer certification and agent trainings

The skills gap behind agentic AI — and how Databricks is closing it with a new context engineer certification and agent trainings

Jul 16 · 4m read

AI hasn’t shifted the bottleneck from coding to code review

AI hasn’t shifted the bottleneck from coding to code review

Jul 16 · 5m read

Quoting Thibault Sottiaux

Quoting Thibault Sottiaux

Jul 16 · 1m read

Open Protocols with the Strands Agents SDK

Open Protocols with the Strands Agents SDK

Jul 16 · 24m read

The Pulse: What can we learn from Bun’s rapid Rust rewrite with AI?

The Pulse: What can we learn from Bun’s rapid Rust rewrite with AI?

Jul 16 · 11m read

Detecting LLM-Generated Texts with “Classical” Machine Learning

Detecting LLM-Generated Texts with “Classical” Machine Learning

Jul 16 · 13m read

Decoy Font

Decoy Font

Jul 16 · 5m read

Visa backs Open USD with new stablecoin platform as Circle faces fresh competition

Visa backs Open USD with new stablecoin platform as Circle faces fresh competition

Jul 16 · 3m read

NotebookLM is now Gemini Notebook

NotebookLM is now Gemini Notebook

Jul 16 · 2m read

Integrating Context-Aware Video AI Agents Into Enterprise Workflows

Integrating Context-Aware Video AI Agents Into Enterprise Workflows

Jul 16 · 14m read

NVIDIA Nemotron 3 Embed Ranks #1 Overall on RTEB, Advancing Agentic Retrieval

NVIDIA Nemotron 3 Embed Ranks #1 Overall on RTEB, Advancing Agentic Retrieval

Jul 16 · 10m read

Create, edit and star in videos with two Google Vids updates

Create, edit and star in videos with two Google Vids updates

Jul 16 · 3m read

Connect more of your apps to Search

Connect more of your apps to Search

Jul 16 · 2m read

CNCF Strengthens Partnership with Broadcom as a Platinum Member to Advance AI-Ready Cloud Native Infrastructure

CNCF Strengthens Partnership with Broadcom as a Platinum Member to Advance AI-Ready Cloud Native Infrastructure

Jul 16 · 3m read

What 10 autonomous film crews taught us about agent teamwork

What 10 autonomous film crews taught us about agent teamwork

Jul 16 · 7m read

Three lessons in accelerating foundation model upgrades

Three lessons in accelerating foundation model upgrades

Jul 16 · 4m read

Cloud CISO Perspectives: How AI leverages deep context as the defender’s advantage

Cloud CISO Perspectives: How AI leverages deep context as the defender’s advantage

Jul 16 · 10m read

Why smarter AI caching sometimes makes everything slower

Why smarter AI caching sometimes makes everything slower

Jul 16 · 16m read

Energy IPOs surge as investors hunt for ways to play AI boom

Energy IPOs surge as investors hunt for ways to play AI boom

Jul 16 · 1m read

Inkling: Our open-weights model

Inkling: Our open-weights model

Jul 16 · 4m read

Linux creator Linus Torvalds tells AI haters to walk away from Linux, or go fork it 

Linux creator Linus Torvalds tells AI haters to walk away from Linux, or go fork it 

Jul 16 · 6m read

How to build long-running MCP tools on Azure Functions

How to build long-running MCP tools on Azure Functions

Jul 16 · 4m read

30 to 70 PRs a Day: How We Managed to Not Wreck Our Systems

30 to 70 PRs a Day: How We Managed to Not Wreck Our Systems

Jul 16 · 26m read

AI Amplifies Your Existing Practices: Lessons from Our Shift to an AI-First Strategy

AI Amplifies Your Existing Practices: Lessons from Our Shift to an AI-First Strategy

Jul 16 · 21m read

OpenAI’s GPT-Red automates prompt injection testing to harden AI agents

OpenAI’s GPT-Red automates prompt injection testing to harden AI agents

Jul 16 · 5m read

Crypto for Advisors: Strengthening defenses against AI fraud

Crypto for Advisors: Strengthening defenses against AI fraud

Jul 16 · 5m read

DataGrip 2026.2: AI Agent Skills, MCP Tools and CLI Commands for Data Source Management, Bundled JDBC Drivers, and Improved Session Control

DataGrip 2026.2: AI Agent Skills, MCP Tools and CLI Commands for Data Source Management, Bundled JDBC Drivers, and Improved Session Control

Jul 16 · 8m read

WebStorm 2026.2: TypeScript 7 Support, GitHub Copilot Integration, Agent Skills, and More

WebStorm 2026.2: TypeScript 7 Support, GitHub Copilot Integration, Agent Skills, and More

Jul 16 · 6m read

A

Kimi K3: Open Frontier Intelligence

Jul 16 · 13m read

Kimi K3 is now live

Kimi K3 is now live

Jul 16 · 1m read

Protecting Privacy in an AI Era

Protecting Privacy in an AI Era

Jul 16 · 1m read

Demystifying AI Exploits: A Blueprint for AI-Assisted Vulnerability Management

Demystifying AI Exploits: A Blueprint for AI-Assisted Vulnerability Management

Jul 16 · 24m read

README, not

README, not

Jul 16 · 2m read

The LLM Critics Are Right. I Use LLMs Anyway

The LLM Critics Are Right. I Use LLMs Anyway

Jul 16 · 19m read

Ledger wants AI agents to manage crypto without holding your keys

Ledger wants AI agents to manage crypto without holding your keys

Jul 16 · 2m read

The Developer Has Changed. So Should Developer Conferences

The Developer Has Changed. So Should Developer Conferences

Jul 16 · 5m read

AI Agents Explained: How to Build with Them Safely

AI Agents Explained: How to Build with Them Safely

Jul 16 · 10m read

“There are no laws, only suggestions”: What AI agents do with your instructions

“There are no laws, only suggestions”: What AI agents do with your instructions

Jul 16 · 9m read

Run a Local AI Model with Ollama in 15 Minutes

Run a Local AI Model with Ollama in 15 Minutes

Jul 16 · 7m read

Code in Space: Redefining Tech Creation with AI and XR

Code in Space: Redefining Tech Creation with AI and XR

Jul 16 · 20m read

Newer Models, Same Advantage

Newer Models, Same Advantage

Jul 16 · 11m read

New Agent Data Injection Attack Can Make AI Agents Misclick or Run Attacker Commands

New Agent Data Injection Attack Can Make AI Agents Misclick or Run Attacker Commands

Jul 16 · 8m read

Securing AI at Enterprise Scale: The Google Kubernetes Engine Blueprint

Securing AI at Enterprise Scale: The Google Kubernetes Engine Blueprint

Jul 16 · 4m read

Loading more…