Designing a Community Skill for AWS Transform Custom: AWS Glue 5.0 Upgrade Readiness

Designing a Community Skill for AWS Transform Custom: AWS Glue 5.0 Upgrade Readiness

TL;DR

I designed a proposed AWS Transform Custom community skill that prepares Glue 2.0, 3.0, and 4.0 repositories for Glue 5.0. It separates safe mechanical transformations from changes that require human evidence, generates a migration report, and preserves already-compatible files unchanged. Because I didn't have live atx access, the benchmarks in this post are explicitly labeled manually simulated, not agent-executed. The proposal is open as issue #75 — not yet merged, not yet a pull request.

The missing data-engineering transformation

AWS Transform Custom can apply agent-driven code transformations across a single repository — or thousands of them at once, via AWS Batch and Fargate. As of July 30, 2026, its public sample repository, aws-samples/aws-transform-custom-samples, contained three community-contributed transformations: an EKS version-upgrade-readiness skill, a JBoss-to-Spring-Boot migration, and a Kubernetes readiness migration. None of them touched data engineering.

Given that most of my day-to-day work sits across AWS data engineering, Databricks, and Delta Lake, that gap was the obvious thing to fill.

What an AWS Transform Custom "skill" looks like

Before writing anything, I studied the deepest existing example, jboss-to-springboot, since the pattern it establishes is effectively an unwritten spec for the other two skills as well:

• README.md — the problem, what the skill does, and how to invoke it via the atx CLI. This is also where the repo draws a clear line: these are readiness transformations. They modify repository artifacts — code and infrastructure-as-code — but they don't deploy jobs, call AWS APIs to change running resources, or claim data-level equivalence. That distinction matters throughout everything below.
• SKILL.md — the agent-facing definition: YAML frontmatter with trigger keywords, an Objective, explicit Non-Goals, Constraints, worked before/after examples, a "signal in source code → reference file" routing table, and a numbered Validation / Exit Criteria checklist.
• references/ — focused deep-dive files the agent pulls in only when a specific signal is detected.
• BENCHMARKS.md — an executive summary table plus a documented, reproducible methodology.

Two design choices were worth copying exactly. First, skills run a conditional Phase 0 that detects which migration phases actually apply before touching anything. Second, they maintain a flag-only category: things the agent must never auto-transform — ambiguous dependencies, custom connectors, non-mechanical behavior changes — and instead has to surface in a report for a human to decide on. That second one ended up mattering more than anything else I built.

Why target Glue 5.0 specifically

AWS Glue 5.1 became generally available in November 2025, running Spark 3.5.6, Python 3.11, Scala 2.12.18, and Java 17. Glue 5.0 (Spark 3.5.4, same Python/Scala/Java versions) remains fully supported. I scoped this first version of the skill — and every fixture and benchmark in it — to Glue 5.0 specifically, matching what's proposed in issue #75. The skill's structure extends cleanly to 5.1 as a follow-up; I'd rather ship a narrower skill with benchmarks I actually ran than a broader one with claims I hadn't validated.

glue-version-upgrade-readiness analyzes AWS Glue ETL jobs (PySpark and Scala) and their IaC, and prepares them for Glue 5.0 from source versions 2.0, 3.0, or 4.0. Gated behind Phase 0 detection flags so only relevant work runs per repository, it covers:

• Spark 2.4/3.1/3.3 → 3.5 compatibility and modernization patterns — deprecated APIs such as registerTempTable (deprecated since Spark 2.0, not removed, but worth cleaning up alongside a runtime upgrade), legacy datetime-parsing flags, and related SQL semantics changes
• Python 3.7/3.10 → 3.11 compatibility, including dependency pin bumps
• Glue-specific API and logging argument changes
• Connector and data lake format updates — native Iceberg/Delta/Hudi support in newer Glue versions
• Terraform / CloudFormation / CDK updates to target the new runtime

And, deliberately, what it refuses to touch automatically: AWS Marketplace connectors, compiled/native dependencies, and anything relying on Spark 2.x behavior with no clean mechanical equivalent. Those get written into a MIGRATION_REPORT.md with a rationale, never silently migrated.

Benchmarking without live access

The intended way to validate a skill is to run it through the real atx CLI against seeded test repositories. I didn't have live AWS Transform Custom access in my environment, so rather than fabricate numbers or skip validation entirely, I seeded three fixture repos with known, planted incompatibilities — a PySpark Glue 2.0 job, a Scala Glue 3.0 job, and a Terraform-managed Glue 4.0 job — and walked each one through the skill's phases by hand, then checked the result with real local tooling: Python 3.11, Terraform, cfn-lint, and sbt.

Every number below is labeled manually simulated, not agent-executed, and that label stayed in the public issue rather than getting smoothed over anywhere:

Fixture	Findings detected	Mechanical fixes validated	Manual-review items	False positives	Result
Glue 2.0 PySpark	5/5	4/4	1/1	0	Partially ready
Glue 3.0 Scala	5/5	3/3	2/2	0	Blocked by connector
Glue 4.0 Terraform	4/4	3/3	1/1	0	Partially ready
Total	14/14	10/10	4/4	0	3/3 controls unchanged

These numbers measure the results of manually applying the skill's prescribed phases against the fixtures. They are not a measurement of AWS Transform Custom's autonomous execution accuracy — I don't have that data yet, and I've said so everywhere the numbers appear.

The Scala result is the one I trust the most, precisely because it isn't a clean pass. That fixture has a deliberately unpublished custom connector planted in it. The skill correctly refused to guess a replacement, flagged it, and the sbt compile genuinely fails on the full build — while the transformed source code itself compiles cleanly in an isolated harness. A skill that reported "ready" there would have been the wrong outcome.

A migration skill shouldn't optimize for the highest possible pass rate. It should optimize for the highest defensible pass rate.

I also ran every embedded code example in the skill's documentation — Python, Terraform, and CloudFormation snippets — through the same real validators, plus a control file in each fixture that should never change, verified by SHA-256 before and after.

Catching my own overreach

Before opening the issue, I ran a separate fact-checking pass against the two claims I was least confident in: whether AWS documentation generally deprecates Glue DynamicFrames (it doesn't — only in specific Lake Formation fine-grained-access-control scenarios), and whether the CDK team prescribes cdk synth as the stability mechanism for the experimental aws-glue-alpha module.

The second one didn't hold up. CDK documents the module as experimental and warns about breaking changes across versions, but it doesn't specifically call out synthesis as a stability guarantee — that was a validation step I'd decided to require, initially written as if it were AWS's own guidance. I reworded it across the skill file, the reference doc, and the proposal so the requirement is now correctly attributed: it's this skill's own safety gate, not a documented CDK recommendation.

That's a small correction on its own, but it's the kind of thing that either gets caught before publication or gets caught by a maintainer in review. Catching it yourself is better either way — it's the difference between a claim someone has to push back on and one they can just verify.

Where it stands

This is a proposal, not a landed contribution. The complete skill, its reference docs, and the three benchmark fixtures are built and validated locally, but nothing has been merged — I'm not linking a branch here because there isn't a public one yet. Per the repository's contributing guidelines, significant work gets discussed in an issue before a pull request, so that's exactly where this sits: issue #75, open for maintainer feedback.

If you're running AWS Glue at scale and have hit your own Glue 5.0 or 5.1 migration edge cases, I'd like to hear about them — either on the issue or in the comments here. And if you're looking at AWS Transform Custom for the first time, community-sourced-transformations/ is a genuinely underexplored on-ramp; there's a lot of room for contributions outside the Java and Kubernetes examples that exist today.