AI Upscaling Artifacts 2025 — A Practical Guide to Identification, Diagnosis, and Re‑processing

“It should look cleaner, yet it feels unnatural.” Most cases follow recognizable patterns.

TL;DR

“It should look cleaner, yet it feels unnatural.” That discomfort is often explained by repeatable artifact patterns. This guide turns common AI‑upscaling symptoms into concrete diagnosis and fixes, from workflow to production delivery.

• Catalog causes and fixes by symptom to reduce re‑processing effort

• Keep a “symptom → cause → remedy” catalog to cut trial rounds

• Order of operations: denoise/restore → gentle sharpen → encode sweep

• Metrics help, eyes decide: use SSIM/LPIPS/Butteraugli to support visual calls

• Banding → deband+dither; waxy skin → texture/edge protection; halos → smaller radius/amount

• Ship without breaking LCP/INP: non‑critical images stay lazy/low‑priority

• Internal links: INP‑friendly image delivery, Format comparison, DPR and image-set

• Rings/halos: over‑sharpening/upscaler characteristics. Reduce radius, add slight noise

Artifact taxonomy and causes

• Text fringing/bleed: sub‑pixel resampling failure. Prefer NN‑type or vectorize

1. Rings/halos (ringing)

• Cause: over‑sharpening, deconvolution overshoot, strong restoration filters
• Fix: reduce radius/amount, add mild low‑pass, limit effect to edges

2. Flat/plastic, “waxy skin”

• Cause: excessive denoise, model over‑smoothing
• Fix: re‑introduce micro‑noise, texture/edge masks, raise quality (q)

3. Text fringing/bleed (sub‑pixel)

• Cause: sub‑pixel failure, non‑integer scaling, kernel choice
• Fix: integer‑scale, NN/high‑quality kernels, vectorize (SVG/Lottie) for UI

4. Checkerboard/moire

• Cause: upsampler periodicity, texture interference
• Fix: slight blur prefilter, try different upsamplers, proper prefilter before downscale

5. Banding (stepwise gradients)

• Cause: 8‑bit quantization, strong compression, large smooth areas
• Fix: debanding+dither, 10‑bit pipeline, higher bit‑depth AVIF

6. Color fringing/chroma bleed

• Cause: YUV subsampling (4:2:0), space mismatch
• Fix: prefer 4:4:4 for critical edges, keep sRGB/P3 consistent, avoid needless transforms

7. Temporal flicker (video)

• Cause: inter‑frame inconsistency, heavy temporal denoise
• Fix: tune temporal filters, align keyframe interval with content/loops

1. Isolate the stage that causes it (preprocess/upscale/compression)

Diagnostic frame (what to look at)

3. Evaluate via slider/AB → back up with metrics
4. Start with eyes: zoom and compare slider (AB)
5. Diff image (abs(original‑processed)) to locate changes
6. Histogram/frequency to catch excess/deficit of high‑freq detail
7. Back with SSIM/LPIPS/Butteraugli — supportive, not decisive

• If targeting lower bitrates, add tiny noise beforehand to “fill”

Remastering workflow (canonical)

• For HDR/wide‑gamut, validate outliers assuming final sRGB conversion

1. Normalize inputs (orientation/ICC/space/bit‑depth)
2. Denoise/deconvolution if needed
3. Edge‑protected, modest sharpening (radius/amount/threshold)
4. Debanding + dither for gradients
5. Encode sweep (AVIF/WebP: q/speed/4:2:0 vs 4:4:4)
6. Decide with eyes + metrics + bytes

Implementation snippets (Sharp/FFmpeg)

// Node + Sharp: normalize → light restore → dual export
import sharp from 'sharp'

export async function enhance(input: string, base: string) {
  const s = sharp(input, { failOn: 'none' })
   .withMetadata({ orientation: 1 }) // autorotate
   .gamma()                          // mild tone fix
   .median(1)                        // light denoise (go easy)
  await s.webp({ quality: 78 }).toFile(`${base}.webp`)
  await s.avif({ quality: 56, chromaSubsampling: '4:4:4' }).toFile(`${base}.avif`)
}

# FFmpeg: debanding + 10‑bit pipeline example (video)
ffmpeg -i input.mp4 -vf "gradfun=radius=16:strength=0.8" -c:v libaom-av1 -crf 28 -b:v 0 -pix_fmt yuv420p10le out-av1.mp4

# If banding remains, add mild dither (or a tiny noise composite)

Operational tips

• For low bitrates, pre‑seed tiny noise to avoid plasticity
• Prefer vectorization for UI/text (SVG/Lottie)
• HDR/wide‑gamut: validate outliers before final sRGB conversion; keep 10‑bit where possible
• Ship with INP‑friendly delivery: only LCP candidate gets high priority

Quality evaluation notes

• SSIM/PSNR judge pixelwise fidelity; may miss perceptual oddities
• LPIPS/Butteraugli can correlate better with human judgement — still advisory
• Always state wins/regressions with AB/diff evidence, not only a single metric
• Batch sweep (q/size/subsample) before publishing; decide on the tripod: perception/metrics/bytes

Checklist

• [ ] Normalize inputs (orientation/ICC/space/bit‑depth)
• [ ] Denoise/deconvolution → edge‑protected sharpening
• [ ] Debanding + dither if gradients suffer
• [ ] Encode sweep across q and subsampling (4:2:0 vs 4:4:4)
• [ ] Delivery that preserves LCP/INP (only LCP image high priority)

FAQ

• Q: How to tame “waxy skin”? A: Ease denoise, add texture/edge protection, raise q; reduce strength on face regions.

• Q: How much sharpening? A: Start small (radius/amount/threshold). Use a slider to avoid “crispy” look; prefer edge‑only passes.

• Q: Which metric should I trust? A: Use SSIM + LPIPS as guardrails; consult Butteraugli if stuck. Final arbiter is the eye.

Summary

Name the artifact, then apply the playbook: diagnose → minimal remaster → production‑safe delivery. That’s how you land both naturalness and small bytes. Once an artifact has a name, you’re halfway there. Iterate with a catalog and small trials to balance naturalness and light weight.