ArXiv

Deep4ge: DNN Training Trajectories for Fault Detection and Diagnosis

2026-07-14 · 15:20 UTC ·Sigma Jahan ·1 min read
Authors
Sigma Jahan
Categories
cs.SE, cs.LG
arXiv
https://arxiv.org/abs/2607.12868v1
PDF
https://arxiv.org/pdf/2607.12868v1

Brief

Deep4ge is a controlled benchmark of per-epoch DNN training runs designed to detect and diagnose implementation faults. The authors collected 59 TensorFlow/Keras programs from Stack Overflow and generated 14,227 runs (9,845 faulty) via 27 source-code transformations across seven fault categories. Each run records 4 evaluation metrics and 26 per-epoch features; dataset and framework released (DOI). Summary based on abstract.

Why it matters

Deep4ge is a public benchmark of 14,227 TensorFlow/Keras training runs derived from 59 adapted DNN programs collected from Stack Overflow; accepted at ICSME Data & Tool Track 2026 and released at https://doi.org/10.5281/zenodo.20337241.

Key details

  • The dataset contains 9,845 faulty runs and 4,382 correct baseline runs produced by 27 source-code transformations that introduce known faults across seven fault categories.
  • For each run Deep4ge records 4 evaluation metrics and 26 per-epoch features (weights, gradients, activations, accuracy/loss trends, learning rate, hardware use), supporting binary fault detection, multi-class diagnosis, and early prediction from partial training.
Source evidence

Abstract

Deep learning systems often fail due to subtle implementation faults that alter training behavior. Recent work has studied how to detect and diagnose such failures from changes observed across training epochs. However, the software engineering community still lacks a public dataset of per-epoch training runs with documented fault history, feature extraction details, and clear reuse support for fault detection and diagnosis tasks. We present Deep4ge, a controlled benchmark of 14,227 training runs generated from 59 adapted TensorFlow/Keras deep neural network (DNN) programs collected from Stack Overflow. We generated faulty variants using 27 source-code transformations that introduce known faults across seven categories. The dataset contains 9,845 faulty runs and 4,382 correct baseline runs. For each run, we record 4 evaluation metrics and 26 features that measure training behavior at every epoch. These features capture weights, gradients, activations, accuracy and loss trends, learning rate, and hardware use. Deep4ge supports binary fault detection, multi-class fault diagnosis, and early fault prediction from partial training runs. We release the dataset and fault-injection framework at https://doi.org/10.5281/zenodo.20337241.

Comment: Accepted at ICSME Data & Tool Track, 2026