ArXiv

DRFLOW: A Deep Research Benchmark for Personalized Workflow Prediction

Authors
Md Tawkat Islam Khondaker, Raymond Li, Muhammad Abdul-Mageed...
Categories
cs.AI, cs.MA
arXiv
https://arxiv.org/abs/2606.18191v1
PDF
https://arxiv.org/pdf/2606.18191v1

Brief

DRFLOW introduces a benchmark for predicting personalized multi-step workflows from heterogeneous evidence. The authors curate 100 tasks across five domains with 1,246 annotated steps grounded in over 3,900 sources and define seven diagnostic evaluation metrics. They present DRFLOW-Agent (DRFA), which achieves up to a 10.02% average F1 gain over strong baselines, yet substantial gaps remain in producing complete, correctly ordered, personalized workflows.

Source evidence

Abstract

Deep research (DR) systems are increasingly used for complex information-seeking tasks, but existing works mainly focus on generating reports and summaries. In contrast, many enterprise tasks instead require an agent to identify concrete workflows which is a sequence of action-steps. For example, rather than summarizing budgeting policies, an agent should be able to determine the steps needed to answer a question such as: "How do I request new headcount given a fixed budget?". Therefore, we introduce DRFLOW, a benchmark for evaluating personalized workflows predicted by agents from heterogeneous sources. Each task requires the agent to identify relevant evidence from scattered sources, then use that evidence to predict the correct action-step sequence for the user's task. DRFLOW contains 100 tasks across five domains, with 1,246 reference workflow steps grounded in more than 3,900 sources. We define seven diagnostic metrics covering factual grounding, step recovery, structural ordering, condition resolution, and personalization. We further present DRFLOW-Agent (DRFA), a workflow-oriented reference agent to predict personalized workflow. We show that although DRFA improves over strong baseline agents (upto 10.02% average F1 score), there is substantial room for improvement remains across these workflow metrics, indicating that predicting complete and correct personalized workflows remains a challenging frontier for deep research.