Toward Decision Support that Respects Personal Preferences: Human Weighting of LLM-Generated Evaluation Axes

Abstract

As a step toward AI-assisted decision support that better respects personal preferences, we examine delegation boundaries by restricting user input to weighting evaluation axes while delegating axis generation and option scoring to an LLM. We present a Simple Additive Weighting (SAW) method in which the LLM produces evaluation axes and numeric scores, and users express their preferences by weighting those axes. In a small exploratory paper-based study on movie selection and company choice, participants described weighting as easy to understand and low-effort. For participants with complete ranking data, similarity analyses suggest that the method sometimes diverged from direct rankings, especially in the movie scenario; interviews and an exploratory follow-up suggest that some divergences prompted articulation of criteria not stated beforehand, although this observation remains preliminary. We also frame weighting as a reflective process that makes value prioritization explicit, which may inform interactive systems for interpretive or creative tasks. Pairwise comparison was perceived as more reliable but also more effortful, suggesting an efficiency-reliability trade-off. Repeated LLM runs under fixed settings produced different axes and scores, suggesting that delegation choices can reshape the decision space even when users retain control over weighting. Rather than claiming effectiveness, we contribute an exploratory design perspective on when weighting may and may not be sufficient.