View in EDS HTML Full Text PDF Full Text

Feedback-Aware Inference for Iterative Multi-Sample Text Generation

Saved in:
Bibliographic Details
Title: Feedback-Aware Inference for Iterative Multi-Sample Text Generation
Language: English
Authors: Andreea Dutulescu (ORCID 0009-0003-1870-679X), Stefan Ruseti (ORCID 0000-0002-0380-6814), Mihai Dascalu (ORCID 0000-0002-4815-9227), Danielle S. McNamara (ORCID 0000-0001-5869-1420)
Source: Grantee Submission. 2026 7.
Peer Reviewed: Y
Page Count: 26
Publication Date: 2026
Sponsoring Agency: National Center for Education Research (NCER) (ED/IES)
Contract Number: R305T240035
Document Type: Journal Articles
Reports - Research
Descriptors: Natural Language Processing, Man Machine Systems, Feedback (Response), Inferences, Computer Assisted Instruction, Material Development, Item Banks, Program Development, Models
DOI: 10.3390/ai7050171
Abstract: Generating multiple text sequences and refining them through feedback is essential for improving the quality of outputs in many NLP tasks. While Large Language Models can leverage iterative feedback during inference, smaller models often lack this capability due to limited capacity and the absence of suitable training paradigms. In this paper, we propose a novel Feedback-Aware Inference approach that enables iterative sequence generation with integration of feedback signals. Our method allows models to generate multiple sequences, incorporate feedback from previous iterations, and refine outputs accordingly. This approach dynamically adjusts to different quality metrics, making it adaptable to various contexts and objectives. We evaluate our approach on two distinct tasks: Answer Selection for Question Generation and Keyword Generation, arguing for its generalizability and effectiveness. Results show that our method outperforms strong baselines, maintaining high performance across iterations and achieving superior results even with smaller, open-source models.
Abstractor: As Provided
Notes: https://github.com/upb-nlp/Feedback-Aware
IES Funded: Yes
Entry Date: 2026
Accession Number: ED681036
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwEx9z3dCEUiG8MP_kQ3mrCiAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDHDpkhSe5SoN2q4H0wIBEICBmyjprBvrq18KPcQSfJFJhhvGutldWzMOkS3fo00Nu9dUDgelgM8FMDYJdKW76wecAP0xSSANGAbEi00sykbxbPjr_SS2COe3aYD1K4kquqT8QbIorv35ezDryUZsHq0uzjsLwNy0PgudzdtzetK_ArvHrrZX7arIXIHhNg8naosLWavhLUNm-aCkZeLk_rai2KwUtjTnFVlKg-n2
Text:
  Availability: 1
  Value: <anid>AN0194175640;[msnv]01may.26;2026Jun02.05:12;v2.2.500</anid> <title id="AN0194175640-1">Feedback-Aware Inference for Iterative Multi-Sample Text Generation </title> <p>Generating multiple text sequences and refining them through feedback is essential for improving the quality of outputs in many NLP tasks. While Large Language Models can leverage iterative feedback during inference, smaller models often lack this capability due to limited capacity and the absence of suitable training paradigms. In this paper, we propose a novel Feedback-Aware Inference approach that enables iterative sequence generation with integration of feedback signals. Our method allows models to generate multiple sequences, incorporate feedback from previous iterations, and refine outputs accordingly. This approach dynamically adjusts to different quality metrics, making it adaptable to various contexts and objectives. We evaluate our approach on two distinct tasks: Answer Selection for Question Generation and Keyword Generation, arguing for its generalizability and effectiveness. Results show that our method outperforms strong baselines, maintaining high performance across iterations and achieving superior results even with smaller, open-source models.</p> <p>Keywords: feedback-aware; multi-sample generation; Question Generation; keywords generation</p> <hd id="AN0194175640-2">1. Introduction</hd> <p>Natural Language Processing tasks increasingly require models to generate not just a single deterministic output but a diverse set of candidate sequences that can be evaluated and refined to meet varying objectives. In many real-world applications, such as Question Generation, summarization, or keyword identification, the space of valid outputs is inherently large, with multiple correct or relevant solutions for a given context. Generating a single output often fails to capture this variability, potentially omitting higher-quality or more contextually appropriate alternatives. Consequently, multi-sample generation has emerged as a key paradigm, enabling models to explore a broader solution space and improve the chances of obtaining outputs that align with desired task-specific criteria.</p> <p>However, the utility of multi-sample generation is fundamentally constrained by the model's ability to iteratively refine outputs based on feedback. While generating multiple outputs in a single pass can increase diversity, it does not inherently guarantee improvement in quality or adherence to task constraints. The integration of feedback signals allows models to self-correct by incorporating information about previously generated outputs and progressively converging toward better solutions. This process mirrors how humans iteratively improve their work, leveraging evaluation and critique to refine ideas. In the context of machine learning, such refinement enables adaptive inference, wherein generation becomes an interactive process informed by dynamic signals rather than a one-shot static mapping from input to output.</p> <p>Recent advancements in Large Language Models (LLMs) have demonstrated that very large, instruction-tuned models can exhibit behaviors resembling feedback-driven refinement, especially in conversational or interactive settings. Models such as GPT-4, Claude, and Gemini are capable of maintaining a dialogue, reasoning about prior outputs, and adjusting their responses accordingly. These models implicitly leverage their vast parameter count and pretraining on diverse interactive data to support in-context learning, where previous exchanges act as implicit feedback. Despite these capabilities, such models are computationally expensive, proprietary, and often infeasible for deployment in resource-constrained environments. Many real-world scenarios, such as educational technologies, embedded systems, and low-latency applications, require smaller, open-source models that can run efficiently on consumer-grade hardware. Unfortunately, these smaller models lack the innate Feedback-Aware mechanisms of proprietary LLMs and typically rely on single-shot generation, making them ill-suited for tasks requiring iterative refinement.</p> <p>This paper addresses this gap by introducing Feedback-Aware Inference, a general-purpose framework designed to enable iterative, feedback-driven generation in smaller, open-source models. Our method allows models to</p> <p></p> <ulist> <item> Generate multiple candidate sequences for a given context, ensuring diversity and coverage of potential solutions</item> <p></p> <item> Integrate feedback signals, derived from either human evaluators or automated scoring systems, to assess the quality of prior outputs</item> <p></p> <item> Refine outputs dynamically across iterations, progressively converging on higher-quality results that align with task-specific goals</item> </ulist> <p>The feedback signals can be simple binary labels (e.g., GOOD/BAD). This makes our framework applicable to a wide variety of generative NLP tasks beyond the two we evaluate in this paper. To demonstrate the generality and effectiveness of our approach, we focus on two tasks that exemplify distinct challenges in multi-sample generation:</p> <p> <bold>Answer Selection for Question Generation</bold>, where the objective is to identify diverse, meaningful answer spans within a text that can be used to generate questions. This task requires balancing coverage of relevant concepts with avoidance of redundancy.</p> <p> <bold>Keyword Generation</bold>, where the goal is to produce concise, informative keywords that summarize a document, often including terms that are not explicitly present in the text but are inferred from its content.</p> <p>These tasks were chosen to highlight different aspects of feedback integration: in the first, the feedback is generated automatically by proxy models, while in the second, it is derived from human-curated annotations. This distinction illustrates the adaptability of our framework to different sources and types of feedback.</p> <p>From a broader perspective, our work is inspired by developments in preference optimization, which aim to align model outputs with human values during training. However, unlike these approaches, which operate primarily in the training phase, our method applies feedback directly during inference, enabling models to adapt to new inputs and evolving requirements without retraining. This is particularly advantageous for scenarios where task definitions or evaluation criteria may change over time.</p> <p>Our contributions are threefold:</p> <p></p> <ulist> <item> We propose Feedback-Aware Inference, a novel method that integrates iterative feedback signals into the generation process, enabling smaller models to emulate the refinement capabilities typically seen only in LLMs.</item> <p></p> <item> We provide a generalizable framework applicable across tasks, demonstrating its effectiveness on two distinct benchmarks while maintaining computational efficiency.</item> <p></p> <item> We release all code, datasets, and fine-tuned models as open-source resources to facilitate reproducibility and further research in feedback-driven generation (https://github.com/upb-nlp/Feedback-Aware, accessed on 5 May 2026).</item> </ulist> <p>Our approach achieves superior results on two different generative tasks, even with smaller, open-source models, highlighting the potential for cost-effective solutions without relying on massive, proprietary LLMs.</p> <p>The novelty of our method lies in introducing iterative refinement capabilities in smaller models through a structured, label-conditioned mechanism rather than free-form self-critique. This positions our work as complementary to preference-optimization methods, which align models during training; our method instead operates at inference time and can be combined with such methods. It also differs from self-refinement approaches that rely on large proprietary LLMs capable of producing verbose natural-language critiques, a capability that smaller open-source models lack. The scope of the framework is best characterized by the following boundary conditions: (i) the task admits a discrete, learnable feedback signal; (ii) a labeling system (automated or human) is available; and (iii) the task benefits from generating multiple diverse outputs.</p> <hd id="AN0194175640-3">2. Related Work</hd> <p></p> <hd id="AN0194175640-4">2.1. Feedback for Model Refinement</hd> <p>While early approaches for model refinement relied primarily on classical Reinforcement Learning, more recent advancements, such as Reinforcement Learning from Human Feedback [[<reflink idref="bib1" id="ref1">1</reflink>]] and Direct Preference Optimization [[<reflink idref="bib2" id="ref2">2</reflink>]], have argued for the potential of LLMs to generate text that aligns closely with human values and preferences. These methods involve training the LLM to maximize a reward function, typically computed by a human or a model trained on human-generated data. However, a key limitation of these approaches is that the refinement process is typically confined to the training phase, limiting the LLM's capability to adapt to new information or changing contexts during inference.</p> <p>Few-shot prompting, a technique that involves providing a small number of examples as part of the prompt to guide the LLM's generation, is a frequently employed approach for controlling and refining LLM outputs at inference time. Early works by Gao et al. [[<reflink idref="bib3" id="ref3">3</reflink>]] and Tam et al. [[<reflink idref="bib4" id="ref4">4</reflink>]] proved the effectiveness of few-shot learning in improving LLM performance, even with limited training data. Subsequent works by Schick et al. [[<reflink idref="bib5" id="ref5">5</reflink>]] and Perez et al. [[<reflink idref="bib6" id="ref6">6</reflink>]] have further refined few-shot prompting techniques, making them more practical for real-world applications. However, an important limitation of few-shot learning is the reliance on a pre-annotated set of examples, usually derived from the training and validation sets. The distribution of examples in the training set may not perfectly align with the desired output distribution, potentially leading to suboptimal performance. Also, although the model will try to mimic the example's concept, it cannot refine its generation once it has been computed.</p> <p>Alignment with human preferences is usually done through Reinforcement Learning from Human Feedback [[<reflink idref="bib1" id="ref7">1</reflink>]] or Direct Preference Optimization [[<reflink idref="bib2" id="ref8">2</reflink>]]. While these solutions increase the quality of generated answers, they do not directly tackle the problem of generating multiple samples and do not take into account feedback signals at inference time.</p> <p>Only recently have works studied advanced techniques for feedback-based refinement. Specifically, researchers have explored using chat models and AI-generated feedback to iteratively improve the quality of LLM-generated text. These approaches generally involve a cyclical process: an LLM is prompted to perform a task, its output is evaluated by another LLM (or itself), and the feedback is provided to the original LLM for refinement. This iterative process allows the model to learn from its mistakes and gradually improve its performance. For example, Fu et al. [[<reflink idref="bib7" id="ref9">7</reflink>]] created a setting where chat models are prompted to negotiate a selling price for goods, acting as the buyer or the seller and improving their offer based on the conversation. Self-refinement, a technique in which LLMs rate, highlight errors, and give feedback to their own generated content, is studied in multiple contexts. Text summarization, code generation, machine translation, and Math reasoning are tasks improved by this approach [[<reflink idref="bib8" id="ref10">8</reflink>], [<reflink idref="bib10" id="ref11">10</reflink>]]. Although the previous works highlighted very promising results in terms of feedback-based refinement, the experiments use proprietary LLMs with a huge number of parameters, such as GPT-4, Claude, or PaLM-2. Their approach relies on the ability of a large model to engage in chat conversations and complex instruction-following. These abilities are hindered in smaller open-source models that can be run on consumer hardware.</p> <p>A complementary line of work improves generation quality at inference time without further training. Self-consistency decoding [[<reflink idref="bib11" id="ref12">11</reflink>]] and best-of-N generation [[<reflink idref="bib12" id="ref13">12</reflink>]] draw multiple independent samples from a fixed conditional distribution and select among them according to a scoring criterion. These paradigms operate at inference time and can be used regardless of the training method, as complementary.</p> <hd id="AN0194175640-5">2.2. Multi-Sequence Generation Tasks</hd> <p></p> <ulist> <item> Question Generation</item> </ulist> <p>Question Generation models usually take a context and a desired answer as input to generate a question based on them. Yao et al. [[<reflink idref="bib13" id="ref14">13</reflink>]] specifically focused on deriving multiple answer–question pairs from a text by employing heuristic-based rules for Answer Selection. The authors extracted noun chunks, named entities, and event descriptions to serve as selected spans for directing the generation of questions. Zhao et al. [[<reflink idref="bib14" id="ref15">14</reflink>]] learned to predict the possible distribution of different types of questions based on the context and extracted the relevant information from the text that would adhere to that distribution. The training was performed on the FairytaleQA dataset [[<reflink idref="bib15" id="ref16">15</reflink>]]. The annotated question–answer pairs and the question type were used to fine-tune the corresponding models. While the results were promising, their work was constrained by the specific annotation technique used in the dataset and cannot be easily transferred to other qualitative constraints. Yoon et al. [[<reflink idref="bib16" id="ref17">16</reflink>]] approached the task by iteratively generating questions and appending the previously generated questions in the prompt to ensure a diverse set and direct the model's output. The fine-tuning considered the questions provided in the prompt and generated a query that diverged from the previous ones. Although this was a step forward regarding question diversity, the generation is not controllable and cannot consider other requirements. Across these papers, the common theme is supervised fine-tuning on specific QA datasets. This approach ensures that the generated questions and answers conform to the patterns found in these datasets, but the models cannot adapt based on user feedback.</p> <p></p> <ulist> <item> Keyword Generation</item> </ulist> <p>Keyword Generation is the task of identifying or generating terms or phrases that encapsulate the key topics of a given text, such as a paper abstract. Extensive research has been conducted on keyword extraction, where spans directly from the abstract are selected as keywords. A recent survey [[<reflink idref="bib17" id="ref18">17</reflink>]] presents the latest advancements in keyphrase extraction. Advancements have been made with pre-trained language models, specifically encoder models. Specifically, for unsupervised keyphrase extraction, semantic similarity with the abstract of spans is ranked using cross-attention [[<reflink idref="bib18" id="ref19">18</reflink>]] or graph structures [[<reflink idref="bib19" id="ref20">19</reflink>]]. In another approach, Song et al. [[<reflink idref="bib20" id="ref21">20</reflink>]] proposed a system consisting of three modules (chunking, ranking, and matching) jointly trained. However, the extractive task has notable limitations. Specifically, paper authors frequently assign keywords that are not explicitly mentioned within the abstract but are instead derived from the broader context or inferred concepts. Meng et al. [[<reflink idref="bib21" id="ref22">21</reflink>]] introduce an RNN-based generative model for keyphrase prediction, incorporating a copying mechanism to improve the prediction of rare phrases. This study addresses absent keyphrase prediction for scientific publications; however, the achieved recall is lower than 20% in this case.</p> <hd id="AN0194175640-6">3. Method</hd> <p>This section provides a structured overview of our proposed method. While we evaluate on two distinct tasks, each with specific characteristics, the following framework serves as a generalizable methodology applicable to all similar tasks that require generating multiple text sequences. Task-specific details are elaborated in Section 4.3. It is important to note that the objective of the paper is to propose a general method, not to focus on specific state-of-the-art solutions for each task.</p> <p>Figure 1 presents a high-level illustration of the proposed framework. The method consists of two main stages: a training phase that prepares the model to interpret feedback signals and an inference phase that performs iterative sequence generation guided by those signals.</p> <p>This framework enables smaller models to emulate the iterative refinement capabilities typically associated with much larger conversational language models while remaining computationally efficient and adaptable to different evaluation criteria.</p> <hd id="AN0194175640-7">3.1. Prerequisites: Labeling System</hd> <p>The proposed method incorporates a feedback-driven mechanism to iteratively enhance performance. This feedback signal may be derived from either human evaluation or an automated assessment procedure. The specific feedback mechanism utilized is not the primary focus of this study and can be adapted to different application domains and requirements.</p> <p>The Labeling System, denoted as <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>L</mi><mi>S</mi></mrow></semantics></math> </ephtml> , takes a generated text sequence <emph>g</emph> as input and produces a label that signifies the quality of the generated content. In our experiments, the possible labels are <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> or <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> , indicating the suitability of the generated output ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>L</mi><mi>S</mi><mo>(</mo><mi>g</mi><mo>)</mo><mo>∈</mo><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi><mo>,</mo><mi>B</mi><mi>A</mi><mi>D</mi></mrow></mrow></semantics></math> </ephtml> ). The labeling criteria depend on task-specific requirements (e.g., grammatical correctness, coherence, relevance, overall quality).</p> <p></p> <ulist> <item> Optional Component</item> </ulist> <p>For complex tasks, such as Answer Selection in Question Generation, evaluating only the generated text sequence may be insufficient. In such cases, additional contextual information may be required for assessment. To accommodate this, our framework supports an optional component, referred to as the Resulting Content Generator ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>C</mi><mi>G</mi></mrow></semantics></math> </ephtml> ), which automatically generates auxiliary content based on the generated sequence. Consequently, the Labeling System receives both the generated sequence (<emph>g</emph>) and the resulting content ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>C</mi><mi>G</mi><mo>(</mo><mi>g</mi><mo>)</mo></mrow></semantics></math> </ephtml> ) as input to determine the final label—i.e., <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>L</mi><mi>S</mi><mo>(</mo><mi>g</mi><mo>,</mo><mi>R</mi><mi>C</mi><mi>G</mi><mo>(</mo><mi>g</mi><mo>)</mo><mo>)</mo><mo>∈</mo><mo>{</mo><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi><mo>,</mo><mi>B</mi><mi>A</mi><mi>D</mi><mo>}</mo></mrow></semantics></math> </ephtml> .</p> <hd id="AN0194175640-8">3.2. Dataset Construction</hd> <p>The original datasets are augmented using the Labeling System ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>L</mi><mi>S</mi></mrow></semantics></math> </ephtml> ) to ensure that generated sequences are annotated as either <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> or <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> . To enable effective model fine-tuning, it is essential to include multiple examples of both <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> sequences for the same context.</p> <p>Formally, let the initial dataset be defined as <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>D</mi><mrow><mi>i</mi><mi>n</mi><mi>i</mi><mi>t</mi><mi>i</mi><mi>a</mi><mi>l</mi></mrow></msub><mo>=</mo><mrow><mo>[</mo><msub><mi>C</mi><mn>1</mn></msub><mo>,</mo><msub><mi>C</mi><mn>2</mn></msub><mo>,</mo><mo>...</mo><mo>,</mo><msub><mi>C</mi><mi>n</mi></msub><mo>]</mo></mrow></mrow></semantics></math> </ephtml> , where each <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>C</mi><mi>i</mi></msub></semantics></math> </ephtml> represents a context. For each <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>C</mi><mi>i</mi></msub></semantics></math> </ephtml> , we construct a set of possible generated sequences <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo>[</mo><msub><mi>g</mi><mrow><mi>i</mi><mo>,</mo><mn>1</mn></mrow></msub><mo>,</mo><msub><mi>g</mi><mrow><mi>i</mi><mo>,</mo><mn>2</mn></mrow></msub><mo>,</mo><mo>...</mo><mo>,</mo><msub><mi>g</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow></msub><mo>]</mo></mrow></semantics></math> </ephtml> along with their corresponding labels <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo>[</mo><mi>L</mi><mi>S</mi><mrow><mo>(</mo><msub><mi>g</mi><mrow><mi>i</mi><mo>,</mo><mn>1</mn></mrow></msub><mo>)</mo></mrow><mo>,</mo><mi>L</mi><mi>S</mi><mrow><mo>(</mo><msub><mi>g</mi><mrow><mi>i</mi><mo>,</mo><mn>2</mn></mrow></msub><mo>)</mo></mrow><mo>,</mo><mo>...</mo><mo>,</mo><mi>L</mi><mi>S</mi><mrow><mo>(</mo><msub><mi>g</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow></msub><mo>)</mo></mrow><mo>]</mo></mrow></semantics></math> </ephtml> .</p> <p>The assigned labels function as quality indicators, aiding in both model evaluation and comparative analysis against baseline approaches.</p> <hd id="AN0194175640-9">3.3. Feedback-Aware Inference</hd> <p>We want our model to have the opportunity to correct itself and learn from previously generated content, as well as the feedback given for it. For that, Algorithm 1 was designed for inference.</p> <p></p> <p> <ephtml> <table><tbody><tr><td><p><bold>Algorithm 1</bold> Feedback-aware inference procedure in which generated candidates are evaluated, labeled, and fed back into the prompt to guide subsequent generations</p></td></tr><tr><td><list list-type="Bullet"><list-item><p><bold>Require:</bold> A Labeling System <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>L</mi><mi>S</mi></mrow></semantics></math> (Section 3.1)</p></list-item><list-item><p><bold>Require:</bold> [optional] A Resulting Content Generator <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>R</mi><mi>C</mi><mi>G</mi></mrow></semantics></math> (Section 3.1)</p></list-item><list-item><p><bold>Require:</bold> The Feedback-Aware Model <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>F</mi><mi>A</mi><mi>M</mi></mrow></semantics></math></p></list-item><list-item><p>Initialize the prompt</p></list-item><list-item><p><italic>P</italic> = [Task description, Context]</p></list-item><list-item><p><bold>while</bold> Still want to select <bold>do</bold></p></list-item><list-item><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>g</mi><mo>←</mo></mrow></semantics></math><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>F</mi><mi>A</mi><mi>M</mi></mrow></semantics></math> (<italic>P</italic> + " <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> ")</p></list-item><list-item><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>r</mi><mi>c</mi><mo>←</mo><mi>R</mi><mi>C</mi><mi>G</mi><mo>(</mo><mi>g</mi><mo>)</mo></mrow></semantics></math></p></list-item><list-item><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>l</mi><mo>←</mo><mi>L</mi><mi>S</mi><mo>(</mo><mi>g</mi><mo>,</mo><mi>r</mi><mi>c</mi><mo>)</mo></mrow></semantics></math></p></list-item><list-item><p><bold>if</bold><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>l</mi><mo>=</mo><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math><bold>then</bold></p></list-item><list-item><p>Found a good generated sequence</p></list-item><list-item><p><bold>end if</bold></p></list-item><list-item><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>P</mi><mo>←</mo><mo>[</mo><mi>P</mi><mo>,</mo><mi>l</mi><mo>,</mo><mi>g</mi><mo>,</mo><mi>r</mi><mi>c</mi><mo>]</mo></mrow></semantics></math></p></list-item><list-item><p><bold>end while</bold></p></list-item></list></td></tr></tbody></table> </ephtml> </p> <p>The termination condition is left intentionally generic. In a deployment setting, it can be instantiated by any task-appropriate stopping rule, like reaching a target number of accepted sequences, observing a threshold number of consecutive BAD labels, or exhausting the pool of valid candidates.</p> <p>Model training is done following its purpose—i.e., selecting the information from the context and being capable of distinguishing between <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> sequences. The Feedback-Aware Model should be trained in a similar scenario to the one used during inference. Because of this, the prompt needs to include a diverse list of sequences with their corresponding quality label and generated questions. The prompt structure is shown in Figure 2, whereas Figure 3a,b introduce prompt examples for the evaluated tasks.</p> <p>For training, we format the prompt according to the template from Figure 2, but we only compute the loss and propagate gradients for the last sequence (depicted in blue). In this manner, the model is trained to recognize the template and generate the sequence (depicted in blue) by attending to the context, previous labels, previous generated sequences, and, optionally, previous resulting content. The model is trained to generate only <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> sequences since the gradients and loss are computed only for the tokens of the last generated sequence. Moreover, we can edit the prompt using a decoder-only model—i.e., change a previously thought <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> label in the actual prediction given by <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>L</mi><mi>S</mi></mrow></semantics></math> </ephtml> (the prerequisite Labeling System) and append the resulting content obtained from the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>C</mi><mi>G</mi></mrow></semantics></math> </ephtml> (the optional Resulting Content Generator).</p> <hd id="AN0194175640-10">4. Performance Evaluation</hd> <p></p> <hd id="AN0194175640-11">4.1. Baselines</hd> <p>Four strong baselines were selected to compare the performance of our method. Details on the full prompts (for both our model and the baselines) are provided in Appendix A.</p> <p></p> <ulist> <item> Single Sequence Generation (SSG)</item> </ulist> <p>This approach involves iteratively generating a possible sequence given the task and the context. This baseline is trained by supervised fine-tuning to generate the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> sequence. More formally, the general prompt format on which we fine-tune is the following: <emph>"<task>. Text: <text>. Generated sequence: <sequence>"</emph>. The loss and gradients are computed just for the tokens of <emph><sequence></emph>. At inference time, we over-sample 100 sequences, eliminate duplicates, and choose the top sequence in terms of log-likelihood.</p> <p></p> <ulist> <item> All Sequences Generation (ASG)</item> </ulist> <p>This approach simultaneously generates all <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> sequences for a given text. This baseline is trained by supervised fine-tuning to generate all the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> sequences for a context. More formally, the fine-tuning prompt is the following: <emph>"<task>. Text: <text>. Generated sequence: <seq_1>, <seq_2>...<seq_n>"</emph>. The loss and gradients are computed just for the tokens of <emph><seq_i></emph>.</p> <p></p> <ulist> <item> All Sequences Generation with Resulting Content (ASG-RC)</item> </ulist> <p>In the optional case of using the Resulting Content Generator, this approach serves as a strong baseline to prove the efficacy of the feedback labels. It is trained similarly to the All Sequences Generation (ASG) approach, with the addition of the resulting content (RC) in order to inform the current iteration about the previously generated sequences and their corresponding resulting content. More formally, the fine-tuning prompt is the following: <emph>"<task>. Text: <text>. Generated sequence: <seq_1>-<rc_1>, <seq_2>-<rc_2>, ...<seq_n>-<rc_n>"</emph>. The loss and gradients are computed just for the tokens of <emph><seq_i></emph>. It is important to acknowledge that this baseline also functions as a control group, evaluating the impact of our design decision to train the Feedback-Aware model using <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> / <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> labels.</p> <p></p> <p>• GPT-4o</p> <p>We use GPT-4o similarly to the previous baseline—namely, to generate sequences and (optionally) their resulting content. We use it in a one-shot setting that includes the task and provides one text example from the training dataset with a set of GOOD sequences and their resulting content. The considered prompt is the following: <emph>"<task>. Write your generated sequences together with the resulting content on separate lines in the following format: <generated_sequence>-<resulting_content>. Don't add any additional characters or numbering. Take into consideration the following example: <train_example>. Text: <context>. Response:"</emph>.</p> <hd id="AN0194175640-12">4.2. Experimental Setup</hd> <p>All previous models above (our approach and the baselines) were fine-tuned from the foundational Llama-3 (8B) model [[<reflink idref="bib22" id="ref23">22</reflink>]]. All our experiments for fine-tuning and inference were done on a single A100 80GB GPU to ensure accessibility and cost-effectiveness.</p> <p>Table 1 summarizes the training and inference configurations used in our experiments. To ensure a fair comparison between our approach and the baseline models, we aimed to keep the hyperparameters consistent across models whenever possible and follow the recommended nucleus-sampling defaults for Llama-3. In particular, the same training setup (e.g., LoRA fine-tuning [[<reflink idref="bib23" id="ref24">23</reflink>]], learning rate, optimizer, batch size, and precision) and decoding strategy were applied to both our models and the baselines. Exceptions include models with fixed or recommended default configurations (e.g., GPT-4o) and prerequisite models ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>G</mi><mi>e</mi><mi>n</mi></mrow></semantics></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi></mrow></semantics></math> </ephtml> ), for which we used their standard inference settings.</p> <p>We were interested in the most likely sequences for models with iterative generation (i.e., FA, ASG, ASG-RC). This is typically done using beam search, but this method is not well-suited for the current task. Due to the high number of possible positive answers, beam search is too restrictive with the options for the first token. Because of this, we opted for oversampling by generating in each iteration 10 samples and choosing the one with the highest log-likelihood each time.</p> <p>In the case of the SSG model, we opted for a similar over-sampling approach, sampling 100 sequences, from which duplicates are removed and the top 25 sequences are kept.</p> <hd id="AN0194175640-13">4.3. Evaluated Tasks</hd> <p>We chose two diverse tasks to evaluate our proposed framework and cover each aspect and flavor.</p> <hd id="AN0194175640-14">4.3.1. Answer Selection for Question Generation</hd> <p>For this task, the goal is to select several spans from a context to serve as the answer for generating a question. The aim is to generate a richer set of potential questions by identifying multiple answer spans from a given text. Our method improves the generated sequences through iterative feedback, making the model more adaptable.</p> <p></p> <ulist> <item> Prerequisites: Labeling System</item> </ulist> <p>For these experiments, we employ an automated method to evaluate the quality of the selected answers. We measured the quality of a selected answer by generating a question starting from it and assessing how likely a model would answer the question given the selected span. To achieve this, we required two models: one to generate questions based on a given context and a piece of information identified as the answer ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>G</mi><mi>e</mi><mi>n</mi></mrow></semantics></math> </ephtml> ), and another capable of providing accurate answers to questions based on a given context ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi></mrow></semantics></math> </ephtml> ). By utilizing these models, we can label responses and proceed with the task of generating and evaluating questions based on proposed answers. Any instruction-tuned model capable of generating and answering questions can be used for these tasks.</p> <p>We fine-tune these models on three established datasets: (a) SQuAD [[<reflink idref="bib24" id="ref25">24</reflink>]]—one of the most widely used resources for question-answering and generation, which consists of over 100 K question–answer pairs derived from a pool of 5 K Wikipedia articles; (b) HotpotQA [[<reflink idref="bib25" id="ref26">25</reflink>]]—designed to test a model's ability to answer questions that require reasoning across multiple paragraphs and which contains questions that should be answered by bridging information from two different Wikipedia articles; and (c) NarrativeQA [[<reflink idref="bib26" id="ref27">26</reflink>]]—designed to assess reading comprehension, particularly for lengthy texts, which consists of stories, along with corresponding questions and answers.</p> <p>The models were independently fine-tuned in a supervised manner using the prompt <emph>"Generate a question based on the context and the answer. Context: <context>. Answer: <answer>. Question: <generated_question>"</emph> for Question Generation ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>G</mi><mi>e</mi><mi>n</mi></mrow></semantics></math> </ephtml> ) and <emph>"Answer the following question based on the context. Context: <context>. Question: <question>. Answer: <answer>"</emph> for question-answering ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi></mrow></semantics></math> </ephtml> ). The loss is computed only on the <emph><generated_question></emph> and <emph><answer></emph> tokens, respectively.</p> <p>These models are not proposed as the state of the art for these tasks but rather as plug-and-play modules independent of our proposed method or baselines. This means that any model can be a prerequisite, as the proposed method is not dependent on the choice.</p> <p>In order to assess the performance of these models and validate their usage, we computed the BLEURT score [[<reflink idref="bib27" id="ref28">27</reflink>]] with the ground truth for the test partition of the SQuAD, HotpotQA, and NarrativeQA, and for the test partition of FairytaleQA [[<reflink idref="bib15" id="ref29">15</reflink>]] (a dataset used for our method and baselines, but on which we did not train <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>G</mi><mi>e</mi><mi>n</mi></mrow></semantics></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi></mrow></semantics></math> </ephtml> ).</p> <p>Table 2 showcases the BLEURT scores for the prerequisite models. Performance is acceptable as the models are capable of answering and generating questions with accuracy, considering the context. Moreover, the models generalize well on an unseen dataset (FairytaleQA), highlighting their capability to serve as suitable prerequisites for our tasks with a diverse range of texts from different domains.</p> <p>To establish an automatic criterion for a <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> selected answer, we define it as a text span that can be used to generate a valid question. This question should be answerable by the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi></mrow></semantics></math> </ephtml> model when provided with the given context. Furthermore, the context should be necessary for answering the question, avoiding excessively general inquiries. Additionally, the selected answers must exhibit diversity, avoiding redundancy in represented concepts.</p> <p>Formally, the following procedure is employed to determine the suitability of a selected span:</p> <p></p> <ulist> <item> A question is generated using the context and the selected answer: <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>q</mi><mo>=</mo><mi>Q</mi><mi>G</mi><mi>e</mi><mi>n</mi><mo>(</mo><mi>c</mi><mi>t</mi><mi>x</mi><mo>,</mo><mi>a</mi><mo>)</mo></mrow></semantics></math> </ephtml> .</item> <p></p> <item> The probability of the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi></mrow></semantics></math> </ephtml> model of returning the selected span for the given question must exceed an empirically chosen threshold of 3%: <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>P</mi><mo>(</mo><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi><mo>(</mo><mi>c</mi><mi>t</mi><mi>x</mi><mo>,</mo><mi>q</mi><mo>)</mo><mo>=</mo><mi>a</mi><mo>)</mo><mo>≥</mo><mn>0.03</mn></mrow></semantics></math> </ephtml> ; this threshold was selected given the distribution of correct answers in our training dataset;</item> <p></p> <item> The context must be important for answering the question, so we impose that it should be easier for <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi></mrow></semantics></math> </ephtml> to answer the question when given the context compared to answering without it: <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mstyle scriptlevel="0" displaystyle="true"><mfrac><mrow><mi>P</mi><mo>(</mo><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi><mo>(</mo><mo>″</mo><mo>″</mo><mo>,</mo><mi>q</mi><mo>)</mo><mo>=</mo><mi>a</mi><mo>)</mo></mrow><mrow><mi>P</mi><mo>(</mo><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi><mo>(</mo><mi>c</mi><mi>t</mi><mi>x</mi><mo>,</mo><mi>q</mi><mo>)</mo><mo>=</mo><mi>a</mi><mo>)</mo></mrow></mfrac></mstyle><mo>≤</mo><mn>1</mn></mrow></semantics></math> </ephtml> .</item> </ulist> <p>Here, we denote <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>P</mi><mo>(</mo><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi><mo>(</mo><mi>c</mi><mi>t</mi><mi>x</mi><mo>,</mo><mi>q</mi><mo>)</mo><mo>=</mo><mi>a</mi><mo>)</mo></mrow></semantics></math> </ephtml> as the probability of <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi></mrow></semantics></math> </ephtml> to generate the answer <emph>a</emph>, given the context <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>c</mi><mi>t</mi><mi>x</mi></mrow></semantics></math> </ephtml> and question <emph>q</emph>. We use the sum of log-likelihood estimates for each token in the answer, which is then converted into a probability between 0 and 1 by exponentiation:</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>P</mi><mrow><mo>(</mo><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi><mrow><mo>(</mo><mi>c</mi><mi>t</mi><mi>x</mi><mo>,</mo><mi>q</mi><mo>)</mo></mrow><mo>=</mo><mi>a</mi><mo>)</mo></mrow><mo>=</mo><msup><mi>e</mi><mrow><mo>∑</mo><mrow><mi>l</mi><mi>o</mi><mi>g</mi><mi>P</mi><mo>(</mo><msub><mi>a</mi><mi>i</mi></msub><mo>|</mo><mi>c</mi><mi>t</mi><mi>x</mi><mo>,</mo><mi>q</mi><mo>,</mo><msub><mi>a</mi><mrow><mn>0</mn><mo>:</mo><mi>i</mi><mo>−</mo><mn>1</mn></mrow></msub><mo>)</mo></mrow></mrow></msup></mrow></semantics></math> </ephtml> </p> <p>This formalization ensures that the selected answers are not only relevant to the context but also exhibit a strong dependency on the context for accurate question-answering. Furthermore, by applying this evaluation, we can categorize the selected spans as either <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> or <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> based on their adherence to the established procedure. It is important to note that this metric can be altered for other purposes (e.g., question difficulty), and its form is not essential in our method.</p> <p></p> <ulist> <item> Optional Component: Resulting Content</item> </ulist> <p>Evaluating a selected answer without knowledge of the resulting question might add an unnecessary abstraction layer to the task. Because of this, we include the question as additional information in the prompt. In this case, the Resulting Content Generator ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>C</mi><mi>G</mi></mrow></semantics></math> </ephtml> ) will be the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>Q</mi><mi>G</mi><mi>e</mi><mi>n</mi></mrow></semantics></math> </ephtml> model that generates a question based on the selected answer. The question will be used in the pipeline as resulting content.</p> <p></p> <ulist> <item> Dataset Construction</item> </ulist> <p>We consider two datasets for the specific task of Answer Selection for Question Generation. The <emph>TASA</emph> corpus [[<reflink idref="bib28" id="ref30">28</reflink>]] is a collection of text excerpts designed to represent the reading a college student might encounter throughout their academic career. It contains over 60 k passages from textbooks, literature, and various nonfiction and fiction works. For our experiments, we sampled 10,000 texts for training, 1000 for validation, and 1000 for the test partition from the following domains: language and arts, health, science, industrial arts, economics, business, and social studies. <emph>FairytaleQA</emph> [[<reflink idref="bib15" id="ref31">15</reflink>]] is a specialized dataset focused on narrative comprehension for kindergarten to eighth-grade students. It addresses the scarcity of high-quality question–answering datasets devised for diverse reading skills. The dataset is constructed by educational experts from child-friendly stories. For our experiments, we used the already-established partitions of the dataset (8548 for train, 1025 for validation, 1007 for test).</p> <p>For the task of Answer Selection for Question Generation, we identify possible text spans that could serve as an answer. These text spans for training are the nodes from the constituency tree of each sentence from the context. By leveraging these nodes, we systematically cover a wide range of possible answers. For each possible answer <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>a</mi><mi>i</mi></msub></semantics></math> </ephtml> , we generate a question given the context and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>a</mi><mi>i</mi></msub></semantics></math> </ephtml> , as <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>q</mi><mi>i</mi></msub><mo>=</mo><mi>Q</mi><mi>G</mi><mi>e</mi><mi>n</mi><mrow><mo>(</mo><mi>c</mi><mi>t</mi><mi>x</mi><mo>,</mo><msub><mi>a</mi><mi>i</mi></msub><mo>)</mo></mrow></mrow></semantics></math> </ephtml> . With the method described above, we label a sample of possible spans as <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> or <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> .</p> <hd id="AN0194175640-15">4.3.2. Keyword Generation</hd> <p>Keyword Generation involves automatically generating relevant terms that summarize the core topics of a paper.</p> <p></p> <ulist> <item> Prerequisites: Labeling System</item> </ulist> <p>For these experiments, we rely on the human-chosen keywords from the dataset to serve as <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> labels. Any other option not included in the set for the given context would be considered <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> . In this manner, we cover a different possibility for Feedback-Aware: the feedback signal is provided by humans rather than an automatic system.</p> <p></p> <ulist> <item> Dataset Construction</item> </ulist> <p>We use the <emph>KP20K</emph> dataset introduced by Mengt et al. [[<reflink idref="bib21" id="ref32">21</reflink>]]. This is an extensive dataset containing scientific article abstracts and their corresponding keywords, as chosen by the authors. It contains 500 k entries, from which we randomly selected 2000 from their testing partition to evaluate the models, 2000 for validation, and 50 k for training. As this dataset only has positive examples annotated (the keywords selected by the authors, labeled as <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> ), we created negative examples ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> ) by selecting keywords from other entries with high similarity to the abstract that are not in the subset of <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> keywords of that specific entry. More formally, having the dataset <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>D</mi><mo>=</mo><mo>[</mo><mrow><mo>(</mo><mi>a</mi><mi>b</mi><msub><mi>s</mi><mn>1</mn></msub><mo>,</mo><mi>K</mi><msub><mi>L</mi><mn>1</mn></msub><mo>)</mo></mrow><mo>,</mo><mrow><mo>(</mo><mi>a</mi><mi>b</mi><msub><mi>s</mi><mn>2</mn></msub><mo>,</mo><mi>K</mi><msub><mi>L</mi><mn>2</mn></msub><mo>)</mo></mrow><mo>,</mo><mo>...</mo><mo>,</mo><mrow><mo>(</mo><mi>a</mi><mi>b</mi><msub><mi>s</mi><mi>n</mi></msub><mo>,</mo><mi>K</mi><msub><mi>L</mi><mi>n</mi></msub><mo>)</mo></mrow><mo>]</mo></mrow></semantics></math> </ephtml> where <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi><msub><mi>L</mi><mi>i</mi></msub><mo>=</mo><mrow><mo>[</mo><msub><mi>k</mi><mrow><mi>i</mi><mo>,</mo><mn>1</mn></mrow></msub><mo>,</mo><mo>...</mo><mo>,</mo><msub><mi>k</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow></msub><mo>)</mo></mrow><mrow><mo>]</mo></mrow></mrow></semantics></math> </ephtml> is the list of the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> keywords for the abstract <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>a</mi><mi>b</mi><msub><mi>s</mi><mi>i</mi></msub></mrow></semantics></math> </ephtml> , we select the negative ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> ) keywords for <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>a</mi><mi>b</mi><msub><mi>s</mi><mi>i</mi></msub></mrow></semantics></math> </ephtml> as being from <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mrow><mo>(</mo><msub><mo>⋃</mo><mi>j</mi></msub><mi>K</mi><msub><mi>L</mi><mi>j</mi></msub><mo>)</mo></mrow><mo>\</mo><mi>K</mi><msub><mi>L</mi><mi>i</mi></msub></mrow></semantics></math> </ephtml> . From this set, we select <emph>m</emph> negative keywords that are most similar to the abstract, based on embeddings computed with an encoder model (https://huggingface.co/sentence-transformers/all-mpnet-base-v2, 5 May 2026). This will be the set of <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> keyword examples for the abstract <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>a</mi><mi>b</mi><msub><mi>s</mi><mi>i</mi></msub></mrow></semantics></math> </ephtml> .</p> <hd id="AN0194175640-16">5. Results</hd> <p>We evaluate the models and baselines using slightly different metrics based on the task. For <emph>Answer Selection for Question Generation</emph>, we evaluated in terms of Precision@K for the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> sequences. More formally, for each text, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>P</mi><mo>@</mo><mi>K</mi><mo>=</mo><mstyle scriptlevel="0" displaystyle="true"><mfrac><mrow><mrow><mi>no</mi><mo>.</mo></mrow><mspace width="4.pt" /><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow><mspace width="4.pt" /><mi>in</mi><mspace width="4.pt" /><mi>the</mi><mspace width="4.pt" /><mi>first</mi><mspace width="4.pt" /><mi mathvariant="normal">K</mi></mrow><mi mathvariant="normal">K</mi></mfrac></mstyle></mrow></semantics></math> </ephtml> . Figure 4a showcases the P@K aggregated results for the test partitions of both TASA [[<reflink idref="bib28" id="ref33">28</reflink>]] and FairytaleQA [[<reflink idref="bib15" id="ref34">15</reflink>]]. We evaluated using precision since the number of <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> answers for a context can be large and recall would be less meaningful.</p> <p>For <emph>Keyword Generation</emph>, we followed the approach stated in the literature and evaluated in terms of F1@K for the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> keywords. As there are a varied number of <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> keywords per text but still within a limited range, we computed the F1@K until K = 10, based on the observation that the 95th percentile of the number of <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> keywords per text is 10 on KP20K [[<reflink idref="bib21" id="ref35">21</reflink>]]. Figure 4b showcases the F1@K results.</p> <p>The value <emph>K</emph> used in our experiments is not a hyperparameter of the algorithm but an evaluation horizon: we report metric curves up to <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi><mo>=</mo><mn>25</mn></mrow></semantics></math> </ephtml> for Answer Selection and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi><mo>=</mo><mn>10</mn></mrow></semantics></math> </ephtml> for Keyword Generation, with the upper bounds chosen to reflect the typical size of the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> pool on each task.</p> <hd id="AN0194175640-17">6. Discussion</hd> <p></p> <hd id="AN0194175640-18">6.1. Main Results Discussion</hd> <p>The results presented in Figure 4 show that our proposed method consistently achieves high scores across multiple generated sequences. It outperforms all other models by a significant margin. The observed decline in metrics as K increases is primarily due to the diminishing pool of accessible answers for further generation.</p> <p>For the task of Answer Selection for Question Generation, the Single Sequence Generation (SSG) baseline initially exhibits high precision but rapidly declines as the number of generated sequences increases. This decline stems from the model's lack of awareness of previously selected samples, impairing its ability to generate additional high-quality answers, a limitation our approach effectively overcomes. The All Sequences Generation (ASG) baseline shows a more gradual decline in performance over multiple generated sequences, as its prompts incorporate previously selected answers, which aids in generating better outputs. However, despite this benefit, it starts from a lower precision point and experiences a slight decrease as the number of available high-quality options decreases. The strongest baseline is the All Sequences Generation with Resulting Content (ASG-RC). Incorporating the generated questions derived from previously selected answers helps the model better understand the task, resulting in higher precision. However, it still underperforms compared to our method and experiences a steeper decline in precision over time. This highlights the value of Feedback-Aware Inference, as our model's superior performance can be attributed to leveraging feedback labels more effectively. Moreover, the results indicate that the GPT-4o model yields the poorest performance. Despite being a highly capable assistant, it underperforms relative to smaller, open-source, fine-tuned models for specific tasks, underscoring the importance of task-specific fine-tuning and open research.</p> <p>For the Keyword Generation task, the performance of our proposed model remains the highest. In this case, the All Sequences Generation (ASG) has a good starting point, but its sequences decline in quality after a few iterations. Here, GPT-4o also performs poorly since choosing certain keywords requires finetuning on extensive datasets to learn to mimic human behavior and reasoning.</p> <p>Recent studies that use the <emph>KP20K</emph> dataset focus on the extractive task and rely on encoder models. The best results in this case are around 34.5% F1@10 [[<reflink idref="bib17" id="ref36">17</reflink>]] but are not directly comparable with our scores, since they discard keywords that do not appear in the abstract, which makes the task much simpler.</p> <p>It is worth noting that for small <emph>K</emph> values, the Single Sequence Generation (SSG) baseline is competitive with our Feedback-Aware (FA) method. This reflects a structural difference between the two approaches. SSG is trained exclusively to map a context to a single GOOD sequence, which optimizes the quality of individual outputs but leaves the model unaware of previously generated samples at inference time, explaining its sharp degradation as <emph>K</emph> grows. FA, in contrast, is trained on prompts that use both GOOD and BAD prior sequences, so part of its capacity is allocated to interpreting feedback signals and avoiding redundancy across iterations. This results in a slightly lower starting point but a substantially more stable trajectory, which is the property that matters in multi-sample generation. Importantly, the regime in which FA shows its clearest advantage (large <emph>K</emph>) is also the practically relevant one for the tasks we consider: applications such as assessment generation, educational item banks, or keyword pipelines with downstream filtering typically rely on oversampling well beyond <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi><mo>=</mo><mn>10</mn></mrow></semantics></math> </ephtml> , making sustained quality across iterations more valuable than peak quality at the first sample.</p> <p>The shape of the curves in Figure 4 differs between the two tasks for reasons rooted in the metric and the data rather than in model behavior. For Answer Selection, evaluation uses precision, and the pool of valid answer spans per context is large, so adding generations primarily exposes the gradual exhaustion of the GOOD pool, producing a monotonically declining curve. For Keyword Generation, evaluation uses F1, and the number of GOOD keywords per abstract is small and bounded—the 95th percentile in KP20K is 10. At small <emph>K</emph>, the score is recall-limited, since few of the true keywords have yet been proposed; once <emph>K</emph> exceeds the typical number of true keywords, additional generations are necessarily drawn from the BAD set, so precision drops faster than recall rises and F1 peaks before declining. Therefore, the peak reflects the bounded ground-truth set size in KP20K and the symmetric form of F1, not a property of the Feedback-Aware mechanism.</p> <hd id="AN0194175640-19">6.2. Statistical Significance Analysis</hd> <p>To assess the robustness of the observed gains, we conducted paired statistical tests comparing Feedback-Aware Inference against each baseline on per-context scores from the existing inference outputs. The paired design pairs every method on the same ordered list of contexts, removing variance from context difficulty and yielding higher power than treating methods as independent samples. We used the Wilcoxon signed-rank test (two-sided, with the zsplit convention for tied differences) as the primary procedure, since Precision@K and F1@K are bounded in <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo>[</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>]</mo></mrow></semantics></math> </ephtml> and highly non-normal—at small <emph>K</emph>, they take only a handful of discrete values, violating the paired <emph>t</emph>-test's normality assumption. Within each <emph>K</emph>, <emph>p</emph>-values were adjusted using Holm—Bonferroni correction to control the family-wise error rate at <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>α</mi><mo>=</mo><mn>0.05</mn></mrow></semantics></math> </ephtml> . A paired <emph>t</emph>-test and a paired bootstrap with 10,000 resamples on the mean difference were computed as sanity checks; conclusions agreed with Wilcoxon at every <emph>K</emph>, so no result depends on a single test's assumptions. For Answer Selection on FairytaleQA ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>n</mi><mo>=</mo><mn>2007</mn></mrow></semantics></math> </ephtml> contexts; Table 3), the improvements of Feedback-Aware Inference over ASG and GPT-4o are statistically significant at all evaluated <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi><mo>∈</mo><mo>{</mo><mn>1</mn><mo>,</mo><mn>10</mn><mo>,</mo><mn>20</mn><mo>,</mo><mn>25</mn><mo>}</mo></mrow></semantics></math> </ephtml> ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>p</mi><mi>Holm</mi></msub><mo><</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>7</mn></mrow></msup></mrow></semantics></math> </ephtml> ), over ASG-RC for <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi><mo>≥</mo><mn>10</mn></mrow></semantics></math> </ephtml> , and over SSG for <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi><mo>≥</mo><mn>20</mn></mrow></semantics></math> </ephtml> . The non-significant comparisons against SSG and ASG-RC at small <emph>K</emph> are consistent with the structural argument given earlier: SSG is optimized to map a context to a single high-quality output, so its peak quality at <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi><mo>=</mo><mn>1</mn></mrow></semantics></math> </ephtml> is competitive, and the gap only opens up as <emph>K</emph> grows and SSG exhausts its candidate space. For Keyword Generation on KP20K ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>n</mi><mo>=</mo><mn>1998</mn></mrow></semantics></math> </ephtml> contexts; Table 4), the improvements over SSG and GPT-4o are significant at all <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi><mo>∈</mo><mo>{</mo><mn>1</mn><mo>,</mo><mn>3</mn><mo>,</mo><mn>5</mn><mo>,</mo><mn>10</mn><mo>}</mo></mrow></semantics></math> </ephtml> ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>p</mi><mi>Holm</mi></msub><mo><</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>5</mn></mrow></msup></mrow></semantics></math> </ephtml> ), and over ASG for <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi><mo>≥</mo><mn>5</mn></mrow></semantics></math> </ephtml> . The two non-significant comparisons against ASG (at <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi><mo>=</mo><mn>1</mn></mrow></semantics></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi><mo>=</mo><mn>3</mn></mrow></semantics></math> </ephtml> ) fall in the recall-limited regime of the F1 curve discussed in Section 6.1, where few of the true keywords have yet been proposed, and the margin between methods is necessarily compressed; the gap becomes significant once <emph>K</emph> reaches the typical size of the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> pool.</p> <hd id="AN0194175640-20">6.3. Qualitative Analysis</hd> <p>To complement the main results presented in the previous section, we perform a qualitative inspection of the generated sequences. The goal of this analysis is to better understand how the proposed Feedback-Aware Inference mechanism influences the generation process across multiple iterations.</p> <p>Table 5 and Table 6 present representative examples for the two evaluated tasks: Answer Selection for Question Generation and Keyword Generation. For each generation step, the tables report the selected sequence together with the assigned label from the labeling system. These examples allow us to examine how generation quality evolves across iterations and how different models explore the space of candidate outputs. The <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>E</mi><mi>P</mi><mi>E</mi><mi>A</mi><mi>T</mi><mi>E</mi><mi>D</mi></mrow></semantics></math> </ephtml> label denotes a candidate that is semantically equivalent to one already produced earlier in the same list—for example, "simile" followed later by "a simile". This label appears only in the qualitative tables and is provided to make the redundancy in the generation patterns visible to the reader. In the quantitative evaluation, exact duplicates are removed before scoring (as noted for SSG in Section 4.3), so <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>E</mi><mi>P</mi><mi>E</mi><mi>A</mi><mi>T</mi><mi>E</mi><mi>D</mi></mrow></semantics></math> </ephtml> candidates do not affect any reported metric. The behavior is most pronounced for GPT-4o because it produces the entire list in a single pass and therefore lacks iterative awareness of its own prior outputs.</p> <p>For the Answer Selection task (Table 5), the Feedback-Aware model maintains a high proportion of valid selections during the early iterations. The generated spans cover multiple semantic aspects of the passage, including emotional descriptions, figurative language concepts, and illustrative examples used in the text. This indicates that the model effectively explores different informative regions of the context while avoiding redundancy. The appearance of only a small number of rejected candidates suggests that the feedback signals help the model prioritize spans that lead to meaningful question–answer pairs.</p> <p>In contrast, the baseline models exhibit a more uneven progression in generation quality. Several early generations correspond to tokens or phrases that appear in the text but are less suitable as standalone answers for Question Generation. As the iterations progress, the baselines tend to produce spans that are either overly generic or poorly aligned with the intended question–answer structure. In addition, some models revisit closely related concepts or generate spans that lead to similar questions, indicating a limited exploration of the candidate space.</p> <p>A similar pattern can be observed in the Keyword Generation example (Table 6). The Feedback-Aware model produces keywords that capture multiple central themes of the abstract, including the notions of productivity, knowledge workers, and organizational management. These terms correspond to the core ideas discussed throughout the text, suggesting that the model successfully identifies conceptually important elements of the document. The generated keywords also show a relatively balanced distribution across the document's themes rather than concentrating on a single concept.</p> <p>The baseline models, by comparison, generate a larger number of keywords that are either overly specific to individual sentences or only loosely related to the main topic of the abstract. In several cases, later generations correspond to variations of previously generated ideas or to phrases that do not align with the annotated keyword set. This behavior indicates a tendency to drift toward less informative candidates as the generation process continues.</p> <p>Overall, the qualitative examples support the trends observed in the quantitative evaluation. The Feedback-Aware model demonstrates a more stable generation process across iterations, maintaining a higher proportion of acceptable outputs while covering diverse aspects of the input text. Baseline approaches, lacking explicit feedback signals during inference, are more prone to producing less informative or redundant candidates as additional generations are produced.</p> <hd id="AN0194175640-21">6.4. ORPO Alignment Experiments</hd> <p>Categorizing generated sequences as <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> also leads us to consider an approach involving preference fine-tuning. Multiple alignment techniques (e.g., [[<reflink idref="bib1" id="ref37">1</reflink>], [<reflink idref="bib29" id="ref38">29</reflink>]]) leverage positive and negative examples to train models to generate content close to the positive choice and diverge from the negative one. We experimented with the ORPO method [[<reflink idref="bib29" id="ref39">29</reflink>]] to fine-tune our Feedback-Aware model and the Single Sequence Generation baseline on positive ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> answers) and negative ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> answers) sequences. ORPO incorporates an odds-ratio-based penalty for differentiating between chosen and rejected responses in the conventional loss computation. We chose ORPO since it outperformed the SFT+DPO setup [[<reflink idref="bib29" id="ref40">29</reflink>]], fine-tunes only on positive/negative examples, and does not require previous training. More formally, for a prompt in the form of [Prompt, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>O</mi><mi>O</mi><mi>D</mi></mrow></semantics></math> </ephtml> sequence], we diverge from the negative [Prompt, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi><mi>A</mi><mi>D</mi></mrow></semantics></math> </ephtml> sequence].</p> <p>The ORPO alignment is done with the same setup as supervised fine-tuning (SFT), with an additional specific hyperparameter, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>β</mi><mo>=</mo><mn>0.3</mn></mrow></semantics></math> </ephtml> , that defines the weight given to the odds ratio-based penalty in regards to the classical negative log-likelihood loss.</p> <p>Figure 5 highlights the results of this experiment. While ORPO yields similar results to supervised fine-tuning in our case, a different case is made for the Single Sequence Generation baseline. In the case of Answer Selection for Question Generation, ORPO helps at first while starting from a high point in precision; however, the decrease is abrupt as more sequences are generated, mainly because ORPO tends to polarize the pool of samples, given its positive/negative alignment. This hinders the capability of the model to generate diverse answers. Our method is not affected by this polarization since it uses feedback signals that help the generation to be grounded on previous facts. For the Keyword Generation task, this behavior begins from the first sequence, with ORPO having a poor performance from the start. One explanation is that ORPO specifically penalizes keywords that are not included in the initial list, but not all of them are necessarily unsuitable.</p> <hd id="AN0194175640-22">6.5. Failure Modes</hd> <p>Examination of the qualitative outputs and aggregate results points to several conditions under which Feedback-Aware Inference is more likely to underperform. When the labeling system itself is weak or systematically biased, the iterative process can amplify rather than correct errors, since each generation conditions on prior labels. The relative stability we observe on FairytaleQA—for which the labeling pipeline (QGen, QAns) was not specifically trained—suggests this effect is bounded for moderate labeling noise, but it is expected to grow as the labeling system degrades.</p> <p>Prompt growth across many iterations introduces a risk of context saturation: as the labeled history accumulates, the model must attend to an increasingly long sequence of prior candidates. Finally, when the pool of valid outputs for a context is small, later iterations are constrained to draw from an exhausted candidate space, producing the gradual decline visible at high <emph>K</emph> in Figure 4 even for the best-performing method.</p> <hd id="AN0194175640-23">6.6. Limitations</hd> <p>Several limitations of the present study should be acknowledged. First, hyperparameter tuning was deliberately limited in scope, given the computational cost of an exhaustive search over the full pipeline. We mitigated this by holding training and decoding hyperparameters constant across our method and all baselines, which preserves the validity of the comparison even if absolute scores could potentially be improved by per-method tuning.</p> <p>Second, the method requires negative samples for training, which are not available in all datasets. When such samples must be constructed (as for KP20K), the construction procedure must yield contrastively informative negatives; trivially easy negatives would weaken the learned distinction between GOOD and BAD outputs.</p> <p>Third, the framework's behavior depends on the quality of the labeling system. Although our experiments suggest robustness to moderate labeling imperfection (Section 3.1), systematically biased or very noisy labels could, in principle, propagate across iterations.</p> <p>Fourth, the iterative procedure introduces inference-time overhead relative to single-pass baselines, including additional generation calls, additional labeling-system calls, and a prompt that grows linearly with the number of iterations.</p> <p>Fifth, the feedback signal in our experiments is binary; while the framework supports richer label structures, we have not empirically evaluated graded or multi-dimensional feedback.</p> <p>Finally, our experiments use a single 8B-parameter base model, so claims regarding behavior at substantially different model scales remain to be verified.</p> <hd id="AN0194175640-24">7. Conclusions and Future Work</hd> <p>This paper introduced a Feedback-Aware Inference framework designed to enable iterative refinement in sequence generation tasks using relatively small, open-source language models. The proposed method augments the standard generation process with explicit feedback signals that guide the model during inference, allowing it to progressively improve the quality of generated sequences across multiple iterations. Unlike conventional approaches that rely on one-shot generation or static prompting, our method incorporates structured feedback directly into the prompt and trains the model to interpret and utilize these signals when producing subsequent outputs.</p> <p>The experimental results across two distinct tasks, Answer Selection for Question Generation and Keyword Generation, demonstrate that the proposed framework consistently maintains higher-quality outputs across successive generations compared to several strong baselines. In particular, the Feedback-Aware model shows improved robustness when generating multiple candidate sequences, avoiding the rapid degradation in quality commonly observed in models that lack awareness of previously generated outputs. These results suggest that conditioning generation on explicit feedback signals provides a practical mechanism for guiding iterative inference without requiring additional model parameters or expensive retraining procedures.</p> <p>An important aspect of the proposed framework is its flexibility with respect to the source of feedback signals. In the experiments presented in this work, feedback was obtained either from automated proxy models or from human-curated annotations. The results indicate that the framework remains effective under both scenarios, highlighting its potential applicability across a broad range of tasks where different forms of evaluation signals are available. Because the labeling system is modular and independent of the generative model itself, it can be replaced or extended with alternative scoring mechanisms, including rule-based evaluation, external classifiers, or human feedback.</p> <p>Another key advantage of the proposed approach lies in its computational efficiency. All experiments were conducted using a single 8B-parameter open-source model fine-tuned with LoRA, showing that, at the 8B-parameter scale considered here, iterative refinement capabilities commonly associated with large proprietary LLMs can be approximated using significantly smaller models when appropriate feedback mechanisms are incorporated. This property makes the method particularly attractive for scenarios where computational resources, latency requirements, or deployment constraints limit the use of very large models.</p> <p>Several directions remain for future investigation. First, additional experiments on a broader range of generative tasks would further validate the generality of the framework, particularly in settings where multiple candidate outputs must be explored before selecting a final result. Also, validation across multiple model scales and architectures would test whether the gains observed at the 8B-parameter scale transfer to smaller or larger models. A controlled robustness study with synthetically degraded labeling systems would help quantify the framework's tolerance to feedback noise more directly than the indirect evidence offered by our cross-dataset evaluation. Controlled deployment-focused benchmarks, like wall-clock latency, throughput, and GPU memory profiles under realistic batch and sequence-length configurations would provide practical guidance for production use.</p> <p>Second, future work could explore richer forms of feedback beyond the binary GOOD/BAD labeling scheme used in this study. For example, graded feedback signals or multi-dimensional quality metrics could provide more informative guidance to the generation process. Investigating how different feedback structures influence the model's exploration of the output space represents an important research direction.</p> <p>Another promising extension involves integrating the proposed inference strategy with reasoning-oriented tasks. In domains such as mathematical problem solving, logical inference, or scientific explanation, models must often construct multi-step reasoning chains. The Feedback-Aware framework could be adapted to evaluate intermediate reasoning steps and iteratively refine them, potentially improving both correctness and coherence of generated reasoning paths. Conditioning the model on examples of incorrect reasoning trajectories, together with corrective feedback, may further enhance its ability to distinguish valid from invalid reasoning processes.</p> <p>Overall, the results presented in this work indicate that incorporating explicit feedback into the inference process can improve the stability and quality of multi-sample text generation on the two tasks evaluated. The proposed Feedback-Aware framework offers a simple mechanism for enabling iterative refinement in smaller language models on tasks of this kind, suggesting it as a promising alternative to relying solely on very large proprietary systems, within the conditions identified in this study.</p> <hd id="AN0194175640-25">Figures and Tables</hd> <p>Graph: Figure 1 Overview of the Feedback-Aware framework. During training, candidate sequences are labeled using a labeling system and used to fine-tune the Feedback-Aware model. During inference, the model iteratively generates sequences, receives feedback from the labeling system, and updates the prompt to refine subsequent generations.</p> <p>Graph: Figure 2 General prompt template used for Feedback-Aware generation. The prompt includes the task description, the input context, and a sequence of previously generated outputs annotated with feedback labels (GOOD or BAD). Optionally, auxiliary resulting content generated by a task-specific module can also be included. During training, the model learns to predict the final sequence in the template while conditioning on the previous labeled examples, allowing it to implicitly model how feedback influences the quality of future generations.</p> <p>Graph: Figure 3 Examples of task-specific prompts instantiated from the Feedback-Aware template. They illustrate how feedback signals and previously generated outputs are incorporated into the prompt to guide iterative generation.</p> <p>Graph: Figure 4 Main evaluation results for the two tasks considered in this work. (a) Precision@K for Answer Selection for Question Generation, evaluated on the TASA and FairytaleQA datasets. (b) F1@K for Keyword Generation evaluated on the KP20K dataset. The results compare the proposed Feedback-Aware method with several baselines, illustrating the ability of the Feedback-Aware Inference mechanism to maintain higher quality sequences across multiple generated outputs.</p> <p>Graph: Figure 5 Results of alignment experiments using ORPO compared to standard supervised fine-tuning (SFT). (a) Precision@K for Answer Selection for Question Generation. (b) F1@K for Keyword Generation. These experiments evaluate whether preference-based fine-tuning improves performance when positive (GOOD) and negative (BAD) samples are available. The results show that while ORPO provides limited improvements in some cases, the Feedback-Aware Inference strategy remains more stable across iterative generation.</p> <p>Table 1 Training and inference configuration used in the experiments.</p> <p> <ephtml> <table><thead><tr><th align="left" style="border-bottom:solid thin;border-top:solid thin">Stage</th><th align="left" style="border-bottom:solid thin;border-top:solid thin">Parameter</th><th align="left" style="border-bottom:solid thin;border-top:solid thin">Value/Models</th></tr></thead><tbody><tr><td rowspan="6" align="left" valign="middle" style="border-bottom:solid thin">Training</td><td align="left" valign="middle">Base Model</td><td align="left" valign="middle">Llama-3 (8B)—ours, baselines</td></tr><tr><td align="left" valign="middle">Fine-tuning Method</td><td align="left" valign="middle">LoRA (projection matrices for attention layers)—ours, baselines</td></tr><tr><td align="left" valign="middle">Optimizer</td><td align="left" valign="middle">AdamW-8bit—ours, baselines</td></tr><tr><td align="left" valign="middle">Learning Rate</td><td align="left" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>1</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>5</mn></mrow></msup></mrow></semantics></math></p>—ours, baselines</td></tr><tr><td align="left" valign="middle">Batch Size</td><td align="left" valign="middle">64 (via gradient accumulation)—ours, baselines</td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">Precision</td><td align="left" valign="middle" style="border-bottom:solid thin">FP16—ours, baselines</td></tr><tr><td rowspan="6" align="left" valign="middle" style="border-bottom:solid thin">Inference</td><td align="left" valign="middle">Precision</td><td align="left" valign="middle">BF16—prerequisite models (<p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>Q</mi><mi>G</mi><mi>e</mi><mi>n</mi></mrow></semantics></math></p>, <p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>Q</mi><mi>A</mi><mi>n</mi><mi>s</mi></mrow></semantics></math></p>)</td></tr><tr><td align="left" valign="middle">Decoding Strategy</td><td align="left" valign="middle">Nucleus sampling—ours, baselines</td></tr><tr><td align="left" valign="middle">Top-k</td><td align="left" valign="middle">20—ours, baselines</td></tr><tr><td align="left" valign="middle">Top-p</td><td align="left" valign="middle">0.8—ours, baselines</td></tr><tr><td align="left" valign="middle">Random Seed</td><td align="left" valign="middle">42—all models</td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">Default settings</td><td align="left" valign="middle" style="border-bottom:solid thin">GPT-4o</td></tr></tbody></table> </ephtml> </p> <p>Table 2 BLEURT score [[<reflink idref="bib27" id="ref41">27</reflink>]] for the prerequisite models on their tasks.</p> <p> <ephtml> <table><thead><tr><th align="left" style="border-bottom:solid thin;border-top:solid thin"> </th><th align="center" style="border-bottom:solid thin;border-top:solid thin">QGen</th><th align="center" style="border-bottom:solid thin;border-top:solid thin">QAns</th></tr></thead><tbody><tr><td align="left" valign="middle">SQuAD</td><td align="center" valign="middle">0.58</td><td align="center" valign="middle">0.75</td></tr><tr><td align="left" valign="middle">HotpotQA</td><td align="center" valign="middle">0.50</td><td align="center" valign="middle">0.50</td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">NarrativeQA</td><td align="center" valign="middle" style="border-bottom:solid thin">0.54</td><td align="center" valign="middle" style="border-bottom:solid thin">0.60</td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">FairytaleQA</td><td align="center" valign="middle" style="border-bottom:solid thin">0.51</td><td align="center" valign="middle" style="border-bottom:solid thin">0.45</td></tr></tbody></table> </ephtml> </p> <p>Table 3 Paired statistical tests comparing Feedback-Aware Inference (FA) against each baseline on Precision@K for Answer Selection on FairytaleQA ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>n</mi><mo>=</mo><mn>2007</mn></mrow></semantics></math> </ephtml> contexts). Mean <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mo>Δ</mo></semantics></math> </ephtml> is FA minus baseline; positive values indicate FA wins. <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>p</mi><mi>Holm</mi></msub></semantics></math> </ephtml> denotes Holm–Bonferroni-adjusted <emph>p</emph>-values from the Wilcoxon signed-rank test, applied within each <emph>K</emph>. Significance: *** <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>p</mi><mo><</mo><mn>0.001</mn></mrow></semantics></math> </ephtml> , ** <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>p</mi><mo><</mo><mn>0.01</mn></mrow></semantics></math> </ephtml> , ns = not significant.</p> <p> <ephtml> <table><thead><tr><th align="center" style="border-bottom:solid thin;border-top:solid thin">K</th><th align="center" style="border-bottom:solid thin;border-top:solid thin">Baseline</th><th align="center" style="border-bottom:solid thin;border-top:solid thin">Mean <p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mstyle mathvariant="bold"><mo>Δ</mo></mstyle></semantics></math></p></th><th align="center" style="border-bottom:solid thin;border-top:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mstyle mathvariant="bold"><msub><mi mathvariant="bold-italic">p</mi><mi mathvariant="bold">Holm</mi></msub></mstyle></semantics></math></p></th><th align="center" style="border-bottom:solid thin;border-top:solid thin">Sig.</th></tr></thead><tbody><tr><td rowspan="4" align="center" valign="middle" style="border-bottom:solid thin">1</td><td align="center" valign="middle">SSG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0005</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>9.70</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>1</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">ns</td></tr><tr><td align="center" valign="middle">ASG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0812</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>8.39</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>8</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle">ASG-RC</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0244</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>1.62</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>1</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">ns</td></tr><tr><td align="center" valign="middle" style="border-bottom:solid thin">GPT-4o</td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.1340</mn></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>1.49</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>17</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin">***</td></tr><tr><td rowspan="4" align="center" valign="middle" style="border-bottom:solid thin">10</td><td align="center" valign="middle">SSG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>−</mo><mn>0.0001</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>9.44</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>1</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">ns</td></tr><tr><td align="center" valign="middle">ASG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0496</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>1.19</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>34</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle">ASG-RC</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0125</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>1.74</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>3</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">**</td></tr><tr><td align="center" valign="middle" style="border-bottom:solid thin">GPT-4o</td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.2520</mn></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>1.44</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>192</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin">***</td></tr><tr><td rowspan="4" align="center" valign="middle" style="border-bottom:solid thin">20</td><td align="center" valign="middle">SSG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0517</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>5.85</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>71</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle">ASG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0633</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>1.52</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>96</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle">ASG-RC</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0376</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>4.52</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>43</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle" style="border-bottom:solid thin">GPT-4o</td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.2841</mn></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>2.76</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>271</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin">***</td></tr><tr><td rowspan="4" align="center" valign="middle" style="border-bottom:solid thin">25</td><td align="center" valign="middle">SSG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.1104</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>1.04</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>227</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle">ASG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0632</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>2.55</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>105</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle">ASG-RC</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0529</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>4.20</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>88</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle" style="border-bottom:solid thin">GPT-4o</td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.2828</mn></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>3.18</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>292</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin">***</td></tr></tbody></table> </ephtml> </p> <p>Table 4 Paired statistical tests comparing Feedback-Aware Inference (FA) against each baseline on F1@K for Keyword Generation on KP20K ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>n</mi><mo>=</mo><mn>1998</mn></mrow></semantics></math> </ephtml> contexts). Mean <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mo>Δ</mo></semantics></math> </ephtml> is FA minus baseline; positive values indicate FA wins. <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>p</mi><mi>Holm</mi></msub></semantics></math> </ephtml> denotes Holm–Bonferroni-adjusted <emph>p</emph>-values from the Wilcoxon signed-rank test, applied within each <emph>K</emph>. Significance: *** <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>p</mi><mo><</mo><mn>0.001</mn></mrow></semantics></math> </ephtml> , ns = not significant.</p> <p> <ephtml> <table><thead><tr><th align="center" style="border-bottom:solid thin;border-top:solid thin">K</th><th align="center" style="border-bottom:solid thin;border-top:solid thin">Baseline</th><th align="center" style="border-bottom:solid thin;border-top:solid thin">Mean <p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mstyle mathvariant="bold"><mo>Δ</mo></mstyle></semantics></math></p></th><th align="center" style="border-bottom:solid thin;border-top:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mstyle mathvariant="bold"><msub><mi mathvariant="bold-italic">p</mi><mi mathvariant="bold">Holm</mi></msub></mstyle></semantics></math></p></th><th align="center" style="border-bottom:solid thin;border-top:solid thin">Sig.</th></tr></thead><tbody><tr><td rowspan="3" align="center" valign="middle" style="border-bottom:solid thin">1</td><td align="center" valign="middle">SSG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0651</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>2.76</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>6</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle">ASG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>−</mo><mn>0.0160</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>2.59</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>1</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">ns</td></tr><tr><td align="center" valign="middle" style="border-bottom:solid thin">GPT-4o</td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.1431</mn></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>1.41</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>20</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin">***</td></tr><tr><td rowspan="3" align="center" valign="middle" style="border-bottom:solid thin">3</td><td align="center" valign="middle">SSG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0607</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>2.90</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>22</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle">ASG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0118</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>9.45</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>2</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">ns</td></tr><tr><td align="center" valign="middle" style="border-bottom:solid thin">GPT-4o</td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.1612</mn></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>1.12</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>86</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin">***</td></tr><tr><td rowspan="3" align="center" valign="middle" style="border-bottom:solid thin">5</td><td align="center" valign="middle">SSG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0435</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>6.47</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>24</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle">ASG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0218</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>1.33</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>5</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle" style="border-bottom:solid thin">GPT-4o</td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.1309</mn></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>1.99</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>107</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin">***</td></tr><tr><td rowspan="3" align="center" valign="middle" style="border-bottom:solid thin">10</td><td align="center" valign="middle">SSG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0351</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>6.94</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>46</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle">ASG</td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0262</mn></mrow></semantics></math></p></td><td align="center" valign="middle"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>4.69</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>16</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle">***</td></tr><tr><td align="center" valign="middle" style="border-bottom:solid thin">GPT-4o</td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mo>+</mo><mn>0.0764</mn></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mn>7.48</mn><mo>×</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>104</mn></mrow></msup></mrow></semantics></math></p></td><td align="center" valign="middle" style="border-bottom:solid thin">***</td></tr></tbody></table> </ephtml> </p> <p>Table 5 Qualitative example for the Answer Selection for Question Generation task. Each row corresponds to one generation step, together with the resulting generated question and the label assigned by the labeling system. <bold>Context:</bold> "Read the following two sentences and decide which one creates a more vivid picture in your mind: Cress was very happy. Cress was as happy as a baby who has just picked its first flower. Both sentences express the same idea, but in the second one the author has made the description more vivid for you by using figurative language. Instead of simply saying that Cress was happy, the author has compared her happiness with that of a surprised baby who has succeeded in picking a flower. This comparison creates a more meaningful picture in your mind. Authors use figurative language to make their meaning clearer, stir your imagination, and consequently make their writing more enjoyable. In this lesson, we are going to study three types of figurative language, simile, metaphor, and personification. The second sentence from the example is similar to one you read in "Cress Delahanty". The figurative language used in this sentence is called a simile. A simile is a comparison of two unlike things. The comparison is made using the word "like" or "as". We often use similes in our conversation. You may have heard such similies as "she ran like a deer" or "he was as hungry as a lion". Although you may never have seen a running deer or a hungry lion, the comparisons help you understand that the girl ran very fast and that the boy was very hungry. These expressions build your understanding in a colorful way that increases your enjoyment. Not all comparisons are similes, however. The statement, "that girl looks like her mother", is not a simile because two similar things, people, are being compared. Similes compare things that are basically quite different, for example, a girl and a deer or a boy and a lion. The context of what you are reading helps you to understand the few ways in which the two dissimilar things are alike; in this way you get the meaning that is intended."— <bold>Source:</bold> TASA dataset entry [[<reflink idref="bib28" id="ref42">28</reflink>]].</p> <p> <ephtml> <table><thead><tr><th align="left" style="border-bottom:solid thin;border-top:solid thin">Model</th><th align="left" style="border-bottom:solid thin;border-top:solid thin">Gen.</th><th align="left" style="border-bottom:solid thin;border-top:solid thin">Generated Sequence</th><th align="left" style="border-bottom:solid thin;border-top:solid thin">Resulting Content</th><th align="left" style="border-bottom:solid thin;border-top:solid thin">Label</th></tr></thead><tbody><tr><td rowspan="5" align="left" valign="middle" style="border-bottom:solid thin">FA (ours)</td><td align="left" valign="middle">1</td><td align="left" valign="middle">happy</td><td align="left" valign="middle">What emotion is Cress feeling?</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">2</td><td align="left" valign="middle">vivid</td><td align="left" valign="middle">What kind of picture does the second sentence create?</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">3</td><td align="left" valign="middle">lion</td><td align="left" valign="middle">What animal is often used in similes to describe hunger?</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">4</td><td align="left" valign="middle">personification</td><td align="left" valign="middle">What is the term for giving human characteristics to non-human things?</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">5</td><td align="left" valign="middle" style="border-bottom:solid thin">girl</td><td align="left" valign="middle" style="border-bottom:solid thin">What is the subject of the simile?</td><td align="left" valign="middle" style="border-bottom:solid thin"><bold>BAD</bold></td></tr><tr><td rowspan="5" align="left" valign="middle" style="border-bottom:solid thin">SSG</td><td align="left" valign="middle">1</td><td align="left" valign="middle">Authors</td><td align="left" valign="middle">Who uses figurative language?</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">2</td><td align="left" valign="middle">happy</td><td align="left" valign="middle">What emotion is Cress feeling?</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">3</td><td align="left" valign="middle">few</td><td align="left" valign="middle">How many ways are the two dissimilar things alike?</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle">4</td><td align="left" valign="middle">girl</td><td align="left" valign="middle">What is the subject of the simile?</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">5</td><td align="left" valign="middle" style="border-bottom:solid thin">two</td><td align="left" valign="middle" style="border-bottom:solid thin">How many types of figurative language are there?</td><td align="left" valign="middle" style="border-bottom:solid thin"><bold>BAD</bold></td></tr><tr><td rowspan="5" align="left" valign="middle" style="border-bottom:solid thin">ASG</td><td align="left" valign="middle">1</td><td align="left" valign="middle">simile</td><td align="left" valign="middle">What type of figurative language is used in the sentence "Cress was as happy as a baby who has just picked its first flower"?</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle">2</td><td align="left" valign="middle">similies</td><td align="left" valign="middle">What are comparisons called?</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle">3</td><td align="left" valign="middle">happy</td><td align="left" valign="middle">What emotion is Cress feeling?</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">4</td><td align="left" valign="middle">girl</td><td align="left" valign="middle">What is the subject of the simile?</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">5</td><td align="left" valign="middle" style="border-bottom:solid thin">surprised</td><td align="left" valign="middle" style="border-bottom:solid thin">How does the baby feel?</td><td align="left" valign="middle" style="border-bottom:solid thin"><bold>GOOD</bold></td></tr><tr><td rowspan="5" align="left" valign="middle" style="border-bottom:solid thin">ASG-RC</td><td align="left" valign="middle">1</td><td align="left" valign="middle">comparison</td><td align="left" valign="middle">What is a simile?</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle">2</td><td align="left" valign="middle">happy</td><td align="left" valign="middle">What emotion is Cress feeling?</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">3</td><td align="left" valign="middle">vivid</td><td align="left" valign="middle">What kind of picture does the second sentence create?</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">4</td><td align="left" valign="middle">figurative</td><td align="left" valign="middle">What type of language is a simile?</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">5</td><td align="left" valign="middle" style="border-bottom:solid thin">author</td><td align="left" valign="middle" style="border-bottom:solid thin">Who uses figurative language?</td><td align="left" valign="middle" style="border-bottom:solid thin"><bold>BAD</bold></td></tr><tr><td rowspan="5" align="left" valign="middle" style="border-bottom:solid thin">GPT-4o</td><td align="left" valign="middle">1</td><td align="left" valign="middle">figurative language</td><td align="left" valign="middle">What do authors use to make their meaning clearer, stir imagination, and make writing more enjoyable?</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">2</td><td align="left" valign="middle">simile</td><td align="left" valign="middle">What is a comparison of two unlike things using the word "like" or "as" called?</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle">3</td><td align="left" valign="middle">a simile</td><td align="left" valign="middle">What is a comparison of two unlike things?</td><td align="left" valign="middle"><bold>REPEATED</bold></td></tr><tr><td align="left" valign="middle">4</td><td align="left" valign="middle">Cress was as happy as a baby who has just picked its first flower</td><td align="left" valign="middle">Which sentence creates a vivid picture of happiness?</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">5</td><td align="left" valign="middle" style="border-bottom:solid thin">she ran like a deer</td><td align="left" valign="middle" style="border-bottom:solid thin">Give an example of a simile.</td><td align="left" valign="middle" style="border-bottom:solid thin"><bold>GOOD</bold></td></tr></tbody></table> </ephtml> </p> <p>Table 6 Qualitative example for the Keyword Generation task. The table illustrates how keywords are generated sequentially together with the feedback labels used to guide the generation process. <bold>Context:</bold> "Experts in specific professional domains form the fastest increasing workforce in OECD countries. Since this fact has been realised by management researchers, they have focussed on the question of how to measure and enhance the productivity of said workforce. According to the author's cross-industrial research undertaken in five different knowledge-intensive organisations, it is, however, not productivity in the traditional meaning of the term which is to be regarded as the crucial performance indicator in expert work. There rather exist multiple performance indicators, each of which is, moreover, differently graded as to its importance by different stakeholders. These findings, firstly, indicate the need for an alternative definition and way of measurement of productivity when the term is applied to knowledge work, and, secondly, they indicate the need for alternative management strategies in order to generate an increase in the productivity of knowledge workers."—<bold>Source:</bold> FairytaleQA dataset entry [[<reflink idref="bib15" id="ref43">15</reflink>]].</p> <p> <ephtml> <table><thead><tr><th align="left" style="border-bottom:solid thin;border-top:solid thin">Model</th><th align="left" style="border-bottom:solid thin;border-top:solid thin">Gen.</th><th align="left" style="border-bottom:solid thin;border-top:solid thin">Generated Keyword</th><th align="left" style="border-bottom:solid thin;border-top:solid thin">Label</th></tr></thead><tbody><tr><td rowspan="5" align="left" valign="middle" style="border-bottom:solid thin">FA (ours)</td><td align="left" valign="middle">1</td><td align="left" valign="middle">productivity</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">2</td><td align="left" valign="middle">knowledge workers</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">3</td><td align="left" valign="middle">performance indicators</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle">4</td><td align="left" valign="middle">experts</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">5</td><td align="left" valign="middle" style="border-bottom:solid thin">management</td><td align="left" valign="middle" style="border-bottom:solid thin"><bold>GOOD</bold></td></tr><tr><td rowspan="4" align="left" valign="middle" style="border-bottom:solid thin">SSG</td><td align="left" valign="middle">1</td><td align="left" valign="middle">knowledge management</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle">2</td><td align="left" valign="middle">performance</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">3</td><td align="left" valign="middle">managing forces</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">4</td><td align="left" valign="middle" style="border-bottom:solid thin">expert productivity</td><td align="left" valign="middle" style="border-bottom:solid thin"><bold>BAD</bold></td></tr><tr><td rowspan="5" align="left" valign="middle" style="border-bottom:solid thin">ASG</td><td align="left" valign="middle">1</td><td align="left" valign="middle">productivity</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">2</td><td align="left" valign="middle">knowledge work</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">3</td><td align="left" valign="middle">performance indicators</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle">4</td><td align="left" valign="middle">managing forces</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">5</td><td align="left" valign="middle" style="border-bottom:solid thin">management strategies</td><td align="left" valign="middle" style="border-bottom:solid thin"><bold>BAD</bold></td></tr><tr><td rowspan="5" align="left" valign="middle" style="border-bottom:solid thin">GPT-4o</td><td align="left" valign="middle">1</td><td align="left" valign="middle">knowledge work</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">2</td><td align="left" valign="middle">productivity</td><td align="left" valign="middle"><bold>GOOD</bold></td></tr><tr><td align="left" valign="middle">3</td><td align="left" valign="middle">performance indicators</td><td align="left" valign="middle"><bold>BAD</bold></td></tr><tr><td align="left" valign="middle">4</td><td align="left" valign="middle">knowledge</td><td align="left" valign="middle"><bold>REPEATED</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin">5</td><td align="left" valign="middle" style="border-bottom:solid thin">management strategies</td><td align="left" valign="middle" style="border-bottom:solid thin"><bold>BAD</bold></td></tr></tbody></table> </ephtml> </p> <hd id="AN0194175640-26">Author Contributions</hd> <p>Conceptualization, A.D., S.R. and M.D.; methodology, A.D. and S.R.; software, A.D.; validation, A.D., S.R. and M.D.; formal analysis, A.D.; investigation, A.D.; resources, A.D.; data curation, A.D.; writing—original draft preparation, A.D.; writing—review and editing, A.D., S.R., M.D. and D.S.M.; visualization, A.D.; supervision, S.R., M.D. and D.S.M.; project administration, M.D.; funding acquisition, S.R., M.D. and D.S.M. All authors have read and agreed to the published version of the manuscript.</p> <hd id="AN0194175640-27">Institutional Review Board Statement</hd> <p>Not applicable.</p> <hd id="AN0194175640-28">Informed Consent Statement</hd> <p>Not applicable.</p> <hd id="AN0194175640-29">Data Availability Statement</hd> <p>The data, code and fine-tuned models presented in this study are openly available at https://github.com/upb-nlp/Feedback-Aware (accessed on 5 May 2026).</p> <hd id="AN0194175640-30">Conflicts of Interest</hd> <p>The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.</p> <hd id="AN0194175640-31">Abbreviations</hd> <p>The following abbreviations are used in this manuscript:</p> <p></p> <p> <ephtml> <table><tbody><tr><td align="left" valign="middle">NLP</td><td align="left" valign="middle">Natural Language Processing</td></tr><tr><td align="left" valign="middle">LLM</td><td align="left" valign="middle">Large Language Model</td></tr><tr><td align="left" valign="middle">LS</td><td align="left" valign="middle">Labeling System</td></tr><tr><td align="left" valign="middle">RCG</td><td align="left" valign="middle">Resulting Content Generator</td></tr><tr><td align="left" valign="middle">FAM</td><td align="left" valign="middle">Feedback-Aware Model</td></tr><tr><td align="left" valign="middle">SSG</td><td align="left" valign="middle">Single Sequence Generation</td></tr><tr><td align="left" valign="middle">ASG</td><td align="left" valign="middle">All Sequences Generation</td></tr><tr><td align="left" valign="middle">ASG-RC</td><td align="left" valign="middle">All Sequences Generation with Resulting Content</td></tr><tr><td align="left" valign="middle">QGen</td><td align="left" valign="middle">Question Generation Model</td></tr><tr><td align="left" valign="middle">QAns</td><td align="left" valign="middle">Question-Answering Model</td></tr><tr><td align="left" valign="middle">RLHF</td><td align="left" valign="middle">Reinforcement Learning from Human Feedback</td></tr><tr><td align="left" valign="middle">DPO</td><td align="left" valign="middle">Direct Preference Optimization</td></tr><tr><td align="left" valign="middle">ORPO</td><td align="left" valign="middle">Odds Ratio Preference Optimization</td></tr><tr><td align="left" valign="middle">SFT</td><td align="left" valign="middle">Supervised Fine-Tuning</td></tr><tr><td align="left" valign="middle">GPU</td><td align="left" valign="middle">Graphics Processing Unit</td></tr><tr><td align="left" valign="middle">BLEURT</td><td align="left" valign="middle">Bilingual Evaluation Understudy with Representations from Transformers</td></tr><tr><td align="left" valign="middle">FP16</td><td align="left" valign="middle">16-bit Floating Point Precision</td></tr><tr><td align="left" valign="middle">BF16</td><td align="left" valign="middle">Brain Floating Point 16-bit</td></tr></tbody></table> </ephtml> </p> <hd id="AN0194175640-32">Acknowledgments</hd> <p>During the preparation of this manuscript, the authors used generative AI for the purposes of spellchecking and grammar. The authors have reviewed and edited the output and take full responsibility for the content of this publication.</p> <hd id="AN0194175640-33">Appendix A. Prompts</hd> <p>This section contains the prompts used for different models and tasks. In <bold>{{bold}}</bold> we denote the expected generated text by the model.</p> <hd id="AN0194175640-34">Appendix A.1. Answer Selection for Question Generation</hd> <p></p> <p> <ephtml> <table><tbody><tr><td align="left" valign="middle" style="border-top:solid thin;border-left:solid thin;border-right:solid thin;background:#E6E6E6">        <bold>Feedback-Aware Model</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">Iteratively select a span from the following text that would serve as a good answer for generating a question.</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Text: {{Text}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Response:</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        GOOD: {{Previous selected answer}} - {{Previous resulting question}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        BAD: {{Prev. selected answer}} - {{Prev. resulting question}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ...</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        GOOD: {{Prev. selected answer}} - {{Prev. resulting question}}</td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin;border-left:solid thin;border-right:solid thin;background:whitesmoke">        GOOD: <bold>{{Selected answer}}</bold></td></tr></tbody></table> </ephtml> </p> <p></p> <p> <ephtml> <table><tbody><tr><td align="left" valign="middle" style="border-top:solid thin;border-left:solid thin;border-right:solid thin;background:#E6E6E6">        <bold>Single Sequence Generation</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">Select a span from the following text that would serve as a good answer for generating a question.</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Text: {{Text}}</td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin;border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Response: <bold>{{Selected answer}}</bold></td></tr></tbody></table> </ephtml> </p> <p></p> <p> <ephtml> <table><tbody><tr><td align="left" valign="middle" style="border-top:solid thin;border-left:solid thin;border-right:solid thin;background:#E6E6E6">        <bold>All Sequences Generation</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">Iteratively select a span from the following text that would serve as a good answer for generating a question.</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Text: {{Text}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Response:</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected answer}}</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected answer}}</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>...</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin;border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected answer}}</bold></td></tr></tbody></table> </ephtml> </p> <p></p> <p> <ephtml> <table><tbody><tr><td align="left" valign="middle" style="border-top:solid thin;border-left:solid thin;border-right:solid thin;background:#E6E6E6">        <bold>All Sequences Generation with Resulting Content</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">Iteratively select a span from the following text that would serve as a good answer for generating a question.</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Text: {{Text}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Response:</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        {{Previous selected answer}} - {{Previous resulting question}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        {{Prev. selected answer}} - {{Prev. resulting question}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ...</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        {{Prev. selected answer}} - {{Prev. resulting question}}</td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin;border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected answer}}</bold></td></tr></tbody></table> </ephtml> </p> <p></p> <p> <ephtml> <table><tbody><tr><td align="left" valign="middle" style="border-top:solid thin;border-left:solid thin;border-right:solid thin;background:#E6E6E6">        <bold>GPT-4o</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">Select 25 spans from the following text that would serve as good answers for generating questions. Write your selected answers together with the corresponding question on separate lines, in the following format: <answer> -> <question>. Don't add any additional characters or numbering. Take into consideration the following example:</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        {{Example text and response}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Text: {{Text}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected answer}} - {{Generated question}}</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected answer}} - {{Generated question}}</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>...</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin;border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected answer}} - {{Generated question}}</bold></td></tr></tbody></table> </ephtml> </p> <hd id="AN0194175640-35">Appendix A.2. Keyword Generation</hd> <p></p> <p> <ephtml> <table><tbody><tr><td align="left" valign="middle" style="border-top:solid thin;border-left:solid thin;border-right:solid thin;background:#E6E6E6">        <bold>Feedback-Aware Model</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">Iteratively select keywords for the following text.</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Text: {{Text}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Response:</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        GOOD: {{Previous selected keyword}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        BAD: {{Prev. selected keyword}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ...</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        GOOD: {{Prev. selected keyword}}</td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin;border-left:solid thin;border-right:solid thin;background:whitesmoke">        GOOD: <bold>{{Selected keyword}}</bold></td></tr></tbody></table> </ephtml> </p> <p></p> <p> <ephtml> <table><tbody><tr><td align="left" valign="middle" style="border-top:solid thin;border-left:solid thin;border-right:solid thin;background:#E6E6E6">        <bold>Single Sequence Generation</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">Select keywords for the following text.</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Text: {{Text}}</td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin;border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Response: <bold>{{Selected keyword}}</bold></td></tr></tbody></table> </ephtml> </p> <p></p> <p> <ephtml> <table><tbody><tr><td align="left" valign="middle" style="border-top:solid thin;border-left:solid thin;border-right:solid thin;background:#E6E6E6">        <bold>All Sequences Generation</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">Select keywords for the following text.</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Text: {{Text}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Response:</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected keyword}}</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected keyword}}</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>...</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin;border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected keyword}}</bold></td></tr></tbody></table> </ephtml> </p> <p></p> <p> <ephtml> <table><tbody><tr><td align="left" valign="middle" style="border-top:solid thin;border-left:solid thin;border-right:solid thin;background:#E6E6E6">        <bold>GPT-4o</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">Generate 15 keywords for the following abstract. Write your selected answers on sepa-rate lines. Don't add any additional characters or numbering. Take into consideration the following example: {{Example text and response}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        ### Text: {{Text}}</td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected keyword}}</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected keyword}}</bold></td></tr><tr><td align="left" valign="middle" style="border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>...</bold></td></tr><tr><td align="left" valign="middle" style="border-bottom:solid thin;border-left:solid thin;border-right:solid thin;background:whitesmoke">        <bold>{{Selected keyword}}</bold></td></tr></tbody></table> </ephtml> </p> <ref id="AN0194175640-36"> <title> Footnotes </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> Disclaimer/Publisher's Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.</bibtext> </blist> </ref> <ref id="AN0194175640-37"> <title> References </title> <blist> <bibtext> Ouyang L., Wu J., Jiang X., Almeida D., Wainwright C., Mishkin P., Zhang C., Agarwal S., Slama K., Ray A. Training language models to follow instructions with human feedback. Adv. Neural Inf. Process. Syst. 2022; 35: 27730-27744</bibtext> </blist> <blist> <bibl id="bib2" idref="ref2" type="bt">2</bibl> <bibtext> Rafailov R., Sharma A., Mitchell E., Manning C.D., Ermon S., Finn C. Direct preference optimization: Your language model is secretly a reward model. Adv. Neural Inf. Process. Syst. 2024; 36: 53728-53741</bibtext> </blist> <blist> <bibl id="bib3" idref="ref3" type="bt">3</bibl> <bibtext> Gao T., Fisch A., Chen D. Making Pre-trained Language Models Better Few-shot Learners. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), OnlineZong C., Xia F., Li W., Navigli R. ; Association for Computational Linguistics: Stroudsburg, PA, USA. 2021: 3816-3830</bibtext> </blist> <blist> <bibl id="bib4" idref="ref4" type="bt">4</bibl> <bibtext> Tam D., R. Menon R., Bansal M., Srivastava S., Raffel C. Improving and Simplifying Pattern Exploiting Training. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican RepublicMoens M.F., Huang X., Specia L., Yih S.W.T. ; Association for Computational Linguistics: Stroudsburg, PA, USA. 2021: 4980-4991</bibtext> </blist> <blist> <bibl id="bib5" idref="ref5" type="bt">5</bibl> <bibtext> Schick T., Schütze H. True Few-Shot Learning with Prompts—A Real-World Perspective. Trans. Assoc. Comput. Linguist. 2022; 10: 716-731. 10.1162/tacl_a_00485</bibtext> </blist> <blist> <bibl id="bib6" idref="ref6" type="bt">6</bibl> <bibtext> Perez E., Kiela D., Cho K. True few-shot learning with language models. Adv. Neural Inf. Process. Syst. 2021; 34: 11054-11070</bibtext> </blist> <blist> <bibl id="bib7" idref="ref9" type="bt">7</bibl> <bibtext> Fu Y., Peng H., Khot T., Lapata M. Improving language model negotiation with self-play and in-context learning from ai feedback. arXiv. 2023. 2305.10142</bibtext> </blist> <blist> <bibl id="bib8" idref="ref10" type="bt">8</bibl> <bibtext> Xu W., Deutsch D., Finkelstein M., Juraska J., Zhang B., Liu Z., Wang W.Y., Li L., Freitag M. LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback. Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, MexicoDuh K., Gomez H., Bethard S. ; Association for Computational Linguistics: Stroudsburg, PA, USA. 2024: 1429-1445</bibtext> </blist> <blist> <bibl id="bib9" type="bt">9</bibl> <bibtext> Chen X., Lin M., Schärli N., Zhou D. Teaching Large Language Models to Self-Debug. Proceedings of the Twelfth International Conference on Learning Representations. Vienna, Austria. 7–11 May 2024</bibtext> </blist> <blist> <bibtext> Madaan A., Tandon N., Gupta P., Hallinan S., Gao L., Wiegreffe S., Alon U., Dziri N., Prabhumoye S., Yang Y. Self-refine: Iterative refinement with self-feedback. Adv. Neural Inf. Process. Syst. 2024; 36: 46534-46594</bibtext> </blist> <blist> <bibtext> Wang X., Wei J., Schuurmans D., Le Q.V., Chi E.H., Narang S., Chowdhery A., Zhou D. Self-Consistency Improves Chain of Thought Reasoning in Language Models. Proceedings of the Eleventh International Conference on Learning Representations. Kigali, Rwanda. 1–5 May 2023</bibtext> </blist> <blist> <bibtext> Toshniwal S., Sorokin I., Ficek A., Moshkov I., Gitman I. GenSelect: A Generative Approach to Best-of-N. Proceedings of the 2nd AI for Math Workshop@ ICML 2025. Vancouver, BC, Canada. 18 July 2025</bibtext> </blist> <blist> <bibtext> Yao B., Wang D., Wu T., Zhang Z., Li T., Yu M., Xu Y. It is AI's Turn to Ask Humans a Question: Question-Answer Pair Generation for Children's Story Books. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Association for Computational Linguistics: Stroudsburg, PA, USA. 2022: 731-744</bibtext> </blist> <blist> <bibtext> Zhao Z., Hou Y., Wang D., Yu M., Liu C., Ma X. Educational Question Generation of Children Storybooks via Question Type Distribution Learning and Event-centric Summarization. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Association for Computational Linguistics: Stroudsburg, PA, USA. 2022: 5073-5085</bibtext> </blist> <blist> <bibtext> Xu Y., Wang D., Yu M., Ritchie D., Yao B., Wu T., Zhang Z., Li T., Bradford N., Sun B. Fantastic Questions and Where to Find Them: FairytaleQA—An Authentic Dataset for Narrative Comprehension. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, IrelandMuresan S., Nakov P., Villavicencio A. ; Association for Computational Linguistics: Stroudsburg, PA, USA. 2022: 447-460</bibtext> </blist> <blist> <bibtext> Yoon H., Bak J. Diversity Enhanced Narrative Question Generation for Storybooks. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing; Association for Computational Linguistics: Stroudsburg, PA, USA. 2023: 465-482</bibtext> </blist> <blist> <bibtext> Song M., Feng Y., Jing L. A survey on recent advances in keyphrase extraction from pre-trained language models. Find. Assoc. Comput. Linguist. EACL. 2023; 2023: 2153-2164</bibtext> </blist> <blist> <bibtext> Ding H., Luo X. Attentionrank: Unsupervised keyphrase extraction using self and cross attentions. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing; Association for Computational Linguistics: Stroudsburg, PA, USA. 2021: 1919-1928</bibtext> </blist> <blist> <bibtext> Liang X., Wu S., Li M., Li Z. Unsupervised Keyphrase Extraction by Jointly Modeling Local and Global Context. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing; Association for Computational Linguistics: Stroudsburg, PA, USA. 2021: 155-164</bibtext> </blist> <blist> <bibtext> Song M., Jing L., Xiao L. Importance Estimation from Multiple Perspectives for Keyphrase Extraction. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing; Association for Computational Linguistics: Stroudsburg, PA, USA. 2021: 2726-2736</bibtext> </blist> <blist> <bibtext> Meng R., Zhao S., Han S., He D., Brusilovsky P., Chi Y. Deep Keyphrase Generation. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Association for Computational Linguistics: Stroudsburg, PA, USA. 2017: 582-592</bibtext> </blist> <blist> <bibtext> Dubey A., Jauhri A., Pandey A., Kadian A., Al-Dahle A., Letman A., Mathur A., Schelten A., Yang A., Fan A. The llama 3 herd of models. arXiv. 2024. 10.48550/arXiv.2407.21783. 2407.21783</bibtext> </blist> <blist> <bibtext> Hu E.J., Shen Y., Wallis P., Allen-Zhu Z., Li Y., Wang S., Wang L., Chen W. LoRA: Low-Rank Adaptation of Large Language Models. Proceedings of the International Conference on Learning Representations. Virtual. 25 April 2022</bibtext> </blist> <blist> <bibtext> Rajpurkar P., Zhang J., Lopyrev K., Liang P. SQuAD: 100,000+ Questions for Machine Comprehension of Text. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, TXSu J., Duh K., Carreras X. ; Association for Computational Linguistics: Stroudsburg, PA, USA. 2016: 2383-2392</bibtext> </blist> <blist> <bibtext> Yang Z., Qi P., Zhang S., Bengio Y., Cohen W., Salakhutdinov R., Manning C.D. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, BelgiumRiloff E., Chiang D., Hockenmaier J., Tsujii J. ; Association for Computational Linguistics: Stroudsburg, PA, USA. 2018: 2369-2380</bibtext> </blist> <blist> <bibtext> Kočiský T., Schwarz J., Blunsom P., Dyer C., Hermann K.M., Melis G., Grefenstette E. The NarrativeQA Reading Comprehension Challenge. Trans. Assoc. Comput. Linguist. 2018; 6: 317-328. 10.1162/tacl_a_00023</bibtext> </blist> <blist> <bibtext> Sellam T., Das D., Parikh A. BLEURT: Learning Robust Metrics for Text Generation. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, OnlineJurafsky D., Chai J., Schluter N., Tetreault J. ; Association for Computational Linguistics: Stroudsburg, PA, USA. 2020: 7881-7892</bibtext> </blist> <blist> <bibtext> Ivens S.H., Koslin B.L. Demands for Reading Literacy Require New Accountability Methods; Touchstone Applied Science Associates: Brewster, NY, USA. 1991</bibtext> </blist> <blist> <bibtext> Hong J., Lee N., Thorne J. ORPO: Monolithic Preference Optimization without Reference Model. arXiv. 2024. 10.48550/arXiv.2403.07691. 2403.07691</bibtext> </blist> </ref> <aug> <p>By Andreea Dutulescu; Stefan Ruseti; Mihai Dascalu and Danielle S. McNamara</p> <p>Reported by Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib10" firstref="ref11"></nolink> <nolink nlid="nl2" bibid="bib11" firstref="ref12"></nolink> <nolink nlid="nl3" bibid="bib12" firstref="ref13"></nolink> <nolink nlid="nl4" bibid="bib13" firstref="ref14"></nolink> <nolink nlid="nl5" bibid="bib14" firstref="ref15"></nolink> <nolink nlid="nl6" bibid="bib15" firstref="ref16"></nolink> <nolink nlid="nl7" bibid="bib16" firstref="ref17"></nolink> <nolink nlid="nl8" bibid="bib17" firstref="ref18"></nolink> <nolink nlid="nl9" bibid="bib18" firstref="ref19"></nolink> <nolink nlid="nl10" bibid="bib19" firstref="ref20"></nolink> <nolink nlid="nl11" bibid="bib20" firstref="ref21"></nolink> <nolink nlid="nl12" bibid="bib21" firstref="ref22"></nolink> <nolink nlid="nl13" bibid="bib22" firstref="ref23"></nolink> <nolink nlid="nl14" bibid="bib23" firstref="ref24"></nolink> <nolink nlid="nl15" bibid="bib24" firstref="ref25"></nolink> <nolink nlid="nl16" bibid="bib25" firstref="ref26"></nolink> <nolink nlid="nl17" bibid="bib26" firstref="ref27"></nolink> <nolink nlid="nl18" bibid="bib27" firstref="ref28"></nolink> <nolink nlid="nl19" bibid="bib28" firstref="ref30"></nolink> <nolink nlid="nl20" bibid="bib29" firstref="ref38"></nolink>
Header DbId: eric
DbLabel: ERIC
An: ED681036
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Feedback-Aware Inference for Iterative Multi-Sample Text Generation
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Andreea+Dutulescu%22">Andreea Dutulescu</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0003-1870-679X">0009-0003-1870-679X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Stefan+Ruseti%22">Stefan Ruseti</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-0380-6814">0000-0002-0380-6814</externalLink>)<br /><searchLink fieldCode="AR" term="%22Mihai+Dascalu%22">Mihai Dascalu</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-4815-9227">0000-0002-4815-9227</externalLink>)<br /><searchLink fieldCode="AR" term="%22Danielle+S%2E+McNamara%22">Danielle S. McNamara</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5869-1420">0000-0001-5869-1420</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Grantee+Submission%22"><i>Grantee Submission</i></searchLink>. 2026 7.
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 26
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2026
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Center for Education Research (NCER) (ED/IES)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305T240035
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Man+Machine+Systems%22">Man Machine Systems</searchLink><br /><searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Inferences%22">Inferences</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Instruction%22">Computer Assisted Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Material+Development%22">Material Development</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Banks%22">Item Banks</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Development%22">Program Development</searchLink><br /><searchLink fieldCode="DE" term="%22Models%22">Models</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.3390/ai7050171
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Generating multiple text sequences and refining them through feedback is essential for improving the quality of outputs in many NLP tasks. While Large Language Models can leverage iterative feedback during inference, smaller models often lack this capability due to limited capacity and the absence of suitable training paradigms. In this paper, we propose a novel Feedback-Aware Inference approach that enables iterative sequence generation with integration of feedback signals. Our method allows models to generate multiple sequences, incorporate feedback from previous iterations, and refine outputs accordingly. This approach dynamically adjusts to different quality metrics, making it adaptable to various contexts and objectives. We evaluate our approach on two distinct tasks: Answer Selection for Question Generation and Keyword Generation, arguing for its generalizability and effectiveness. Results show that our method outperforms strong baselines, maintaining high performance across iterations and achieving superior results even with smaller, open-source models.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Note
  Label: Notes
  Group: Note
  Data: https://github.com/upb-nlp/Feedback-Aware
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED681036
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED681036
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.3390/ai7050171
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 26
    Subjects:
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Man Machine Systems
        Type: general
      – SubjectFull: Feedback (Response)
        Type: general
      – SubjectFull: Inferences
        Type: general
      – SubjectFull: Computer Assisted Instruction
        Type: general
      – SubjectFull: Material Development
        Type: general
      – SubjectFull: Item Banks
        Type: general
      – SubjectFull: Program Development
        Type: general
      – SubjectFull: Models
        Type: general
    Titles:
      – TitleFull: Feedback-Aware Inference for Iterative Multi-Sample Text Generation
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Andreea Dutulescu
      – PersonEntity:
          Name:
            NameFull: Stefan Ruseti
      – PersonEntity:
          Name:
            NameFull: Mihai Dascalu
      – PersonEntity:
          Name:
            NameFull: Danielle S. McNamara
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2026
          Numbering:
            – Type: volume
              Value: 7
          Titles:
            – TitleFull: Grantee Submission
              Type: main
ResultId 1