🤖 AI Tools
Ragas Evaluation Metric Map from Dataset Inventory (No Invented Scores)
Map Ragas evaluation metrics from a dataset inventory only. No invented faithfulness scores, context precision, or latency claims beyond the inventory.
0Reviews
Prompt
Act as a Ragas evaluation design aide who only uses a pasted dataset inventory. You write an evaluation metric map the inventory already supports. You do not invent faithfulness scores, context precision, answer relevancy, or API keys. This is not a live eval run and not a model quality certification. You work only from Inputs. Do not invent stats, citations, quotes, URLs, names, IDs, or records that are not in Inputs. Inputs: - Dataset inventory I lock (sample stubs, metric cues, judge notes): [Inventory] - Ragas / LangChain version notes I lock: [Version] - Experiment label I may quote (or UNKNOWN): [Experiment] - Required metric names I may quote (or UNKNOWN): [Metrics] - Words I must not use: [Banned] - What I must never invent (faithfulness scores, context precision, answer relevancy, API keys): [Never] - Output format: [Format] - Language: [Lang] Generate: 1. Honesty ledger: Inventory nouns, Version, Experiment, Metrics, Lang. Forbidden: invented faithfulness scores, context precision, answer relevancy, API keys. Banner: not a live eval run; not a model quality certification. 2. Evaluation metric map table: one row per Inventory sample stub or metric cue. Missing judge notes write NOT IN INPUTS. 3. Metric name set: only names in Metrics. Unnamed metrics stay NOT IN INPUTS. Never print score VALUES not in Inventory. 4. Version lock: print Version. Refuse Ragas objects newer than Version if Version is named. 5. Refuse list: inventing faithfulness scores, inventing context precision, inventing answer relevancy, inventing API keys. 6. Compliance pass: quote Banned and Never hits. Cut them. Format as Format. Constraints: - Map from Inventory only. No invented score VALUES. - Honor Version. No emojis.
Instructions
Replace every [bracket] with your details before running. Works on ChatGPT, Claude, and Gemini.
Generated Output
This image was generated using the prompt above.
Examples
Example Input
Inventory: sample Harbor Quay PPE FAQ as pasted metric cue faithfulness as pasted judge note llm-as-judge as pasted; sample River Ops Policy QA as pasted metric cue context_precision as pasted. Faithfulness score NOT IN INPUTS. Answer relevancy NONE. Version: Ragas 0.1 as pasted (do not invent unreleased metrics) Experiment: Harbor Quay RAG Sprint as pasted Metrics: faithfulness as pasted; context_precision as pasted. latency UNKNOWN. Banned: guaranteed score, faithfulness invent, precision invent, api key invent Never: invent faithfulness scores, invent context precision, invent answer relevancy, invent API keys Format: ledger + table + metrics + version + refuse + compliance Lang: English
Example Output
1. Ledger. Inventory: sample Harbor Quay PPE FAQ metric cue faithfulness judge note llm-as-judge; sample River Ops Policy QA metric cue context_precision. Faithfulness score NOT IN INPUTS. Answer relevancy NONE. Version Ragas 0.1. Experiment Harbor Quay RAG Sprint. Metrics faithfulness; context_precision; latency UNKNOWN. Lang English. Banner: not a live eval run; not a model quality certification. Forbidden: invented faithfulness scores, context precision, answer relevancy, API keys, guaranteed score, faithfulness invent, precision invent, api key invent. 2. Evaluation metric map. 1. Harbor Quay PPE FAQ | metric faithfulness as pasted | judge llm-as-judge as pasted | score NOT IN INPUTS | relevancy NONE. 2. River Ops Policy QA | metric context_precision as pasted | judge NOT IN INPUTS | score NOT IN INPUTS | relevancy NONE. API keys not printed. Answer relevancy not invented. 3. Metric name set. faithfulness; context_precision as Metrics. latency UNKNOWN so write latency NOT IN INPUTS. No score VALUES printed. No third metric invented. 4. Version lock. Ragas 0.1 as pasted. Unreleased metrics not used. Dataset schema NOT IN INPUTS. 5. Refuse. Faithfulness invent: refused. Precision invent: refused. Relevancy invent: refused. API key invent: refused. Guaranteed score: refused. 6. Compliance. Banned hits none. Format ledger+table+metrics+version+refuse+compliance. Gaps: judge for River Ops Policy QA, latency decision, ground-truth path, batch size, model name if any. Missing-data policy: if a field was blank, write NOT IN INPUTS rather than guessing. Lock any tool version named in Inputs; if unnamed, write unknown. No invented testimonials, star ratings, or press logos. If legal, clinical, insurance, HR, education-plan, or veterinary content appears, add a one-line not-advice and de-identify banner. Quote banned-word hits and cut them. End with a gaps list of five bullets the user still owes you. Character and byte caps in the job are hard; print counts when relevant. Refuse to backfill DOIs, exam dumps, PHI, PII, or compensation promises not in Inputs.