{"attribution":{"digest":{"sha256":"ba4f668e72c3fba74a58b8ee56064568fb9c6aae1441e4f0f7a8f5edba498ee9"},"mediaType":"text/yaml","name":"EverMind-AI/EverMemOS prompt source","uri":"https://github.com/EverMind-AI/EverMemOS/blob/1f2f083d9fd07fd6580064bbdfc7b97da39c47bb/evaluation/config/prompts.yaml"},"instrumentId":"audited","license":{"digest":{"sha256":"348894226f41573907d120f8796c91c6951656459a98be955c7bc2ab8f73e0f2"},"mediaType":"text/plain","name":"Apache-2.0","uri":"https://raw.githubusercontent.com/EverMind-AI/EverOS/1f2f083d9fd07fd6580064bbdfc7b97da39c47bb/LICENSE"},"messages":[{"role":"developer","segments":[{"literal":"You are an expert grader that determines if answers to questions match a gold standard answer"}]},{"role":"user","segments":[{"literal":"Your task is to label an answer to a question as 'CORRECT' or 'WRONG'. You will be given the following data:\n    (1) a question (posed by one user to another user),\n    (2) a 'gold' (ground truth) answer,\n    (3) a generated answer\nwhich you will score as CORRECT/WRONG.\n\nThe point of the question is to ask about something one user should know about the other user based on their prior conversations.\nThe gold answer will usually be a concise and short answer that includes the referenced topic, for example:\nQuestion: Do you remember what I got the last time I went to Hawaii?\nGold answer: A shell necklace\nThe generated answer might be much longer, but you should be generous with your grading - as long as it touches on the same topic as the gold answer, it should be counted as CORRECT.\n\nFor time related questions, the gold answer will be a specific date, month, year, etc. The generated answer might be much longer or use relative time references (like \"last Tuesday\" or \"next month\"), but you should be generous with your grading - as long as it refers to the same date or time period as the gold answer, it should be counted as CORRECT. Even if the format differs (e.g., \"May 7th\" vs \"7 May\"), consider it CORRECT if it's the same date.\n\nNow it's time for the real question:\nQuestion: "},{"field":"question"},{"literal":"\nGold answer: "},{"field":"referenceAnswer"},{"literal":"\nGenerated answer: "},{"field":"candidateAnswer"},{"literal":"\n\nFirst, provide a short (one sentence) explanation of your reasoning, then finish with CORRECT or WRONG.\nDo NOT include both CORRECT and WRONG in your response, or it will break the evaluation script.\n\nJust return the label CORRECT or WRONG in a json format with the key as \"label\"."}]}],"model":{"adapter":"jinn-openai","generation":{"background":false,"fallbackModels":[],"maxOutputTokens":512,"metadata":null,"persistedConversation":false,"promptCacheIdentifier":null,"retries":0,"serviceTier":"default","store":false,"stream":false,"temperature":0,"tools":[]},"requested":"gpt-4o-mini-2024-07-18"},"promptSource":{"digest":{"sha256":"ba4f668e72c3fba74a58b8ee56064568fb9c6aae1441e4f0f7a8f5edba498ee9"},"mediaType":"text/yaml","name":"EverMemOS prompts.yaml at 1f2f083d","uri":"https://raw.githubusercontent.com/EverMind-AI/EverMemOS/1f2f083d9fd07fd6580064bbdfc7b97da39c47bb/evaluation/config/prompts.yaml"},"promptTemplateSha256":"sha256:56882c1bf0911a72d5e4766830bd7402233b2ddadfaa808fe0624f22e6f5bf34","protocol":"https://spec.jinn.network/binary-judgment/judge-instrument/v1","response":{"invalidOutputDecision":"INVALID","mediaType":"text/plain; charset=utf-8","parser":{"digest":"sha256:f3d9ca6df8bfc263e64e25500c300d24124175c1c76596effc8be1c3780c1fb6","id":"network.jinn.parser.binary-evermem-json-label","version":"2.0.0"}}}