
Snowflake Semantic Views: Validate the Business Answer
A practical guide to metric definitions, reference queries, and Cortex acceptance tests.
Get the complete guide
Use the worked metric example, question bank, and acceptance worksheets to decide where one Snowflake AI use case is ready for business users.
- Reconcile the reference calculation.
- Test the semantic query and actual user answer.
- Record a bounded release decision.
Complete the form to open the 12-page PDF. The article follows below.
KCM Solutions | Research checked October 2, 2026
Consider this situation: an operations director asks, “What percentage of our September orders left the warehouse on time?”
The assistant answers 76%. The approved operations report says 89.09%. Both calculations use the same source data. Both look reasonable. Only one answers the company's question.
The difference is how the calculation treats warehouses of different sizes. An average of their individual percentages gives a small warehouse the same influence as a large one. Counting all eligible orders produces a different result.
This is the decision behind a Snowflake AI rollout: can business users rely on the answer for the defined task, with their own access and the explanation they actually receive?
KCM recommends approving a specific set of business questions only after three things agree: a reconciled reference result, the semantic definition, and the response shown to the user. Give exceptions an explicit outcome, such as asking for clarification or referring the question to an analyst.
This guide shows how to assemble that evidence. It includes a calculation, a question bank, and worksheets for accepting one use case. It is intended for Snowflake platform owners, analytics leaders, and the business owners of the metrics being exposed.
The worked figures do not benchmark Snowflake.
Why this deserves attention now
Snowflake made Semantic Studio generally available on September 30, 2026. It provides an environment in Workspaces for authoring semantic views, editing YAML, deploying definitions, and using Git version control. A separate release on the same day made DEFINE SEMANTIC VIEW in DCM Projects generally available, supporting declarative management of semantic definitions. Semantic Studio release · DCM release
Snowflake's current Cortex Analyst documentation recommends semantic views for new implementations while retaining legacy staged YAML models for compatibility. Snowflake also recommends transitioning from Cortex Analyst to Cortex Agents. That recommendation should inform a new design; it does not establish a retirement deadline or remove the need to test an existing application. Cortex Analyst documentation · Transition recommendation
As authoring and deployment become easier, teams need a repeatable way to approve what a definition means. A successfully deployed object is only the start of that decision.
Begin with the business meaning
A semantic view stores business concepts as a Snowflake schema object. It can describe entities, relationships, dimensions, and metrics. That gives applications more context than technical table names alone. Semantic view overview
The business owner still needs to settle the definition. “On time,” “active,” “revenue,” and “available” can each mean several things inside one organization.
For an initial rollout, choose a small operational domain with an accountable owner and an existing reconciliation process. Snowflake's modeling guidance similarly emphasizes business scope, clear descriptions, and explicit relationships. Modeling guidance
Write the answer rules before adding more tables. For example, the following rules could govern an on-time dispatch metric.
| Definition choice | Business rule |
|---|---|
| Unit being counted | One customer order, regardless of its number of lines or shipments |
| Warehouse assignment | Each order belongs to one responsible warehouse, recorded at order acceptance |
| Eligible population | Orders with an original promised dispatch deadline in September 2026 |
| Time basis | September 1 at 00:00 through October 1 at 00:00, using America/Toronto time; the end is exclusive |
| Success | All required order items left the warehouse by the original promised dispatch deadline |
| Exclusions | Orders cancelled before that deadline; test orders |
| Late or incomplete orders | Remain in the denominator and fail the on-time test |
| Promise changes | Use the promise captured at order acceptance, not a later revised promise |
| Data cut-off | A reconciled, frozen snapshot as of October 1, 2026 at 09:00 America/Toronto |
| Empty population | Show “no eligible orders,” not 0% or 100% |
| Meaning of the label | Dispatch from the warehouse; it says nothing about arrival at the customer |
These rules are a design example, not an industry standard. A business that measures carrier handover, first shipment, or final delivery needs a different definition.
Also decide where the approved calculation already lives. It may be in a reporting model, a transformation, or a governed database view. Compare its actual logic with the proposed semantic definition. A shared metric name does not establish that the filters, dates, or aggregation agree.
Work through the calculation before testing the assistant
For this snapshot, all included September deadlines have passed. The source owner has reconciled eligibility, cancellations, complete dispatch events, and the data cut-off.
| Warehouse | Eligible orders | Orders dispatched on time | Warehouse rate |
|---|---|---|---|
| East | 100 | 92 | 92.00% |
| West | 10 | 6 | 60.00% |
| Company total | 110 | 98 | 89.09% |
The company metric is:
98 on-time orders ÷ 110 eligible orders × 100 = 89.09%, rounded to two decimal places.
The unweighted average of the warehouse rates is:
(92% + 60%) ÷ 2 = 76%.
The second calculation answers a different question: the average warehouse percentage when each warehouse receives equal weight. It is 13.09 percentage points below the order-based company rate in this example.
Define the metric using its numerator and denominator at the required level of aggregation. Do not let the display of percentages quietly determine the business calculation.
Then inspect what one source row represents. If a join turns one order into three shipment rows, counting rows can change the denominator. If the source only records completed dispatches, undispatched or incomplete orders can disappear from the population. If the current promise overwrites the original promise, a late order can become apparently on time.
Resolve those issues in the data model and reference calculation. Instructions to the assistant cannot reconstruct facts that the source no longer retains.
The explanation is part of the answer
For this example, the acceptable company response is:
For September 2026, 98 of 110 eligible orders were fully dispatched by their original promised deadline, an on-time dispatch rate of 89.09%. The result uses the snapshot as of October 1 at 09:00 America/Toronto. It measures warehouse dispatch, not customer delivery.
This response was not captured from Snowflake.
A response of “89.09% of customers received their order on time” is still incorrect. The number matches, but the explanation substitutes delivery for dispatch.
A response of “92% company-wide” would also fail if it came from an East-only data scope. A correct regional calculation needs a regional label.
Check three points in the same path
Use the same definition version, snapshot, period, and authorized population throughout the comparison.
| Evidence | What to inspect | What a failure tells you |
|---|---|---|
| Reference calculation | Approved SQL result, source reconciliation, eligibility counts, and numerator | The expected answer or underlying data needs correction |
| Direct semantic query | Metric result by warehouse and in total, filters, join behavior, and empty populations | The semantic implementation does not match the approved meaning |
| Actual user response | Generated SQL or trace where available, returned values, explanation, period, and access scope | The natural language or application path changes the meaning or presentation |
This is KCM's proposed acceptance method.
Snowflake documents both direct queries of semantic views and a Cortex path that reads the definition and generates SQL against physical tables. KCM's conclusion is that a successful direct semantic query cannot, by itself, prove that every natural language response follows the intended calculation. Test both paths. Semantic view overview
Snowflake also validates semantic definitions against structural and expression rules. Those checks are useful. They do not decide whether your business wants an order-weighted rate or an equal-weight average. That choice belongs in the approved definition and its test evidence. Validation rules
Avoid choosing an expected result simply because it matches the existing dashboard. If the dashboard and semantic query disagree, reconcile the difference against the business rules. The older report may need correction too.
Build a question bank that can expose a wrong answer
Ask intended users how they phrase the decision. Include ordinary language and the boundaries that could change its meaning. Reserve some questions for acceptance after the team finishes its initial tuning.
Snowflake's evaluation guidance recommends representative inputs and reviewed expected responses. Generated questions can supplement that work, but the business owner must verify what the answer should be. Evaluation design guidance
The following bank extends the same example.
| Question or test | Expected behavior |
|---|---|
| “What was the company on-time dispatch rate for September 2026?” | 98 of 110; 89.09%; correct period, snapshot, and dispatch label |
| “Show September 2026 by warehouse.” | East 92 of 100; West 6 of 10; reconcile the counts to the company total |
| “What is the average of the two warehouse percentages?” | 76%; explicitly distinguish this from the company order-based rate |
| “How many September orders missed the original dispatch promise?” | 12 orders; include eligible incomplete orders |
| “Did we deliver on time in September?” | Clarify whether the user means dispatch or customer arrival; do not answer a delivery question with dispatch data |
| East manager asks for the company total | Follow the approved access behavior: return only a clearly labeled permitted scope, or explain that the company result is unavailable to that user |
| Ask about a test population with no eligible orders | Report no eligible orders; do not imply perfect or zero performance |
| Add a synthetic order with two shipment rows | Preserve the order count and use the approved complete-dispatch rule |
| Follow “show East” with “now company-wide” | Resolve the changed scope through the actual application and permissions |
| Ask for a period after the approved data cut-off | Explain the available data period or decline the unsupported calculation |
For each case, store the expected value or behavior, the source snapshot, the authorized identity, and the business reason it matters. A numeric tolerance should cover intentional rounding, not excuse a different population or formula.
Make the conditions difficult enough to distinguish competing calculations. A test where every warehouse has the same number of orders would hide the weighted-average problem.
Keep three kinds of question separate:
- Examples used to help the application interpret the business.
- Reserved acceptance questions used to assess a candidate.
- Newly observed user questions that may reveal a gap after release.
When a reserved question becomes part of tuning, record that change and add fresh acceptance cases.
Use Snowflake evaluations for the evidence they provide
Check the generated SQL
The Verified Query Repository pairs business questions with reference SQL. Review the SQL and its result before treating it as trusted evidence. Snowflake requires verified queries to use the logical names defined in the semantic model. An incorrect reference can reward the wrong behavior. Verified Query Repository
Cortex Analyst evaluations use selected verified queries to assess SQL generation. Snowflake temporarily removes those selected examples from a copy of the semantic view for the run. Unselected examples can still guide generation. Results include SQL correctness, regressions, latency, and expected versus generated SQL. Each run covers one semantic view, and evaluation questions are independent; multi-turn conversations are not evaluated by this mechanism. Analyst evaluations
For the dispatch example, inspect whether the generated SQL divides total successful orders by total eligible orders and applies the approved time and cancellation rules. Read individual failures even when the overall score improves.
Check what the agent tells the user
For a Cortex Agent application, evaluate the final answer as well. Current native evaluations include answer correctness, logical consistency, and custom metrics. Tool selection and tool execution metrics remain Public Preview in the documentation checked for this guide. Answer correctness can assess response content; it does not validate how a chart is rendered. Agent evaluations
Use the expected answer to require the right units, period, permitted scope, and distinction between dispatch and delivery. Have a business reviewer inspect the cases where the number is correct but the wording could cause a wrong decision.
Keep the SQL and response results separate. A SQL correctness score is not the same measure as an agent answer score.
Reproduce the conditions, then examine variation
Snowflake supports targeting an agent version and selecting versions of system evaluation metrics. A pinned judge configuration does not freeze generated SQL or agent behavior. Runs can still differ. Version targeting
For a controlled comparison, record the committed agent version, semantic definition revision, evaluation questions, metric version, and data snapshot. A movable production alias or a refreshed table can change the conditions.
Run important cases more than once. Review which questions change between runs, rather than relying on a single average. If a minor rephrasing repeatedly changes the denominator, the metric is not ready for broad use.
Test access and conversation behavior through the real application
Use the actual user identities and roles planned for the rollout. Test both permitted and prohibited requests, including whether a regional result is labeled clearly.
Current Cortex Agent evaluations do not pass session attributes. Snowflake therefore excludes evaluation of agents whose queries depend on those values, including related row access policy cases. Native evaluations also do not exercise MCP server tools. Build separate application tests for those paths. Agent evaluation limits
For a session-dependent East manager, a native batch result cannot establish that the production answer will show only East data. Log in through the intended application with the representative identity and check the visible result, supporting records, and response wording.
Test follow-up questions in the application as conversations. In the question bank, “now company-wide” changes the scope after “show East.” A collection of independent prompts does not prove that this interaction works.
Record missing evidence explicitly. If the acceptance environment cannot reproduce the intended identity or policy conditions, leave that use case out of the approved rollout until it can.
Make a bounded release decision
Review failures by consequence before looking at the aggregate score.
For the dispatch use case, KCM would propose the following acceptance rules for the owner to approve:
- Every designated critical calculation case must reconcile to the reference counts and formula.
- No observed response may expose prohibited records or present a regional result as a company total.
- Dispatch and delivery must remain distinct in the question handling and answer.
- Missing, ambiguous, and empty populations must receive the agreed response.
- Remaining noncritical defects must have a named owner and a visible usage restriction.
These are proposed rules for this example, not Snowflake defaults or a universal accuracy threshold. Passing a finite set does not prove that every future question will be correct.
An unresolved 76% company answer should block this metric's rollout. A correct 89.09% answer with an incorrect delivery claim should also block it. Improving the overall score while retaining either defect does not resolve the business problem.
Where the evidence supports only a narrow scope, approve that scope explicitly: for example, historical dispatch questions for operations analysts, with an escalation route for delivery questions. Make the supported period, population, and limitations visible in the user experience.
Before expanding access, name the person who can pause the use case and the analyst who will resolve disputed answers.
Keep the meaning stable when the implementation changes
Treat the approved metric, its reference cases, and its consumers as a maintained data product. Snowflake's development guidance covers ownership, deployment, access, and BI integrations for semantic views. Development guidance
Retest when the underlying table, join, business definition, semantic description, reference query, agent instructions, permissions, or application behavior changes.
For this example, changing the numerator from “all required items dispatched” to “first shipment dispatched” changes the metric. It needs business approval even if every deployment check passes.
Use Semantic Studio or the team's chosen development process to retain the reviewed definition and change history. If adopting DCM Projects, decide who owns the deployed definition and how changes move through review. Avoid parallel editing processes that can overwrite one another.
Record the version used by the application and the one used by any comparison report. Test an actual consumer before claiming that an existing dashboard now shares the semantic definition automatically.
Keep a small set of questions whose failure would make the metric unsafe to use. Add real misunderstandings and corrected defects to that set. Revisit the expected results when the approved business definition changes.
Worksheet: one question, complete evidence
Duplicate this record for each critical acceptance case.
| Field | Record |
|---|---|
| Business question and case ID | |
| Decision the answer supports | |
| Business owner and technical owner | |
| Metric definition and revision | |
| Grain, population, exclusions, time zone, and period | |
| Source snapshot and reconciliation evidence | |
| User identity, active role, and relevant session conditions | |
| Reference SQL location and approved result | |
| Direct semantic query result | |
| Generated SQL or trace reference | |
| Expected response, units, scope, and explanation | |
| Permitted rounding; prohibited interpretations | |
| Actual response and repeated-run observations | |
| Result: accepted, incorrect, or not tested | |
| Defect owner, correction, and retest evidence |
An untested access case is not a pass. An unresolved expected answer is not an evaluation result.
Worksheet: approve the use case
| Decision item | Record |
|---|---|
| Exact questions, users, and periods proposed for release | |
| Critical cases and acceptance rules agreed before testing | |
| Definition, agent, application, dataset, and scoring revisions | |
| Correctness evidence and unresolved disagreements | |
| Actual-identity access tests and conversation tests completed | |
| Known limitations visible to users | |
| Operating owner, escalation route, and pause procedure | |
| Change events that require another acceptance review | |
| Decision: approve stated scope, narrow scope, or hold | |
| Business approver, technical approver, and review date |
Start with one metric that already causes reconciliation work. Bring the current report, its definition, and ten real user questions. The immediate objective is to make the expected answer testable.
Talk to KCM about Snowflake analytics, governed models, and AI readiness.
Sources and scope
Product sources were checked on October 2, 2026. Availability, evaluation limits, and interfaces can change. The acceptance method and worksheets are KCM recommendations. No Snowflake account or customer system was tested for this guide.
- Semantic Studio: September 30, 2026 GA release
- DCM Projects DEFINE SEMANTIC VIEW: September 30, 2026 GA release
- Cortex Analyst and semantic-view guidance
- Recommendation to transition from Cortex Analyst to Cortex Agents
- Overview of semantic views
- Best practices for modeling semantic views
- How Snowflake validates semantic views
- Cortex Analyst Verified Query Repository
- Cortex Analyst evaluations
- Cortex Agent evaluations
- Version targeting for Cortex Agent and Cortex Analyst evaluations
- Best practices for evaluating Cortex Agents
- Best practices for developing and deploying semantic views
