Individual
Bordeaux colour
https://w3id.org/cqa/wine-benchmark/cq-02
Instance Graph 27 / 41
Curated instance graph (A-box): individuals conforming to this schema.
Graph
The individuals above as a connected A-box: nodes wear the same symbols their classes and enumerations have in the schema graph, and edges are the object-property assertions between them.
Drag to pan • Scroll to zoom
Individuals
Bordeaux colourCompetencyQuestionAnswer
Cabernet Sauvignon with seafoodCompetencyQuestionAnswer
Wine for grilled meatCompetencyQuestionAnswer
Characteristics affecting a pairingCompetencyQuestionAnswer
Vintage variationCompetencyQuestionAnswer
Good vintages for Napa ZinfandelCompetencyQuestionAnswer
Wine competency questionsBenchmark
Individual
Cabernet Sauvignon with seafood
https://w3id.org/cqa/wine-benchmark/cq-03
Individual
Wine for grilled meat
https://w3id.org/cqa/wine-benchmark/cq-04
Individual
Characteristics affecting a pairing
https://w3id.org/cqa/wine-benchmark/cq-05
Individual
Vintage variation
https://w3id.org/cqa/wine-benchmark/cq-06
Individual
Good vintages for Napa Zinfandel
https://w3id.org/cqa/wine-benchmark/cq-07
Individual
Wine competency questions
https://w3id.org/cqa/wine-benchmark
The seven competency questions of the wine ontology, less the one its schema answers, each paired with the specification of its correct answer over wine's worked-example graph.
Schema Graph
Initializing visualization...
Drag to pan • Scroll to zoom
Namespaces 11
Namespace prefixes used in this ontology.
| Prefix | Namespace IRI |
|---|---|
|
https://w3id.org/cqa/
|
cco |
https://www.commoncoreontologies.org/
|
cqa |
https://w3id.org/cqa/
|
earl |
http://www.w3.org/ns/earl#
|
linkml |
https://w3id.org/linkml/
|
obo |
http://purl.obolibrary.org/obo/
|
skos |
http://www.w3.org/2004/02/skos/core#
|
owl |
http://www.w3.org/2002/07/owl#
|
rdf |
http://www.w3.org/1999/02/22-rdf-syntax-ns#
|
rdfs |
http://www.w3.org/2000/01/rdf-schema#
|
xsd |
http://www.w3.org/2001/XMLSchema#
|
Classes 3
Classes defined in this ontology.
-
Class
Benchmark
A named set of competency question answers, evaluated against one knowledge graph.
- Subclass of (external)
- Information Content Entity
- Mappings
- close: earl:TestRequirement
- Slots
-
competency_question_answers
:
CompetencyQuestionAnswer multivalued
The questions this benchmark holds.
-
description
:
string
A longer prose account of what this benchmark covers.
-
id
:
string refined here
The short, stable identifier a record is known by.
-
name
:
string
A short human-readable label.
-
target_dataset
:
string required
Which instance graph the anchors resolve in, named as the target package publishes it: a dataset name from that package's publish manifest. One schema can have several, and they need not hold the same records, so naming the schema and its version does not identify one.
-
target_dataset_version
:
string required
The version of the dataset the anchors were written against: the release identifier of whatever publishes it, which is the schema's own version when the two ship together. It fixes which records exist, since records can be added, corrected or removed without the schema changing.
-
target_schema
:
uri required
The schema the target graph's records conform to, as a URI.
-
target_schema_version
:
string required
The version of the target schema the anchors were written against. It fixes how identifiers are minted, which can change between releases.
-
Class
CompetencyQuestionAnswer
A competency question paired with the specification of its correct answer.
- Subclass of (external)
- Information Content Entity
- Mappings
- close: earl:TestCase
- Rules
-
A closed-world negative states the connection it claims is absent.
when
answer_kindhas valueclosed-world-negative, thenunconnected_anchorsis present -
An attribution names the source its answer rests on.
when
answer_kindhas valueattribution, thenexpected_citationsis present -
A comparison sets two or more records against each other.
when
answer_kindhas valuecomparison, thenexpected_anchorshas at least 2 value(s) -
A synthesis draws on two or more records, since a pattern no single record states needs more than one to state it.
when
answer_kindhas valuesynthesis, thenexpected_anchorshas at least 2 value(s)
- Slots
-
answer_kind
:
AnswerKindEnum required multivalued
The kinds of answering this question calls for. Every kind named is a separate check, and all of them must pass.
-
connecting_class
:
uri
The class a joining record would belong to, as a URI in the target schema. Optional, because a benchmark can be written against a graph whose schema its author does not have: without it the claim is that no record of any kind joins the anchors, and with it the claim is narrowed to one kind.
-
expected_anchors
:
DomainRecord required multivalued refined here
The records a correct answer must retrieve, as IRIs into the target graph. For an answer that reports no such record exists, these are the records whose absent connection is the answer, and reaching them is what distinguishes that answer from a system that retrieved nothing.
-
expected_citations
:
DomainRecord multivalued
The records a correct answer must rest on, as IRIs into the target graph. Always among the anchors: retrieving a record is not the same as answering from it. Empty is a real answer.
-
ground_truth
:
string required
The correct answer in prose. A generated answer is scored against this; the anchors below are what retrieval is scored against.
-
id
:
string refined here
The short, stable identifier a record is known by.
-
name
:
string
A short human-readable label.
-
question
:
string required
The competency question, in the words it would be asked in.
-
unconnected_anchors
:
DomainRecord multivalued
The anchors that no single record joins, for an answer whose correctness is that the graph holds no such connection. A subset of the anchors: a claim about a record retrieval was never required to reach cannot be checked against what was retrieved. Records held by a container do not count as joined by it.
-
Class
DomainRecord
A record in the knowledge graph that a benchmark is evaluated against. Its instances belong to that graph, not to this schema.
- Slots
-
id
:
string
The short, stable identifier a record is known by.
Slots 14
Slots defined in this ontology.
answer_kind
connecting_class
description
expected_anchors
expected_citations
ground_truth
id
name
question
target_dataset
target_dataset_version
target_schema
target_schema_version
unconnected_anchors
Slot
answer_kind
The kinds of answering this question calls for. Every kind named is a separate check, and all of them must pass.
-
A closed-world negative states the connection it claims is absent.
when
answer_kindhas valueclosed-world-negative, thenunconnected_anchorsis present -
An attribution names the source its answer rests on.
when
answer_kindhas valueattribution, thenexpected_citationsis present -
A comparison sets two or more records against each other.
when
answer_kindhas valuecomparison, thenexpected_anchorshas at least 2 value(s) -
A synthesis draws on two or more records, since a pattern no single record states needs more than one to state it.
when
answer_kindhas valuesynthesis, thenexpected_anchorshas at least 2 value(s)
Slot
connecting_class
The class a joining record would belong to, as a URI in the target schema. Optional, because a benchmark can be written against a graph whose schema its author does not have: without it the claim is that no record of any kind joins the anchors, and with it the claim is narrowed to one kind.
Slot
description
A longer prose account of what this benchmark covers.
Slot
expected_anchors
The records a correct answer must retrieve, as IRIs into the target graph. For an answer that reports no such record exists, these are the records whose absent connection is the answer, and reaching them is what distinguishes that answer from a system that retrieved nothing.
-
A comparison sets two or more records against each other.
when
answer_kindhas valuecomparison, thenexpected_anchorshas at least 2 value(s) -
A synthesis draws on two or more records, since a pattern no single record states needs more than one to state it.
when
answer_kindhas valuesynthesis, thenexpected_anchorshas at least 2 value(s)
Slot
expected_citations
The records a correct answer must rest on, as IRIs into the target graph. Always among the anchors: retrieving a record is not the same as answering from it. Empty is a real answer.
-
An attribution names the source its answer rests on.
when
answer_kindhas valueattribution, thenexpected_citationsis present
Slot
ground_truth
The correct answer in prose. A generated answer is scored against this; the anchors below are what retrieval is scored against.
Slot
id
The short, stable identifier a record is known by.
Slot
name
A short human-readable label.
Slot
question
The competency question, in the words it would be asked in.
Slot
target_dataset
Which instance graph the anchors resolve in, named as the target package publishes it: a dataset name from that package's publish manifest. One schema can have several, and they need not hold the same records, so naming the schema and its version does not identify one.
Slot
target_dataset_version
The version of the dataset the anchors were written against: the release identifier of whatever publishes it, which is the schema's own version when the two ship together. It fixes which records exist, since records can be added, corrected or removed without the schema changing.
Slot
target_schema
The schema the target graph's records conform to, as a URI.
Slot
target_schema_version
The version of the target schema the anchors were written against. It fixes how identifiers are minted, which can change between releases.
Slot
unconnected_anchors
The anchors that no single record joins, for an answer whose correctness is that the graph holds no such connection. A subset of the anchors: a claim about a record retrieval was never required to reach cannot be checked against what was retrieved. Records held by a container do not count as joined by it.
-
A closed-world negative states the connection it claims is absent.
when
answer_kindhas valueclosed-world-negative, thenunconnected_anchorsis present
Enumerations 1
Enumerations defined in this ontology.
Enum
AnswerKindEnum
The kinds of answering a competency question can call for. Each names a different check on a correct answer.
-
attribution
triggers 1 rule on CompetencyQuestionAnswer
An answer that must carry the source it rests on. The question is answerable without it, but not answerable well.
-
closed-world-negative
triggers 1 rule on CompetencyQuestionAnswer
The correct answer is that the graph holds no such record. Absence of a record is not evidence against the claim, only absence of a claim for it.
-
comparison
triggers 1 rule on CompetencyQuestionAnswer
Two or more records set against each other on a shared slot.
-
lookup
A value read from a single record.
-
synthesis
triggers 1 rule on CompetencyQuestionAnswer
A pattern across several records that no single record states.