Refinement and Validation
This chapter applies Step 7 of Ontology Development 101 (Noy & McGuinness, 2001) to scimantic, and with it closes the rebuild.
The last step is creating individual instances of classes in the hierarchy. Defining an individual instance of a class requires (1) choosing a class, (2) creating an individual instance of that class, and (3) filling in the slot values.
Step 7 is instances, but here instances are a means rather than the end. Building them out against a real study is how the schema gets validated, and refined in the same motion: each filled-in slot value tests whether the ontology can carry the study, and the gaps that surface are the refinements to make. Validation and refinement are not two phases here. They are one interleaved activity.
That refinement comes from two directions. One is dogfooding: validating the schema against Appendix A’s worked study, instance by instance, surfaces what it still lacks. The other is reflection, stepping back from the finished hierarchy to ask whether every class it accumulated still earns its place. This is the one kind of refinement that needs no instances to motivate it. N&M’s second rule, that ontology development is necessarily iterative (Chapter 1), makes both expected. The chapter opens with the reflection, because the graph alone made it plain.
Culling the uncertainty cluster
At the end of Chapter 6 the schema was complete enough to review as a
whole. Its provenance classes form a single connected component in the
graph: claims, acts, and artifacts linked by hasInput, hasOutput, and
the act chain. Four classes stand outside it. Uncertainty, Credibility,
and UncertaintyModel form a fragment the provenance chain never reaches,
and StatisticalMethod, though placed under AnalyticalMethod, is used by
no act. All four were added in Step 4 and have gone unused since. The
question is whether they belong in scimantic at all.
Two reasons say they do not.
The first is a category error. Uncertainty and Credibility were
grounded as BFO qualities (obo:BFO_0000019), and Chapter 4 reasoned that
a result’s uncertainty “inheres in the result.” But a quality inheres only
in an independent continuant, and the thing it was meant to qualify, a
Result, is not one. A result is an information content entity, a
generically dependent continuant, and a quality cannot inhere in it.
BFO splits continuants three ways by how they depend on other things. An independent continuant exists in its own right and can bear qualities: an organism, a sample, an instrument. A specifically dependent continuant, such as a quality, exists only by inhering in one particular bearer. A generically dependent continuant is a pattern of information that can be copied across bearers; an ICE is one. Inherence (Chapter 4) runs from a quality to an independent continuant, so grounding the uncertainty of an information artifact as a quality mismatches the categories.
The grounding was incorrect from the start, independent of the graph; the disconnected component only made it easy to notice.
The second is a layering error. Uncertainty is a property of the data,
not of the provenance chain. A distribution over a data point or a
confidence interval on a result belongs with the data itself, a layer below
the one scimantic models. That layer already has a vocabulary for it:
scimantic’s Dataset is a dcat:Dataset, and DCAT’s quality companion,
the W3C Data Quality Vocabulary (DQV),
attaches quality measurements and annotations to a dataset. A consumer
records a result’s uncertainty as one such measurement, against the data
rather than the provenance graph. scimantic records that a result was
produced, by
which act and from which dataset; how far it could vary is the data layer’s
to express.
So all four classes are removed, along with the slots only they bore
(quantifies, family, parameters, confidenceLevel, and nature),
the UncertaintyNature enum, and the now-unused urref: prefix.
Removing a class is not the same as denying what it named, and two of the
four need a note on where their content goes. Credibility is not lost,
only relocated. An EvidenceAssessment already weighs a piece of evidence
and, on acceptance, confers an AcceptedState on it; credibility is the
judgment that assessment makes, read from the conferred state, not a
separate quality that needs its own class. (If a graded score is wanted
over the plain accept-or-reject verdict, it is a slot on
EvidenceAssessment, where the judgment is made.) StatisticalMethod was
is_a AnalyticalMethod, but statistics is orthogonal to the
analytical/experimental split rather than a subtype of one side. If a
concrete statistical method type is needed later, it returns as a mixin
composed onto the relevant method, added on demand rather than now.
The removal also settles two threads earlier chapters left open. Chapter
2’s twelfth competency question, what is the uncertainty model for a given
result, and how was it derived?, falls to the data layer rather than
scimantic, and the validation pass will mark it out of scope. The [0, 1]
bound Chapter 6 placed on confidenceLevel is removed with the class that
carried it; the matching bound on strength is unaffected.
--- scimantic-yaml-v11
+++ scimantic-yaml-v12
@@ -23,13 +23,11 @@
cco: https://www.commoncoreontologies.org/
# Reused domain vocabularies: metadata, annotation, datasets,
- # nanopublication, uncertainty, claim relations
+ # nanopublication, claim relations
dcterms: http://purl.org/dc/terms/
oa: http://www.w3.org/ns/oa#
dcat: http://www.w3.org/ns/dcat#
np: http://www.nanopub.org/nschema#
- urref: http://eturwg.c4i.gmu.edu/files/ontologies/URREF.owl#
cito: http://purl.org/spar/cito/
@@ -311,39 +309,6 @@
polarity, a strength, and the asserting act, through which
the agent and time are reached.
- Uncertainty:
- subclass_of: obo:BFO_0000019
- description: >-
- A quality of a result quantifying how much it could vary;
- borne by the result, quantified by an UncertaintyModel.
-
- Credibility:
- subclass_of: obo:BFO_0000019
- description: >-
- A graded quality of a piece of evidence expressing how far it
- can be relied on; conferred by an EvidenceAssessment.
-
- UncertaintyModel:
- subclass_of: cco:ont00000853
- slots:
- - quantifies
- - family
- - parameters
- - confidenceLevel
- - nature
- description: >-
- A Descriptive ICE that quantifies a result's uncertainty
- quality: a family, its parameters, a confidence level, and a
- nature (aleatory or epistemic).
-
- StatisticalMethod:
- is_a: AnalyticalMethod
- description: >-
- An analytical method prescribing how to derive a result's
- uncertainty and credibility from data.
-
EvidenceLine:
subclass_of: cco:ont00000853
@@ -528,35 +493,6 @@
The act that asserted this relation; its agent and time are
reached through the act.
- quantifies:
- range: Uncertainty
- required: true
- description: The uncertainty quality this model quantifies.
-
- family:
- range: string
- description: >-
- The distribution or method family — Gaussian, bootstrap, and
- so on. An open string, not enumerated.
-
- parameters:
- range: string
- multivalued: true
- description: >-
- The model's parameters; free strings here, with structured
- typing left to a later increment.
-
- confidenceLevel:
- range: float
- minimum_value: 0
- maximum_value: 1
- description: >-
- The confidence level the model reports, in [0, 1] (e.g. 0.95).
-
- nature:
- range: UncertaintyNature
- description: Whether the uncertainty is aleatory or epistemic.
-
members:
range: Evidence
multivalued: true
@@ -597,14 +533,3 @@
refines:
meaning: cito:extends
description: One claim refines or extends another.
-
- UncertaintyNature:
- description: >-
- Whether a result's uncertainty is aleatory (irreducible
- randomness) or epistemic (reducible lack of knowledge). A
- plain enum for now; a URREF grounding is deferred.
- permissible_values:
- aleatory:
- description: Irreducible variability in the phenomenon.
- epistemic:
- description: Reducible uncertainty from limited knowledge.
What a data file needs that a schema did not
The dogfood direction starts before the first record is written. scimantic’s worked example is a data file: LinkML instance data, records conforming to the schema’s classes, held apart from the schema itself. Trying to write one surfaced three things six chapters of schema building never needed, and each lands here as a refinement — new freezes, not retro-edits to earlier snapshots.
The first is an identifier. Instances form a provenance graph with
shared references: one act’s output is another act’s input, a state
points back at the act that established it, an evidential relation
names two claims. A reference needs something to point at, and no
scimantic class had a key. The new id slot (7.2.3)
is marked identifier: true and joins every referenceable class, so
hasInput: [upstream-result] in a record means the node carrying
that id. Identifiers are unique across a whole file, not per class — two
records may not share one even when their classes differ.
The second is a root. A YAML file holds one document, and a
document conforming to a schema must be an instance of some class —
so the schema needs a class whose instances are data files.
ProvenanceRecord (7.2.2) is that class, marked
tree_root: true, holding one multivalued collection per concrete
class plus a title and description for the file itself. Its
collections are attributes: rather than entries in the shared
slots: block: they exist only on the container, and the container
exists only because a data file needs a root.
The third the schema had seen coming. Chapter 6’s relational
discussion left open whether Study should stay a sibling of Act —
both grounded in CCO Planned Act, with Study carrying only
hasPart — or become an act itself. Building the record forced the
call: a study has a principal investigator, which is agent; it runs
over a timespan, which is performedAt; and one question goes in and
one concluding claim comes out, which are hasInput and hasOutput
narrowed to Question and Conclusion. So Study is now is_a Act
(7.2.1), inheriting the act slots and the Planned Act
grounding through Act instead of declaring its own.
One more change rides this freeze without being schema design at all. Between the last freeze and this one the book was renumbered: the Introduction became an unnumbered prefix chapter, so each chapter now carries the number of the N&M step it applies. Two slot descriptions that pointed at “Chapter 7” for the per-act narrowing now say Chapter 6, and the diff shows the edit. The frozen listings of earlier chapters keep the bytes they were frozen with — where one renders the old number, a note beside it says why.
--- scimantic-yaml-v12
+++ scimantic-yaml-v13
@@ -46,6 +46,7 @@
subclass_of: cco:ont00000853
slots:
+ - id
- supports
- contradicts
- refines
@@ -80,6 +81,7 @@
subclass_of: cco:ont00000228
slots:
+ - id
- agent
- performedAt
- hasInput
@@ -208,6 +210,8 @@
Method:
abstract: true
subclass_of: cco:ont00000965
+ slots:
+ - id
description: >-
A prescriptive information content entity that prescribes how
to carry out an act, realized by the acts that execute it.
@@ -230,6 +234,7 @@
subclass_of: cco:ont00000819
slots:
+ - id
- qualifies
- establishedBy
- occursOver
@@ -261,6 +266,8 @@
Agent:
subclass_of: cco:ont00001017
+ slots:
+ - id
description: >-
A material entity that performs acts and bears responsibility
for them — a researcher, group, or institution.
@@ -268,6 +275,8 @@
TemporalInterval:
subclass_of: obo:BFO_0000038
+ slots:
+ - id
description: >-
The stretch of time an act occupies, with a beginning and an
end rather than a single instant.
@@ -275,6 +284,8 @@
Question:
subclass_of: cco:ont00000958
+ slots:
+ - id
description: >-
An information content entity expressing what an inquiry seeks
to answer; the output of a QuestionFormation and the start of
@@ -283,6 +294,8 @@
Result:
subclass_of: cco:ont00000853
+ slots:
+ - id
description: >-
A finding produced by an Analysis; the input a ResultAssessment
weighs in drawing a conclusion.
@@ -290,6 +303,8 @@
SourceDocument:
subclass_of: cco:ont00001298
+ slots:
+ - id
description: >-
A document a LiteratureSearch surfaces and an Annotation marks
up; the bearer a piece of Evidence is extracted from.
@@ -298,6 +313,7 @@
EvidentialRelation:
subclass_of: cco:ont00000853
slots:
+ - id
- subject
- object
- polarity
@@ -313,6 +329,7 @@
EvidenceLine:
subclass_of: cco:ont00000853
slots:
+ - id
- members
- strength
- bearsOn
@@ -320,31 +337,142 @@
A Descriptive ICE grouping several pieces of evidence under
one strength and asserting their joint bearing on a claim.
Study:
- subclass_of: cco:ont00000228
+ is_a: Act
slots:
+ - id
- hasPart
+ slot_usage:
+ hasInput:
+ range: Question
+ hasOutput:
+ range: Conclusion
description: >-
- A planned process whose parts are the formations, searches,
- experiments, and analyses of one question-to-conclusion cycle.
+ A composite act whose parts are the formations, searches,
+ experiments, and analyses of one question-to-conclusion cycle;
+ it takes the motivating question as input and produces the
+ concluding claim.
Dataset:
class_uri: dcat:Dataset
+ slots:
+ - id
description: >-
A dataset reused from W3C DCAT: the input to an analysis or
experiment, or the output a result is drawn from.
Annotation:
class_uri: oa:Annotation
+ slots:
+ - id
description: >-
An annotation on a source document, reused from the W3C Web
Annotation model; an EvidenceExtraction draws a piece of
Evidence from one.
+ ProvenanceRecord:
+ tree_root: true
+ attributes:
+ title:
+ range: string
+ description: A short name for the instance file's contents.
+ description:
+ range: string
+ description: What study or studies the instance file records.
+ questions:
+ range: Question
+ multivalued: true
+ hypotheses:
+ range: Hypothesis
+ multivalued: true
+ evidence:
+ range: Evidence
+ multivalued: true
+ conclusions:
+ range: Conclusion
+ multivalued: true
+ results:
+ range: Result
+ multivalued: true
+ datasets:
+ range: Dataset
+ multivalued: true
+ annotations:
+ range: Annotation
+ multivalued: true
+ sourceDocuments:
+ range: SourceDocument
+ multivalued: true
+ questionFormations:
+ range: QuestionFormation
+ multivalued: true
+ literatureSearches:
+ range: LiteratureSearch
+ multivalued: true
+ evidenceExtractions:
+ range: EvidenceExtraction
+ multivalued: true
+ evidenceAssessments:
+ range: EvidenceAssessment
+ multivalued: true
+ hypothesisFormations:
+ range: HypothesisFormation
+ multivalued: true
+ designOfExperiments:
+ range: DesignOfExperiment
+ multivalued: true
+ experimentations:
+ range: Experimentation
+ multivalued: true
+ analyses:
+ range: Analysis
+ multivalued: true
+ resultAssessments:
+ range: ResultAssessment
+ multivalued: true
+ evidentialRelations:
+ range: EvidentialRelation
+ multivalued: true
+ evidenceLines:
+ range: EvidenceLine
+ multivalued: true
+ openStates:
+ range: OpenState
+ multivalued: true
+ acceptedStates:
+ range: AcceptedState
+ multivalued: true
+ retractedStates:
+ range: RetractedState
+ multivalued: true
+ experimentalMethods:
+ range: ExperimentalMethod
+ multivalued: true
+ analyticalMethods:
+ range: AnalyticalMethod
+ multivalued: true
+ studies:
+ range: Study
+ multivalued: true
+ agents:
+ range: Agent
+ multivalued: true
+ intervals:
+ range: TemporalInterval
+ multivalued: true
+
slots:
+ id:
+ identifier: true
+ description: >-
+ The identifier a record is referenced by in instance data;
+ unique across every record in a file.
+
supports:
range: Claim
@@ -417,7 +545,7 @@
- cco:ont00001921
description: >-
An artifact an act consumes; ranged broadly over the schema's
- artifacts here, narrowed per-act in Chapter 7.
+ artifacts here, narrowed per-act in Chapter 6.
hasOutput:
@@ -434,7 +562,7 @@
- cco:ont00001986
description: >-
An artifact an act produces; ranged broadly here, narrowed
- per-act in Chapter 7's facets.
+ per-act in Chapter 6's facets.
qualifies:
any_of: