Refinement and Validation

This chapter applies Step 7 of Ontology Development 101 (Noy & McGuinness, 2001) to scimantic, and with it closes the rebuild.

Step 7 is instances, but here instances are a means rather than the end. Building them out against a real study is how the schema gets validated, and refined in the same motion: each filled-in slot value tests whether the ontology can carry the study, and the gaps that surface are the refinements to make. Validation and refinement are not two phases here. They are one interleaved activity.

That refinement comes from two directions. One is dogfooding: validating the schema against Appendix A’s worked study, instance by instance, surfaces what it still lacks. The other is reflection, stepping back from the finished hierarchy to ask whether every class it accumulated still earns its place. This is the one kind of refinement that needs no instances to motivate it. N&M’s second rule, that ontology development is necessarily iterative (Chapter 2), makes both expected. The chapter opens with the reflection, because the graph alone made it plain.

Culling the uncertainty cluster

At the end of Chapter 7 the schema was complete enough to review as a whole. Its provenance classes form a single connected component in the graph: claims, acts, and artifacts linked by hasInput, hasOutput, and the act chain. Four classes stand outside it. Uncertainty, Credibility, and UncertaintyModel form a fragment the provenance chain never reaches, and StatisticalMethod, though placed under AnalyticalMethod, is used by no act. All four were added in Step 4 and have gone unused since. The question is whether they belong in scimantic at all.

Two reasons say they do not.

The first is a category error. Uncertainty and Credibility were grounded as BFO qualities (obo:BFO_0000019), and Chapter 5 reasoned that a result’s uncertainty “inheres in the result.” But a quality inheres only in an independent continuant, and the thing it was meant to qualify, a Result, is not one. A result is an information content entity, a generically dependent continuant, and a quality cannot inhere in it.

The grounding was incorrect from the start, independent of the graph; the disconnected component only made it easy to notice.

The second is a layering error. Uncertainty is a property of the data, not of the provenance chain. A distribution over a data point or a confidence interval on a result belongs with the data itself, a layer below the one scimantic models. That layer already has a vocabulary for it: scimantic’s Dataset is a dcat:Dataset, and DCAT’s quality companion, the W3C Data Quality Vocabulary (DQV), attaches quality measurements and annotations to a dataset. A consumer records a result’s uncertainty as one such measurement, against the data rather than the provenance graph. scimantic records that a result was produced, by which act and from which dataset; how far it could vary is the data layer’s to express.

So all four classes are removed, along with the slots only they bore (quantifies, family, parameters, confidenceLevel, and nature), the UncertaintyNature enum, and the now-unused urref: prefix.

Removing a class is not the same as denying what it named, and two of the four need a note on where their content goes. Credibility is not lost, only relocated. An EvidenceAssessment already weighs a piece of evidence and, on acceptance, confers an AcceptedState on it; credibility is the judgment that assessment makes, read from the conferred state, not a separate quality that needs its own class. (If a graded score is wanted over the plain accept-or-reject verdict, it is a slot on EvidenceAssessment, where the judgment is made.) StatisticalMethod was is_a AnalyticalMethod, but statistics is orthogonal to the analytical/experimental split rather than a subtype of one side. If a concrete statistical method type is needed later, it returns as a mixin composed onto the relevant method, added on demand rather than now.

The removal also settles two threads earlier chapters left open. Chapter 2’s twelfth competency question, what is the uncertainty model for a given result, and how was it derived?, falls to the data layer rather than scimantic, and the validation pass will mark it out of scope. The [0, 1] bound Chapter 7 placed on confidenceLevel is removed with the class that carried it; the matching bound on strength is unaffected.

--- scimantic-yaml-v11
+++ scimantic-yaml-v12
@@ -23,13 +23,11 @@
   cco: https://www.commoncoreontologies.org/
 
   # Reused domain vocabularies: metadata, annotation, datasets,
-  # nanopublication, uncertainty, claim relations
+  # nanopublication, claim relations
   dcterms: http://purl.org/dc/terms/
   oa: http://www.w3.org/ns/oa#
   dcat: http://www.w3.org/ns/dcat#
   np: http://www.nanopub.org/nschema#
-  urref: http://eturwg.c4i.gmu.edu/files/ontologies/URREF.owl#
   cito: http://purl.org/spar/cito/
 
@@ -311,39 +309,6 @@
       polarity, a strength, and the asserting act, through which
       the agent and time are reached.
 
-  Uncertainty:
-    subclass_of: obo:BFO_0000019
-    description: >-
-      A quality of a result quantifying how much it could vary;
-      borne by the result, quantified by an UncertaintyModel.
-
-  Credibility:
-    subclass_of: obo:BFO_0000019
-    description: >-
-      A graded quality of a piece of evidence expressing how far it
-      can be relied on; conferred by an EvidenceAssessment.
-
-  UncertaintyModel:
-    subclass_of: cco:ont00000853
-    slots:
-      - quantifies
-      - family
-      - parameters
-      - confidenceLevel
-      - nature
-    description: >-
-      A Descriptive ICE that quantifies a result's uncertainty
-      quality: a family, its parameters, a confidence level, and a
-      nature (aleatory or epistemic).
-
-  StatisticalMethod:
-    is_a: AnalyticalMethod
-    description: >-
-      An analytical method prescribing how to derive a result's
-      uncertainty and credibility from data.
-
   EvidenceLine:
     subclass_of: cco:ont00000853
@@ -528,35 +493,6 @@
       The act that asserted this relation; its agent and time are
       reached through the act.
 
-  quantifies:
-    range: Uncertainty
-    required: true
-    description: The uncertainty quality this model quantifies.
-
-  family:
-    range: string
-    description: >-
-      The distribution or method family — Gaussian, bootstrap, and
-      so on. An open string, not enumerated.
-
-  parameters:
-    range: string
-    multivalued: true
-    description: >-
-      The model's parameters; free strings here, with structured
-      typing left to a later increment.
-
-  confidenceLevel:
-    range: float
-    minimum_value: 0
-    maximum_value: 1
-    description: >-
-      The confidence level the model reports, in [0, 1] (e.g. 0.95).
-
-  nature:
-    range: UncertaintyNature
-    description: Whether the uncertainty is aleatory or epistemic.
-
   members:
     range: Evidence
     multivalued: true
@@ -597,14 +533,3 @@
       refines:
         meaning: cito:extends
         description: One claim refines or extends another.
-
-  UncertaintyNature:
-    description: >-
-      Whether a result's uncertainty is aleatory (irreducible
-      randomness) or epistemic (reducible lack of knowledge). A
-      plain enum for now; a URREF grounding is deferred.
-    permissible_values:
-      aleatory:
-        description: Irreducible variability in the phenomenon.
-      epistemic:
-        description: Reducible uncertainty from limited knowledge.