Dimitris Kontokostas Jens Lehmann Patrick Westphal Roland Cornelissen Sebastian Hellmann Sören Auer 2013-10-01 The RDFUnit ontology describes concepts used in RDFUnit, a test driven RDF Validation framework that can run automatically generated (based on a schema) and manually generated test cases against an endpoint. It's best described in "NLP data cleansing based on Linguistic Ontology constraints" (ESWC 2014) p5-7, http://jens-lehmann.org/files/2014/eswc_rdfunit_nlp.pdf, includes the ontology diagram. 2013-10-01 2014-01-10 This ontology is licensed under Apache 2.0 (http://www.apache.org/licenses/LICENSE-2.0) and CC-BY (http://creativecommons.org/licenses/by/3.0/) Test-Driven Data Validation Ontology rut http://rdfunit.aksw.org/ns/core# The RDFUnit ontology describes concepts used in RDFUnit, a test driven RDF Validation framework that can run automatically generated (based on a schema) and manually generated test cases against an endpoint. It's best described in "NLP data cleansing based on Linguistic Ontology constraints" (ESWC 2014) p5-7, http://jens-lehmann.org/files/2014/eswc_rdfunit_nlp.pdf, includes the ontology diagram. Test-Driven RDF Validation Ontology 0.4.1 Data Quality Test Pattern (DQTP): a query with embedded %%parameter's%% and a corresponding list of rut:parameter's Pattern 1 1 1 1 The WHERE part of a sparql query with %%parameter%% placeholders. Must bind variables ?resource and ?message (see rut:sparqlWhere) sparqlWherePattern A sparql query to COUNT the number of instances being tested. May reuse %%parameter's%% from rut:sparqlWherePattern. May be empty but should be present (TODO: why?) sparqlPrevalencePattern List of Parameters for the Pattern parameter A pattern parameter. Has dct:identifier equal to the %%parameter%% placeholder. rut:parameterConstraint and rut:constraintPattern determine the possible values Parameter 1 1 1 What can be substituted for the parameter, one of None (means anything), Operator, Resource, Property, Class parameterConstraint Regexp that further qualifies a rut:parameterConstraint, e.g. '<|<=|>|>=|=|!=' for a rut:Operator constraintPattern What can be substituted for a parameter, one of None (means anything), Operator, Resource, Property, Class ParameterConstraint Generates TestCases based on a Pattern and a SPARQL query that instantiates the pattern Parameters TestGenerator 1 1 SPARQL SELECT query that instantiates test cases based on a Pattern. Must return variables named the same as the Pattern's parameters, plus a ?DESCRIPTION sparqlGenerator The Pattern instantiated by a PatternBasedTestCase or a TestGenerator basedOnPattern Superclass of all test cases. Defines properties that all cases should have, but concrete implementation occurs on subclasses. TestCase 1 1 1 1 1 What does a test case apply to: Schema, EnrichedSchema, Dataset or Application appliesTo How the test was created: Automatically generated or Manually made. generated The Classes and Properties involved in the described test case. references A schema (ontology), dataset or application that this test case was created for source rlog:Level that this TestCase will return in case of failure, or StatusTestCaseResult has recorded. Analogous to rlog:level, which applies to rlog:Entry and rut:RLOGTestCaseResult testCaseLogLevel A test case that is based on a Pattern instantiated with Bindings for each Parameter PatternBasedTestCase 1 1 Binding of a pattern Parameter to a value binding Binding of a pattern Parameter to a value Binding 1 1 the value provided for the bound Parameter bindingValue A test case where the SPARQL is defined manually ManualTestCase 1 1 The WHERE part of a SPARQL query. Must be enclosed in brackets {} and bind the following variables: ?resource: the instance (URI) where the constraint violation occurs; ?message: a logging message. sparqlWhere A SPARQL query to COUNT the number of instances being tested sparqlPrevalence How a test case was created: Automatically generated or Manually made TestGenerationType What does a test case apply to: Schema, EnrichedSchema, Dataset or Application TestAppliesTo A suite of TestCases, linked to it using prov:hadMember TestSuite 1 Links an ontology to a TestSuite for validating data adhering to that ontology. Usually each TestCase in the suite will have rut:appliesTo rut:Schema (or rut:EnrichedSchema), and link back to the ontology using rut:source testSuite A dependency between two test cases TestCaseDependency Test case that preconditions the execution of another test case dependencyFrom Test case whose execution depends on another test case dependencyTo The rut:ResultStatus of the dependencyFrom case that triggers this dependency dependencyCondition Whether to execute the dependencyTo case. If a case has multiple dependencies, it is executed only if all dependencyCondition's are satisfied, and all dependencyExecute's are true. TODO: IS THIS TRUE? Else: Only useful while debugging test suites dependencyExecute Holds the TestCaseResults of a TestSuite execution against a dataset. The results are linked to TestExecution by prov:wasGeneratedBy TestExecution 1 1 1 Total tests run in a TestExecution testsRun Total tests that succedded in a TestExecution testsSucceeded Total tests that failed in a TestExecution testsFailed Total tests that timed out in a TestExecution testsTimeout Total tests that raised an error in a TestExecution testsError The result of a TestExecution. Each result links to the TestCase (rut:testCase) it originated from, and to the TestExecution (prov:wasGeneratedBy) it is part of TestCaseResult 1 1 The TestCase that this TestCaseResult originated from testCase The status and log level of a single TestCaseResult. TODO: Why is rut:testCaseLogLevel mandatory, doesn't it apply only when resultStatus=Fail? It's optional in rut:TestCase StatusTestCaseResult 1 1 1 result status. One of Success, Fail, Timeout, Error resultStatus Result Status. One of Success, Fail, Timeout, Error ResultStatus Aggregated results of a single TestCase execution AggregatedTestCaseResult 1 1 The number of results having the given rut:resultStatus in StatusTestCaseResult execution. -1 means there was a timeout and -2 an error. resultCount The total COUNT of instances tested by a TestCase execution. -1 means there was a timeout and -2 an error. resultPrevalence A TestCaseResult at the level of an individual rlog:resource (the resource that was tested), with rlog:message and rlog:level RLOGTestCaseResult 1 1 1 An extended TestCaseResult that includes additional information: spin:violationRoot (offending resource), spin:violationPath (offending property), spin:violationValue (offending value), rut:errorPropertyContext (property also involved in the offense), and error classification, source and type. These are provided through rut:ResultAnotation ExtendedTestCaseResult 1 Property(ies) that also contributed to the offense (e.g. dbo:deathDate when spin:violationPath is dbo:birthDate). errorPropertyContext High-level classification of the error. Usually a skos:Concept errorClassification Source of the error, eg data parsing, data publishing, mapping, pre processing, post processing (if known beforehand). Usually a skos:Concept errorSource More specific error type, eg missing property, redundant property, inaccurate property. Usually a skos:Concept errorType Used by a TestCase, TestAutoGenerator or Pattern to add properties to an ExtendedTestCaseResult. ResultAnnotation Property to add to ExtendedTestCaseResult, eg spin:violationPath, spin:violationValue, rut:errorPropertyContext annotationProperty Value to add to ExtendedTestCaseResult. Either constant (eg rlog:WARN) or a string designating the query '?variable' or pattern '%%parameter%%' to use annotationValue ResultAnnotation to apply to all results produced by a TestCase, TestAutoGenerator, or Pattern resultAnnotation The parameter can be anything, even a free variable (?var) None The parameter can be an operator. Operator The pattern can be a resource. Resource The parameter is restricted to a property. Property The parameter is restricted to a Class. Class Schema: When the tests apply to an ontology / vocabulary Schema Dataset: When the tests apply to a dataset only (i.e. a SPARQL Endpoint) Dataset EnrichedSchema: When the tests apply to an ontology / vocabulary enriched with external (semi-)automatic approaches EnrichedSchema Application: When the tests are specific to an application only Application ManuallyGenerated: This test was created manually ManuallyGenerated AutoGenerated: This test was created automatically AutoGenerated ResultStatusSuccess: the test succeeded ResultStatusSuccess ResultStatusFail: the test failed. rut:testCaseLogLevel further specifies the rlog:Level ResultStatusFail ResultStatusTimeout: the test timed out ResultStatusTimeout ResultStatusError: the test raised an error (eg HTTP error code) ResultStatusError