semantica.evals is planned as a comprehensive evaluation framework for measuring extraction accuracy, graph quality, and pipeline performance.
semantica.evals is not yet implemented. The module is a placeholder with __all__ = []. No classes or functions are available for import. This page describes the planned API only.

​
Planned Features

When released, semantica.evals will provide:
Planned ClassRole
KGEvaluatorCompleteness, consistency, schema compliance, coverage, and orphan node detection
ExtractionEvaluatorNER precision / recall / F1 and relation extraction metrics against gold datasets
PipelineBenchmarkThroughput (docs/sec), per-step latency, peak memory, and error rate
RegressionTrackerRecord runs and compare metrics across commits or config changes
EvalReportStructured report: {scores, regressions, recommendations}
DeduplicationEvaluatorMerge precision, false positive / false negative rates
ReasoningEvaluatorInference accuracy, rule coverage, and derivation depth

​
Current Workaround

Until semantica.evals ships, use semantica.ontology.OntologyEvaluator for ontology quality metrics: 
from semantica.ontology import OntologyEvaluator

evaluator = OntologyEvaluator()

# evaluate_ontology takes the ontology dict only
result = evaluator.evaluate_ontology(ontology)

print("Coverage:    ", result.coverage_score)
print("Completeness:", result.completeness_score)
print("Gaps:        ", result.gaps)
print("Suggestions: ", result.suggestions)

# Full report with class granularity and relation completeness
report = evaluator.generate_report(ontology)
print("Coverage score:    ", report["evaluation"]["coverage_score"])
print("Completeness score:", report["evaluation"]["completeness_score"])
print("Relation coverage: ", report["relation_completeness"]["relation_coverage"])
EvaluationResult fields returned by evaluate_ontology():
FieldTypeDescription
coverage_scorefloatFraction of competency questions answerable by the ontology
completeness_scorefloatAverage of class and property completeness scores
gapsList[str]Identified gaps in coverage
suggestionsList[str]Improvement suggestions
metricsdictDetailed sub-metrics

Semantic Extract

Extraction module.

Knowledge Graph

Graph quality assessment.

Pipeline

Pipeline performance metrics.

Ontology Evaluator

Available now for ontology quality metrics.