MarkdownCaseLoader#

Module: agent_framework_evaluator.case_markdown

class agent_framework_evaluator.case_markdown.MarkdownCaseLoader(base_dir, glob_pattern, evaluator_registry=None, *, initializer_ref=None, resolver=None)[source]#

Bases: object

Discover *.md cases under base_dir with a glob; cache invalidates on path/mtime changes.

Pass initializer_ref to automatically skip cases whose initializer frontmatter field is set to a different initializer (stem comparison, so foo.py matches foo). Cases with no initializer frontmatter field always match.

Parameters:

  • base_dir (Path)

  • glob_pattern (str)

  • evaluator_registry (Mapping[str, Callable[..., Any]] | None)

  • initializer_ref (str | None)

  • resolver (FileReferenceResolver | None)

get_test_cases()[source]#
Return type:

list[dict[str, Any]]