The project contains the evaluation of the loan approval scenario. The scenario focuses on the compliance analysis using the compliance AI agent framework. The scenario contains:
- original dataset
loan_dataset.csvfrom https://www.kaggle.com/datasets/arbaaztamboli/loan-approval-dataset. - the output specifications (
outputs.json) produced by the agent for the analysis of the project context, dataset and trained classifier model objectives. - interactive session in VSCode Copilot for the analysis of the project (
session_analyis.md, GPT-6.1 Luna model), evaluation of the dataset compliance properties (session_data_evaluation.md, GPT-6.1 Luna model), and the evaluation of model compliance objectives (session_model_evaluaition.md, GPT-6.1 Sol model). - the generated compliance evaluator for the dataset (
compliance_report.py). - the original version of the source code for the classifier training (
loan_approval_train_original.py). - the modified version of the source code for the classifier training (
loan_approval_train.py) to incorporate the necessary compliance metrics. - the generated evaluator definitions driving the agents (
evaluation/). - the generated compliance evaluator for the model (
loan_model_compliance.py).