Evaluate Model on Observed Dataset

This section demonstrates how to evaluate simulated model outputs against an observed dataset using evaluate().

Import Observed Data

Observed data may be loaded from a CSV file or any pandas DataFrame. For demonstration purposes, we use a mimicked dataset from the test suite.

from apsimNGpy.tests.unittests.test_factory import obs
from apsimNGpy.core.apsim import ApsimModel

Initialize and Run the Model

Replace 'Maize' with the path to your own .apsimx file, or customize the model configuration as needed.

with ApsimModel('Maize') as model:

    # Add simulation year to the report
    model.add_report_variable(
        variable_spec='[Clock].Today.Year as year',
        report_name='Report'
    )

    # Run the simulation
    model.run()

    # Evaluate simulation results against observed data
    metrics = model.evaluate(
        ref_data=obs,
        table='Report',
        index_col=['year'],
        target_col='Yield',
        ref_data_col='observed',
        verbose=True
    )

Output

If verbose=True, the evaluation summary will be printed to the console. Otherwise, all metrics are returned as a dictionary.

Example console output:

Model Evaluation Metrics
----------------------------------------
BIAS   :  -0.0001
CCC    :   1.0000
MAE    :   0.0003
ME     :   1.0000
MSE    :   0.0000
R2     :   1.0000
RMSE   :   0.0003
RRMSE  :   0.0000
SLOPE  :   1.0000
WIA    :   1.0000

Returned Metrics

The returned dictionary contains:

• BIAS — Mean bias error

• MAE — Mean absolute error

• MSE — Mean squared error

• RMSE — Root mean squared error

• RRMSE — Relative RMSE

• R2 — Coefficient of determination

• CCC — Concordance correlation coefficient

• ME — Model efficiency

• WIA — Willmott’s index of agreement

• SLOPE — Regression slope

See also

evaluate()