GitHub

Example: Using the POMATWO Data Reporting System

Overview

The POMATWO package now includes a comprehensive data reporting system that provides detailed validation feedback when loading electricity market data. This system helps identify data quality issues early and provides clear guidance for fixing them.

Basic Usage

Loading Data with Reports

using POMATWO

# Your data file paths
data_files = Dict{Symbol,String}(
    :plants => "data/plants.csv",
    :nodes => "data/nodes.csv", 
    :zones => "data/zones.csv",
    :demand => "data/demand.csv",
    :types => "data/planttypes.csv"
)

# Option 1: Use the new reporting function
params, report = load_data_with_report(data_files)

# Check if there were any issues
print_report(report)

Report Structure

Report Levels

  • NOTES: Informational messages (data loaded successfully, defaults applied)
  • WARNINGS: Potential issues that don't prevent execution (unusual values, optional files missing)
  • ERRORS: Critical issues that prevent proper functioning (missing files, invalid data)

Report Categories

  • file_access: File existence and readability
  • missing_columns: Required or optional columns not found
  • missing_values: Missing data in required fields
  • data_type: Non-numeric data in numeric columns
  • range_validation: Values outside expected ranges
  • duplicate_values: Duplicate indices or identifiers
  • configuration_error: Critical setup issues
  • incomplete_data: Rows with missing critical information

Detailed Validations

Plants Data Validation

  • Required columns: index, plant_type, node, g_max, eta
  • Positive values: g_max >= 0, storage_capacity >= 0, storage_power >= 0
  • Efficiency range: eta ∈ [0, 1]
  • No duplicate plant indices
  • Proper data types in numeric columns

Nodes Data Validation

  • Required columns: index, zone, slack
  • Slack bus configuration: exactly one node with slack = 1
  • Binary values: slack ∈ {0, 1}
  • No duplicate node indices
  • Coordinate validation (if provided)

Zones Data Validation

  • Required columns: index
  • No duplicate zone indices

Types Data Validation

  • Required columns: index, dispatchable
  • Binary values: dispatchable, prosumer, storage ∈ {0, 1}
  • Positive values: fuel_price > 0, co2content > 0

Input Data Reporting API

POMATWO.DataReport — Type
DataReport

Container for collecting data validation reports during data loading.

Fields

  • items::Vector{DataReportItem}: Collection of all report items
  • has_errors::Bool: Quick check if any errors were reported
source
POMATWO.DataReportItem — Type
DataReportItem

Structure representing a single data validation report item.

Fields

  • level::DataReportLevel: The severity level (NOTE, WARNING, ERROR)
  • category::String: Category of the validation issue (e.g., "fileaccess", "datatype", "range")
  • message::String: Descriptive message about the validation issue
  • location::String: Location where the issue was found (e.g., file path, column name)
source
POMATWO.print_report — Function
print_report(report::DataReport; show_notes::Bool=true, show_warnings::Bool=true, show_errors::Bool=true)

Print a formatted summary of the data report.

source
POMATWO.get_errors — Function
get_errors(report::DataReport)

Return all error-level items from the data report.

Arguments

  • report::DataReport: The data report to filter.

Returns

  • Vector{DataReportItem}: All items with level ERROR.
source
POMATWO.get_warnings — Function
get_warnings(report::DataReport)

Return all warning-level items from the data report.

Arguments

  • report::DataReport: The data report to filter.

Returns

  • Vector{DataReportItem}: All items with level WARNING.
source
POMATWO.get_notes — Function
get_notes(report::DataReport)

Return all note-level items from the data report.

Arguments

  • report::DataReport: The data report to filter.

Returns

  • Vector{DataReportItem}: All items with level NOTE.
source
POMATWO.load_data_with_report — Function
load_data_with_report(data::Dict)

Extended version of load_data that returns both Parameters and a DataReport.

Usage

params, report = loaddatawithreport(datafiles)

source