ARPA-E Grid Optimization (GO) Competition Challenge 3

Description

Synthetic Input Data and Team Results for the GO Competition Challenge 3 for Events 1 - 4 and the Sandbox, along with problem and format descriptions and code to validate data and solutions, are available here. Data for industry scenarios will not be made public. The Grid Optimization (GO) Competition Challenge 3 focused on the security-constrained optimal power flow (SCOPF) problem. It is part of a continuing effort begun with Challenges 1 and 2, to successfully discover, develop, and test innovative and disruptive software solutions for critical energy challenges and to overcome existing barriers. The broader goal of the of the GO Competition is to accelerate the development of transformational and disruptive methods for solving problems related to the electric power grid and to provide a transparent, fair, and comprehensive evaluation of new solution methods. Challenge 3 used multiperiod dynamic markets, including advisory models for extreme weather events, day-ahead markets, and the real-time markets with an extended look-ahead. In Event 4, whose submission window was August 31-September 4, 2023, 14 teams solved for the objective values of 669 scenarios (39 scenarios required solutions both with and without line switching being allowed). The 591 synthetic scenarios from 9 network models (3.6 GB) are available here. Ten teams were funded to participate and 7 won prizes totaling $2,400,000. The largest prize ($550,000) went to Mississippi State University. An additional $600,000 was awarded in Event 3 (6/15-16/2023). No prizes were awarded in Events 1 (1/25-27/2023) or 2 (4/13-14/2023). For more information on the competition and challenge see the "GO Competition Challenge 3 Information" resource below. Challenge 1 and Challenge 2 information can be found in the resources linked below.

Resources

Name Format Description Link
57 Challenge 3 Event 4 24 8316-bus Division 2 synthetic dataset scenarios each consisting of 3 files: scenario.json (input), pop_solution.json (simple feasible solution), and popsolution.log (solution log). The datasets were developed by UW-M, GT and NREL. See source files tab on "GO Competition Challenge 3 Event 4 Results Summary.xlsx" resource for details. https://data.openei.org/files/5997/C3E4N08316D2_20231002.zip
57 The zip file contains 31 Event 3 Division 1 scenarios from 7 network models with 617-, 1576-, 4224-, 6049-, 6717-, 8316-, and 23643-buses. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3E3.1D1_20240606.zip
21 Supporting ARPA-E Power Grid Optimization (Final Report). https://doi.org/10.2172/2404530
21 Full information and data for Challenge 1. https://data.openei.org/submissions/6153
21 Full information and data for Challenge 2. https://data.openei.org/submissions/6197
33 Additional information about the ARPA-E Power Grid Optimization project. https://data.openei.org/files/5997/GO%20Competition%20Additional%20Info.pdf
21 This repository contains code to handle problem and solution data for the GO Competition Challenge 3. The main functionality includes: read problem data, check problem data formatting and properties, scrub problem data to remove unnecessary fields and anonymize string values, read solution data, check solution data formatting and properties, and evaluate solution to problem, i.e. constraint feasibility, objective value, properties of interest. Once a GO Competition Challenge 3 dataset is generated in the JSON format, the syntax and correctness are verified by the “checker” (check_data.py). The syntax and correctness of the JSON solution files is verified by the “evaluator” (evaluation.py), which also generates the solution score. Two Github repositories are used: the GO-3-data-model owned by Elaine Hale (NREL) was made publicly available September 26, 2022, and has been stable since Nov. 8, 2022; and the C3DataUtilities, owned by Steve Elbert (PNNL), was made public September 12, 2022, and has been stable since September 6, 2023. https://github.com/GOCompetition/C3DataUtilities
53 Event 3 Results Summary Spreadsheet with tabs data, edata, sdata,switching, obj vs time, Top 5 scores, Top 5 speeds, summary, source mapping, and Exit Codes. The "data" tab is based on Arun's event_results_all_E3.1_2024_05_29_10_41_36_837817275.csv collected from all Event 3 runs and contains 1989 rows (14 teams x (109 scenarios plus 30 with switching not allowed and 3 extra time runs; total 142) plus the tabulation row. The Data tab is sorted by team, model, scenario, and allow switching. Tab edata is the same data sorted by model, scenario, SW, score, and team. Tab sdata is sorted by speed instead of score. Speed is defined as score/runtime in seconds. Tab switching analyses the impact of switching. Tab obj vs time plots score vs runtime. Tab Top 5 scores shows the 5 best results for each of the 139 scenarios. Tab Top 5 speeds does the same for speed. Tab summary has rows for each scenario and columns rows for each scenario and columns for objective, feasibility, score, and scaled score (fraction of best score) for each team. Below that are summaries used in the Event 3 Leaderboard. Tab source mapping gives the location on Globus of each of the 109 scenarios and which team generated the data (GaTech, NREL, TAMU, or UW-Madison). The Exit Codes tab provides descriptions of the exit codes listed in the data tab as well as a description of the Deception computer. Column V in the data tab gives the URL of a tar.gz file on rcdtn1.pnl.gov that contains the results from each run on PNNL's Deception machine. The tar file typically contains the feasiblity.log from running the Evaluation code, a jobs.log, messages.log, the solver log, the solution.json, and a evaluation summary in csv and json formats. This csv feeds into the master csv Arun generates along with the runtime information (times, exit codes, reruns, etc.). Industry datasets do not include the solver log or the solution file. https://data.openei.org/files/5997/E3LB_master_20240605.xlsx
57 Challenge 3 Event 4 48 1576-bus synthetic dataset scenarios each consisting of 3 files: scenario.json (input), pop_solution.json (simple feasible solution), and popsolution.log (solution log). The zip file contains 24 datasets for Division 1 and 24 for Division 3. There are no Division 2 scenarios. The datasets were developed by UW-M and NREL. See source files tab on "GO Competition Challenge 3 Event 4 Results Summary.xlsx" resource for details. https://data.openei.org/files/5997/C3E4N01576_20231002.zip
33 Description of the GO Competition Challenge 3 data format including naming conventions, parameters, and input/output data descriptions. For additional Data Format information see: https://gocompetition.energy.gov/challenges/challenge-3/data_format. https://data.openei.org/files/5997/Challenge3_Data_Format_20230124.pdf
57 The zip file contains 41 Event 3 Division 2 scenarios from 7 network models with 73-, 617-, 1576-, 4224-, 6049-, 8316-, and 23643-buses. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3E3.1D2_20230629%20%281%29.zip
57 The zip file contains Event 2 (all divisions) scenarios for 2000- and 6717-bus systems that are replacements for the E2 files released May 10, 2023. The older files do not comply because they do not meet the positivity requirement of the current Problem Formulation and therefore fail the current data checker. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3E2.1_20230515.zip
57 Challenge 3 Sandbox scenarios were made available before each Event for practice purposes. These Sandbox 4 switching scenarios were released 8/09/2023 before Event 4 (August 31-September 4, 2023). The zip file contains 9 617-bus Division 1 scenarios and 4 73-bus Division 2 scenarios constructed to demonstrate the advantages of using line switching. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3S4X_20230809%20%281%29.zip
57 Challenge 3 Event 4 76 4224-bus synthetic dataset scenarios each consisting of 3 files: scenario.json (input), pop_solution.json (simple feasible solution), and popsolution.log (solution log). The zip file contains 28 datasets for Division 1, 24 for Division 2, and 24 for Division 3. The datasets were developed by UW-M, GT and NREL. See source files tab on "GO Competition Challenge 3 Event 4 Results Summary.xlsx" resource for details. https://data.openei.org/files/5997/C3E4N04224_20231002.zip
57 Challenge 3 Event 4 48 8316-bus Division 1 synthetic dataset scenarios each consisting of 3 files: scenario.json (input), pop_solution.json (simple feasible solution), and popsolution.log (solution log). The datasets were developed by UW-M, GT and NREL. See source files tab on "GO Competition Challenge 3 Event 4 Results Summary.xlsx" resource for details. https://data.openei.org/files/5997/C3E4N08316D1_20231002.zip
53 Event 1 spreadsheet with tabs data, summary, and Time Outs. The "data" tab is based on Arun's event_best_results_all_2023_02_09_09_50_39_231442037.csv collected from all Event 1 runs and contains 3373 rows (12 teams x 281 scenarios plus 1 tabulation row) sorted by team, model, scenario, and allow switching. Tab summary has rows for each scenario and columns for objective, feasibility, score and scaled score (fraction of best score) for each team. Below that are summaries used in the Event 1 Leaderboard. The Time Outs tab summarizes the completion status for each scenario for each team. Column R in the data tab gives the URL of a tar.gz file on rcdtn1.pnl.gov that contains the results from each run on PNNL's deception machine. It typically contains the feasiblity.log from running the Evaluation code, a jobs.log, messages.log, the solver log, the solution.json, and an evaluation summary in csv and json formats. This csv feeds into the master csv Arun generates along with the runtime information (times, exit codes, reruns, etc.). Industry datasets do not include the solver log or the solution file. https://data.openei.org/files/5997/E1LB_Master_20240529%20%281%29.xlsx
57 Challenge 3 Event 4 78 6049-bus synthetic dataset scenarios each consisting of 3 files: scenario.json (input), pop_solution.json (simple feasible solution), and popsolution.log (solution log). The zip file contains 36 datasets for Division 1, 18 for Division 2, and 24 for Division 3. The datasets were developed by UW-M, GT and NREL. See source files tab on "GO Competition Challenge 3 Event 4 Results Summary.xlsx" resource for details. https://data.openei.org/files/5997/C3E4N06049_20231002.zip
57 Challenge 3 Event 4 57 6717-bus synthetic dataset scenarios each consisting of 3 files: scenario.json (input), pop_solution.json (simple feasible solution), and popsolution.log (solution log). The zip file contains 36 datasets for Division 1, 18 for Division 2, and 3 for Division 3. The datasets were developed by TAMU and NREL. See source files tab on "GO Competition Challenge 3 Event 4 Results Summary.xlsx" resource for details. https://data.openei.org/files/5997/C3E4N06717_20231002.zip
57 Challenge 3 Event 4 44 8316-bus Division 3 synthetic dataset scenarios each consisting of 3 files: scenario.json (input), pop_solution.json (simple feasible solution), and popsolution.log (solution log). The datasets were developed by UW-M, GT and NREL. See source files tab on "GO Competition Challenge 3 Event 4 Results Summary.xlsx" resource for details. https://data.openei.org/files/5997/C3E4N08316D3_20231002.zip
57 The zip file contains 32 Event 2 Division 1 scenarios comprised of 6 network models consisting of 617-, 1576-, 2000-, 4224-, 6049-, and 6717-buses from the S1 and S2 datasets used in Challenge 3 Event 2, with variations on load profiles. The 2000- and 6717-bus scenarios are OBSOLETE, see C3E2.1_20230515 for the current versions. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3E2D1_20230510.zip
57 The zip file contains 49 Division 3 scenarios comprised of 7 network models consisting of 73-, 617-, 1576-, 2000-, 4224-, 6049-, and 6717-buses from the S1 and S2 datasets used in Challenge 3 Event 2, with variations on load profiles. The 2000- and 6717-bus scenarios are OBSOLETE. They do not meet the positivity requirement of the current Problem Formulation and therefore fail the current data checker. See Challenge 3 Event 2 Synthetic Dataset Scenarios Update.zip from C3E2.1_20230515.zip for the current versions. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3E2D3_20230510.zip
57 The zip file contains 67 Event 2 Division 2 scenarios comprised of 7 network models consisting of 73-, 617-, 1576-, 2000-, 4224-, 6049-, and 6717-buses from the S0, S1 and S2 datasets used in Challenge 3 Event 2, with variations on load profiles. The 2000- and 6717-bus scenarios are OBSOLETE. They do not meet the positivity requirement of the current Problem Formulation and therefore fail the current data checker. See Challenge 3 Event 2 Synthetic Dataset Scenarios Update.zip from C3E2.1_20230515.zip for the current versions. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3E2D2_20230510.zip
57 Challenge 3 Sandbox scenarios were made available before each Event for practice purposes. Sandbox 0 was originally released 12/18/2022 and contains mostly small problems not used in any events. The zip file contains 12 scenarios from 4 network models with 3-, 4-, 37-, and 73-buses. These are the updated S0.1 scenarios that comply with the positivity requirement of the current Problem Formulation, not the original S0 versions. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3S0.1_20230804.zip
57 Challenge 3 Sandbox scenarios were made available before each Event for practice purposes. Sandbox 1 was released 12/22/2022 before Event 1. The zip file contains 12 scenarios from 4 network models with 600-, 1576-, 4224-, and 6049-buses. These are the updated S1.1 scenarios that comply with the positivity requirement of the current Problem Formulation, not the original S1 versions. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3S1.1_20230807.zip
57 Challenge 3 Event 4 88 73-bus synthetic dataset scenarios each consisting of 3 files: scenario.json (input), pop_solution.json (simple feasible solution), and popsolution.log (solution log). The zip file contains data for all Divisions; 24 for Division 1, 40 for Division 2, and 24 for Division 3. The datasets were developed by NREL and PNNL. See source files tab on "GO Competition Challenge 3 Event 4 Results Summary.xlsx" resource for details. https://data.openei.org/files/5997/C3E4N00073_20231002.zip
57 Challenge 3 Event 4 39 2000-bus synthetic dataset scenarios each consisting of 3 files: scenario.json (input), pop_solution.json (simple feasible solution), and popsolution.log (solution log). The zip file contains 18 datasets for Division 1, 18 for Division 2, and 3 for Division 3. The datasets were developed by TAMU and NREL. See source files tab on "GO Competition Challenge 3 Event 4 Results Summary.xlsx" resource for details. https://data.openei.org/files/5997/C3E4N02000_20231002.zip
57 Challenge 3 Event 4 6 23,643-bus synthetic dataset scenarios each consisting of 3 files: scenario.json (input), pop_solution.json (simple feasible solution), and popsolution.log (solution log). The zip file contains 2 datasets for Division 1, 2 for Division 2, and 2 for Division 3. The datasets were developed by TAMU. See source files tab on "GO Competition Challenge 3 Event 4 Results Summary.xlsx" resource for details. https://data.openei.org/files/5997/C3E4N23643_20231002.zip
57 The zip file contains 257 scenarios comprised of 8 network models consisting of 3-, 14-, 37-, 73-, 600-, 1576-, 4200- and 6049-buses from the S0 and S1 datasets, with variations on load profiles. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3E1_20230214.zip
57 Challenge 3 Sandbox scenarios were made available before each Event for practice purposes. Sandbox 2 was released 3/16/2023 before Event 2 (April 13-14, 2023). The zip file contains 6 scenarios from 2 network models with 2000- and 6717-buses. Labeled S2.1 for consistency with the other Sandbox datasets, these are the original S2.0 scenarios that comply with the positivity requirement of the current Problem Formulation. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3S2.1_20230316.zip
33 In depth description of the GO Challenge 3 problem formulation, as well as data description, data format, solver requirements, and evaluation methods. https://data.openei.org/files/5997/Challenge3_Problem_Formulation_20240122.pdf
57 The zip file contains 28 Event 3 Division 3 scenarios from 7 network models with 73-, 617-, 1576-, 4224-, 6049-, 8316-, and 23643-buses. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3E3.1D3_20230629%20%281%29.zip
57 Challenge 3 Sandbox scenarios were made available before each Event for practice purposes. Sandbox 4 (shown on the website as S3.1-20230821) was released 8/21/2023 before Event 4 (August 31-September 4, 2023). The zip file contains 16 scenarios from 6 network models with 73-, 1576-, 2000-, 6049-, 6717-, and 8316-buses. These scenarios comply with the positivity requirement of the current Problem Formulation. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3S3.1_20230821.zip
21 Description of GO Competition Challenge 3, problem overview, and associated papers. The competing teams and prizes can be found here: https://gocompetition.energy.gov/challenges/Challenge-3/Leaderboards. https://gocompetition.energy.gov/challenges/challenge-3
57 Challenge 3 Event 4 87 617-bus synthetic dataset scenarios each consisting of 3 files: scenario.json (input), pop_solution.json (simple feasible solution), and popsolution.log (solution log). The zip file contains data for all Divisions; 39 for Division 1, 24 for Division 2, and 24 for Division 3. The datasets were developed by TAMU, NREL and PNNL. See source files tab on "GO Competition Challenge 3 Event 4 Results Summary.xlsx" resource for details. https://data.openei.org/files/5997/C3E4N00617_20231002.zip
21 Repository for model formulation of Grid Optimization Competition #3. Pydantic models for the GO Competition Challenge 3 json format. Support for loading, validation, editing, saving. Should often be used in conjunction with C3DataUtilities (https://github.com/GOCompetition/C3DataUtilities) Once a GO Competition Challenge 3 dataset is generated in the JSON format, the syntax and correctness are verified by the “checker” (check_data.py). The syntax and correctness of the JSON solution files is verified by the “evaluator” (evaluation.py), which also generates the solution score. Two Github repositories are used: the GO-3-data-model owned by Elaine Hale (NREL) was made publicly available September 26, 2022, and has been stable since Nov. 8, 2022; and the C3DataUtilities, owned by Steve Elbert (PNNL), was made public September 12, 2022, and has been stable since September 6, 2023. https://github.com/Smart-DS/GO-3-data-model
53 Spreadsheet with tabs data, data2, source files, file summary, score summary, rank summary, 73, 617, 1576, 2000, 4224, 6049, 6708, 6717, 8316, 23643 (tab number indicates the number of buses in dataset). The "data" tab is based on Arun's event_best_results_all_E4_2023_09_18_08_42.csv, collected from all Event 4 runs, and contains 10,039 rows (15 teams x 669 scenarios plus 4 tabulation rows) sorted by team, uuid, model, scenario and SW. Tab "data2" is similar but sorted by model, scenario, SW, and objective, which give the team ranking for each scenario. The spreadsheet includes data from a German team (Quasimodo) not mentioned elsewhere; there was a problem with their solver so they asked to withdraw. Column S in the data tab (T in data2) gives the URL of a tar.gz file on rcdtn1.pnl.gov that contains the results from each run on PNNL's deception machine. It typically contains the feasiblity.log from running the Evaluation code, a jobs.log, messages.log, the solver log, the solution.json, and a evaluation summary in csv and json formats. This csv feeds into the master csv Arun generates along with the runtime information (times, exit codes, reruns, etc.). Industry datasets do not include the solver log or the solution file. https://data.openei.org/files/5997/E4LB_Master_20240506.xlsx
53 Event 2 Spreadsheet with tabs data and summary. The "data" tab is based on Arun's event_best_results_all_2023_04_21_04_30_0505.csv collected from all Event 2 runs and contains 2562 rows (13 teams x 197 scenarios plus 1 tabulation row) sorted by team, model, scenario, and allow switching. Tab summary has rows for each scenario and columns for objective, feasibility, score and scaled score (fraction of best score) for each team. Below that are summaries used in the Event 2 Leaderboard. Column S in the data tab gives the URL of a tar.gz file on rcdtn1.pnl.gov that contains the results from each run on PNNL's deception machine. It typically contains the feasiblity.log from running the Evaluation code, a jobs.log, messages.log, the solver log, the solution.json, and a evaluation summary in csv and json formats. This csv feeds into the master csv Arun generates along with the runtime information (times, exit codes, reruns, etc.). Industry datasets do not include the solver log or the solution file. https://data.openei.org/files/5997/E2LB_Master_20240529.xlsx
57 Challenge 3 Sandbox scenarios were made available before each Event for practice purposes. Sandbox 3 was released 6/6/2023 before Event 3 (June 15-16, 2023). The zip file contains 41 scenarios from 6 network models with 14-, 37-, 1576-, 2000-, 8316-, and 23643-buses. These scenarios comply with the positivity requirement of the current Problem Formulation. On June 7, 2023, it was discovered that the following contain invalid data and should not be used: S3.1/D*/C3S3N00037D*/scenario_001.json; S3.1/D1/C3S3N01576D1/scenario_007.json; and S3.1/D*/C3S3N23643D*/scenario_001.json. Each scenario consists of 3 files: scenario.json (input), pop_solution.json (simple Prior Operating Point feasible solution), and popsolution.log (solution log). https://data.openei.org/files/5997/C3S3.1_20230606.zip
33 Description of GO Competition Challenge 3 Sandbox access and use. The Sandbox was the mechanism competitors use to test their solvers on the PNNL Evaluation Platform using both synthetic and industry datasets. The document includes submission.conf syntax and usage statistics for all 4 Challenge 3 Events and post Event 4. https://data.openei.org/files/5997/The%20GO%20Competition%20Sandbox.pdf

Tags

  • arpa-e
  • optimization
  • multiperiod
  • multiperiod-dynamic-markets
  • grid
  • unit-commitment
  • grid-optimization
  • go-competition
  • security-constrained-optimal-power-flow
  • computational-science
  • power
  • model
  • acopf
  • competition
  • challenge-3
  • energy-model
  • energy

Topics

Categories