Evaluation and Optimization API
For an introduction, refer to
and the accompanying: ADORE Evaluation Docs
Evaluator
adore.optimization.api.evaluator.GraphApiEvaluator
GraphApiEvaluator(translator: DesignProblemTranslator, save_to_project=False, **_)
Evaluator defining an architecture optimization problem and the implementation for evaluating a generated
architecture. Implement the _evaluate function.
Normally, it is best to instantiate an evaluator using the from_file method.
Results are automatically cached if save_to_project=True. For equal design vectors, analysis is then not
repeated.
Typical usage:
from typing import Dict
from adore.optimization.api import GraphApiEvaluator
from adore.api.schema import * # Import last to avoid conflicts
class MyArchitectureEvaluator(GraphApiEvaluator):
def _evaluate(self, architecture: Architecture,
arch_qois: List[ArchQOI], **_) -> Dict[ArchQOI, Union[float, Quantity]]:
"""
Implement evaluation code here, taking care of:
- Interpreting the architecture and converting it to your design problem
- Running the evaluation/simulation
- Extracting output (the requested output is given as a list of ArchQOI's)
"""
if __name__ == '__main__':
# Load the *.adore file
evaluator = MyArchitectureEvaluator.from_file('path/to/model.adore', save_to_project=True)
# Optionally define which design problem you want to use
evaluator.use_design_problem(evaluator.project.design_problems[0])
# Connect to SBArchOpt for optimization
from sb_arch_opt.algo.pymoo_interface import get_nsga2
from pymoo.optimize import minimize
problem = evaluator.get_arch_opt_problem()
algorithm = get_nsga2(pop_size=50) # Normally, you'd also add a `results_folder` here!
result = minimize(problem, algorithm, termination=('n_gen', 10), verbose=True)
# Save ADORE evaluator results
evaluator.to_file('results.adore')
evaluator.save_results_csv('results.csv')
all_design_vars
all_design_vars: List[DesignVar]
Get the list of adore.api.schema.adore_schema.DesignVar (including fixed) of the design
problem.
constraints
constraints: List[Constraint]
Get the list of adore.api.schema.adore_schema.Constraint of the design problem.
design_problem
design_problem: DesignProblem
Get the loaded adore.api.schema.adore_schema.DesignProblem.
design_vars
design_vars: List[DesignVar]
Get the list of active (non-fixed) adore.api.schema.adore_schema.DesignVar of the design
problem.
helper
helper: SchemaHelper
Get the adore.api.helper.SchemaHelper for search for elements in an ADORE project.
objectives
objectives: List[Objective]
Get the list of adore.api.schema.adore_schema.Objective of the design problem.
_evaluate
_evaluate(architecture: Architecture, arch_qois: List[ArchQOI], **_) -> Dict[ArchQOI, Union[float, Quantity]]
Implement this function to enable evaluating architectures for your design problem. The architecture should
be converted to whatever format your design problem needs for analysis. arch_qois contains a list of QOIs
(Quantities of Interest) for which output is requested. A dictionary mapping ArchQOI to their values should
be returned.
Values can either be provided as float values or as Quantities, which are values associated with units.
To create a Quantity object, use the .quantity(value, 'unit') function.
correct_design_vector
correct_design_vector(design_vector: DesignVector) -> Tuple[DesignVector, IsActiveVector]
Imputes a design vector and returns which design variables are active, without actually creating an architecture and adding it to the list
evaluate
evaluate(architecture: Architecture, **kwargs) -> MetricsData
Evaluate an architecture, caching results.
evaluate_get_input
evaluate_get_input(architecture: Architecture, **kwargs)
Implements the ask-side of architecture evaluation: returns input created for a given architecture.
evaluate_process_output
evaluate_process_output(output_data, dv: DesignVector = None, architecture: Architecture = None, **kwargs) -> Tuple[List[float], List[float], Architecture]
Implements the tell-side of architecture evaluation: processes the output data to update the architecture and return objective and constraint values.
fix_design_var
fix_design_var(design_var: DesignVar, value: Union[int, float])
Fix a design variable with a specific value. Provide an int for a discrete design variable, a float for a
continuous design variable.
from_db
from_db(database: Database, save_to_project=True, sync_translator=True, design_problem_id=None, **kwargs) -> T
Instantiate the evaluator from a Database.
A specific design problem to be used can be selected.
Enable save_to_project to automatically save generated architectures and results back to the ADORE model.
to_file can be used to save the results back to an *.adore file.
from_design_problem
from_design_problem(project: Project, schema_factory: SchemaFactory, design_problem: DesignProblem, save_to_project=True, sync_translator=True, **kwargs) -> T
Instantiate the evaluator from a DesignProblem.
Enable save_to_project to automatically save generated architectures and results back to the ADORE model.
to_file can be used to save the results back to an *.adore file.
from_file
from_file(path: str, save_to_project=True, use_existing_design_problem=True, sync_translator=True, **kwargs)
Instantiate the evaluator from an *.adore file. If use_existing_design_problem is true and a design problem
has already been defined, this design problem is used.
Enable save_to_project to automatically save generated architectures and results back to the ADORE model.
to_file can be used to save the results back to an *.adore file.
from_project
from_project(project: Project, schema_factory: SchemaFactory, save_to_project=True, use_existing_design_problem=True, sync_translator=True, **kwargs) -> T
Instantiate the evaluator from a Project. If use_existing_design_problem is true and a design problem
has already been defined, this design problem is used.
Enable save_to_project to automatically save generated architectures and results back to the ADORE model.
to_file can be used to save the results back to an *.adore file.
get_arch_opt_problem
Helper function to get an SBArchOpt optimization problem.
get_architecture
get_architecture(design_vector: DesignVector) -> ArchData
Generate the architecture corresponding to a design vector.
get_design_vector
get_design_vector(architecture: Architecture) -> Tuple[Optional[DesignVector], Optional[IsActiveVector]]
Gets the design vector used to generate the architecture (if not created manually)
get_fixed_value
get_fixed_value(design_var: DesignVar) -> Optional[Union[int, float]]
Get the value of the fixed design variable.
get_qoi_value
Get the QOI value as Quantity (value + units)
get_results
Return a pandas DataFrame with analysis results (design variables, QOIs, output) of the design problem.
get_results_csv
Get a string formatted as CSV of the results.
has_architecture
Return whether the provided output is associated to an evaluated architecture.
has_in_thread_execution
Whether the actual resource-heavy evaluation is performed in the same thread as the evaluator or not. If True, it is assumed that parallelization should be achieved using a process pool, otherwise only a thread pool is used.
is_discrete_categorical
is_discrete_categorical(design_var: DiscreteDesignVar) -> bool
Returns whether the discrete design variable is a categorical or integer design var. Integer variables have a notion of order and distance (e.g. 1, 2, 3, 4 engines), whereas categorical variables don't have this (e.g. T-tail, V-tail, normal tail).
launch_in_gui
Launch the GUI with this project loaded in memory. Note that any edits done are then not synchronized with the original project anymore.
qoi_to_units
Get a copy of a QOI with values converted to other units.
quantity
Create a Quantity from a numerical value and units.
save_results_csv
Write results to a file in CSV format.
show_project
show_project(save_path: str = None, height: int = None, show_architecture_id: int = None, externalize_js: Union[bool, str] = False)
Generate and view the HTML export showing the project. Also supports Jupyter notebooks.
externalize_js=True places the frontend code in a separate file in the same directory as the notebook,
thereby greatly reducing the size of the notebook itself.
However, this only works when the notebook is exported to HTML and then served. Within the interactive notebook
itself it does not work!
use_design_problem
use_design_problem(design_problem: DesignProblem)
Select another adore.api.schema.adore_schema.DesignProblem to use.
adore.optimization.api.evaluator.ArchData
ArchData = Tuple[Architecture, DesignVector, IsActiveVector]
adore.graph.api.design_problem.DesignProblemTranslator
DesignProblemTranslator(project: Project, design_problem: DesignProblem, schema_factory: SchemaFactory, adore_graph: AdoreGraph, encoding_timeout: float = None, sync=True)
Class that provides translation services between the design space and design problems, using the optimization module.
correct_design_vector
correct_design_vector(design_vector: DesignVector) -> Tuple[DesignVector, IsActiveVector]
Imputes a design vector and returns which design variables are active, without actually creating an architecture and adding it to the list
design_problem_conforms
design_problem_conforms(project: Project, design_problem: DesignProblem, helper: SchemaHelper = None) -> bool
Check whether a design problem conforms to the design space model without regenerating the design problem.
from_new
from_new(project: Project, schema_factory: SchemaFactory, adore_graph: AdoreGraph, name: str = None, sync=True, append=True) -> DesignProblemTranslator
Create a DesignProblemTranslator for a new DesignProblem.
get_architecture
get_architecture(design_vector: DesignVector, append=False) -> Tuple[Architecture, DesignVector, IsActiveVector]
Get the architecture corresponding to the given design vector, either existing or a newly created.
get_pareto_architectures
get_pareto_architectures() -> List[Architecture]
Get architectures in the Pareto front (for the current design problem)
get_qoi_value
Get the QOI value as Quantity (value + units)
get_results
Returns a DataFrame with the inputs (design vector) and outputs for all architectures in the project.
qoi_to_units
Get a copy of a QOI with values converted to other units.
quantity
Create a Quantity from a numerical value and units.
write_evaluation_results
write_evaluation_results(architecture: Architecture, metric_qoi_values: Dict[ArchQOI, Union[float, Quantity]], project: Project, helper: SchemaHelper = None)
Write evaluation results to ArchQOIs, related objectives/constraints, and set the architecture evaluation/feasible flags.
Ensure same logic as src/architectures.ts::setArchQoiMetricValue
Optimization
adore.optimization.bridge.problem.AdoreArchOptProblem
AdoreArchOptProblem(evaluator: GraphApiEvaluator, n_parallel=None, eval_kwargs=None)
SBArchOpt wrapper for an ADORE API problem. Note that under the hood,
SBArchOpt uses pymoo.
The connection is made between SBArchOpt's ArchOptProblemBase class and the ADORE GraphApiEvaluator class,
which contains all information for running an ADORE architecture optimization problem.
Parallel processing is possible by setting n_parallel to a number higher than 1. Note that evaluations will be
done using multithreading to correctly preserve object references.
Example usage:
from pymoo.optimize import minimize
from sb_arch_opt.algo.pymoo_interface import get_nsga2
evaluator = ... # Instantiate the evaluator
algorithm = get_nsga2(pop_size=100)
problem = evaluator.get_arch_opt_problem()
# OR equivalently:
# problem = AdoreArchOptProblem(evaluator)
result = minimize(problem, algorithm, termination=('n_eval', 500))