Benchmark a set of planners on a problem instance. More...

#include <ompl/tools/benchmark/Benchmark.h>

Classes

struct  CompleteExperiment
 This structure holds experimental data for a set of planners. More...
 
struct  PlannerExperiment
 The data collected after running a planner multiple times. More...
 
struct  Request
 Representation of a benchmark request. More...
 
struct  Status
 This structure contains information about the activity of a benchmark instance. If the instance is running, it is possible to find out information such as which planner is currently being tested or how much. More...
 

Public Types

using RunProperties = std::map< std::string, std::string >
 The data collected from a run of a planner is stored as key-value pairs.
 
using RunProgressData = std::vector< std::map< std::string, std::string > >
 
using PreSetupEvent = std::function< void(const base::PlannerPtr &)>
 Signature of function that can be called before a planner execution is started.
 
using PostSetupEvent = std::function< void(const base::PlannerPtr &, RunProperties &)>
 Signature of function that can be called after a planner execution is completed.
 

Public Member Functions

 Benchmark (geometric::SimpleSetup &setup, const std::string &name=std::string())
 Constructor needs the SimpleSetup instance needed for planning. Optionally, the experiment name (name) can be specified.
 
 Benchmark (control::SimpleSetup &setup, const std::string &name=std::string())
 Constructor needs the SimpleSetup instance needed for planning. Optionally, the experiment name (name) can be specified.
 
void addExperimentParameter (const std::string &name, const std::string &type, const std::string &value)
 Add an optional parameter's information to the benchmark output. Useful for aggregating results over different benchmark instances, e.g., parameter sweep. type is typically "BOOLEAN", "INTEGER", or "REAL".
 
const std::map< std::string, std::string > & getExperimentParameters () const
 Get all optional benchmark parameters. The map key is 'name type'

 
std::size_t numExperimentParameters () const
 Return the number of optional benchmark parameters.
 
void setExperimentName (const std::string &name)
 Set the name of the experiment.
 
const std::string & getExperimentName () const
 Get the name of the experiment.
 
void addPlanner (const base::PlannerPtr &planner)
 Add a planner to use.
 
void addPlannerAllocator (const base::PlannerAllocator &pa)
 Add a planner allocator to use.
 
void clearPlanners ()
 Clear the set of planners to be benchmarked.
 
void setPlannerSwitchEvent (const PreSetupEvent &event)
 Set the event to be called before any runs of a particular planner (when the planner is switched)
 
void setPreRunEvent (const PreSetupEvent &event)
 Set the event to be called before the run of a planner.
 
void setPostRunEvent (const PostSetupEvent &event)
 Set the event to be called after the run of a planner.
 
virtual void benchmark (const Request &req)
 Benchmark the added planners on the defined problem. Repeated calls clear previously gathered data. More...
 
const StatusgetStatus () const
 Get the status of the benchmarking code. This function can be called in a separate thread to check how much progress has been made.
 
const CompleteExperimentgetRecordedExperimentData () const
 Return all the experiment data that would be written to the results file. The data should not be changed, but it could be useful to quickly extract cartain statistics.
 
virtual bool saveResultsToStream (std::ostream &out=std::cout) const
 Save the results of the benchmark to a stream.
 
bool saveResultsToFile (const char *filename) const
 Save the results of the benchmark to a file.
 
bool saveResultsToFile () const
 Save the results of the benchmark to a file. The name of the file is the current date and time.
 

Protected Attributes

geometric::SimpleSetupgsetup_
 The instance of the problem to benchmark (if geometric planning)
 
control::SimpleSetupcsetup_
 The instance of the problem to benchmark (if planning with controls)
 
std::vector< base::PlannerPtr > planners_
 The set of planners to be tested.
 
CompleteExperiment exp_
 The collected experimental data (for all planners)
 
Status status_
 The current status of this benchmarking instance.
 
PreSetupEvent plannerSwitch_
 Event to be called when the evaluated planner is switched.
 
PreSetupEvent preRun_
 Event to be called before the run of a planner.
 
PostSetupEvent postRun_
 Event to be called after the run of a planner.
 

Detailed Description

Benchmark a set of planners on a problem instance.

Definition at line 112 of file Benchmark.h.

Member Function Documentation

◆ benchmark()

void ompl::tools::Benchmark::benchmark ( const Request req)
virtual

Benchmark the added planners on the defined problem. Repeated calls clear previously gathered data.

Parameters
reqThe parameters for the execution of the benchmark
Note
The values returned for memory consumption may be misleading. Memory allocators often free memory in a lazy fashion, so the returned values for memory consumption indicate the increase in memory usage for each run. Since not all the memory for the previous run was freed, the increase in usage may be close to 0. To get correct averages for memory usage, use req.runCount = 1 and run the process multiple times.

Definition at line 353 of file Benchmark.cpp.


The documentation for this class was generated from the following files: