LayeredRL Logo

Getting Started

  • Overview
    • Core Concepts
      • Levels
      • The Hierarchy
    • Getting Started
    • Key Takeaways
  • Installation
    • Basic Installation
    • Development Installation
    • Verification
  • Quickstart
    • Basic Hierarchy
    • Logging with Tensorboard and Weights & Biases
    • Testing periodically during training
    • Saving and loading hierarchies
    • Next Steps

User Guide

  • Hierarchies
    • Creating a Hierarchy
      • Basic Structure
      • Observation Maps
      • Data Collection and Training
    • API Reference
  • Levels
    • Available Level Types
      • TD-MPC2 Level
      • Tianshou Level
      • DADS Level
      • SPlaTES Level
      • Planner Level
    • Custom Levels
    • API Reference
  • Models and Predictors
    • Models
      • Probabilistic Ensemble
    • Predictors
      • Static Predictor
      • Reward Predictor
    • Encoders
      • Identity Encoder
      • Fixed Encoder
      • Learned Encoder
    • Predictor training
    • API Reference

Examples

  • Examples
    • Flat TD-MPC2 Hierarchy
    • SPlaTES + Planning
    • DADS + Planning
    • Model-Based Planning
    • Ensemble Rollout
    • Blackbox Planning
    • Running the Examples
    • LayeredRL on More Challenging Environments
    • Tests
    • Next Steps

API Reference

  • Hierarchies API
    • Hierarchy
      • Hierarchy.__init__()
      • Hierarchy.eval()
      • Hierarchy.get_action()
      • Hierarchy.get_copy()
      • Hierarchy.learn()
      • Hierarchy.load()
      • Hierarchy.load_buffers()
      • Hierarchy.process_transition()
      • Hierarchy.reset()
      • Hierarchy.save()
      • Hierarchy.save_buffers()
      • Hierarchy.set_n_env_instances()
      • Hierarchy.soft_reset()
      • Hierarchy.train()
    • RandomHierarchy
      • RandomHierarchy.__init__()
    • FlatTianshouHierarchy
      • FlatTianshouHierarchy.__init__()
  • Levels API
    • Level
      • Level.__init__()
      • Level.add_transitions()
      • Level.eval()
      • Level.get_action()
      • Level.get_action_shape_and_type()
      • Level.get_copy()
      • Level.get_input_space()
      • Level.get_level_state()
      • Level.get_level_state_dims()
      • Level.get_reward()
      • Level.initialize()
      • Level.learn()
      • Level.load()
      • Level.load_buffers()
      • Level.process_transition()
      • Level.register_env_reward()
      • Level.reset()
      • Level.sample_from_action_space()
      • Level.save()
      • Level.save_buffers()
      • Level.set_n_env_instances()
      • Level.soft_reset()
      • Level.train()
    • RandomLevel
      • RandomLevel.get_action()
      • RandomLevel.get_input_space()
    • TianshouLevel
      • TianshouLevel.__init__()
      • TianshouLevel.eval()
      • TianshouLevel.get_action()
      • TianshouLevel.get_input_space()
      • TianshouLevel.initialize()
      • TianshouLevel.learn()
      • TianshouLevel.load()
      • TianshouLevel.load_buffers()
      • TianshouLevel.prepare_config()
      • TianshouLevel.process_transition()
      • TianshouLevel.save()
      • TianshouLevel.save_buffers()
      • TianshouLevel.set_n_env_instances()
      • TianshouLevel.train()
    • DADSLevel
      • DADSLevel.__init__()
      • DADSLevel.get_action()
      • DADSLevel.get_copy()
      • DADSLevel.get_fixed_std()
      • DADSLevel.get_input_space()
      • DADSLevel.get_reward()
      • DADSLevel.get_tdmpc2_obs_shape()
      • DADSLevel.initialize()
      • DADSLevel.process_transition()
      • DADSLevel.reset()
      • DADSLevel.sample_from_skill_prior()
      • DADSLevel.set_n_env_instances()
      • DADSLevel.soft_reset()
    • SPlaTESLevel
      • SPlaTESLevel.__init__()
      • SPlaTESLevel.get_action()
      • SPlaTESLevel.get_copy()
      • SPlaTESLevel.get_fixed_std()
      • SPlaTESLevel.get_input_space()
      • SPlaTESLevel.get_level_state()
      • SPlaTESLevel.get_level_state_dims()
      • SPlaTESLevel.get_reward()
      • SPlaTESLevel.get_tdmpc2_obs_shape()
      • SPlaTESLevel.initialize()
      • SPlaTESLevel.learn()
      • SPlaTESLevel.process_transition()
      • SPlaTESLevel.reset()
      • SPlaTESLevel.sample_from_skill_prior()
      • SPlaTESLevel.set_n_env_instances()
      • SPlaTESLevel.soft_reset()
    • PlannerLevel
      • PlannerLevel.__init__()
      • PlannerLevel.eval()
      • PlannerLevel.get_action()
      • PlannerLevel.get_copy()
      • PlannerLevel.get_input_space()
      • PlannerLevel.get_reward()
      • PlannerLevel.initialize()
      • PlannerLevel.learn()
      • PlannerLevel.load()
      • PlannerLevel.load_buffers()
      • PlannerLevel.process_transition()
      • PlannerLevel.reset()
      • PlannerLevel.save()
      • PlannerLevel.save_buffers()
      • PlannerLevel.set_n_env_instances()
      • PlannerLevel.soft_reset()
      • PlannerLevel.train()
    • ConstantLevel
      • ConstantLevel.__init__()
      • ConstantLevel.get_action()
      • ConstantLevel.get_input_space()
    • TDMPC2Level
      • TDMPC2Level.__init__()
      • TDMPC2Level.get_action()
      • TDMPC2Level.get_input_space()
      • TDMPC2Level.get_tdmpc2_obs_shape()
      • TDMPC2Level.initialize()
      • TDMPC2Level.learn()
      • TDMPC2Level.load()
      • TDMPC2Level.load_buffers()
      • TDMPC2Level.process_transition()
      • TDMPC2Level.reset()
      • TDMPC2Level.save()
      • TDMPC2Level.save_buffers()
      • TDMPC2Level.set_n_env_instances()
      • TDMPC2Level.soft_reset()
    • ActionSequenceLevel
      • ActionSequenceLevel.__init__()
      • ActionSequenceLevel.get_action()
      • ActionSequenceLevel.get_input_space()
      • ActionSequenceLevel.process_transition()
      • ActionSequenceLevel.reset()
  • Planners API
    • Planner
      • Planner.__init__()
      • Planner.get_aux_rewards()
      • Planner.plan()
      • Planner.reset()
      • Planner.set_predictor()
      • Planner.shift_initialization()
    • BlackboxPlanner
      • BlackboxPlanner.__init__()
      • BlackboxPlanner.cost()
      • BlackboxPlanner.plan()
      • BlackboxPlanner.reset()
    • CEMPlanner
      • CEMPlanner.__init__()
      • CEMPlanner.plan()
      • CEMPlanner.shift_initialization()
  • Policies API
    • Policy
      • Policy.__init__()
      • Policy.get_action()
      • Policy.get_log_prob()
      • Policy.reset()
      • Policy.transform_action()
      • Policy.untransform_action()
    • UniformPolicy
      • UniformPolicy.reset()
    • TianshouPolicy
      • TianshouPolicy.__init__()
      • TianshouPolicy.get_log_prob()
      • TianshouPolicy.get_value()
      • TianshouPolicy.reset()
      • TianshouPolicy.update()
  • Models API
    • Model
      • Model.__init__()
      • Model.get_log_prob()
      • Model.get_parameters()
      • Model.get_prob()
      • Model.learn()
      • Model.loss()
      • Model.predict()
      • Model.rollout()
      • Model.sample()
      • Model.set_n_total_env_steps()
    • ProbabilisticEnsemble
      • ProbabilisticEnsemble.__init__()
      • ProbabilisticEnsemble.get_log_prob()
      • ProbabilisticEnsemble.get_member_log_prob()
      • ProbabilisticEnsemble.get_parameters()
      • ProbabilisticEnsemble.learn()
      • ProbabilisticEnsemble.loss()
      • ProbabilisticEnsemble.predict()
      • ProbabilisticEnsemble.rollout()
      • ProbabilisticEnsemble.sample()
  • Predictors API
    • Predictor
      • Predictor.__init__()
      • Predictor.encode()
      • Predictor.learn()
      • Predictor.loss()
      • Predictor.predict()
      • Predictor.predict_reward()
      • Predictor.rollout()
      • Predictor.sample_partial_trajectories()
      • Predictor.value()
    • PredictorFactory
      • PredictorFactory.__call__()
      • PredictorFactory.__init__()
    • RewardPredictor
      • RewardPredictor.__init__()
      • RewardPredictor.compute_targets()
      • RewardPredictor.get_standardized_log_prob()
      • RewardPredictor.learn()
      • RewardPredictor.loss()
      • RewardPredictor.sample_model_batch()
      • RewardPredictor.soft_update()
      • RewardPredictor.update_target_nets()
    • StaticPredictor
      • StaticPredictor.learn()
      • StaticPredictor.loss()
    • get_default_predictor_factory()
  • Neural Networks API
    • ConcatNet
      • ConcatNet.__init__()
      • ConcatNet.forward()
    • Critic
      • Critic.__init__()
      • Critic.forward()
    • ProbFCDynamics
      • ProbFCDynamics.__init__()
      • ProbFCDynamics.forward()
    • RandomDynamics
      • RandomDynamics.__init__()
      • RandomDynamics.forward()
    • FixedEncoderNet
      • FixedEncoderNet.__init__()
      • FixedEncoderNet.context_dim
      • FixedEncoderNet.decode()
      • FixedEncoderNet.forward()
      • FixedEncoderNet.latent_state_dim
    • Encoder
      • Encoder.__init__()
      • Encoder.context_dim
      • Encoder.latent_state_dim
    • EncoderNet
      • EncoderNet.__init__()
      • EncoderNet.context_dim
      • EncoderNet.forward()
      • EncoderNet.latent_state_dim
    • ValueNet
      • ValueNet.__init__()
      • ValueNet.forward()
    • RewardNet
      • RewardNet.__init__()
      • RewardNet.forward()
    • IdentityEncoder
      • IdentityEncoder.__init__()
      • IdentityEncoder.context_dim
      • IdentityEncoder.decode()
      • IdentityEncoder.forward()
      • IdentityEncoder.latent_state_dim
  • Collectors API
    • Collector
      • Collector.__init__()
      • Collector.collect()
      • Collector.reset()
      • Collector.save_checkpoint()
      • Collector.test()
  • Environments and Wrappers API
    • LogRewWrapper
      • LogRewWrapper.__init__()
      • LogRewWrapper.reward()
    • AntFlippedWrapper
      • AntFlippedWrapper.reset()
      • AntFlippedWrapper.step()
    • AntNoWallFlippedWrapper
      • AntNoWallFlippedWrapper.reset()
      • AntNoWallFlippedWrapper.step()
    • AffineRewWrapper
      • AffineRewWrapper.__init__()
      • AffineRewWrapper.reward()
    • Maze2DEnv
      • Maze2DEnv.__init__()
      • Maze2DEnv.close()
      • Maze2DEnv.metadata
      • Maze2DEnv.render()
      • Maze2DEnv.reset()
      • Maze2DEnv.set_plans()
      • Maze2DEnv.step()
    • create_simple_maze()
    • create_corridor_maze()
    • create_medium_maze()
  • Optimizers API
    • Optimizer
      • Optimizer.__init__()
      • Optimizer.optimize()
      • Optimizer.reset()
    • CEM
      • CEM.__init__()
      • CEM.optimize()
      • CEM.reset()
    • ICEM
      • ICEM.__init__()
  • Utilities API
    • ToDeviceReplayBuffer
      • ToDeviceReplayBuffer.__getitem__()
      • ToDeviceReplayBuffer.__init__()
      • ToDeviceReplayBuffer.sample()
    • cdf_normal()
    • sample_truncated_normal()
    • get_normal_prob()
    • get_normal_log_prob()
    • get_writer()
    • VideoLogger
      • VideoLogger.__init__()
      • VideoLogger.log_frame()
      • VideoLogger.save_video()
    • RangesMap
      • RangesMap.__call__()
      • RangesMap.__init__()
    • ConcatMap
      • ConcatMap.__call__()
      • ConcatMap.__init__()
    • to_torch()
    • to_numpy()
    • copy_torch_or_numpy()
    • temp_eval_mode()
    • get_key_indices()
    • get_obs_indices()
    • RunningBatchNorm
      • RunningBatchNorm.__init__()
      • RunningBatchNorm.forward()
    • Standardizer
      • Standardizer.__init__()
      • Standardizer.forward()
      • Standardizer.get_symm_mat()
    • Schedule
    • RampSchedule
      • RampSchedule.__init__()
    • ZeroThenLinearSchedule
      • ZeroThenLinearSchedule.__init__()
    • PiecewiseLinearSchedule
      • PiecewiseLinearSchedule.__init__()
    • PiecewiseLogLinearSchedule
      • PiecewiseLogLinearSchedule.__init__()
    • ConstantSchedule
      • ConstantSchedule.__init__()

Additional Information

  • Citation
    • BibTeX
    • Related Publications
      • SPlaTES
      • TD-MPC2
      • DADS
  • Contributing
    • Development Setup
    • Running Tests
    • Code Style
      • Pull Requests
    • Adding a New Level Type
    • Documentation
      • Building Documentation Locally
    • Code of Conduct
LayeredRL
  • Python Module Index

Python Module Index

l
 
l
- layeredrl
    layeredrl.collectors
    layeredrl.envs
    layeredrl.hierarchies
    layeredrl.levels
    layeredrl.models
    layeredrl.nets
    layeredrl.optimizers
    layeredrl.planners
    layeredrl.policies
    layeredrl.predictors
    layeredrl.utils

© Copyright 2025, LayeredRL Contributors.

Built with Sphinx using a theme provided by Read the Docs.