Yann LeCun states that inference-time optimization underpins Energy-Based Models and Objective-Driven AI (ODAI). He asserts that for continuous latent or action variables, gradient-based optimization is the appropriate inference method, and points to world model–based systems as a practical ODAI example using gradient-based planning.
Using optimization at inference time is a foundational concept of Energy-Based Models (EBM) and Objective-Driven AI architectures (ODAI).
Using optimization at inference time is a foundational concept of Energy-Based Models (EBM) and Objective-Driven AI architectures (ODAI).
When the variables to be inferred are continuous, it makes sense to use gradient-based optimization.
A good instance of ODAI is world model-based systems that use gradient-based optimization for planning.