# Hyper-Decision Transformer for Efficient Online Policy Adaptation

**Authors:** Mengdi Xu, Yuchen Lu, Yikang Shen, Shun Zhang, Ding Zhao, Chuang Gan  
**arXiv:** [2304.08487](https://arxiv.org/abs/2304.08487)  
**Date:** 2023-04-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.08487v1/

## Key points

- The Hyper-Decision Transformer (HDT) is a new framework proposed to improve the generalization capability of large transformer-based agents with limited online interactions and computation budgets.
- Existing large transformer-based agents may have sub-optimal performances when directly deployed in novel tasks.
- HDT can generalize to novel unseen tasks from a handful of demonstrations in a data- and parameter-efficient manner.
- The base Decision Transformer (DT) is augmented with an adaptation module, whose parameters are initialized by a hyper-network.
- HDT adapts faster to unseen tasks than fine-tuning the whole DT model with only 0.5% of DT parameters fine-tuned after a single expert demonstration.
- HDT outperforms state-of-the-art baselines in terms of task success rates by a large margin even when expert actions are not available.
- Efficient adaptations of large transformer-based agents can be achieved through meta RL techniques such as few-shot learning or transfer learning.
- The Hyper Decision Transformer framework provides an effective solution for adapting quickly to novel unseen tasks while maintaining data and parameter efficiency.

## Summary

The Hyper-Decision Transformer (HDT) is a new framework proposed to improve the generalization capability of large transformer-based agents with limited online interactions and computation budgets. While existing large transformer-based agents have demonstrated strong performances in training tasks, they may have sub-optimal performances when directly deployed in novel tasks. To address this challenge, HDT can generalize to novel unseen tasks from a handful of demonstrations in a data- and parameter-efficient manner. The base Decision Transformer (DT) is augmented with an adaptation module, whose parameters are initialized by a hyper-network. When encountering unseen tasks, the hyper-network takes a handful of demonstrations as inputs and initializes the adaptation module accordingly. This initialization enables HDT to efficiently adapt to novel tasks by only fine-tuning the adaptation module. The authors validate HDT's generalization capability on object manipulation tasks and find that with a single expert demonstration and fine-tuning only 0.5% of DT parameters, HDT adapts faster to unseen tasks than fine-tuning the whole DT model. Furthermore, they explore a more challenging setting where expert actions are not available, and show that HDT outperforms state-of-the-art baselines in terms of task success rates by a large margin. The problem formulation for efficient adaptation from observations is discussed in detail, highlighting how existing large transformer based agents may have suboptimal performance when directly deployed in novel tasks due to discrepancies between training and testing tasks. The authors propose that efficient adaptations of large transformer based agents can be achieved through meta RL techniques such as few shot learning or transfer learning. Overall, the Hyper Decision Transformer framework provides an effective solution for adapting quickly to novel unseen tasks while maintaining data and parameter efficiency. It has promising applications in various domains where rapid adaptation is crucial for successful task completion.