# AirObject: A Temporally Evolving Graph Embedding for Object Identification

**Authors:** Nikhil Varma Keetha, Chen Wang, Yuheng Qiu, Kuan Xu, Sebastian Scherer  
**arXiv:** [2111.15150](https://arxiv.org/abs/2111.15150)  
**Date:** 2021-11-30  
**Summary page:** https://www.summarizepaper.com/en/paper/2111.15150v1/

## Key points

- Object encoding and identification are crucial for various robotic tasks
- Most existing approaches are limited to a "fixed" partial object representation from a single viewpoint
- AirObject proposes a novel temporal 3D object encoding approach that generates global keypoint graph-based embeddings of objects
- AirObject uses a temporal convolutional network across structural information obtained from a graph attention-based encoding method to generate global 3D object embeddings
- This approach enables the creation of temporally "evolving" global object representations built as the robot observes the object from multiple viewpoints
- AirObject achieves state-of-the-art performance for video object identification and is robust to severe occlusion, perceptual aliasing, viewpoint shift, deformation, and scale transform
- It outperforms existing single-frame and sequential descriptors while also being class-agnostic
- AirObject is one of the first temporal object encoding methods that aggregate structural knowledge across multiple instances
- Several spatio-temporal representation techniques exist in related fields of research that use LSTM, GRUs, Graph Convolutional Networks (GCNs), and Temporal Convolutional Networks to model temporal relations
- In terms of visual place recognition (VPR), some approaches leverage spatio-temporal information in terms of landmarks or bio-inspired memory cells
- However, these approaches do not consider explicit geometry available from interest points like SuperPoint does
- Overall, AirObject provides an innovative solution for generating temporally evolving global object representations that can be used for various robotic tasks such as semantic scene understanding and re-localization.

## Summary

Object encoding and identification are crucial for various robotic tasks, including autonomous exploration, semantic scene understanding, and re-localization. The recent success of convolutional neural networks (CNN) in computer vision has led to the development of deep learned features-based image retrieval methods that have shown significant improvements over handcrafted features. However, most existing approaches are limited to a "fixed" partial object representation from a single viewpoint. To address this limitation, researchers propose a novel temporal 3D object encoding approach called AirObject that generates global keypoint graph-based embeddings of objects. AirObject uses a temporal convolutional network across structural information obtained from a graph attention-based encoding method to generate global 3D object embeddings. This approach enables the creation of temporally "evolving" global object representations built as the robot observes the object from multiple viewpoints. The proposed framework achieves state-of-the-art performance for video object identification and is robust to severe occlusion, perceptual aliasing, viewpoint shift, deformation, and scale transform. It outperforms existing single-frame and sequential descriptors while also being class-agnostic. AirObject is one of the first temporal object encoding methods that aggregate structural knowledge across multiple instances. While single frame representations have been extensively used across literature, there has been limited attention given to temporal information for compact representations in robotics. However, several spatio-temporal representation techniques exist in related fields of research that use LSTM, GRUs, Graph Convolutional Networks (GCNs), and Temporal Convolutional Networks to model temporal relations. In terms of visual place recognition (VPR), some approaches leverage spatio-temporal information in terms of landmarks or bio-inspired memory cells. However, these approaches do not consider explicit geometry available from interest points like SuperPoint does. Overall, AirObject provides an innovative solution for generating temporally evolving global object representations that can be used for various robotic tasks such as semantic scene understanding and re-localization.