Pose2Seg: Detection Free Human Instance Segmentation

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Very close Builds on this paper Cites this paper Uses this paper in depth Not relevant

   PoSeg: Pose-Aware Refinement Network for Human Instance Segmentation

   Desen Zhou, Qian He · 2020 · IEEE Access · cited 11 times · Not summarized yet

   PoSeg extends pose-aware human instance segmentation by combining top-down detection proposals with bottom-up cues, and cites Pose2Seg.

   Abstract

   Human instance segmentation is a core problem for human-centric scene understanding and segmenting human instances poses a unique challenge to vision systems due to large intra-class variations in both appearance and shape, and complicated occlusion patterns. In this paper, we propose a new pose-aware human instance segmentation method. Compared to the previous pose-aware methods which first predict bottom-up poses and then estimate instance segmentation on top of predicted poses, our method integrates both top-down and bottom-up cues for an instance: it adopts detection results as human proposals and jointly estimates human pose and instance segmentation for each proposal. We develop a modu…

2. Very close* Same question Not relevant

   Pose2Seg: Human Instance Segmentation Without Detection

   Ruilong Li, Xin Dong, Zixi Cai et al. · 2018 · arXiv.org · preprint, not peer reviewed · cited 6 times · Not summarized yet

   Pose2Seg addresses human instance segmentation without detection, the same pose-based segmentation problem named in the paper.

3. Very close Another approach to the problem Cites this paper Uses this paper in depth Not relevant

   Joint Human Pose Estimation and Instance Segmentation with PosePlusSeg

   Niaz Ahmad, Jawad Khan, Jeremy Yuhyun Kim et al. · 2022 · AAAI Conference on Artificial Intelligence · cited 8 times · Not summarized yet

   PosePlusSeg jointly estimates human pose and instance masks, using keypoint and body heatmaps to segment people.

   Abstract

   Despite the advances in multi-person pose estimation, state-of-the-art techniques only deliver the human pose structure.Yet, they do not leverage the keypoints of human pose to deliver whole-body shape information for human instance segmentation. This paper presents PosePlusSeg, a joint model designed for both human pose estimation and instance segmentation. For pose estimation, PosePlusSeg first takes a bottom-up approach to detect the soft and hard keypoints of individuals by producing a strong keypoint heat map, then improves the keypoint detection confidence score by producing a body heat map. For instance segmentation, PosePlusSeg generates a mask offset where keypoint is defined as a c…

4. Very close Broader context Cites this paper Not relevant

   ROM-Pose: restoring occluded mask image for 2D human pose estimation

   Yun-Ju Lee, Jihie Kim · 2025 · PeerJ Computer Science · cited 3 times · Not summarized yet

   ROM-Pose addresses bounding-box errors and occlusion in 2D human pose estimation rather than human instance segmentation.

   Abstract

   Human pose estimation (HPE) is a field focused on estimating human poses by detecting key points in images. HPE includes methods like top-down and bottom-up approaches. The top-down approach uses a two-stage process, first locating and then detecting key points on humans with bounding boxes, whereas the bottom-up approach directly detects individual key points and integrates them to estimate the overall pose. In this article, we address the problem of bounding box detection inaccuracies in certain situations using the top-down method. The detected bounding boxes, which serve as input for the model, impact the accuracy of pose estimation. Occlusions occur when a part of the target’s body is o…

5. Very close Broader context Cites this paper Uses this paper in depth Not relevant

   Eigenpose: Occlusion-Robust 3D Human Mesh Reconstruction

   Mi-Gyeong Gwon, Gi-Mun Um, Won-Sik Cheong et al. · 2025 · IEEE Transactions on Image Processing · cited 4 times · Not summarized yet

   Eigenpose reconstructs 3D human meshes from single images while addressing occlusion, rather than segmenting human instances.

   Abstract

   A new approach for occlusion-robust 3D human mesh reconstruction from a single image is introduced in this paper. Since occlusion has emerged as a major problem to be resolved in this field, there have been meaningful efforts to deal with various types of occlusions (e.g., person-to-person occlusion, person-to-object occlusion, self-occlusion, etc.). Although many recent studies have shown the remarkable progress, previous regression-based methods still have respective limitations to handle occlusion problems due to the lack of the appearance information. To address this problem, we propose a novel method for human mesh reconstruction based on the pose-relevant subspace analysis. Specificall…

6. Very close Same question Not relevant

   Human Instance Segmentation Based on Omega- Shape Using Deep Learning

   Huma Sheraz, Zuhaib Ahmed Khan, Muhammad Awais · 2025 · Journal of Science, Technology and Engineering Research · Not summarized yet

   The omega-shape method segments people in cluttered, occluded environments, addressing human instance segmentation.

   Abstract

   Human detection and segmentation in an unconstrained environment is a very important and difficult task, having many important applications including tracking human beings, pedestrian detection, head count etc. Human detection in a single object environment is quite easy, but the problem becomes quite cumbersome in a crowded and cluttered environment. Most of the existing algorithms work on the detection and segmentation of the whole person in a scene. However, the performance of such algorithms degrades in case of occlusion and cluttered environment. To increase the performance in such an environment a technique is available that detects a distinct part of the human body which is “omega sha…

7. Very close Shared data, tool or method Cited by this paper Not relevant

   CrowdHuman: A Benchmark for Detecting Human in a Crowd

   Shuai Shao, Zijian Zhao, Boxun Li et al. · 2018 · arXiv.org · preprint, not peer reviewed · cited 919 times · Not summarized yet

   CrowdHuman provides a richly annotated crowd-detection dataset with occlusions, complementing the paper’s OCHuman benchmark.

   Abstract

   Human detection has witnessed impressive progress in recent years. However, the occlusion issue of detecting human in highly crowded environments is far from solved. To make matters worse, crowd scenarios are still under-represented in current human detection benchmarks. In this paper, we introduce a new dataset, called CrowdHuman, to better evaluate detectors in crowd scenarios. The CrowdHuman dataset is large, rich-annotated and contains high diversity. There are a total of $470K$ human instances from the train and validation subsets, and $~22.6$ persons per image, with various kinds of occlusions in the dataset. Each human instance is annotated with a head bounding-box, human visible-regi…

8. Very close Same method, another problem Not relevant

   Pose2Body: Pose-Guided Human Parts Segmentation

   Zhong Li, Xin Chen, Wangyiteng Zhou et al. · 2019 · IEEE International Conference on Multimedia and Expo · cited 6 times · Not summarized yet

   Pose2Body uses pose estimates to guide human-part segmentation, applying pose-guided segmentation to body parts rather than whole instances.

   Abstract

   Reliable human parts segmentation on 2D images plays an important role in many human-centric computer vision tasks. While significant achievements have been made on human pose estimation, the performance on human parts segmentation remains low. In this paper, we present a novel technique that we call Pose2Body that robustly conducts human parts segmentation based on the pose estimation results. We partition an image into superpixels and set out to assign a segment label to each superpixel most consistent with the pose. We design special feature vectors for every superpixel-label assignment as well as superpixel-superpixel pairs and model optimal labeling as to solve for a conditional random …

Prior work it builds on

1. Close Groundwork for this paper Used in depth by this paper A method it uses Not relevant

   Mask R-CNN

   Kaiming He, Georgia Gkioxari, Piotr Dollár et al. · 2017 · Computer Vision and Pattern Recognition · cited 33,854 times · AI summary available

   Mask R-CNN detects objects and predicts instance masks jointly, the detection-based segmentation approach discussed by the paper.

   Abstract

   We present a conceptually simple, flexible, and general framework for object instance segmentation. Our approach efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask R-CNN is simple to train and adds only a small overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to generalize to other tasks, e.g., allowing us to estimate human poses in the same framework. We show top results in all three tracks of the COCO suite of challenges, in…

2. Close Another approach to the problem Used in depth by this paper A method it uses Not relevant

   PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model

   G. Papandreou, Tyler Lixuan Zhu, Liang-Chieh Chen et al. · 2018 · European Conference on Computer Vision · cited 615 times · Not summarized yet

   PersonLab jointly estimates pose and instance masks with a bottom-up, part-based model, unlike Pose2Seg’s pose-based instance separation.

   Abstract

   We present a box-free bottom-up approach for the tasks of pose estimation and instance segmentation of people in multi-person images using an efficient single-shot model. The proposed PersonLab model tackles both semantic-level reasoning and object-part associations using part-based modeling. Our model employs a convolutional network which learns to detect individual keypoints and predict their relative displacements, allowing us to group keypoints into person pose instances. Further, we propose a part-induced geometric embedding descriptor which allows us to associate semantic person pixels with their corresponding person instance, delivering instance-level person segmentations. Our system …

3. Close Groundwork for this paper Used in depth by this paper A method it uses Not relevant

   Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

   Zhe Cao, T. Simon, S. Wei et al. · 2016 · Computer Vision and Pattern Recognition · cited 7,609 times · Not summarized yet

   Part Affinity Fields provide bottom-up multi-person pose estimation, a pose-estimation approach cited by the paper.

   Abstract

   We present an approach to efficiently detect the 2D pose of multiple people in an image. The approach uses a nonparametric representation, which we refer to as Part Affinity Fields (PAFs), to learn to associate body parts with individuals in the image. The architecture encodes global context, allowing a greedy bottom-up parsing step that maintains high accuracy while achieving realtime performance, irrespective of the number of people in the image. The architecture is designed to jointly learn part locations and their association via two branches of the same sequential prediction process. Our method placed first in the inaugural COCO 2016 keypoints challenge, and significantly exceeds the pr…

4. Close Another approach to the problem Not relevant

   Pose2Instance: Harnessing Keypoints for Person Instance Segmentation

   Subarna Tripathi, Maxwell D. Collins, Matthew A. Brown et al. · 2017 · arXiv.org · preprint, not peer reviewed · cited 21 times · Not summarized yet

   Pose2Instance uses keypoints as priors for person instance segmentation, addressing the same task through a deep segmentation framework.

   Abstract

   Human keypoints are a well-studied representation of people.We explore how to use keypoint models to improve instance-level person segmentation. The main idea is to harness the notion of a distance transform of oracle provided keypoints or estimated keypoint heatmaps as a prior for person instance segmentation task within a deep neural network. For training and evaluation, we consider all those images from COCO where both instance segmentation and human keypoints annotations are available. We first show how oracle keypoints can boost the performance of existing human segmentation model during inference without any training. Next, we propose a framework to directly learn a deep instance segme…

5. Close Groundwork for this paper A method it uses Not relevant

   Towards Accurate Multi-person Pose Estimation in the Wild

   G. Papandreou, Tyler Lixuan Zhu, Nori Kanazawa et al. · 2017 · Computer Vision and Pattern Recognition · cited 928 times · Not summarized yet

   This work estimates multi-person poses with a top-down detector and keypoint model, providing pose-estimation groundwork cited by the paper.

   Abstract

   We propose a method for multi-person detection and 2-D pose estimation that achieves state-of-art results on the challenging COCO keypoints task. It is a simple, yet powerful, top-down approach consisting of two stages. In the first stage, we predict the location and scale of boxes which are likely to contain people, for this we use the Faster RCNN detector. In the second stage, we estimate the keypoints of the person potentially contained in each proposed bounding box. For each keypoint type we predict dense heatmaps and offsets using a fully convolutional ResNet. To combine these outputs we introduce a novel aggregation procedure to obtain highly localized keypoint predictions. We also use…

Papers that cite it

1. Close Broader context Uses this paper in depth Not relevant

   Occluded Human Mesh Recovery

   Rawal Khirodkar, Shashank Tripathi, Kris Kitani · 2022 · Computer Vision and Pattern Recognition · cited 95 times · Not summarized yet

   Occluded Human Mesh Recovery uses spatial context to recover 3D human meshes under person-person occlusion, not instance masks.

   Abstract

   Top-down methods for monocular human mesh recovery have two stages: (1) detect human bounding boxes; (2) treat each bounding box as an independent single-human mesh recovery task. Unfortunately, the single-human assumption does not hold in images with multi-human occlusion and crowding. Consequently, top-down methods have difficulties in recovering accurate 3D human meshes under severe person-person occlusion. To address this, we present Occluded Human Mesh Recovery (OCHMR) - a novel top-down mesh recovery approach that incorporates image spatial context to overcome the limitations of the single-human assumption. The approach is conceptually simple and can be applied to any existing top-down…

2. Close Shared data, tool or method Uses this paper in depth Not relevant

   When Human Pose Estimation Meets Robustness: Adversarial Algorithms and Benchmarks

   Jiahang Wang, Sheng Jin, Wentao Liu et al. · 2021 · Computer Vision and Pattern Recognition · cited 73 times · Not summarized yet

   This work introduces OCHuman-C as a benchmark for pose robustness to image corruptions and cites Pose2Seg.

   Abstract

   Human pose estimation is a fundamental yet challenging task in computer vision, which aims at localizing human anatomical keypoints. However, unlike human vision that is robust to various data corruptions such as blur and pixelation, current pose estimators are easily confused by these corruptions. This work comprehensively studies and addresses this problem by building rigorous robust benchmarks, termed COCO-C, MPII-C, and OCHuman-C, to evaluate the weaknesses of current advanced pose estimators, and a new algorithm termed AdvMix is proposed to improve their robustness in different corruptions. Our work has several unique benefits. (1) AdvMix is model-agnostic and capable in a wide-spectrum…

3. Close Broader context Uses this paper in depth Not relevant

   Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation

   Rawal Khirodkar, Visesh Chari, Amit Agrawal et al. · 2021 · IEEE International Conference on Computer Vision · cited 84 times · Not summarized yet

   MIPNet predicts multiple poses within a bounding box to handle crowded occlusions, and evaluates on OCHuman.

   Abstract

   A key assumption of top-down human pose estimation approaches is their expectation of having a single person/instance present in the input bounding box. This often leads to failures in crowded scenes with occlusions. We propose a novel solution to overcome the limitations of this fundamental assumption. Our Multi-Instance Pose Network (MIPNet) allows for predicting multiple 2D pose instances within a given bounding box. We introduce a Multi-Instance Modulation Block (MIMB) that can adaptively modulate channel-wise feature responses for each instance and is parameter efficient. We demonstrate the efficacy of our approach by evaluating on COCO, CrowdPose, and OCHuman datasets. Specifically, we…

4. Close Same question Uses this paper in depth Not relevant

   MultiPoseSeg: Feedback Knowledge Transfer for Multi-Person Pose Estimation and Instance Segmentation

   Niaz Ahmad, Jawad Khan, Jeremy Yuhyun Kim et al. · 2022 · International Conference on Pattern Recognition · cited 3 times · Not summarized yet

   MultiPoseSeg addresses multi-person pose estimation and instance segmentation, focusing on knowledge transfer for challenging multi-person images.

   Abstract

   Multi-person pose estimation and instance segmentation suffer large performance loss when images are with an increasing number of people and their uncontrolled complex appearance. Yet, existing models cannot efficiently leverage unbalanced training images, i.e., few of them are with multi-person, and most are with single-person, making them ineffective for challenging multi-person scenarios. To tackle multi-person cases with a limited portion of them, we propose MultiPoseSeg, a data preparation and feedback knowledge transfer system designed for multi-person pose estimation and instance segmentation. First, MultiPoseSeg categorically performs random data reduction to reduce the single-person…

5. Close Broader context Uses this paper in depth Not relevant

   Peeking into occluded joints: A novel framework for crowd pose estimation

   Ling-Teng Qiu, Xuanye Zhang, Yanran Li et al. · 2020 · European Conference on Computer Vision · cited 92 times · Not summarized yet

   This framework estimates occluded joints for crowd pose estimation and proposes a benchmark with more occlusions.

   Abstract

   Although occlusion widely exists in nature and remains a fundamental challenge for pose estimation, existing heatmap-based approaches suffer serious degradation on occlusions. Their intrinsic problem is that they directly localize the joints based on visual information; however, the invisible joints are lack of that. In contrast to localization, our framework estimates the invisible joints from an inference perspective by proposing an Image-Guided Progressive GCN module which provides a comprehensive understanding of both image context and pose structure. Moreover, existing benchmarks contain limited occlusions for evaluation. Therefore, we thoroughly pursue this problem and propose a novel …