{"id":103,"date":"2025-12-05T15:40:59","date_gmt":"2025-12-05T15:40:59","guid":{"rendered":"https:\/\/labs.ri.cmu.edu\/tbd\/?page_id=103"},"modified":"2025-12-08T15:54:38","modified_gmt":"2025-12-08T15:54:38","slug":"tbd-social-navigation-datasets","status":"publish","type":"page","link":"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/","title":{"rendered":"TBD Pedestrian Dataset"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-page\" data-elementor-id=\"103\" class=\"elementor elementor-103\" data-elementor-post-type=\"page\">\n\t\t\t\t<div class=\"elementor-element elementor-element-458fbfc e-flex e-con-boxed e-con e-parent\" data-id=\"458fbfc\" data-element_type=\"container\" data-settings=\"{&quot;background_background&quot;:&quot;classic&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-4898be5 elementor-widget elementor-widget-heading\" data-id=\"4898be5\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">TBD Pedestrian Dataset<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-f441ff5 e-flex e-con-boxed e-con e-child\" data-id=\"f441ff5\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-61bafbd elementor-widget__width-initial elementor-widget elementor-widget-text-editor\" data-id=\"61bafbd\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<hr class=\"short\" \/>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-928f0c8 e-flex e-con-boxed e-con e-parent\" data-id=\"928f0c8\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-b70edfe elementor-widget elementor-widget-text-editor\" data-id=\"b70edfe\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p class=\"has-text-align-center has-medium-font-size\" style=\"text-align: center;\">Welcome! The TBD pedestrian dataset is a dataset primarily collected in pedestrian-rich environments. Our dataset highlights the following components: human-verified labels grounded in the metric space, a combination of top-down and ego-centric views, and naturalistic human behavior in the presence of a socially appropriate \u201crobot\u201d.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-72ebf61 elementor-widget elementor-widget-text-editor\" data-id=\"72ebf61\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\">\n<div class=\"wp-block-embed__wrapper\"><iframe title=\"TBD Pedestrian Dataset Demo\" src=\"https:\/\/www.youtube.com\/embed\/YwGMpf24LEM?feature=oembed\" width=\"1154\" height=\"649\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\" data-mce-fragment=\"1\"><\/iframe><\/div>\n<figcaption class=\"wp-element-caption\">This video demonstrates our synchronized sensor views. We annotate the trajectories from the top-down view videos using a semi-autonomous pipeline with human verification. The trajectories are in metric units and are projected onto the ego-centric sensor views.<\/figcaption><\/figure>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-c9f45de e-flex e-con-boxed e-con e-parent\" data-id=\"c9f45de\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-15c3997 elementor-widget elementor-widget-heading\" data-id=\"15c3997\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Dataset Highlights<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4d81398 elementor-widget elementor-widget-heading\" data-id=\"4d81398\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Human verified metric space labels:\n<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-95b5863 elementor-widget elementor-widget-text-editor\" data-id=\"95b5863\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tHaving labels grounded in metric space eliminates the possibility that camera poses might have an effect on the scale of the labels. It also makes the dataset useful for robot navigation related research because robots plan in the metric space rather than image space.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-74ff588 elementor-widget elementor-widget-heading\" data-id=\"74ff588\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Top-down and ego-centric views:<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-758f6b1 elementor-widget elementor-widget-text-editor\" data-id=\"758f6b1\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tA dataset that contains both top-down views and ego-centric views will be useful for research projects that rely on ego-centric views. This allows ego-centric inputs to their models, while still having access to ground truth knowledge of the entire scene.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-60a3c34 elementor-widget elementor-widget-heading\" data-id=\"60a3c34\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Naturalistic human behavior with the presence of a \u201crobot\u201d:<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ba6b65d elementor-widget elementor-widget-text-editor\" data-id=\"ba6b65d\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The \u201crobot\u201d that provides ego-centric view data collection is a cart being pushed by human. Doing so reduces the novelty effects from the surrounding pedestrians. Having the \u201crobot\u201d being pushed by humans also ensures safety for the pedestrians and its own motion has more natural human behavior. As such, the pedestrians also react naturally around the robot by treating it as another human agent.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-f56f768 e-flex e-con-boxed e-con e-parent\" data-id=\"f56f768\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-2e953d7 elementor-widget elementor-widget-heading\" data-id=\"2e953d7\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Example Pedestrian Interactions<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-18ba7ae elementor-widget elementor-widget-text-editor\" data-id=\"18ba7ae\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<figure class=\"aligncenter size-full\"><img fetchpriority=\"high\" decoding=\"async\" class=\"aligncenter wp-image-177 size-large\" src=\"https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_examples-1-1024x446.jpg\" alt=\"Eight example images from an overhead cameras. Traces of pedestrian motion are shown on each image.\" width=\"800\" height=\"348\" srcset=\"https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_examples-1-1024x446.jpg 1024w, https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_examples-1-300x131.jpg 300w, https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_examples-1-768x334.jpg 768w, https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_examples-1.jpg 1402w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><figcaption class=\"wp-element-caption aligncenter\">Example scenes from the TBD pedestrian dataset (labeled at 2.5 Hz). a) a dynamic group. b) a static conversational group. c) a large tour group with 14 pedestrians. d) a pedestrian affecting other pedestrians\u2019 navigation plans by asking them to come to the table. e) pedestrians stop and look at their phones. f) two pedestrians change their navigation goals and turn towards the table. g) a group of pedestrians change their navigation goals multiple times. h) a crowded scene where pedestrians are heading towards different directions.<\/figcaption><\/figure>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-34ac89c e-flex e-con-boxed e-con e-parent\" data-id=\"34ac89c\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-8c5d959 elementor-widget elementor-widget-heading\" data-id=\"8c5d959\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Downloads<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-39ac65b elementor-widget elementor-widget-text-editor\" data-id=\"39ac65b\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The button below will take you to our datasets on KiltHub. Descriptions of the data organization and label files are in the readme file. Set 2 is the larger and newer dataset.<\/p><p>If you are a member of the Robotics Institute or Carnegie Mellon, please contact <a href=\"https:\/\/www.cs.cmu.edu\/~astein\/\">Aaron Steinfeld<\/a> for local access options.<\/p><p>This effort is IRB approved under Carnegie Mellon University protocols STUDY2019_00000116 and STUDY2021_00000199. Please contact <a href=\"https:\/\/www.cs.cmu.edu\/~astein\/\">Aaron Steinfeld<\/a> for IRB related issues.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4d1897d elementor-widget elementor-widget-button\" data-id=\"4d1897d\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/kilthub.cmu.edu\/authors\/TBDLab_Admin\/18437643\" target=\"_blank\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Click here<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-b07f99b e-flex e-con-boxed e-con e-parent\" data-id=\"b07f99b\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-7adacc3 elementor-widget elementor-widget-heading\" data-id=\"7adacc3\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Hardware \u2013 Set 2<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9258e33 elementor-widget elementor-widget-text-editor\" data-id=\"9258e33\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Set 2 is the recommended set to use. It is larger and contains 3D LiDAR data. The robotic suitcase has better localization capability, so the label synchronization between the top-down view and the ego-centric view is very accurate.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-614caf8 elementor-widget elementor-widget-text-editor\" data-id=\"614caf8\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<figure class=\"aligncenter size-large\"><img decoding=\"async\" class=\"aligncenter wp-image-178 size-large\" src=\"https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/new_setup-1024x498.jpg\" alt=\"Left: overhead corner view of the atrium with highlights on cameras and other features. Right, top: Static camera equipment. Right, bottom: robot suitcase.\" width=\"800\" height=\"389\" srcset=\"https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/new_setup-1024x498.jpg 1024w, https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/new_setup-300x146.jpg 300w, https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/new_setup-768x373.jpg 768w, https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/new_setup-1536x746.jpg 1536w, https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/new_setup.jpg 1743w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><figcaption class=\"wp-element-caption\">Hardware setup used while collecting the TBD pedestrian dataset. Blue circles indicate the positions of RGB cameras. Green box shows our suitcase robot with a 360 camera and stereo camera. The robot is manually pulled along the side of a researcher during recording. The white area is where trajectory labels are collected.<\/figcaption><\/figure>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-fd7d31b elementor-widget elementor-widget-text-editor\" data-id=\"fd7d31b\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>We positioned three FLIR Blackfly RGB cameras surrounding the scene on the upper floors overlooking the ground level at roughly 90 degrees apart from each other. Compared to a single overhead camera, multiple cameras ensure better pedestrian labeling accuracy. The RGB cameras are connected to portable computers powered by lead-acid batteries. We also positioned three more units on the ground floor but did not use them for pedestrian labeling.<\/p>\n<p>In addition, we pulled a robotic suitcase side-by-side through the scene. The robotic suitcase is a converted carry-on rolling suitcase. It is equipped with an IMU and a 3D LiDAR sensor. The robot\u2019s computer, batteries, and all its internal components are hidden inside the suitcase, so pushing the robot resembles pushing a suitcase. We selected this robot because of its inconspicuous design to reduce curious, unnatural reactions from nearby pedestrians, as curious pedestrians may intentionally block robots or display other unnatural movements. In addition, a ZED stereo camera was mounted to collect both ego-centric RGB views and depth information about the scene. A GoPro Fusion 360 camera for capturing high-definition 360 videos of nearby pedestrians was mounted above the ZED. Data from the onboard cameras are useful in capturing pedestrian pose data and facial expressions (annotation in progress).<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-f86218e e-flex e-con-boxed e-con e-parent\" data-id=\"f86218e\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-4e64b86 elementor-widget elementor-widget-heading\" data-id=\"4e64b86\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Hardware \u2013 Set 1<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b1db8ed elementor-widget elementor-widget-text-editor\" data-id=\"b1db8ed\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tSet 1 is an older version of our dataset. It is smaller and the synchronization between the top-down view and the ego-centric view is less accurate, but it still contains interesting pedestrian interaction data. It can be used to complement set 2.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3dfb635 elementor-widget elementor-widget-text-editor\" data-id=\"3dfb635\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<figure class=\"aligncenter size-full\"><img decoding=\"async\" class=\"aligncenter wp-image-179 size-large\" src=\"https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_hardware-5-1024x533.jpg\" alt=\"Left: overhead corner view of the atrium with highlights on cameras and other features. Right, top: Static camera equipment. Right, bottom: mobile cart\" width=\"800\" height=\"416\" srcset=\"https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_hardware-5-1024x533.jpg 1024w, https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_hardware-5-300x156.jpg 300w, https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_hardware-5-768x400.jpg 768w, https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_hardware-5.jpg 1249w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><figcaption class=\"wp-element-caption\">Hardware setup used while collecting the TBD pedestrian dataset (set 1). Blue circles indicate the positions of RGB cameras. Green box shows our mobile cart with a 360 camera and stereo camera which imitate a mobile robot sensor suite. The cart is manually pushed by a researcher during recording. The white area is where trajectory labels are collected.<\/figcaption><\/figure>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4dc0994 elementor-widget elementor-widget-text-editor\" data-id=\"4dc0994\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tInstead of the suitcase robot, we pushed a cart through the scene, which was equipped with the same ZED stereo camera. The same GoPro Fusion 360 camera was mounted above the ZED at human chest level. The ZED camera was powered by a laptop with a power bank.\n\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-4892474 e-flex e-con-boxed e-con e-parent\" data-id=\"4892474\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-6e3180d elementor-widget elementor-widget-heading\" data-id=\"6e3180d\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Statistics<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2898f81 elementor-widget elementor-widget-text-editor\" data-id=\"2898f81\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Due to the larger size, Set 2 is the recommended set to use.<\/p>\n\n<figure class=\"wp-block-table is-style-stripes\">\n<table>\n<tbody>\n<tr>\n<td>Set<\/td>\n<td>Duration (min)<\/td>\n<td>Duration w Robot\/Cart (min)<\/td>\n<td># Of Pedestrians<\/td>\n<td>Label Frequency (Hz)<\/td>\n<\/tr>\n<tr>\n<td>2<\/td>\n<td>626<\/td>\n<td>213<\/td>\n<td>10300<\/td>\n<td>10<\/td>\n<\/tr>\n<tr>\n<td>1<\/td>\n<td>133<\/td>\n<td>51<\/td>\n<td>1416<\/td>\n<td>60<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-7a6ed9c e-flex e-con-boxed e-con e-parent\" data-id=\"7a6ed9c\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-135baae elementor-widget elementor-widget-heading\" data-id=\"135baae\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Publication<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3f09448 elementor-widget elementor-widget-text-editor\" data-id=\"3f09448\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div>Set 2 \u2013 Wang, A., Sato, D., Corzo, Y., Simkin, S., &amp; Steinfeld, A. (2024). TBD Pedestrian Data Collection: Towards Rich, Portable, and Large-Scale Natural Pedestrian Data. 2024 IEEE International Conference on Robotics and Automation. IEEE, 2024. [<a href=\"https:\/\/arxiv.org\/abs\/2309.17187\">link<\/a>]<\/div><div>\u00a0<\/div><div>Set 1 \u2013 Wang, A., Biswas, A., Admoni, H., &amp; Steinfeld, A. (2022). Towards Rich, Portable, and Large-Scale Pedestrian Data Collection. ArXiv, abs\/2203.01974. [<a href=\"https:\/\/arxiv.org\/abs\/2203.01974\">link<\/a>]<\/div>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-5f5607c e-flex e-con-boxed e-con e-parent\" data-id=\"5f5607c\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-9a1b660 elementor-widget elementor-widget-heading\" data-id=\"9a1b660\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Acknowledgments<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b7ac17f elementor-widget elementor-widget-text-editor\" data-id=\"b7ac17f\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tThis project was supported by grants (IIS-1734361 and IIS-1900821) from the National Science Foundation. Additional support was provided by the Office of Naval Research (N00014-18-1-2503) and the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR grant number 90DPGE0003).\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-1940414 e-flex e-con-boxed e-con e-parent\" data-id=\"1940414\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-c4b7caa elementor-widget elementor-widget-heading\" data-id=\"c4b7caa\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Contacts<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-cf810b5 elementor-widget elementor-widget-text-editor\" data-id=\"cf810b5\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><a href=\"https:\/\/www.ri.cmu.edu\/ri-people\/allan-wang\/\">Allan Wang<\/a><\/p><p><a href=\"https:\/\/www.ri.cmu.edu\/ri-faculty\/aaron-steinfeld\/\">Aaron Steinfeld<\/a><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>TBD Pedestrian Dataset Welcome! The TBD pedestrian dataset is a dataset primarily collected in pedestrian-rich environments. Our dataset highlights the following components: human-verified labels grounded in the metric space, a combination of top-down and ego-centric views, and naturalistic human behavior in the presence of a socially appropriate \u201crobot\u201d. This video demonstrates our synchronized sensor views. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":94,"menu_order":0,"comment_status":"closed","ping_status":"open","template":"elementor_header_footer","meta":{"footnotes":""},"class_list":["post-103","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>TBD Pedestrian Dataset - TBD Lab<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"TBD Pedestrian Dataset - TBD Lab\" \/>\n<meta property=\"og:description\" content=\"TBD Pedestrian Dataset Welcome! The TBD pedestrian dataset is a dataset primarily collected in pedestrian-rich environments. Our dataset highlights the following components: human-verified labels grounded in the metric space, a combination of top-down and ego-centric views, and naturalistic human behavior in the presence of a socially appropriate \u201crobot\u201d. This video demonstrates our synchronized sensor views. [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/\" \/>\n<meta property=\"og:site_name\" content=\"TBD Lab\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-08T15:54:38+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_examples-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1402\" \/>\n\t<meta property=\"og:image:height\" content=\"610\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/resources\\\/tbd-social-navigation-datasets\\\/\",\"url\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/resources\\\/tbd-social-navigation-datasets\\\/\",\"name\":\"TBD Pedestrian Dataset - TBD Lab\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/resources\\\/tbd-social-navigation-datasets\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/resources\\\/tbd-social-navigation-datasets\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/wp-content\\\/uploads\\\/sites\\\/38\\\/2025\\\/12\\\/web_examples-1-1024x446.jpg\",\"datePublished\":\"2025-12-05T15:40:59+00:00\",\"dateModified\":\"2025-12-08T15:54:38+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/resources\\\/tbd-social-navigation-datasets\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/resources\\\/tbd-social-navigation-datasets\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/resources\\\/tbd-social-navigation-datasets\\\/#primaryimage\",\"url\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/wp-content\\\/uploads\\\/sites\\\/38\\\/2025\\\/12\\\/web_examples-1.jpg\",\"contentUrl\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/wp-content\\\/uploads\\\/sites\\\/38\\\/2025\\\/12\\\/web_examples-1.jpg\",\"width\":1402,\"height\":610},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/resources\\\/tbd-social-navigation-datasets\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Resources\",\"item\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/resources\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"TBD Pedestrian Dataset\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/#website\",\"url\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/\",\"name\":\"TBD Lab\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/tbd\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"TBD Pedestrian Dataset - TBD Lab","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/","og_locale":"en_US","og_type":"article","og_title":"TBD Pedestrian Dataset - TBD Lab","og_description":"TBD Pedestrian Dataset Welcome! The TBD pedestrian dataset is a dataset primarily collected in pedestrian-rich environments. Our dataset highlights the following components: human-verified labels grounded in the metric space, a combination of top-down and ego-centric views, and naturalistic human behavior in the presence of a socially appropriate \u201crobot\u201d. This video demonstrates our synchronized sensor views. [&hellip;]","og_url":"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/","og_site_name":"TBD Lab","article_modified_time":"2025-12-08T15:54:38+00:00","og_image":[{"width":1402,"height":610,"url":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_examples-1.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/","url":"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/","name":"TBD Pedestrian Dataset - TBD Lab","isPartOf":{"@id":"https:\/\/labs.ri.cmu.edu\/tbd\/#website"},"primaryImageOfPage":{"@id":"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/#primaryimage"},"image":{"@id":"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/#primaryimage"},"thumbnailUrl":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_examples-1-1024x446.jpg","datePublished":"2025-12-05T15:40:59+00:00","dateModified":"2025-12-08T15:54:38+00:00","breadcrumb":{"@id":"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/#primaryimage","url":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_examples-1.jpg","contentUrl":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-content\/uploads\/sites\/38\/2025\/12\/web_examples-1.jpg","width":1402,"height":610},{"@type":"BreadcrumbList","@id":"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/tbd-social-navigation-datasets\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/labs.ri.cmu.edu\/tbd\/"},{"@type":"ListItem","position":2,"name":"Resources","item":"https:\/\/labs.ri.cmu.edu\/tbd\/resources\/"},{"@type":"ListItem","position":3,"name":"TBD Pedestrian Dataset"}]},{"@type":"WebSite","@id":"https:\/\/labs.ri.cmu.edu\/tbd\/#website","url":"https:\/\/labs.ri.cmu.edu\/tbd\/","name":"TBD Lab","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/labs.ri.cmu.edu\/tbd\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-json\/wp\/v2\/pages\/103","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-json\/wp\/v2\/comments?post=103"}],"version-history":[{"count":40,"href":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-json\/wp\/v2\/pages\/103\/revisions"}],"predecessor-version":[{"id":232,"href":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-json\/wp\/v2\/pages\/103\/revisions\/232"}],"up":[{"embeddable":true,"href":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-json\/wp\/v2\/pages\/94"}],"wp:attachment":[{"href":"https:\/\/labs.ri.cmu.edu\/tbd\/wp-json\/wp\/v2\/media?parent=103"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}