{"id":7,"date":"2019-07-02T13:41:36","date_gmt":"2019-07-02T13:41:36","guid":{"rendered":"https:\/\/labs.ri.cmu.edu\/argo-ai-center\/research\/"},"modified":"2020-08-06T18:10:46","modified_gmt":"2020-08-06T18:10:46","slug":"research","status":"publish","type":"page","link":"https:\/\/labs.ri.cmu.edu\/av-center\/research\/","title":{"rendered":"Research Themes"},"content":{"rendered":"
\n
\n
\n
\n
\n
<\/div>\n
\n
\n
<\/div>\n
Multimodal 3D Scene Understanding<\/strong><\/a><\/div>\n

shape understanding, sensor fusion, motion estimation<\/p>\n

A central challenge in scalable autonomy is multimodal, dynamic 3D scene understanding. Specific challenges include integration of multiple sensors (that can operate that different spatial and temporal resolutions), semantic understanding of a dynamic scene, and integration of prior knowledge in the form of a map (that may be outdated and require updates). Finally, it is crucial for such understanding to happen in a streaming setting with low latency, robust performance, and graceful degradation and error handling.<\/p>\n

<\/p>\n<\/div>\n<\/div>\n

<\/div>\n<\/div>\n<\/article>\n
\n
\n
<\/div>\n
\n
\n
<\/div>\n
Naturalistic Behavior Modeling<\/strong><\/a><\/div>\n

forecasting, multi-agent modeling, rules of behaviour<\/p>\n

Autonomy in the open-world requires understanding and forecasting the behaviors, intentions, and goals of other agents. This is particularly challenging due to the multi-agent nature of the problem; the behavior of any one agent can have a profound effect on others. Forecasting future behaviors requires encoding and representing uncertainties about the multitude of ways the world could evolve. Our key strategy here is to make use of prior knowledge and experience, obtained by observing past examples of naturalistic behavior.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/article>\n

\n
\n
<\/div>\n
\n
\n
Scalable End-to-End Learning<\/strong><\/a><\/div>\n

reinforcement learning, simulation, validation and verification<\/p>\n

Traditional autonomous vehicle pipelines are highly modularized with different subsystems for localization, perception, actor prediction, planning, and control. Though this approach provides ease of interpretation, it can be difficult to generalize to unseen environments and scale to new environments and cities without hand-engineering. Motivated by this observation, the center is also exploring end-to-end learning approaches that make use of historical data and simulators, to enable autonomy stacks that can be self-tuned and are adaptive.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/article>\n<\/div>\n

<\/div>\n<\/div>\n

Public Datasets<\/h1>\n
To support advancements in 3D tracking, motion forecasting, and other perception tasks for autonomous vehicles, Argo AI offers sensor data and HD maps to the public, free of charge under a creative commons share-alike license. Read more about these resources and upcoming competitions at argoverse.org<\/a>.<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Multimodal 3D Scene Understanding shape understanding, sensor fusion, motion estimation A central challenge in scalable autonomy is multimodal, dynamic 3D scene understanding. Specific challenges include integration of multiple sensors (that can operate that different spatial and temporal resolutions), semantic understanding of a dynamic scene, and integration of prior knowledge in the form of a map … Continued<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-7","page","type-page","status-publish","hentry"],"yoast_head":"\nResearch Themes - CMU AV Center<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/labs.ri.cmu.edu\/av-center\/research\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Research Themes - CMU AV Center\" \/>\n<meta property=\"og:description\" content=\"Multimodal 3D Scene Understanding shape understanding, sensor fusion, motion estimation A central challenge in scalable autonomy is multimodal, dynamic 3D scene understanding. Specific challenges include integration of multiple sensors (that can operate that different spatial and temporal resolutions), semantic understanding of a dynamic scene, and integration of prior knowledge in the form of a map … Continued\" \/>\n<meta property=\"og:url\" content=\"https:\/\/labs.ri.cmu.edu\/av-center\/research\/\" \/>\n<meta property=\"og:site_name\" content=\"CMU AV Center\" \/>\n<meta property=\"article:modified_time\" content=\"2020-08-06T18:10:46+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/av-center\\\/research\\\/\",\"url\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/av-center\\\/research\\\/\",\"name\":\"Research Themes - CMU AV Center\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/av-center\\\/#website\"},\"datePublished\":\"2019-07-02T13:41:36+00:00\",\"dateModified\":\"2020-08-06T18:10:46+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/av-center\\\/research\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/labs.ri.cmu.edu\\\/av-center\\\/research\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/av-center\\\/research\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/av-center\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Research Themes\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/av-center\\\/#website\",\"url\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/av-center\\\/\",\"name\":\"CMU AV Center\",\"description\":\"CMU Center for Autonomous Vehicle Research\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/labs.ri.cmu.edu\\\/av-center\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Research Themes - CMU AV Center","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/labs.ri.cmu.edu\/av-center\/research\/","og_locale":"en_US","og_type":"article","og_title":"Research Themes - CMU AV Center","og_description":"Multimodal 3D Scene Understanding shape understanding, sensor fusion, motion estimation A central challenge in scalable autonomy is multimodal, dynamic 3D scene understanding. Specific challenges include integration of multiple sensors (that can operate that different spatial and temporal resolutions), semantic understanding of a dynamic scene, and integration of prior knowledge in the form of a map … Continued","og_url":"https:\/\/labs.ri.cmu.edu\/av-center\/research\/","og_site_name":"CMU AV Center","article_modified_time":"2020-08-06T18:10:46+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/labs.ri.cmu.edu\/av-center\/research\/","url":"https:\/\/labs.ri.cmu.edu\/av-center\/research\/","name":"Research Themes - CMU AV Center","isPartOf":{"@id":"https:\/\/labs.ri.cmu.edu\/av-center\/#website"},"datePublished":"2019-07-02T13:41:36+00:00","dateModified":"2020-08-06T18:10:46+00:00","breadcrumb":{"@id":"https:\/\/labs.ri.cmu.edu\/av-center\/research\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/labs.ri.cmu.edu\/av-center\/research\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/labs.ri.cmu.edu\/av-center\/research\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/labs.ri.cmu.edu\/av-center\/"},{"@type":"ListItem","position":2,"name":"Research Themes"}]},{"@type":"WebSite","@id":"https:\/\/labs.ri.cmu.edu\/av-center\/#website","url":"https:\/\/labs.ri.cmu.edu\/av-center\/","name":"CMU AV Center","description":"CMU Center for Autonomous Vehicle Research","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/labs.ri.cmu.edu\/av-center\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/labs.ri.cmu.edu\/av-center\/wp-json\/wp\/v2\/pages\/7","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/labs.ri.cmu.edu\/av-center\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/labs.ri.cmu.edu\/av-center\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/labs.ri.cmu.edu\/av-center\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/labs.ri.cmu.edu\/av-center\/wp-json\/wp\/v2\/comments?post=7"}],"version-history":[{"count":29,"href":"https:\/\/labs.ri.cmu.edu\/av-center\/wp-json\/wp\/v2\/pages\/7\/revisions"}],"predecessor-version":[{"id":195,"href":"https:\/\/labs.ri.cmu.edu\/av-center\/wp-json\/wp\/v2\/pages\/7\/revisions\/195"}],"wp:attachment":[{"href":"https:\/\/labs.ri.cmu.edu\/av-center\/wp-json\/wp\/v2\/media?parent=7"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}