Abstract: Enabling autonomous robots to access, navigate and broadly operate in perceptually-degraded industrial or natural environments represents a strenuous and daunting challenge. Motivated by this fact, this talk focuses on methods and systems toward instilling resilient autonomy - using both classical and data-driven methods - across diverse robot configurations with the aim to seamlessly access and operate anywhere and subject to any conditions. Results and experiences from the victorious journey of Team CERBERUS in the DARPA Subterranean Challenge are presented, lessons learned are outlined and a multitude of experimental studies from follow-up research activities are discussed.

Abstract: For robot manipulators to move out of industrial settings and into human environments, they will need physical intelligence for their interactions with the environment and humans and they will also need the dexterous capabilities of the human hand. This raises many unsolved problems, from designing the mechanisms and actuator technologies for such dexterous manipulators to their fine motor control with force and tactile sensing capabilities. These problems are interlinked: the mechanism for a manipulator is interdependent with its control which is interdependent with its sensing capabilities. This talk will present my past and recent work towards solving these problems.

Abstract: Humans and other biological agents interact with the environment to obtain the information for their tasks. Differently, robots are “afraid” of contacting the environment, strongly relying on passive modes of information gathering, which restrict their capabilities. In this talk, I will present my past and recent work to endorse robots with interactive capabilities to perceive their environment, for example, to find objects of interest and manipulate them. I will also present our recent effort to develop an affordable robotic hand that is friendly with the type of behavior my lab wants to create in robots: full of contact and where learning is enabled by physical interactions.

Abstract: What architecture will scale to human-level robot intelligence? Classical perception-planning-control methods often assume perfect models and tracktable optimization; learning-based methods are data hungry and often fail to generalize. In this talk I will introduce the Differentiable Algorithm Network (DAN), a compositional framework that fuses classical algorithmic architectures and deep neural networks. A DAN is composed of neural network modules that each encode a differentiable robot algorithm, and it is trained end-to-end from data. I will illustrate the potentials of the DAN framework through applications including visual robot navigation and autonomous vehicle control.

Abstract: Recent advances in artificial learning and perception enable robots to achieve semantic understanding and contextual awareness of their surroundings through multimodal interactive sensing and learning. This panel discussion explores the cutting-edge field of robot learning and perception techniques that actively plan observations and interactions to acquire informative data from the environment, thereby enhancing navigation and manipulation skills. Our panelists, Coline, Efi, Peter, and Roberto, will discuss the challenges and emerging opportunities in robot learning and perception for navigation and manipulation. The panel will be chaired by Nikolay Atanasov.

Abstract: Robotic manipulation of diverse objects in unstructured environments remains a major challenge in the field of robotics. Humans adeptly search, grasp, and manipulate objects across a variety of challenging scenarios, whereas current state-of-the-art robotic systems fail to achieve human-level proficiency. The overarching goal of my research is to develop computational learning models that enable robots to perceive the world, acquire new skills, and perform dexterous manipulation at or beyond human capabilities. In this workshop, I will present recent progress from my research group toward the research goal. Specifically, I will discuss (1) object searching and grasping under occlusion and clutter and (2) interactive manipulation for object segmentation. Both projects aim to integrate perception and decision-making to address key challenges in robotic perception and manipulation.

Abstract: Robot learning has been difficult to scale due to cost of obtaining data for each additional task. This talk will discuss how we can instead benefit from broad, robot-agnostic knowledge about the world and then improve by reducing the cost of acquiring each next task.

Abstract: This study presents a viable approach for outdoor mobile robots by integrating perception, planning, and experimentation. For the perception of mobile robots, we develop a real-time depth estimation algorithm that is gaining interest as a viable alternative to large and heavy sensors, such as LiDAR (light detection and ranging) sensors. To achieve this goal, we first designed a depth estimation network for a wide-FOV stereo camera. Then, we estimated the depth image using a convolutional neural network and improved the accuracy using stereo-matching. By exploiting our proposed planning algorithm with an optimization approach, we conducted experiments using a real drone and ground mobile robot in an outdoor environment to prove the performance. The experimental results are analyzed, and we further discuss precautions for operating outdoor mobile robots.

Abstract: The long-standing ambition for autonomous, intelligent service robots that are seamlessly integrated into our everyday environments is yet to become a reality. Humans develop comprehension of their embodiments by interpreting their actions within the world and acting reciprocally to perceive it ---- the environment affects our actions, and our actions simultaneously affect our environment. Besides great advances in robotics and Artificial Intelligence (AI), e.g., through better hardware designs or algorithms incorporating advances in Deep Learning in robotics, we are still far from achieving robotic embodied intelligence. The challenge of attaining artificial embodied intelligence — intelligence that originates and evolves through an agent's sensorimotor interaction with its environment — is a topic of substantial scientific investigation and is still an open challenge. In this talk, I will walk you through our recent research works for enabling humanoid mobile manipulation robots with spatial intelligence through perception and interaction to coordinate and acquire skills that are necessary for their promising real-world applications. In particular, we will see how we can use robotic priors for learning to coordinate mobile manipulation robots, how neural representations can allow for learning safe interactions, and, at the crux, how we can leverage those representations to allow the robot to understand and interact with a scene, or guide it to acquire more “information” while acting in a task-oriented manner.

Abstract: Deploying intelligent mobile robots in the real world has been a longstanding goal in robotics and AI. These environments are often dense, heterogeneous, constrained, and unstructured. In this talk, I'll discuss my research on enabling intelligent mobile robots to navigate such complex environments by instilling human-like mobility in robots. In particular, my talk will describe advanced computer vision and machine learning techniques for improved tracking of dynamic entities in dense traffic. It will also introduce an innovative model for estimating drivers' risk preferences by conceptualizing traffic as an undirected dynamic graph and applying the risk estimation algorithm to resolve conflicts between drivers at unsignalized intersections and during merging. Lastly, the talk will offer insights into the creation of simulators, tools, and datasets to spur further research, providing the audience with a comprehensive understanding of the advancements and innovations in intelligent mobile robot navigation.

Abstract: Autonomous decision-making for navigation and manipulation in complex situations necessitates contextual and situational awareness. Robots must possess the ability to translate their semantic understanding of the environment into meaningful planning objectives and control constraints. At a high level, perception-aware planning can produce contextually relevant robot behavior. At a lower level, perception-aware control can facilitate contextually and physically safe robot motion. This panel discussion explores the cutting-edge field of perception-aware robot planning and control techniques that actively utilize geometric and semantic perceptual information in generating, executing, and adapting robot actions. Our panelists, Changhyun, Georgia, Kostas, and Rohan, will discuss the opportunities and challenges of perception-aware robot navigation and manipulation. The panel will be chaired by Rafael Papallas.