Harry Freeman

I am a first year PhD student in the Robotics Institute at Carnegie Mellon University supervised by Professor George Kantor. My research interests are in robot learning, semantic scene understanding, and data-efficiency. I seek to explore generalizable and adaptable methods that allow robots to reason about objects in their surrounding in order to perform complex tasks in diverse and unstructured environments.

I completed my Master of Science in Robotics at CMU working with Professor George Kantor. My research focused on computer vision, 3D reconstruction, and next-best-view planning for phenotyping small crops in agriculture.

Prior to CMU, I was a senior embedded software engineer for Amazon Web Services in their AI Devices division. I worked on the AWS Panorama appliance that brings computer vision and machine learning capabilities to IP cameras. I completed my undergraduate degree at Cornell University where I studied Electrical and Computer Engineering.

Email  /  CV  /  Google Scholar  /  LinkedIn  /  Github

(representative papers are highlighted)

Journals

We develop a computer vision-based method to size and track the growth rates of apple fruitlets. Fruitlets are sized and temporally associated using a combination of deep learning-based and classical methods.

Peer-reviewed Conferences

Developed a novel next-best-view planning approach to enable a 7 DoF robotic arm to autonomously capture images of apple fruitlets. Utilized a coarse and fine dual-map representation along with an attention-guided information gain formulation to determine the next best camera pose. Presented a robust estimation and graph clustering approach to associate fruit detections across images in the presence of wind and sensor error.

We develop a method for creating high-quality 3D models of sorghum panicles to non-destructively estimate seed counts. This is acheived using seeds as semantic 3D landmarks for global registration and a novel density-based clustering approach. Additionally, we present an unsupervised metric to assess point cloud reconstruction quality in the absence of ground truth.

We build a real-time aerial system for multi-camera control that can reconstruct human motions in natural environments without the use of special-purpose markers. This is acheived with a multi-robot coordination scheme that maintains the optimal flight formation for target reconstruction quality amongst obstacles.

Workshops

We develop a next-best-view planning approach to capture images of and size apple fruitlets. Our planner utilizes both coarse and fine octrees to map the environment and to calculate the information gain of sampled viewpoints. Fruitlet sizing is performed by reprojecting extracted fruitlet surfaces onto 2D images and fitting ellipses.

We demonstrate how the use of semantics and environmental priors can help in constructing accurate 3D maps for downstream agricultural tasks with the target application of phenotyping Sorghum.

Thesis

We present computer vision-based methods to non-destructively measure phenotypes of smaller grains and fruit, specifically sorghum seed counts and apple fruitlet sizes. We do this by leveraging semantic information to improve tasks such as localization, association, and viewpoint planning.

A few selected projects from a mix of academic and personal.

Applied deep reinforcement learning to learn a unified policy that controls a quadruped mounted with a camera on a mobile arm with the task of tracking a moving target.

Applied deep reinforcement learning techniques towards informative path planning for drone mapping.

Applied NeuS and metalearning towards watertight mesh reconstruction of plants from sparse viewpoints.

Utilized Generative-Adversarial Networks and Swapping AutoEncoders for coarse style and scene data augmentation.

Built a real-time face detection system in Cython and C using custom implementation of Histogram of Oriented Gradients.