Six teams of engineers and scientists from the Newport Naval Undersea Warfare Center Division gathered in a warehouse at Naval Station Newport for the “Spot Robot Warehouse Challenge Innovation Event” that culminated the competition on June 21 . The month-long challenge provided an opportunity to demonstrate an effective ending. machine learning pipeline all the way.
The process enabled teams to develop technology solutions to some core problems, such as item identification and tracking, misplaced item detection, and identification of lost or changed items in a warehouse environment. The teams, assisted by Newport Division photographers, collected images, pre-planned with the right tools, tagged hundreds of images, built/deployed the correct machine learning model architecture, verified performance, and then deployed that solution. “in production” on Spot, a robot dog.
A product of Boston Dynamics, Spot is a four-legged robot designed to automate routine inspection tasks and data capture safely, accurately and frequently. Engineers Gary Huntress and Eugene Chabot of the Department of Undersea Warfare (USW) Platform and Payload Integration acquired Spot so that Newport Division scientists and engineers can develop their skills autonomously. For this challenge, teams used Spot to identify objects of different sizes, shapes, and visual characteristics.
The US Navy faces many complex and dynamic environments, from detecting mines in cluttered landscapes to identifying changes in our critical infrastructure, event organizers said. Naval logistics is an example of a critical function to supply the materials needed to keep the fleet on mission.
“NAVSEA Logistic Command’s warehouse located at Naval Station Newport is a complex and dynamic environment that requires a lot of coordination to complete this challenge,” said Huntress. “Autonomy and artificial intelligence provide new opportunities to reduce job demands and potential human error.”
The event successfully showcased the ability of teams to work on algorithms that are the foundation of artificial intelligence and machine learning. Support from the Philadelphia Division of the Naval Surface Warfare Center (NSWC PD), the University of Massachusetts Lowell, and Brown University, who all brought their own Spot robots to the challenge, provided collaboration beyond the science and technology community of the Newport Division. At some point, war center teams will collaborate to explore the idea of building on each other’s strengths. NSWC PD would use its advanced autonomy as a detection platform, while Division Newport would provide a machine learning classifier via a suitable docker container for deployment to its Spot.
Adam Sherman of the USW Payload and Platform Integration Department hosted the event as part of his three-month NAVSEA Journey-Level Leadership (JLL) rotation.
“When I was approached to host a ‘Spot Robot Warehouse Challenge Innovation Event’ as part of my JLL rotation, I had no idea what I was getting into, but it sounded interesting and different,” said Sherman. “I could never have imagined how much work and detail would need to go into this effort to make this event a success. I was very lucky to have a dedicated team that was always available. Each member of the team had different attributes that were monumental in making sure everything that needed to be accomplished was completed.”
The challenge
Each participant was required to have some skills in Python programming, imaging, and autonomy-based machine learning. Each participant also had to dedicate 32 hours to the event, which included training, help sessions, and the challenge event. The teams had to work within the allotted time for the challenge and had limited access to the warehouse.
The core of the challenge was based on identifying objects in different ways. There was a fixed set of exactly 30 known objects and a well-defined autonomy task. Teams were told what to send after each race and scored as evenly as possible. Some tasks were more difficult than others and more points were awarded accordingly. Because the Innovation Event was also a competition, event organizers came up with a challenge and scoring rubric.
“We challenged 30 people to take a robot they had never used into an unfamiliar environment and teach it how to search for and identify objects using a collection of machine learning techniques most had never tried,” said Huntress. “The simplest approach would have been to use Spot’s built-in autonomy to record a pre-defined route and use the baseline pre-trained (but not optimized) machine learning model provided to them. Neither team did this. All teams worked hard to extend autonomy and build a better machine learning classifier and all teams were successful.
“This is important because the pipeline is very easy to modify,” said Huntress. “If a new object of interest were identified, then it would be easy to get/label new images and re-run the model training without modification. That is, the pipeline is more important than the machine learning itself.”
Ultimately, getting the teams to better understand the end-to-end machine learning pipeline made the event worthwhile. Where possible transitioned to US Navy use, this event showed that Spot could help with shipyard maintenance in addition to storage.
“The result was everything I expected. I had that ‘aha!’ moment when I saw how much more valuable that pipeline was,” Huntress said. “There are nuances to the autonomy aspect. You can have a lot of tools in the toolbox, but until you put them into process, you have nothing.”
NUWC Newport is the nation’s oldest warfighting center, dating back to the Naval Torpedo Station established on Goat Island in Newport Harbor in 1869. Commanded by Capt. Chad Hennings, NUWC Newport maintains major detachments in West Palm Beach, Florida , and Andros. Island in the Bahamas, as well as testing facilities at Seneca Lake and Fisher’s Island, New York, Leesburg, Florida, and Dodge Pond, Connecticut.
Join our team! NUWC Division Newport, one of the 20 largest employers in Rhode Island, employs a diverse, highly skilled, educated and skilled workforce. We are continually looking for engineers, scientists, and other STEM professionals, as well as business, finance, logistics, and other support experts who want to be at the forefront of subsea research and development. Connect with NUWC Division Newport Recruiting on this site- https://www.navsea.navy.mil/Home/Warfare-Centers/NUWC-Newport/Career-Opportunities/ and follow us on LinkedIn @NUWC-Newport and on Facebook @NUWCNewport.