SA¹ú¼Ê´«Ã½

An engineering professor uses machine learning to improve human-centered robotics

Robotics has come a long way in the last decade, going from rare novelties to everyday helpers doing everything from vacuuming homes to performing intricate surgeries. And if you ask Assistant Professor Maria Kyrarini, this is just the tip of the iceberg. She believes that within a few years, robots will be in every person’s home and workplace.

Unlike programmable robots, people are rarely predictable, and no two individuals have the same needs. So, given the increasing interdependence between humans and robots, Kyrarini explains that the bots of the future will need the ability to personalize their behavior based on their user’s emotional and physical state.

Using the power of machine learning and biosensors, Kyrarini’s cutting-edge research at SA¹ú¼Ê´«Ã½’s Robotic Systems Lab is helping robots detect when people with paralysis or other mobility impairments are fatigued, allowing the robots to take on more predictive tasks. It’s a sneak peek, she says, into the sci-fi promise of human-centered robotics.

Cura personalis, through robotics

When Kyrarini reflects on when she knew engineering was right for her, she admits it was later in her career than most.

Her undergraduate degree in her native Greece had been mostly theoretical due to the high costs of securing hardware in the classroom. But, during her Master’s at the University of Bremen in Germany, she finally got to dig into "real robots!” That first encounter, so to speak, would shape the rest of her career.

Staying in Bremen for her Ph.D., she was welcomed onto large projects developing robots that would work for and alongside humans. This included voice-controlled robotic systems for people who might be using their hands in manufacturing settings and, later, for people with disabilities.

“That was the most exciting project for me, because I had a colleague who had multiple sclerosis and was not able to move from the neck down,” Kyrarini says. “Whatever I developed, she would test and give me direct feedback. Having this co-designing process was really helpful.”

Since then, she’s continued to design robots with the holistic needs of her end-user in mind, understanding that if a person with paralysis relies on voice commands to get robotic assistance, then cognitive fatigue might be a user’s greatest challenge.

Recently, Kyrarini, her students, and her partners at the University of Texas at Arlington have collaborated on a system that allows a mobile robotic arm to interpret biological data from wearable sensors that measure electrocardial and electrodermal activity to determine when the user is tired, and then adjust its behavior accordingly.

“We’re using machine learning to help these robots process these biological signals and figure out whether to ask the user more questions about their needs, or if they’re tired, take care of things for them and simply let the user hear and approve its plan for the day,” explains Kyrarini.

Her partners at UT Arlington have recruited test users from the school’s two nationally-ranked for wheelchair users; meanwhile, Kyrarini recruits non-STEM majors as an unbiased control group to compare against the UT Arlington results.

The next frontier

While Kyrarini says this technology is still in its infancy, she’s still incredibly proud of the ways her team’s work has pushed the potential of this technology.

“For example, I have a Master’s student who is trying to detect cognitive fatigue by only using speech. He’s only gotten an accuracy of 62%, so it’s not great yet, but we are hoping that if we get more data, he’ll get better results.”

In the future, they might extend the research into recognizing different emotions, like sadness or frustration, through speech and facial recognition, but there’s a lot of data their team has to process first.

Other areas the team has explored include: 1) a robotic system that coordinates multiple arm bases, allowing for two-handed actions or multi-tasking, 2) replacing the extensive biosensors with a smart watch, or 3) a robot that can use a person’s schedule to predict what items they might need to get ready for the day.

Because the project relies on AI technology, Kyrarini is excited that SA¹ú¼Ê´«Ã½ not only gives undergrads access to hardware she didn’t have until grad school, but the school also offers a new Master’s degree in AI—a boon for students who want to be competitive applicants in the growing robotics industry.

More opportunities for hands-on learning are on the horizon, she adds. This year, Kyrarini received two NSF grants to use her cognitive fatigue research to improve the way we manufacture.

The biggest is a to design an AI-powered robotic-based manufacturing system to create products from recycled metal. While smarter, human-centered robots like Kyrarini’s would be an asset for any industry because of increased human safety and productivity, she’s particularly excited to be working in an industry connected to sustainability.

Kyrarini and her School of Engineering colleague, Associate Professor Fatemeh Davoudi, will adjust similar robotic arms for manufacturing settings and ergonomic requirements. From there, the scope of this work goes beyond just building a robot—at its core, the “recyclofacturing” project, as the team calls it, is about building a new way to build things. 

“We have so many interdisciplinary people on this project—computer scientists, engineers, metal specialists, economists, sociologists, and metal recycling stakeholders—so not only will this be very interesting, but I think it’s helpful that our engineering students will be exposed to so many other disciplines. Robotics is the direction the world is moving, but to succeed in designing robots, you have to really understand the world first.”