The team has already developed a system, called Motion2Vec, which has watched and analyzed many publicly available surgery videos and used that knowledge to teach a da Vinci surgical robot to apply sutures across incisions. Their findings, which have not yet been reviewed but are published in arXiv, show an “85.5% segmentation accuracy on average suggesting performance improvement over several state-of-the-art baselines, while kinematic pose imitation gives 0.94 centimeter error in position per observation on the test set.”
The team relied on a set of modern computing techniques, including neural networks and Siamese networks, the latter being a way to compare and group images that are alike, to provide a way to teach robots tasks that only humans can presently perform. Perhaps with enough videos to watch, the system may one day even help with performing routine procedures.
Study in arXiv: Motion2Vec: Semi-Supervised Representation Learning from Surgical Videos
Link: Motion2Vec project info page…
(hat tip: Engadget)
