News

Researchers develop a new way to instruct dance in Virtual Reality

The new virtual reality technique makes it easier to join a dance routine without first having to learn choreography.
The WAVE technique developed by the researchers is based on anticipating future movement, such as a turn. Picture: Markus Laatta
The WAVE technique developed by the researchers is based on anticipating future movement, such as a turn. Picture: Markus Laattala.

Researchers at Aalto University were looking for better ways to instruct dance choreography in virtual reality. The new WAVE technique they developed will be presented in May at the CHI conference, a major venue for human-computer interaction research. 

Previous techniques have largely relied on pre-rehearsal and simplification. 

‘In virtual reality, it is difficult to visualise and communicate how a dancer should move. The human body is so multi-dimensional, and it is difficult to take in rich data in real time,’ says Professor Perttu Hämäläinen.

The researchers started by experimenting with visualisation techniques familiar from previous dance games. But after several prototypes and stages, they decided to try out the audience wave, familiar from sporting events, to guide the dance.

‘The wave-like movement of the model dancers allows you to see in advance what kind of movement is coming next. And you don't have to rehearse the movement beforehand,’ says PhD researcher Markus Laattala.

In general, one cannot follow a new choreography in real time because of the delay in human perceptual motor control. The WAVE technique developed by the researchers, on the other hand, is based on anticipating future movement, such as a turn.

‘No one had figured out how to guide a continuous, fluid movement like contemporary dance. In the choreography we implemented, making a wave is communication, a kind of micro-canon in which the model dancers follow the same choreography with a split-second delay,’ says Hämäläinen.

From tai chi to exaggerated movements

A total of 36 people took part in the one-minute dance test, comparing the new WAVE visualization to a traditional virtual version in which there was only one model dancer to follow. The differences between the techniques were clear.

‘This implementation is at least suitable for slow-paced dance styles. The dancer can just jump in and start dancing without having to learn anything beforehand. However, in faster movements, the visuals can get confusing, and further research and development is needed to adapt and test the approach with more dance styles’ says Hämäläinen.

In addition to virtual dance games, the new technique may be applicable to music videos, karaoke, and tai chi.

‘It would be optimal for the user if they could decide how to position the model dancers in a way that suits them. And if the idea were taken further, several dancers could send each other moves in social virtual reality. It could become a whole new way of dancing together’, says Laattala.

‘Current mainstream VR devices only track the movement of the headset and handheld controllers. On the other hand, machine learning data can sometimes be used to infer how the legs move,’ says Hämäläinen.

‘But in dance, inference is more difficult because the movements are stranger than, for example, walking,’ adds Laattala.

On the other hand, if you have a mirror in the real dance space, you can follow the movement of your feet using machine vision. The dancer's view could be modified using a virtual mirror. 

‘A dancer's virtual performance can be improved by exaggeration, for example by increasing flexibility, height of the jumps, or hip movement. This can make them feel that they are more skilled than they are, which research shows has a positive impact on physical activity motivation,’ says Hämäläinen.

The virtual dance game has been developed using the Magics infrastructure's motion capture kit, where the model dancer is dressed in a costume with sensors. These have been used to record the dance animation.

The WAVE dance game can be downloaded for Meta Quest 2 and 3 VR devices here.  The Github repository  also includes the open source code that anyone can use to develop the game further.

Reference:

Laattala, M., Piitulainen, R., Ady, N., Tamariz, M., & Hämäläinen, P. (2024). Anticipatory Movement Visualization for VR Dancing. ACM SIGCHI Annual Conference on Human Factors in Computing Systems

PDF of the study available here

Contact information:

Aalto computer scientists in CHI 2024

Conference on Human Factors in Computing Systems.

News
FCAI

Finnish Center for Artificial Intelligence (external link)

The Finnish Center for Artificial Intelligence FCAI is a research hub initiated by Aalto University, the University of Helsinki, and the Technical Research Centre of Finland VTT. The goal of FCAI is to develop new types of artificial intelligence that can work with humans in complex environments, and help modernize Finnish industry. FCAI is one of the national flagships of the Academy of Finland.

  • Published:
  • Updated:

Read more news

Photo: Ville Heirola/Aalto University.
Appointments, Research & Art Published:

Call for two Assistant Professors in Computational Physics

The Department of Applied Physics at Aalto University School of Science invites applications for two Assistant professor positions in Computational Physics.
A lit motorway crossing the sea, with buildings and sky visible
Cooperation, Press releases, Research & Art, University Published:

Finland is strongly investing in AI and establishing an ELLIS Institute to accelerate research and attract talent

Public and private sector investment are coming together to launch ELLIS Institute Finland.
Samuel Girmay in a laboratory, wearing a white lab coat and safety glasses.
Research & Art Published:

Virtual laboratories improve accessibility and language awareness in science education

Doctoral researcher Samuel Girmay has received notable funding from the Jenny and Antti Wihuri Foundation. His project aims to enhance accessibility and language awareness in secondary school science education using virtual laboratories.
A person holds a phone showing the Twitter profile, with mask emojis in the header. A plant is in the background.
Research & Art Published:

Malicious social media bots increased significantly during the COVID19 pandemic

A groundbreaking joint study by Aalto University and the Finnish Institute for Health and Welfare (THL) has revealed critical insights into the role of bots in shaping public health discourse during the COVID-19 pandemic, particularly on Twitter.