Osu
BCI Cursor Control task that accurately cues any trajectory and velocity to investigate granular features of movement
ROLE
Research Engineer
TIMELINE
Feb 2024, - Aug, 2024 (6 Months)
TEAM
Chethan Pandarinath (Principal Investigator)
Mattia Rigotti-Thompson (PhD candidate)
SKILLS
Python
Redis
Pyglet
Numpy
Matplotlib
PROBLEM
Previous cursor control tasks couldn't investigate granular features of movement
Previous cursor control tasks are usually of the form "attempt a movement from position A to position B." We weren't able to specifically instruct BCI participants to go at certain speeds or take complex trajectories. As result, we’re unable to investigate questions about how finer features of movement (speed, direction, etc) are encoded in motor cortex.
SOLUTION
Osu, the cursor control task inspired by the music rhythm game.
We invented a new cursor control task that allows better instruction of the participant's attempted trajectories and velocities. There are 2 main goals of the task:
1. To be able to cue any trajectory in advance
2. To cue velocities in such a way that allows a good match between cued and attempted movements
CHALLENGES
We needed to convey 3 dimensions of data on a 2D display
The main challenge was conveying 3 dimensions of data (X, Y, and speed) on a 2D screen. In the beginning of each trial, we wanted the user to be able to understand what speed they should go at and prepare for the movement. We iterated on over a dozen different graphic options and decided to use the 3 following methods to indicate speed:
1 | Targets are shown and color coded to indicate velocity before trial begins
All targets are shown before the trial so the user can prepare, plan, and execute the movement. Green targets indicate slowest speed, orange targets for medium speed, and red targets for fast speed.
2 | Targets fill up and become teal when full
When targets become closer to being selected, they fill up to the ring. When they're full, they become teal, indicating to the user that they should be at that target in the path.
3 | Ghost Cursor makes a first pass
The ghost cursor is a gray cursor that shows the ideal velocity and trajectory of the trial. It makes a first pass through the target path so the user has a better understanding of the trajectory and velocity for this trial.
Can generate any complex trajectory
To be able to cue any trajectory in advance, I built an algorithm to generate a path of targets from any starting point to ending point. Parameters such as the angle of the curve, spacing between targets, target ring radius, and target width can all be customized and changed in real time during the experiment
MY CONTRIBUTION
Task Design & Development
As the sole Research Engineer, I was responsible for the entire end-to-end development of the task. I built the frontend of the task with Python and Pyglet, and connected it to BRAND, our backend for real time asynchronous neural decoding. I experimented with dozens of features and graphics before settling on the best user experience for the participant.
GOAL
Validate and run task in a clinical session by Aug 2024
Session time with our clinical participant is invaluable, so experiments must be bug-free and validated with our Clinical Neurotechnology Research Assistant, Payton. The Crowdstrike issue brought down the main development station I was using 2 weeks before validation, but thanks to our IT staff & rest of the BrainGate team, we were still able to get the task validated and deployed in session!
