NEC Pioneers Technology Allowing Lagless Virtual Rehearsals And Performances During COVID-19

New England Conservatory (NEC) and Grammy Award winning vocal faculty member Ian Howell have revolutionized low-latency music-making using cutting edge and affordable technology for musicians and music educators. By pioneering virtually lagless audio-visual elements to enhance digital music making with a setup anyone can assemble at home, NEC's research and experimentation is offering a life-line to musicians struggling to rehearse or perform during the COVID-19 era.

When NEC went into virtual mode in the spring, faculty members went to work to figure out how to bring the world-class music education to students during lockdown. With an entrepreneurial mindset and determination to make modern technology work for musicians, Howell and his team began testing various technology solutions that would support music making at the highest level. They sensed the widespread feeling of loss among the international music community and knew a solution was in sight.

After testing numerous platforms, the NEC team discovered if they combine a free software called SoundJack with an affordable at-home hardware setup, their AV innovation allows for musicians to rehearse and perform with virtually no lag in synchronous rehearsing and performing. "For me, SoundJack is the best compromise of features, flexibility, and complexity," Howell notes on his website.

Howell and his team are now advising other music institutions and colleagues around the globe on how to set up and implement the software, created by Dr. Alexander Carôt. The usage rate of this setup has soared during the pandemic with Howell spearheading its widespread use. The system, which originally only had 60 users, has soared to over 11,000.

Ian Howell says, "The core of live performance is spontaneity and communication between performers and the connection with a live audience. We can still make beautiful art digitally that's within the social distancing limitations established in the current times. Sharing our research in low-latency technology has filled a need in the worldwide music community. The SoundJack guide I wrote and published in June has received more than 21,000 pageviews to date. Demonstration videos I shared widely on Facebook have received more than 15,000 views. Musicians are hungry for this information, and eager to make music in real-time with their colleagues or students. We're glad to share this knowledge with colleagues across the globe."

"This is a no-risk option for music-making," says David Zoffer, Jazz Department Chair of NEC's Preparatory & Continuing Education Program, and an avid user, who has coached and played in several ensembles for both kids and adults. "I think it will change rehearsing forever. This is something that's huge not just during the pandemic, but beyond. Our students will no longer have to travel to Boston to attend rehearsals. They can play with world-class musicians in ensembles and lessons from their homes."

Other NEC faculty agree. "I felt like my hands were finally untied and I could do my job again. I cried tears of joy," says collaborative pianist and NEC faculty member Chelsea Whitaker on the first time she used this new AV setup. Kayla Gautereaux, Assistant Vocal Pedagogy Director adds, "When you're performing, you forget about the technology and can focus on musical collaboration again."

As long as the musicians or student and teacher are within about 500 miles of each other, and both have the AV setup installed, they are able to perform with extreme low-latency. As an example, this setup allows musicians who are within 500 miles of each other to perform or rehearse together with a lag of just 15-20ms-or the equivalent of standing about 15-20 feet away from another person. Normal music-making in person is calculated at a lag of 3-10ms. On campus, our rooms are connected with a latency of about 2-10ms, which is a close as in person. So this experience is the closest to real-time that exists. Considering the lag of other software traditionally used can be greater than 100ms (or like standing more than 100 feet away), this easy hardware setup is giving thousands of musicians around the world the next best thing to in-person music making.

After piloting this low-latency approach during the summer of 2020 through the Preparatory School and School of Continuing Education, NEC has adopted this as part of its hybrid remote-and-in-person plan for College students and faculty. Creating a musical collaborative option that is both synchronous and physically separate has created flexibility for coachings, classes, and lessons. A voice class might include a student singing without a mask in one of the Low Latency Rooms established on campus, while the coach and non-singing students gather masked in another room to work with that student. Chamber musicians can work together in real-time from separate locations, and vocal lessons can take place with the singer, pianist, and vocal coach each in a separate location-either working in one of NEC's Low Latency Rooms, or working from their own setup at home. NEC is even running repertoire classes with performers enjoying a Low Latency connection in Boston, connected with high fidelity sound over SoundJack to their teacher in Germany who broadcasts the audio back to the other course participants distributed up and down the east coast of the United States. Going under the Atlantic ocean twice slows the audio down enough to appear in sync with Zoom video between the performers and observers in Boston.

The SoundJack software is free; the cost of the required technology elements range from approximately $200-$600 depending on the level of sophistication of audio interface, microphones, and headphones the student or musician desires, including smaller costs like ethernet cables, XLR cable and adapters. A microcomputer called a FastMusic Box can be assembled for $170 that can replace an older computer not strong enough to run the system.

"We are really just getting started," says Howell. "Now that the semester is underway we can move onto the next round of research and development which will include infrastructure for real-time, idiomatically expressive broadcast performances from distributed locations, optimization of video algorithms to cut video latency as low as possible, and the expansion of this technology into Boston public schools with arts programs gutted by the pandemic. All the work we have done up to now has set us and our partners up to do what is next."

NEC Vocal Pedagogy Director and vocalist Ian Howell and collaborative pianist Chelsea Whitaker recorded "Flow My Tears" by John Dowland in real time over a distance of 16 miles: https://youtu.be/_7bCmtoz800

Four NEC jazz faculty recently gathered online with SoundJack and recorded this short jam clip. This clip is performed live from their own homes across New England but it sounds like they are in the same room. No mixing, splicing, editing, just live playing: https://youtu.be/GfcdRxBRNfU.

Don't Miss a Boston News Story
Sign up for all the news on the Spring season, discounts & more...