TRIUMF’s Communications team today launched a new platform for exploring the tools and techniques physicists use to analyze nuclear structure - from tracing colliding atoms to deducing the inner workings of high-energy cosmic events - developed using immersive 3D technology: The ISAC-II Research Hall in Matterport.
Matterport is an interactive media-driven exploration that presents a ‘virtual twin’ of the ISAC-II space, an area not commonly part of the lab’s public tour program. A concept that brings tagged descriptions of experimental facilities, together with beamlines and design schematic treatments in video loops, all produced by a team of creative collaborators who have been contributing to the project since the fall of 2019.
This portal into the ISAC-II hall provides an immersive view into TRIUMF’s science infrastructure where visitors can now:
Access the Hall through any web browser enabled device
Navigate 360° using a point-and-click interface
Rotate around a virtual twin using on screen controls
The accessible space, due to constraints in this demo, is limited to the area directly in front of the TRIUMF-ISAC Gamma Ray Suppressed Spectrometer (TIGRESS) facility - viewable here in full 4K resolution.
(Image: the ISAC-II virtual interface)
For this project, TRIUMF collaborated with Emily Carr University of Art + Design (ECU) faculty and audiovisual artist Scott Mallory.
Scott’s name may be a familiar one to the TRIUMF community; he participated in the TRIUMF-ECU Leaning Out of Windows collaboration, which paired groups of artists with groups of physicists to co-design, curate, test, and analyze models of collaboration for art and science. His work entitled ‘Physics at the Limits’ takes you on a virtual walk through antimatter.
“I was always impressed with TRIUMF's magnificent technology and felt compelled to draw more people into it at a different scale,” said Mallory. “We have the tools, we have creatives in the community who are inspired by this content, and TRIUMF has the drive to connect science with society, so I wanted to take this opportunity to create new public access points to science. I'm even more curious to see how initiatives like this can take root at scale and continue to create impact that we can measure by bringing together the different sectors.”
Now working with TRIUMF as an Arts & Culture Curator, Mallory set out to re-imagine guided tour experiences through new and novel access points to TRIUMF’s science. Putting together a team of cross-disciplinary creatives with expertise in interaction design, architecture, and film to help experiment with opportunities to engage the general public in exploring complex scientific concepts.
(Image: still from a 3D animation of the interior workings of the TIGRESS gamma ray spectrometer based on TRIUMF Design Shop drawings)
This integrated media approach was developed by first accessing the Matterport framework to blend these captured assets through workflows used in creative media. The intricate effort included a combination of 3D model scans, 360° photography, 3D animation, visual effects (VFX), and production design.
The ISAC-II virtual twin was generated by Matterport camera scans of the space. This process uses artificial intelligence (AI) to generate a 3D model of the surrounding area, while capturing high dynamic range (HDR) 360° panoramic images, combined for a dimensionally accurate visualization of the space.
(Image: Capturing TIGRESS in 3D scans and 4K HDR 360° panoramic)
(Image: Space mapping driven by mobile device attached to Matterport Pro2 camera)
(Image: Matterport HDR 360° photograph)
The 360° images were taken into a separate VFX software used for cinema. Here they were ‘wrapped’ onto spheres in a CG environment to create video ‘scenes’ or shots by custom animating CG camera moves inside the spheres.
The beam activity was stylized and animated using a combination of 2D and 3D tools, then tracked and integrated into the camera moves of each scene according to location, dimension, and color.
(Image: 2D digital compositing in NukeX)
Because of its high level of detail accessing the TIGRESS 3D design schematic model, created by TRIUMF’s Design Shop, required an industrial design workflow to extract its core components before bringing them into the VFX pipeline.
(Image: TIGRESS 3D model optimization from Solidworks, into both Rhino and Maya workflows; Robert-Martyn Fraser and Alex Swanson pictured)
(Image: TIGRESS design schematic wireframe analysis)
After design schematic optimization, the core model was placed into a 360° VFX scene and matched to an animated CG camera move. Several variations of the tracked model were isolated as video clips to be re-combined and customized in the next production phase. The clips were imported as ‘layers’ into compositing software to be further stylized and seamlessly integrated into the sequence of beam activity visualizations.
(Image: 360 camera scene in After Effects)
The Matterport 3D model was extracted then integrated into the featured site map transitions using dynamic simulation and pixel blending techniques between 3D, 360°, and graphic design of the site.
(Image: Matterport model and 360° image ‘fly in’ pixel blend)
(Image: 360° image pixel scatter)
Finally, these results were assembled into the video clips linked in the Matterport interface alongside text and web links, providing a multifaceted sequence of information that allows the user to experience ISAC-II at their own pace.
(Image: ISAC-II virtual 3D twin)
Mallory adds: “This opportunity to deconstruct and reconstruct modern visual technology and information design into a completely new workflow has given us a lot of momentum to develop other new and more widely accessible learning experiences."
TRIUMF Communications team of collaborators include:
Scott Mallory, Production Lead, Digital Compositor;
Ray Sun, Matterport Specialist;
Ivan Li, CG Artist;
Declan Reilly-Gordon, Interaction Designer;
Robert-Martyn Fraser, Industrial Designer;
Alex Swanson, Industrial Designer
With this project, TRIUMF joins a number of partner laboratories across the global physics community (including McMaster, Fermilab, CERN and others) in bringing various elements of on-site tours, programming, and outreach online for virtual audiences. For TRIUMF, these endeavours bring the opportunity to significantly enhance accessibility to the lab and extend the reach of education and engagement initiatives to communities across Canada and around the world.
Mallory and the Communications team will continue collaborations that explore advanced technologies and virtual interfaces for opportunities to bring other TRIUMF locations online in future.
“This initiative shows tremendous promise for driving high-quality, interactive public engagement for our tours and outreach programming,” said Jonathan Bagger, TRIUMF Director. “It’s a joy to see TRIUMF’s unique infrastructure and experimental apparatuses brought to life online, and I anticipate that this new experience will be very welcome to students, educators, researchers, and others across Canada and around the world.”