2024-09-26
Today I gave a virtual presentation at the INCF Neuroinformatics Assembly, held in Austin. For the talk, I focused on LINDI, but I prepared three documents covering LINDI, Neurosift, and Dendro.
Leveraging LINDI for efficient and non-redundant NWB access on DANDI
LINDI is a cloud-friendly format for representing NWB files in a way that separates metadata from large binary data to optimize streaming and reduce redundancy. This presentation covers the various LINDI file formats and highlights the performance improvements in streaming NWB files using precomputed LINDI files. The talk also touches on future integration of LINDI with DANDI and Neurosift.
Exploring NWB Datasets on DANDI with Neurosift
Here I introduce Neurosift, a browser-based tool designed to explore NWB files stored in the DANDI Archive. Neurosift enables users to interactively visualize neurophysiology data by lazy-loading NWB files, offering efficient data streaming even for large files. It integrates seamlessly with DANDI, allowing users to open files directly from the archive and explore them using various visualization plugins. Neurosift leverages the LINDI format to pre-index Dandisets, speeding up metadata loading, and provides an intuitive interface for navigating neurophysiology data.
Analyzing NWB Datasets on DANDI with Dendro
Dendro is a framework designed to facilitate the collaborative analysis of NWB files stored on DANDI. Dendro runs containerized jobs on local or remote compute resources, integrating seamlessly with Neurosift and DANDI to provide a streamlined experience. This presentation showcases an example of computing autocorrelograms for neural units. Additionally, it demonstrated the setup and use of custom compute clients for distributed processing and presented early-stage developments in spike sorting and other advanced analyses.
2024-09-13
In this workshop at the McGovern Institute for Brain Research (MIT), I walked participants through the process of exploring and analyzing NWB datasets on DANDI using Neurosift and Dendro.