Web-based visualization and analysis of neurophysiology data

Jeremy Magland, Center for Computational Mathematics, Flatiron Institute

Open Data in Neurophysiology (ODIN) Symposium 2023, McGovern Institute for Brain Research, MIT

Why web-based software?

• Easy to use: no installation
• Easy to share: copy-paste the link
• Cross-platform: all desktop options and mobile
• Development cycle advantages: simplifies distribution, etc.
• Integrates naturally with cloud resources: e.g., DANDI

• Limitations: no native access to local files/software, requires internet connection, limited access to previous versions, requires coding in JavaScript

I will introduce three web-based tools for neurophysiology visualization and analysis

• Figurl: framework for creating shareable interactive visualizations
• Neurosift: static web app for visualizing NWB files hosted in the cloud (DANDI)
• Protocaas: prototype web app for analyzing neurophysiology data in the cloud (or with cluster/local compute)

Figurl: overview

• Simplifies sharing of interactive figures
  • Run a Python script to generate a shareable URL
• Create custom visualization plugins
  • Static HTML bundles in the cloud
  • React/typescript
• Promotes scientific collaboration, communication, reproducibility
• Focus on neurophysiology, but can be used for other domains

Figurl: Plotly example

import plotly.express as px
import figurl as fig

# Load the iris dataset and create a Plotly figure
iris = px.data.iris()
ff = px.scatter_3d(iris, x='sepal_length',
		y='sepal_width', z='petal_width',
		color='species')

# Create and print the figURL
url = fig.Plotly(ff).url(label='plotly example - iris 3d')
print(url)

# Output: 
# https://figurl.org/f?v=gs://figurl/plotly-1
# &d=sha1://5c6ec276ce9a3b20b208aaff911b037ce4052e51
# &label=plotly%20example%20-%20iris%203d

Figurl link

Figurl architecture

Figurl / SpikeInterface integration (with Alessio Buccino)

Figurl / SpikeInterface integration

import spikeextractors as se

# Load the recording and sorting
recording, sorting = ...

# prepare SpikeInterface widget
widget = ...

# Prepare and print the figURL
url = widget.url(label='example')
print(url)

Figurl / SpikeInterface integration (figurl)

Figurl: Other examples - gallery

Figurl: Other examples - gallery

Figurl: Other examples - gallery

Figurl uses Kachery

Kachery is a Content Addressable Storage database in the cloud

• Minimal configuration for upload
• Download from anywhere
• Python client or Command-line client
• Serverless infrastructure
• Organized into zones (labs can host zones / pay for storage)

Storing kachery data

echo "test-content" > test_content.txt
kachery-cloud-store test_content.txt
# output:
# sha1://b971c6ef19b1d70ae8f0feb989b106c319b36230?label=test_content.txt

From Python

uri = kcl.store_text('example text', label='example.txt')
# uri = "sha1://d9e989f651cdd269d7f9bb8a215d024d8d283688?label=example.txt"

Retrieving kachery data

kachery-cloud-load sha1://b971c6ef19b1d70ae8f0feb989b106c319b36230

w = kcl.load_text('sha1://d9e989f651cdd269d7f9bb8a215d024d8d283688?label=example.txt')

x = kcl.load_json('sha1://d0d9555e376ff13a08c6d56072808e27ca32d54a?label=example.json')

y = kcl.load_npy("sha1://bb55205a2482c6db2ace544fc7d8397551110701?label=example.npy")

z = kcl.load_pkl("sha1://20d178d5a1264fc3267e38ca238c23f3e2dcd5d2?label=example.pkl")

Neurosift: overview

• Visualize / browse NWB files hosted on DANDI (and elsewhere)
• Also view local NWB files
• No installation required
• No server backend required (client-only)
• Efficiently read data lazily from HDF5 files (h5wasm fork)
• Similar to NWBWidgets
• Developed in collaboration with CatalystNeuro

Neurosift / DANDI integration

Browse to a DANDI NWB file and click to open in Neurosift

Neurosift: NWB file browser

link

Neurosift: ElectricalSeries

link

Neurosift: lazy loading from remote HDF5 files

Forked version of h5wasm that uses efficient/smart chunking optimized for reading HDF5 using WebAssembly and web workers.

https://github.com/usnistgov/h5wasm

https://github.com/flatironinstitute/neurosift/tree/main/gui/src/pages/NwbPage/RemoteH5File/h5wasm

Neurosift: Raster plot (spike trains)

link

Neurosift: Synchronized views

Neurosift: Peri-stimulus time histogram (PSTH)

link

Neurosift examples

https://github.com/flatironinstitute/neurosift/wiki/Neurosift-DANDI-Examples

Neurodata types: ImageSegmentation | SpatialSeries | TwoPhotonSeries | TimeSeries | TimeIntervals | PSTH | ElectricalSeries | LabeledEvents | ImageSeries | RasterPlot | Autocorrelograms | Images | BehavioralEvents | ...

Neurosift: viewing local files

• Prerequisite: NodeJS v16 or higher
• Install the neurosift Python package: pip install --upgrade neurosift
• Run the following command in a terminal window:

neurosift view-nwb /path/to/file.nwb

Will open Neurosift in a browser.

Protocaas: overview

Protocaas is a prototype web-based tool for analyzing neurophysiology data in the cloud (or with cluster/local compute).

• See also: NeuroCAAS, SpikeInterface-cloud
• Under heavy development with Ben Dichter and Luiz Tauffer (last couple of months)
• Enable labs to use cloud resources to run analysis pipelines on their data
• Also allow using local machines or compute clusters
• Initial focus is spike sorting, but will expand
• Tight integration with NWB and DANDI
• Uses SpikeInterface and spikeinterface-dockerfiles

Protocaas: spike sorting steps

• Step 1: Prepare NWB files, create a dandiset, upload raw data (or use existing)
• Step 2: Create a Protocaas project and import the dandiset (does not create a copy)
• Step 3: Launch spike sorting from the web GUI (configure compute resource)
• Step 4: Browse outputs using Neurosift
• Step 5: Upload outputs to dandiset (existing or new)

Protocaas spike sorting: Create a project

• Log in with GitHub
• Give it a try - just a prototype

Protocaas spike sorting: Import the raw NWB files from DANDI

Does not create a copy (links only)

Protocaas spike sorting: Select files for sorting

Protocaas spike sorting: Select sorting parameters

Protocaas spike sorting: Monitor running jobs

• Jobs are running on the compute resource (AWS, Slurm, or local machine)
• Monitor their status through the web GUI

Protocaas spike sorting: Browse results in Neurosift

• When jobs complete, they upload results to a cloud bucket
• Output files are added to the Protocaas project (pointers to the cloud outputs)
• Protocaas project contains all provenance information

Protocaas spike sorting: Upload results to DANDI

Protocaas spike sorting: Provenance stored in DANDI metadata

Protocaas: advantages

• Scalability: you can spike sort many sessions at once
• Reproducibility: use AWS and get the exact same results
• Provenance: Jobs store all necessary metadata
• Compute flexibility: local, slurm cluster, cloud
• Access to sorters: not always easy to set up even when using SpikeInterface
• Collaboration: add other GitHub users to a workspace
• Integration with DANDI

Protocaas: Initial experiments

https://protocaas.vercel.app/project/4b4d3486?tab=project-files

Protocaas: containerized processors

• Each spike sorter is a processor within a Protocaas app
• Existing apps: Kilosort 2.5, Kilosort 3 and MountainSort 5
  • SpikeInterface
  • SpikeInterface-dockerfiles
• Apps live in containers for production, or on file systems during development
• It's relatively easy to make custom processor apps

Protocaas: compute resources

• Users can use the default compute resources
• Or configure their own compute resources
• A compute resource is a daemon that runs on some server or machine
  • Watches for new jobs in the Protocaas database
  • Spawns the jobs based on the configuration (AWS, Slurm, local machine)
• Jobs are autonomous
  • Capable of reporting their own status to the central system
  • Does not need to communicate with the daemon

Summary

Some tools for visualization and analysis of neurophysiology data:

• Figurl
  • Stable
• Neurosift
  • Initial version is stable
• Protocaas
  • Prototype
  • Under active development, with CatalystNeuro
  • Almost ready for beta testers
  • Please submit your name ideas!

Thank you

Figurl

Jeff Soules, Alessio Buccino, Loren Frank, Eric denovellis, Kyu Hyun Lee, Alison Comrie, Michael Coulter

Neurosift

Jeff Soules, Ben Dichter, Cody Baker

Protocaas

Ben Dichter, Luiz Tauffer

Thanks to the NWB and DANDI teams!