Skip to content

Scripting

The data managment system through DIVE and Girder may not be enough for very large datasets or complicated datasets. Scripting the uploading/downloading and processing of data may become necessary. This can be done utilizing the DIVE/Girder Rest Endpoints and a Python pacakge called 'GirderClient' that helps with interfacing these endpoints.

Endpoints

The endpoints documentation provides a more comprehensive list of commonly used endpoints when scripting. These are endpoints for uploading/downloading data as well as running postprocess, pipelines and training on DIVE Datasets.

GirderClient

The documentation for GirderClient and some introductury scripts for authentication along with some commonly used functions for DIVE Scripting

There are several example scripts provided in the repository including:

  • userCount.py - A script utilized by admins to download information about all of the datasets in the system and determine the ttoal number of users that are avaialable.
  • setAnnotationFPS.py - Sets the annotation FPS on a sample folder by modification of the metdata on the DIVE Dataset Folder
  • uploadScript - The process of uploading a new JSON or ViameCSV formatted file to a DIVE Dataset folder and running the postprocess endpoint to add these new annotations.
  • syncAnnotationsScript.py - Script to sync a folder hierarchy of Annotation files with a similar folder structur within DIVE
  • exportAnnotations.py - Script that takes in a list of base FolderIds, recursively finds the DIVE Datasets under the hierarchy and then downloads all annotations to a folder structure mimicing the girder structure.