Skip to content

Pipeline Import and Export

Model packs

Open Models in DIVE Web or DIVE Desktop. On desktop, Models is immediately beside Training and contains the model management controls previously on Training.

  • Import (top right of Trained Models) accepts a ZIP of .pipe files, model weights, and supporting files. Imported packs appear in Trained Models and in the Trained pipeline list. Web imports belong to the current user and are private.
  • Export to ZIP downloads or saves all files in the selected model pack, including other pipelines and nested model folders. The ZIP can be imported into either DIVE Web or Desktop.
  • Convert to ONNX starts the existing ONNX conversion job. This is separate from exporting the complete pack.
  • Delete removes the whole pack, including its pipelines and weights. If a pack has multiple pipelines listed, deleting any one of them removes the pack.

Accepted ZIP layouts

A ZIP must contain at least one .pipe file. These layouts are accepted:

1
2
configs/pipelines/example.pipe     pipelines/example.pipe     example.pipe
configs/pipelines/models/...       pipelines/models/...       models/...

The third layout may put weights beside the .pipe files instead of in models/. Any layout may also be inside one enclosing folder. The importer removes the layout's outer folders and preserves paths beneath the pipeline directory, so references such as models/weights.onnx continue to point to the same files. Files outside the pipeline directory in a VIAME add-on are not installed. Dependencies on the base VIAME installation must be installed separately.

Pack names come from the ZIP filename. Importing the same name again creates a new name with a numeric suffix; it does not replace an existing model. Invalid archives are rejected, and a failed import does not leave a partial model pack.

On desktop, packs are stored under ${Project Data Storage Path}/DIVE_Pipelines. On web, they are stored in your VIAME/VIAME Training Results folder, which can also be opened with Browse for managing files and sharing permissions.

Advanced pipeline configuration

Accepting input

If your pipe must accept input, set the pipeline folder metadata property requires_input to true .

Including base pipelines

User-uploaded pipelines may depend on any pipe already installed from the base image or an addon using include <pipename>.pipe . Depending on other user-uploaded pipes is not supported.

Tip

KWIVER pipe files can be exported for use with DIVE using kwiver pipe-config

Pipe file headers

DIVE reads optional comment headers at the top of a .pipe file to decide how to present and run the pipeline. Headers are case-insensitive. Pipes without these headers behave as they always have.

Example:

1
2
3
4
5
6
# Description: Sea-lion registration stabilizer
# Metadata File: stabilizer:flight_log
# Image List Keys: stabilizer:image_list{cam}
# Input: image
# Output: image
# Requires Calibration: False
Header Meaning
# Description: Human-readable summary shown in the Run Pipeline UI. May continue on following # comment lines until the next named header.
# Input: / # Output: Declared media/annotation kinds for the pipe (used when discovering and categorizing static pipelines).
# Requires Calibration: True Restricts the pipe to stereo datasets that have a calibration file attached. Values true, yes, and 1 are accepted.
# Calibration Keys: <k> [k…] Opt-in: binds the dataset's stereo calibration file to each listed KWIVER config key at run time (one -s <k>=<cal-path> per key). Keys may be space- or comma-separated. Use this when the pipe's calibration consumer is not the conventional measurer:calibration_file / calibration_reader:file pair (those are used when the header is unset). Needed because $CONFIG{global:…} indirection cannot receive -s overrides (macros expand at parse time; -s blocks are appended last).
# Metadata File: <block>:<key> Opt-in: when the dataset has an attached Metadata File, DIVE appends a KWIVER override -s <block>:<key>=<path> at run time. The same CSV/TXT attachment is also considered for Frame Metadata. Without this header, no metadata file is injected.
# Image List Keys: <k> [k…] Opt-in: binds the run's per-camera input image list(s) to each listed KWIVER key. Keys may be space- or comma-separated. A key containing {cam} is expanded once per camera (1-based), e.g. stabilizer:image_list{cam} → image_list1, image_list2, …. A key without {cam} receives camera 1's list only.
# Camera Order: <cam> [cam…] 2-cam/3-cam pipes only: labels the role expected on each inputN, in order (e.g. # Camera Order: EO, UV, IR → input1 optical, input2 ultraviolet, input3 thermal). Role tokens are EO, IR, UV (aliases: eo/rgb/optical/color/vis, ir/thermal/lwir/flir, uv/ultraviolet); any other token (e.g. left, right) labels the slot literally. See Multicam camera assignment for the pre-run dialog where you confirm which dataset camera fills each slot. Camera 1 is the frame the pipe's warp processes map onto: each other camera's registration (Camera Registration tab) onto camera 1 is written to the job as <camera>_to_<camera1>_registration.json and bound to warpN.

Multicam camera assignment

Which dataset camera feeds which input of a 2-cam/3-cam pipe is confirmed in a dialog before the run:

  1. Camera roles. Each camera of a multicam dataset carries a sensor role (eo, ir, uv) in cameraRoles, inferred once at import from the camera (subfolder) name and, failing that, from tokens in its image file names (KAMERA style …_rgb.jpg, …_ir.tif, …_uv.jpg). Only a unanimous answer is recorded; ambiguous cameras get no role.
  2. Assignment step. When you run a 2-cam/3-cam pipe, DIVE shows one row per pipeline input (from the # Camera Order: header, or plain input1..N when the pipe has none) with the dataset camera it proposes — matched by role when both sides have one, else by name — and any slot that would otherwise stay empty is prefilled from display order (input N gets the Nth camera) instead of leaving a blank picker. You confirm or change it before anything runs. Unfilled or duplicated slots block the run. Confirming a role-labelled slot saves the roles back onto the dataset (uncheck Save these as the dataset's camera roles to skip), so a corrected role wins over a misleading name next time and for every other pipeline.
  3. Registration check. DIVE reads which inputs the pipe warps onto camera 1 (process warpN :: warp_detections / warp_image in the pipe body) and, in the same dialog, shows for each such row whether the chosen camera has a fitted registration onto camera 1. A missing one blocks the run with "Register X → Y in the Camera Registration tab first" — the pipe never gets to fail at configure time on a missing registration_cameraN_to_camera1.json. The same check runs server-side (and on desktop) before the job is created, so CLI runs get the same message.
  4. Job. The confirmed order is what the job runs with (cameraOrder in the pipeline params; visible in the desktop job manifest). API callers that omit it get the dataset's stored camera order, as before.

Metadata File vs Configuration File

These are different files with different jobs:

Import field What it is Consumed by
Configuration File (config) DIVE JSON (config.json, or legacy meta.json / *.meta.json / *.config.json: attributes, styles, FPS, …) DIVE itself
Metadata File (metadata) One dataset attachment (.json, .txt, or .csv), e.g. a UAV flight log Pipelines that declare # Metadata File:; CSV/TXT files with matching rows also appear as Frame Metadata

Desktop project store vs Configuration File

On DIVE Desktop, each imported dataset also has a dataset.json under DIVE_Projects/ that records media paths and other local project state. That file is not portable and is not the Configuration File described here.

Pick the file at import (UI or CLI --metadata), or give it one of the reserved frame-metadata names so ingestion paths without an upload field — assetstore, zip archive, desktop folder — find it on their own. Any other filename is used only when it is selected explicitly. The KWIVER process behind the pipe owns the schema.

DIVE additionally reads a selected CSV/TXT attachment as Frame Metadata for image-sequence and video datasets; if no rows match, it shows no frame values while pipelines still receive the file unchanged. Metadata is available on single-camera and multicamera imports alike; it is not stereo-gated (unlike calibration).

Note

Injection is opt-in and silent when either side is missing: if a pipe declares # Metadata File: but the dataset has no attached file (or the reverse), the job still runs without that -s override. Prefer attaching the sidecar at import for registration pipes that need it.

Tip

# Image List Keys: is intended for image-sequence (and multicam image) runs where DIVE builds line-separated image manifests. On video runs the bound value is the video path, which may not be what a registration-style process expects.