Skip to content

Data Formats

DIVE Desktop and Web support a number of annotation and configuration formats. The following formats can be uploaded or imported alongside your media and will be automatically parsed.

  • DIVE Annotation JSON (default annotation format)
  • DIVE Configuration JSON
  • VIAME CSV
  • KPF (KWIVER Packet Format)
  • COCO and KWCOCO

DIVE Annotation JSON

Info

The current DIVE schema version is v2. Version 2 was introduced in DIVE version 1.9.0. It is backward-compatible with v1.

Files are typically named result_{dataset-name}.json. Their schema is described as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
/** AnnotationSchema is the schema of the annotation DIVE JSON file */
interface AnnotationSchema {
  tracks: Record<string, TrackData>;
  groups: Record<string, GroupData>;
  version: 2;
}

interface TrackData {
  id: AnnotationId;
  meta: Record<string, unknown>;
  /**
   * Track-level attributes. Can contain arbitrary key-value pairs.
   * 
   * Reserved attribute names (cannot be created by users):
   * - `userCreated`: Internal flag indicating user creation status.
   */
  attributes: Record<string, unknown>;
  confidencePairs: Array<[string, number]>;
  begin: number;
  end: number;
  features: Array<Feature>;
}

interface GroupData {
  id: AnnotationId;
  meta: Record<string, unknown>;
  attributes: Record<string, unknown>;
  confidencePairs: Array<[string, number]>;
  begin: number;
  end: number;
  /**
   * members describes the track members of a group,
   * including sub-intervals that they are participating in the group.
   */
  members: Record<AnnotationId, {
    ranges: [number, number][];
  }>;
}

interface Feature {
  frame: number;
  flick?: Readonly<number>;
  interpolate?: boolean;
  keyframe?: boolean;
  bounds?: [number, number, number, number]; // [x1, y1, x2, y2] as (left, top), (bottom, right)
  geometry?: GeoJSON.FeatureCollection<GeoJSON.Point | GeoJSON.Polygon | GeoJSON.LineString | GeoJSON.Point>;
  fishLength?: number;
  /**
   * Detection attributes. Can contain arbitrary key-value pairs.
   * 
   * Reserved attribute names (cannot be created by users):
   * - `rotation`: Rotation angle in radians for rotated bounding boxes (counter-clockwise).
   *   When present, the `bounds` field represents an axis-aligned bounding box, and the
   *   actual rotated rectangle is computed by applying this rotation around the bbox center.
   *   Only stored if rotation is significant (|rotation| > 0.001 radians).
   * - `userModified`: Internal flag indicating user modification status.
   */
  attributes?: Record<string, unknown>;
  head?: [number, number];
  tail?: [number, number];
  /**
   * Free-form note text for this detection (frame). Stored as a string
   * array for format compatibility; DIVE typically uses a single entry.
   * Omitted when the detection has no note.
   */
  notes?: string[];
}

Reserved Attribute Names

Reserved Attribute Names

Certain attribute names are reserved by DIVE and cannot be used when creating custom attributes. Attempting to create attributes with these names will result in an error.

Reserved Detection Attributes (stored in Feature.attributes): - rotation: Used to store the rotation angle in radians for rotated bounding boxes. When present, the bounds field represents an axis-aligned bounding box, and the actual rotated rectangle is computed by applying this rotation around the bbox center. Only stored if rotation is significant (|rotation| > 0.001 radians). - userModified: Internal flag used by DIVE to track user modification status.

Reserved Track Attributes (stored in TrackData.attributes): - userCreated: Internal flag used by DIVE to track user creation status.

These reserved names are enforced at both the UI level (when creating attributes) and the API level (when saving attributes). If you need to use similar names, consider alternatives like rotationAngle, isUserModified, or isUserCreated.

The full source TrackData definition can be found here as a TypeScript interface.

Example JSON File

This is a relatively simple example, and many optional fields are not included.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
{
  "version": 2,

  "tracks": {
    // Track 1 is a true multi-frame track
    "1": {
      "id": 1,
      "meta": {},
      "attributes": {},
      "confidencePairs": [["fish", 0.87], ["rock", 0.22]],
      "features": [
        { "frame": 0, "bounds": [0, 0, 10, 10], "interpolate": true },
        { "frame": 3, "bounds": [10, 10, 20, 20], "attributes": { "rotation": 0.785 } },
      ],
      "begin": 0,
      "end": 2,
    },
    // Track 2 is a simple single-frame bounding box detection
    "2": {
      "id": 2,
      "meta": {},
      "attributes": {},
      "confidencePairs": [["scallop", 0.67]],
      "features": [
        { "frame": 3, "bounds": [10, 10, 20, 20] },
      ],
      "begin": 3,
      "end": 3,
    },
  },

  "groups": {
    "1": {
      "id": 1,
      "meta": {},
      "attributes": {},
      "confidencePairs": [["underwater-stuff", 1.0]],
      "members": {
        // The fish is a group member on frame 0, 1, and 3.
        // The scallop is only a group member at frame 3.
        "1": { "ranges": [[0, 1], [3, 3]] },
        "2": { "ranges": [[3, 3]] },
      },
      "begin": 0,
      "end": 2,
    }
  }
}

DIVE Configuration JSON

This information provides the specification for an individual dataset. It consists of the following.

  • Allowed types (or labels) and their appearances are defined by customTypeStyling and customGroupStyling.
  • Preset confidence filters for those types are defined in confidenceFilters
  • Track and Detection attribute specifications are defined in attributes
  • Free-form, dataset-level metadata (cruise id, station id, location, …) is stored in datasetInfo as a key/value object.
  • Edited from the Dataset Info panel.
  • Included in DIVE Configuration JSON as datasetInfo.
  • Included in VIAME CSV and COCO / KWCOCO export, and restored on import.

When importing a DIVE Configuration JSON with datasetInfo, Overwrite import (the default) replaces the existing datasetInfo block; an additive import merges it per-key (imported values win). A configuration file with no datasetInfo entry leaves existing dataset metadata untouched.

The full DatasetMetaMutable definition can be found here.

1
2
3
4
5
6
7
8
9
interface DatasetMetaMutable {
  version: number;
  customTypeStyling?: Record<string, CustomStyle>;
  customGroupStyling?: Record<string, CustomStyle>;
  confidenceFilters?: Record<string, number>;
  imageEnhancements?: ImageEnhancements;
  attributes?: Readonly<Record<string, Attribute>>;
  datasetInfo?: Record<string, unknown>;
}

imageEnhancements stores viewer display settings (brightness, contrast, saturation, sharpen, and optional percentile stretch bounds). See Image Enhancements for platform support of high bit-depth stretch.

Media frame metadata

Each frame in an image-sequence or multicam dataset may carry a timestamp field (epoch seconds) parsed from the filename at load time. When every frame on every camera in a multicam dataset has a timestamp, DIVE builds a global aligned timeline for playback. See Aligned playback and timestamps.

1
2
3
4
5
6
interface FrameImage {
  url: string;
  filename: string;
  id?: string; // large-image item id (web tiled TIFF)
  timestamp?: number; // capture time in epoch seconds, when parseable from filename
}

VIAME CSV

Read the VIAME CSV Specification.

Warning

VIAME CSV is the format that DIVE exports to. It doesn't support all features of the annotator (like groups) so you may need to use the DIVE Json format. It's easier to work with.

Dataset metadata in the header

DIVE writes a # metadata comment line near the top of the CSV carrying dataset-level values such as fps. When a dataset has Dataset Info custom metadata, the whole datasetInfo object is added to that line as a single nested JSON entry keyed dataset_info:

1
# metadata,fps: 23.976,"dataset_info: {""gfishsite_id"": ""2024TXN012"", ""year"": ""2024""}", ...
  • On import the # metadata line is parsed back into dataset metadata.
  • fps and the dataset_info block are restored; other fields (such as exported_by) are ignored.
  • Overwrite import (the default) replaces the existing dataset_info block; an additive import merges it per-key (imported values win).
  • A CSV with no dataset_info entry leaves existing metadata untouched.
  • This is how dataset context, for example a gfishsite_id used to re-link annotations to an external database, travels with the exported annotations without renaming files. See the Dataset Info panel for how to populate it.

VIAME CSV polygons and length

DIVE extends standard VIAME CSV with additional geometry and measurement fields:

Polygons — one or more (poly) columns per row, each followed by flat x y coordinate pairs:

1
0,1.png,0,100,100,500,500,1.0,-1,fish,1.0,(poly) 100 100 200 100 200 200 100 200

Multiple polygons — additional (poly) columns on the same row:

1
..., (poly) 100 100 200 100 200 200 100 200, (poly) 300 300 400 300 400 400 300 400

Holes — (hole) columns follow the outer (poly) they belong to:

1
..., (poly) 100 100 500 100 500 500 100 500, (hole) 200 200 400 200 400 400 200 400

Multiple holes are supported with additional (hole) columns.

Length measurements — the standard VIAME length column (8th numeric field) stores stereo fish-length values. DIVE also reads and writes a length entry in detection attributes; on export, the resolved value is written to both the column and attributes when present. Interactive stereo in DIVE Desktop populates these values during annotation.

VIAME CSV notes

DIVE stores a free-form note on each detection (Feature.notes). In VIAME CSV it is written as a (note) column on the detection row:

1
0,1.png,0,100,100,500,500,1.0,-1,fish,1.0,(note) primary observation
  • The (note) column is followed by the note text (whitespace after (note) is trimmed on import).
  • Notes are per-detection (CSV row), not track-level. There is no track-scoped note token.
  • On import, (note) text is stored in Feature.notes (typically a single-element string[]).
  • On export, the note is emitted last among the token columns so DIVE Desktop and Web produce identical row order.
  • If a row contains more than one (note) column, all values are imported into Feature.notes and preserved on re-export, but the DIVE UI edits a single combined note string.

KWIVER Packet Format (KPF)

DIVE supports MEVA KPF

Info

KPF is typically broken into 3 files, but DIVE only supports annotations being loaded as a single file. However, the 3-file breakdown is just convention and KPF can be loaded from a single combined file.

1
2
# Example: create a sinlge KPF yaml annotation file for use in DIVE
cat 2018-03-07.11-05-07.11-10-07.school.G339.*.yml > combined.yml

COCO and KWCOCO

DIVE Web and Desktop can import and export COCO for a single dataset at a time (an image-sequence dataset or a single video dataset). KWCOCO-compatible files are also accepted on import.

Export Notes

  • For image-sequence datasets, exported images[].file_name uses dataset image filenames.
  • For video datasets, DIVE exports per-frame synthetic names (for example, frame_000123.jpg) because base COCO does not define a canonical video container field.

DIVE COCO Attribute Extensions

COCO does not define standard fields for arbitrary track or detection attributes or free-form notes. To preserve DIVE attributes and notes during COCO export/import, DIVE uses extension fields on each COCO annotation object:

  • dive_detection_attributes: Detection/frame-level attributes (maps to Feature.attributes)
  • dive_track_attributes: Track-level attributes (maps to Track.attributes)
  • dive_notes: Per-detection note (maps to Feature.notes)

These extension keys are declared in the COCO info object as:

  • info.dive_extensions = ["dive_detection_attributes", "dive_track_attributes", "dive_notes"]

Dataset-level metadata (datasetInfo)

The dataset's free-form Dataset Info metadata (e.g. gfishsite_id, cruise, station) is written to the COCO info block under a single dive_dataset_info key and advertised in info.dive_extensions:

  • info.dive_dataset_info = { "gfishsite_id": "2024TXN012", "year": "2024", ... }

Extension Field Details

The DIVE extension fields are JSON objects with user-defined key/value pairs. Values are typically strings, numbers, or booleans.

  • annotation.dive_detection_attributes
  • Scope: one COCO annotation (one frame-level detection)
  • DIVE mapping: Track.features[i].attributes
  • annotation.dive_track_attributes
  • Scope: logical track identity across frames (track_id)
  • DIVE mapping: Track.attributes
  • annotation.dive_notes
  • Scope: one COCO annotation (one frame-level detection)
  • Type: string[] (typically one entry; a single non-empty string is also accepted on import)
  • DIVE mapping: Track.features[i].notes
  • Legacy alias: on import, if dive_notes is absent, DIVE also reads notes

When importing, DIVE merges any keys in the attribute objects into the target detection/track attribute dictionaries. If the same key appears in multiple annotations belonging to the same track, later imported entries may overwrite earlier values for that track-level key. The note is attached to the feature for that annotation only.

Round-Trip Behavior

For COCO files produced by DIVE:

  • DIVE writes info.dive_extensions to advertise the extension keys used.
  • DIVE writes dive_detection_attributes and dive_track_attributes on each annotation when attributes are present.
  • DIVE writes dive_notes on each annotation when that feature has a note.
  • Re-importing that file into DIVE preserves those attributes and notes.

For COCO files not produced by DIVE:

  • DIVE still imports standard COCO fields (bbox, optional polygon segmentation, optional keypoints), and reads extension fields when present.

Supported / Unsupported COCO Features

  • Supported:
  • Bounding boxes (bbox)
  • Polygon segmentations in list format (segmentation: [[x1, y1, ...]]); if bbox is omitted, DIVE derives it from the polygon's axis-aligned bounds
  • Head/tail keypoints from category keypoint labels
  • Partially supported:
  • COCO has no direct equivalent for DIVE groups, so groups are not represented in COCO export.
  • Partially supported:
  • Run-length encoded segmentations (RLE): bounding boxes and other fields import, but masks are skipped and a warning is shown.

Example COCO Annotation with DIVE Extensions

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
{
  "info": {
    "description": "DIVE export for my-dataset",
    "dive_extensions": ["dive_detection_attributes", "dive_track_attributes", "dive_notes"]
  },
  "images": [
    { "id": 1, "file_name": "frame_000000.jpg", "frame_index": 0 }
  ],
  "categories": [
    { "id": 1, "name": "fish", "keypoints": ["head", "tail"] },
    { "id": 2, "name": "crab" }
  ],
  "annotations": [
    {
      "id": 1,
      "image_id": 1,
      "category_id": 1,
      "bbox": [100, 200, 50, 80],
      "score": 0.97,
      "track_id": 42,
      "dive_detection_attributes": {
        "visibility": "poor",
        "occluded": true
      },
      "dive_track_attributes": {
        "reviewed": true,
        "source": "analyst"
      },
      "dive_notes": ["primary observation"]
    },
    {
      "id": 2,
      "image_id": 1,
      "category_id": 2,
      "bbox": [320, 140, 120, 90],
      "score": 0.91,
      "track_id": 77,
      "segmentation": [
        [320, 140, 360, 130, 430, 170, 440, 220, 360, 230, 325, 200]
      ],
      "keypoints": [350, 150, 2, 410, 210, 2],
      "num_keypoints": 2,
      "dive_detection_attributes": {
        "visibility": "clear"
      },
      "dive_track_attributes": {
        "species_confidence_note": "manual QA"
      }
    }
  ]
}