Widgets

Widgets allow users to interact with scene objects and drive functionality
through clicking, dragging, and touch.

VTK.js currently has two flavors of widgets: the first, located in
Sources/Interaction/Widgets, is the VTK C++ style widgets that were
semi-ported over. The second, located in Sources/Widgets, is the new-style
VTK.js widgets that were architected from the ground-up to address shortcomings
with the original widget architecture.

The rest of this document will be only concerned with the new-style widgets, as
the old-style widgets are deprecated (but won’t be removed in the near-term).

Widget Architecture Overview

The widget architecture follows the Model-View-Controller (MVC) pattern. In
mapping the MVC pattern to VTK.js widgets, there are three primary components
involved:

• vtkWidgetState is the model and the central source of truth for a widget.
  When a widget is available across different renderers, they all refer to the
  same widget state object for updates.
• vtkWidgetRepresentation is the base class for the view, which listens to a
  widget state object and generates the appropriate VTK.js scene objects that
  represent the state.
• vtkAbstractWidget is the base class that contains widget behavior,
  interaction code, and a reference to the widget state. This object is closest
  to the controller.

In addition to the three components above, there are widget manipulators, a
widget manager, and an abstract widget factory. In brief, manipulators take
events (e.g. mouse events) and translate them into some meaningful 3D
coordinate. For widgets, this is usually located within the widget behavior
code. The widget manager is a per-renderer manager that manages the lifecycle
of widgets attached to one specific renderer. The abstract widget factory is
the primary object that creates the widget state, constructs subclasses of
vtkAbstractWidget to be attached to a widget manager, and constructs the
associated representations with the view/renderer.

The following diagram demonstrates the relationships between the different
components of the widgets. The example shows off the case of a widget being
shown in two different renderers, with a single shared widget state.

Widget Factory

The widget factory is used to construct widgets, widget state, and
representations. From the widget factory’s perspective, it starts by building a
single vtkWidgetState. When a widget is requested via getWidgetForView(),
the factory creates a new widget object, attaches the widget state to the
widget, creates the appropriate representations, and sets those representations
on the widget.

Widget developers should subclass vtkAbstractWidgetFactory in order to create
widgets. vtkAbstractWidget is meant as an internal class by the widget
factory, and thus should not be subclassed.

Widget State

The widget state is the central store and source of truth for widget data. By
separating widget data out from vtkAbstractWidget, many widgets across many
renderers can be kept in sync by referring to a single widget state. States can
also be nested with sub-states, leading to a tree of states. Changes to any
node within the state tree should propagate up and reflect as a change event in
the root state.

While constructing a widget state by hand is possible, it is much easier (and
recommended) to use vtkStateBuilder for the majority of use cases.
vtkStateBuilder simplifies the construction of widget states through the use
of a handful of APIs: addField(), addStateFromMixin(), etc. More
information can be found in the vtkStateBuilder docs.

Mixins are self-contained state bundles that further assist state creation for
common state properties. They can be applied to a sub-state via the
addStateFromMixin() builder API. Examples of commonly used properties include
position, visibility (mixin named “visible”), color, and size (mixin named
“scale1”).

Widget Representation

Representations are scene objects that render based on the attached widget
state. These are not to be instantiated directly; rather, these should be
referenced in an implementation of
vtkAbstractWidgetFactory.getRepresentationsForViewType.

There are two types of representation behaviors: handle and context. The
primary difference is handles are pickable and contexts are not pickable. If
you want to create a non-pickable representation, extend
vtkContextRepresentation which sets the representation behavior as a context.
Otherwise, extend vtkHandleRepresentation to get pickable handles.

Widget representations are implemented as VTK.js algorithms. That is, they must
implement requestData(inData, outData), where inData contains the widget
state and outData should be set with the resultant vtkPolyData. Here,
requestData is intended to generate a relevant representation given a widget
state, e.g. placing spheres at points provided as a list in the widget state.

When using a representation, care must be taken to ensure that the state has
certain properties available. As an example, vtkSphereHandleRepresentation
expects “origin” and “position” mixins to be specified, but optionally can take
the “visible” and “scale1” mixins. Each representation specifies their
dependency in their documentation and their requestData method.

Widget representations must generate actors to be added to a scene. Internally,
all actors should be appended to the model.actors array in order to be
rendered.

Scaling representations to have fixed pixel size

In order to scale a representation such that it retains the same size in
display space, a widget representation should use the scaleInPixels property
and the getPixelWorldHeightAtCoord(coord) method. When scaleInPixels is set
to true, a widget representation should multiply whatever scaling they perform
by the output of getPixelWorldHeightAtCoord(coord).

Look at the SphereHandleRepresentation as an example for how
getPixelWorldHeightAtCoord is used.

If scaleInPixels is false, then all scale values should be interpreted as
world scales.

Widget Manager

The widget manager manages the lifecycle of a widget associated with a
view/renderer. In order to add a widget into a renderer, a widget manager must
first exist for that renderer (via the widget manager’s setRenderer(ren)
call). The widget manager’s addWidget() method takes in a
vtkAbstractWidgetFactory and uses the factory to construct a widget and
associated representations for the particular view.

Widget manipulators

Manipulators are objects that implement a single method, handleEvent(callData, glRenderWindow). This method takes a mouse event (in callData) and
transforms it into a 3D coordinate position, returning an object containing at
least a worldCoords and eventually other values. This is different from
interactivity manipulators in that interactivity manipulators change how the
scene is viewed via manipulating the camera, whereas widget manipulators
transform mouse input into meaningful world coordinates, e.g. snapping
positions to a line.

An example manipulator would be a plane manipulator. When instantiated with a
plane point and normal, the plane manipulator will project mouse events onto the
plane in 3D and return that projected point.

SVG Representation Support

vtk.js no longer supports SVG-based widget representations, as managing the SVG
rendering mechanisms are better left to a separate library. All of the examples
that used SVG representations have been rewritten to use a rendering shim based
on snabbdom, located in Examples/Utilities/SVGHelpers.js.

If you are looking for a reasonably easy drop-in for rendering SVG
representations, check out the above SVGHelpers.js, which requires snabbdom
to be installed. An example usage can be found in the vtkPolyLineWidget
example.