Texture

Introduction

vtkTexture is an image algorithm that handles loading and binding of texture
maps. It obtains its data from an input image data dataset type. Thus you can
create visualization pipelines to read, process, and construct textures. Note
that textures will only work if texture coordinates are also defined, and if
the rendering system supports texture.

This class is used in both WebGL and WebGPU rendering backends, but the
implementation details may vary. In WebGL, it uses HTMLImageElement and
HTMLCanvasElement for textures, while in WebGPU, it uses HTMLImageElement,
HTMLCanvasElement, and ImageBitmap.

Methods

extend

Method use to decorate a given object (publicAPI+model) with vtkTexture characteristics.

Argument	Type	Required	Description
publicAPI		Yes	object on which methods will be bounds (public)
model		Yes	object on which data structure will be bounds (protected)
initialValues	ITextureInitialValues	No	(default: {})

generateMipmaps

Generates mipmaps for a given GPU texture using a compute shader.

This function iteratively generates each mip level for the provided texture,
using a bilinear downsampling compute shader implemented in WGSL. It creates
the necessary pipeline, bind groups, and dispatches compute passes for each
mip level.

Argument	Type	Required	Description
device	GPUDevice	Yes	- The WebGPU device used to create resources and submit commands.
texture	GPUTexture	Yes	- The GPU texture for which mipmaps will be generated.
mipLevelCount	number	Yes	- The total number of mip levels to generate (including the base level).

getCanvas

Returns the canvas used by the texture.

getEdgeClamp

Returns true if the texture is set to clamp at the edges.

getImage

Returns the image used by the texture.

getImageBitmap

Returns an ImageBitmap object.

getImageLoaded

Returns true if the image is loaded.

getInputAsJsImageData

Returns the input image data object.

getInterpolate

Returns true if the texture is set to interpolate between texels.

getMipLevel

Returns the current mip level of the texture.

getRepeat

Returns true if the texture is set to repeat at the edges.

getResizable

Returns true if the texture can be resized at run time.
This is useful for dynamic textures that may change size based on user
interaction or other factors.

newInstance

Method use to create a new instance of vtkTexture.

Argument	Type	Required	Description
initialValues	ITextureInitialValues	No	for pre-setting some of its content

setCanvas

Returns the canvas used by the texture.

setEdgeClamp

Sets the texture to clamp at the edges.

Argument	Type	Required	Description
edgeClamp		Yes

setImage

Sets the image used by the texture.

Argument	Type	Required	Description
image		Yes

setImageBitmap

Sets the image as an ImageBitmap object.
Supported in WebGPU only.

Argument	Type	Required	Description
imageBitmap		Yes

setInterpolate

Sets the texture to interpolate between texels.

Argument	Type	Required	Description
interpolate		Yes

setJsImageData

Sets the input image data as a JavaScript ImageData object.

Argument	Type	Required	Description
imageData		Yes

setMipLevel

Sets the current mip level of the texture.

Argument	Type	Required	Description
level		Yes

setRepeat

Sets the texture to repeat at the edges.

Argument	Type	Required	Description
repeat		Yes

Source

index.d.ts

import { vtkAlgorithm } from '../../../interfaces';
import { Nullable } from '../../../types';

/**
 *
 * @param {boolean} [resizable] Must be set to true if texture can be resized at run time (default: false)
 */
export interface ITextureInitialValues {
  repeat?: boolean;
  interpolate?: boolean;
  edgeClamp?: boolean;
  imageLoaded?: boolean;
  mipLevel?: number;
  resizable?: boolean;
}

export interface vtkTexture extends vtkAlgorithm {
  /**
   * Returns the canvas used by the texture.
   */
  getCanvas(): Nullable<HTMLCanvasElement>;

  /**
   * Returns true if the texture is set to repeat at the edges.
   */
  getRepeat(): boolean;

  /**
   * Returns true if the texture is set to clamp at the edges.
   */
  getEdgeClamp(): boolean;

  /**
   * Returns true if the texture is set to interpolate between texels.
   */
  getInterpolate(): boolean;

  /**
   * Returns the image used by the texture.
   */
  getImage(): Nullable<HTMLImageElement>;

  /**
   * Returns an ImageBitmap object.
   */
  getImageBitmap(): Nullable<ImageBitmap>;

  /**
   * Returns true if the image is loaded.
   */
  getImageLoaded(): boolean;

  /**
   * Returns the input image data object.
   */
  getInputAsJsImageData(): Nullable<
    ImageData | ImageBitmap | HTMLCanvasElement | HTMLImageElement
  >;

  /**
   * Returns the current mip level of the texture.
   */
  getMipLevel(): number;

  /**
   * Returns true if the texture can be resized at run time.
   * This is useful for dynamic textures that may change size based on user
   * interaction or other factors.
   */
  getResizable(): boolean;

  /**
   * Returns the canvas used by the texture.
   */
  setCanvas(canvas: HTMLCanvasElement): void;

  /**
   * Sets the texture to clamp at the edges.
   * @param edgeClamp
   * @default false
   */
  setEdgeClamp(edgeClamp: boolean): boolean;

  /**
   * Sets the texture to interpolate between texels.
   * @param interpolate
   * @default false
   */
  setInterpolate(interpolate: boolean): boolean;

  /**
   * Sets the image used by the texture.
   * @param image
   * @default null
   */
  setImage(image: HTMLImageElement): void;

  /**
   * Sets the image as an ImageBitmap object.
   * Supported in WebGPU only.
   * @param imageBitmap
   */
  setImageBitmap(imageBitmap: ImageBitmap): void;

  /**
   * Sets the input image data as a JavaScript ImageData object.
   * @param imageData
   */
  setJsImageData(imageData: ImageData): void;

  /**
   * Sets the current mip level of the texture.
   * @param level
   */
  setMipLevel(level: number): boolean;

  /**
   * Sets the texture to repeat at the edges.
   * @param repeat
   * @default false
   */
  setRepeat(repeat: boolean): boolean;
}

/**
 * Method use to decorate a given object (publicAPI+model) with vtkTexture characteristics.
 *
 * @param publicAPI object on which methods will be bounds (public)
 * @param model object on which data structure will be bounds (protected)
 * @param {ITextureInitialValues} [initialValues] (default: {})
 */
export function extend(
  publicAPI: object,
  model: object,
  initialValues?: ITextureInitialValues
): void;

/**
 * Method use to create a new instance of vtkTexture.
 * @param {ITextureInitialValues} [initialValues] for pre-setting some of its content
 */
export function newInstance(initialValues?: ITextureInitialValues): vtkTexture;

/**
 * Generates mipmaps for a given GPU texture using a compute shader.
 *
 * This function iteratively generates each mip level for the provided texture,
 * using a bilinear downsampling compute shader implemented in WGSL. It creates
 * the necessary pipeline, bind groups, and dispatches compute passes for each
 * mip level.
 *
 * @param {GPUDevice} device - The WebGPU device used to create resources and submit commands.
 * @param {GPUTexture} texture - The GPU texture for which mipmaps will be generated.
 * @param {number} mipLevelCount - The total number of mip levels to generate (including the base level).
 */
export function generateMipmaps(
  device: any,
  texture: any,
  mipLevelCount: number
): Array<Uint8ClampedArray>;

/**
 * vtkTexture is an image algorithm that handles loading and binding of texture
 * maps. It obtains its data from an input image data dataset type. Thus you can
 * create visualization pipelines to read, process, and construct textures. Note
 * that textures will only work if texture coordinates are also defined, and if
 * the rendering system supports texture.
 *
 * This class is used in both WebGL and WebGPU rendering backends, but the
 * implementation details may vary. In WebGL, it uses HTMLImageElement and
 * HTMLCanvasElement for textures, while in WebGPU, it uses HTMLImageElement,
 * HTMLCanvasElement, and ImageBitmap.
 */
export declare const vtkTexture: {
  newInstance: typeof newInstance;
  extend: typeof extend;
};
export default vtkTexture;

index.js

/* eslint-disable no-bitwise */
import macro from 'vtk.js/Sources/macros';

// ----------------------------------------------------------------------------
// vtkTexture methods
// ----------------------------------------------------------------------------

function vtkTexture(publicAPI, model) {
  // Set our className
  model.classHierarchy.push('vtkTexture');

  publicAPI.imageLoaded = () => {
    model.image.removeEventListener('load', publicAPI.imageLoaded);
    model.imageLoaded = true;
    publicAPI.modified();
  };

  publicAPI.setJsImageData = (imageData) => {
    if (model.jsImageData === imageData) {
      return;
    }

    // clear other entries
    if (imageData !== null) {
      publicAPI.setInputData(null);
      publicAPI.setInputConnection(null);
      model.image = null;
      model.canvas = null;
      model.imageBitmap = null;
    }

    model.jsImageData = imageData;
    model.imageLoaded = true;
    publicAPI.modified();
  };

  publicAPI.setImageBitmap = (imageBitmap) => {
    if (model.imageBitmap === imageBitmap) {
      return;
    }

    // clear other entries
    if (imageBitmap !== null) {
      publicAPI.setInputData(null);
      publicAPI.setInputConnection(null);
      model.image = null;
      model.canvas = null;
      model.jsImageData = null;
    }

    model.imageBitmap = imageBitmap;
    model.imageLoaded = true;

    publicAPI.modified();
  };

  publicAPI.setCanvas = (canvas) => {
    if (model.canvas === canvas) {
      return;
    }

    // clear other entries
    if (canvas !== null) {
      publicAPI.setInputData(null);
      publicAPI.setInputConnection(null);
      model.image = null;
      model.imageBitmap = null;
      model.jsImageData = null;
    }

    model.canvas = canvas;
    publicAPI.modified();
  };

  publicAPI.setImage = (image) => {
    if (model.image === image) {
      return;
    }

    // clear other entries
    if (image !== null) {
      publicAPI.setInputData(null);
      publicAPI.setInputConnection(null);
      model.canvas = null;
      model.jsImageData = null;
      model.imageBitmap = null;
    }

    model.image = image;
    model.imageLoaded = false;

    if (image.complete) {
      publicAPI.imageLoaded();
    } else {
      image.addEventListener('load', publicAPI.imageLoaded);
    }

    publicAPI.modified();
  };

  publicAPI.getDimensionality = () => {
    let width = 0;
    let height = 0;
    let depth = 1;

    if (publicAPI.getInputData()) {
      const data = publicAPI.getInputData();
      width = data.getDimensions()[0];
      height = data.getDimensions()[1];
      depth = data.getDimensions()[2];
    }
    if (model.jsImageData) {
      width = model.jsImageData.width;
      height = model.jsImageData.height;
    }
    if (model.canvas) {
      width = model.canvas.width;
      height = model.canvas.height;
    }
    if (model.image) {
      width = model.image.width;
      height = model.image.height;
    }
    if (model.imageBitmap) {
      width = model.imageBitmap.width;
      height = model.imageBitmap.height;
    }

    const dimensionality = (width > 1) + (height > 1) + (depth > 1);
    return dimensionality;
  };

  publicAPI.getInputAsJsImageData = () => {
    if (!model.imageLoaded || publicAPI.getInputData()) return null;

    if (model.jsImageData) {
      return model.jsImageData;
    }

    if (model.imageBitmap) {
      return model.imageBitmap;
    }

    if (model.canvas) {
      const context = model.canvas.getContext('2d');
      const imageData = context.getImageData(
        0,
        0,
        model.canvas.width,
        model.canvas.height
      );
      return imageData;
    }

    if (model.image) {
      const width = model.image.width;
      const height = model.image.height;
      const canvas = new OffscreenCanvas(width, height);
      const context = canvas.getContext('2d');
      context.translate(0, height);
      context.scale(1, -1);
      context.drawImage(model.image, 0, 0, width, height);
      const imageData = context.getImageData(0, 0, width, height);
      return imageData;
    }

    return null;
  };
}

/**
 * Generates mipmaps for a given GPU texture using a compute shader.
 *
 * This function iteratively generates each mip level for the provided texture,
 * using a bilinear downsampling compute shader implemented in WGSL. It creates
 * the necessary pipeline, bind groups, and dispatches compute passes for each
 * mip level.
 *
 * @param {GPUDevice} device - The WebGPU device used to create resources and submit commands.
 * @param {GPUTexture} texture - The GPU texture for which mipmaps will be generated. Must be created with mip levels.
 * @param {number} mipLevelCount - The total number of mip levels to generate (including the base level).
 */
const generateMipmaps = (device, texture, mipLevelCount) => {
  const computeShaderCode = `
    @group(0) @binding(0) var inputTexture: texture_2d<f32>;
    @group(0) @binding(1) var outputTexture: texture_storage_2d<rgba8unorm, write>;

    @compute @workgroup_size(8, 8)
    fn main(@builtin(global_invocation_id) global_id: vec3<u32>) {
      let texelCoord = vec2<i32>(global_id.xy);
      let outputSize = textureDimensions(outputTexture);

      if (texelCoord.x >= i32(outputSize.x) || texelCoord.y >= i32(outputSize.y)) {
        return;
      }

      let inputSize = textureDimensions(inputTexture);
      let scale = vec2<f32>(inputSize) / vec2<f32>(outputSize);

      // Compute the floating-point source coordinate
      let srcCoord = (vec2<f32>(texelCoord) + 0.5) * scale - 0.5;

      // Get integer coordinates for the four surrounding texels
      let x0 = i32(floor(srcCoord.x));
      let x1 = min(x0 + 1, i32(inputSize.x) - 1);
      let y0 = i32(floor(srcCoord.y));
      let y1 = min(y0 + 1, i32(inputSize.y) - 1);

      // Compute the weights
      let wx = srcCoord.x - f32(x0);
      let wy = srcCoord.y - f32(y0);

      // Fetch the four texels
      let c00 = textureLoad(inputTexture, vec2<i32>(x0, y0), 0);
      let c10 = textureLoad(inputTexture, vec2<i32>(x1, y0), 0);
      let c01 = textureLoad(inputTexture, vec2<i32>(x0, y1), 0);
      let c11 = textureLoad(inputTexture, vec2<i32>(x1, y1), 0);

      // Bilinear interpolation
      let color = mix(
        mix(c00, c10, wx),
        mix(c01, c11, wx),
        wy
      );

      textureStore(outputTexture, texelCoord, color);
    }
  `;

  const computeShader = device.createShaderModule({
    code: computeShaderCode,
  });

  const bindGroupLayout = device.createBindGroupLayout({
    entries: [
      {
        binding: 0,
        // eslint-disable-next-line no-undef
        visibility: GPUShaderStage.COMPUTE,
        texture: { sampleType: 'float' },
      },
      {
        binding: 1,
        // eslint-disable-next-line no-undef
        visibility: GPUShaderStage.COMPUTE,
        storageTexture: { format: 'rgba8unorm', access: 'write-only' },
      },
      {
        binding: 2,
        // eslint-disable-next-line no-undef
        visibility: GPUShaderStage.COMPUTE,
        sampler: { type: 'filtering' },
      },
    ],
  });

  const pipelineLayout = device.createPipelineLayout({
    bindGroupLayouts: [bindGroupLayout],
  });

  const pipeline = device.createComputePipeline({
    label: 'ComputeMipmapPipeline',
    layout: pipelineLayout,
    compute: {
      module: computeShader,
      entryPoint: 'main',
    },
  });

  const sampler = device.createSampler({
    magFilter: 'linear',
    minFilter: 'linear',
  });

  // Generate each mip level
  for (let mipLevel = 1; mipLevel < mipLevelCount; mipLevel++) {
    const srcView = texture.createView({
      baseMipLevel: mipLevel - 1,
      mipLevelCount: 1,
    });

    const dstView = texture.createView({
      baseMipLevel: mipLevel,
      mipLevelCount: 1,
    });

    const bindGroup = device.createBindGroup({
      layout: pipeline.getBindGroupLayout(0),
      entries: [
        { binding: 0, resource: srcView },
        { binding: 1, resource: dstView },
        { binding: 2, resource: sampler },
      ],
    });

    const commandEncoder = device.createCommandEncoder({
      label: `MipmapGenerateCommandEncoder`,
    });
    const computePass = commandEncoder.beginComputePass();

    computePass.setPipeline(pipeline);
    computePass.setBindGroup(0, bindGroup);

    const mipWidth = Math.max(1, texture.width >> mipLevel);
    const mipHeight = Math.max(1, texture.height >> mipLevel);
    const workgroupsX = Math.ceil(mipWidth / 8);
    const workgroupsY = Math.ceil(mipHeight / 8);

    computePass.dispatchWorkgroups(workgroupsX, workgroupsY);
    computePass.end();

    device.queue.submit([commandEncoder.finish()]);
  }
};

// ----------------------------------------------------------------------------
// Object factory
// ----------------------------------------------------------------------------

const DEFAULT_VALUES = {
  image: null,
  canvas: null,
  jsImageData: null,
  imageBitmap: null,
  imageLoaded: false,
  repeat: false,
  interpolate: false,
  edgeClamp: false,
  mipLevel: 0,
  resizable: false, // must be set at construction time if the texture can be resizable
};

// ----------------------------------------------------------------------------

export function extend(publicAPI, model, initialValues = {}) {
  Object.assign(model, DEFAULT_VALUES, initialValues);

  // Build VTK API
  macro.obj(publicAPI, model);
  macro.algo(publicAPI, model, 6, 0);

  macro.get(publicAPI, model, [
    'canvas',
    'image',
    'jsImageData',
    'imageBitmap',
    'imageLoaded',
    'resizable',
  ]);

  macro.setGet(publicAPI, model, [
    'repeat',
    'edgeClamp',
    'interpolate',
    'mipLevel',
  ]);

  vtkTexture(publicAPI, model);
}

// ----------------------------------------------------------------------------

export const newInstance = macro.newInstance(extend, 'vtkTexture');
export const STATIC = { generateMipmaps };

// ----------------------------------------------------------------------------

export default { newInstance, extend, ...STATIC };