HPCCloud

Redux

Redux handles our data flow and storage throughout the application. We trigger actions which can can call asyncronus client calls and other actions which in turn get dispatched to reducers which updates our data store. What follows is an annotated version of our Redux data model as outlined here. The term “… object” (e.g. “simulation object”) refers to the plain object you would get back from the Girder endpoint.

Auth

auth: {
  pending: false,
  user: {},
},

• pending determines if there is an async auth action in progress.
• user is the user object we use throughout the application.

Network

network: {
  pending: {
    [id] : { id, name, progress },
  },
  success: {},
  error: {},
  backlog: [ {contents of outdated pending, success, and errors} ],
  progress: { [fileId]: {current: number, total: number}, ... }
  progressReset: bool
},

• pending keeps track of network calls that are still in progress, once they are complete they are moved into either success or error
• success is a store of successfull network calls
• error is a store of failed network calls
• progress is an object keeping track of file uploads
• progressReset is a boolean which controls the opacity and progress of the progress bar.

Preferences

preferences: {
  clusters: {
    active: 0,
    list: [
      { name, config, status }
    ],
    mapById: {
      [_id]: { name, 
        config, 
        status,
        log: []
      },
    }
    pending: false,
    presets: {
      [name]: { prop: value, ... }
    }
  },
  aws: {
    active: 0,
    list: [],
    mapById: {
      [_id]: { },
    }
    pending: false,
  },
  statuses: {
    ec2: [ {...} ],
    clusters: [ {...} ],
  }
},

Each property is self explanatory for what the data contains, they have some common properties too.

• mapById This is the main container for the parent property categorized by their id.
• pending if there are any pending network calls, we use this to make sure we don’t poll the endpoints more than necessary.
• list This is the source for the item list in the ActiveList panel.
• active determines which item in the list is active, -1 if none are active.

Under clusters there’s presets this is a list of preset machines so you don’t have to enter them everytime. We have some preset machines at Kitware setup, if you’d like to add your own machines you can edit the source file for the presets at dist/clusters-presets.json.

Taskflows

taskflows: {
  pending: false,
  mapById: {
    [id]: {
      flow: { taskflow... },
      taskMapById: {
        [id] : { task... }
      },
      jobMapById: {
        [id] : { job... }
      },
      log: [ ...log entries ],
      simulation: id,
      primaryJob: 'pyfr_run',
      stepName: 'Simulation',
      actions: [ 'rerun', 'terminate' ],
      allComplete: false,
      outputDirectory: '/opt/hpccloud-jobs/23452345234',
    }
  },
  taskflowMapByTaskId: {
    [taskId]: ${taskflowId},
  },
  taskflowMapByJobId: {
    [jobId]: ${taskflowId},
  },
  updateLogs: [ taskflowId, ... ],
},

The taskflow store is the most complicated store because of the amount of relations a single taskflow can have to other objects.

• mapById is similar to mapById in clusters however it contains some extra information and the actual taskflow object is stored in flow, all other properties are effectively metadata.
  • taskMayById contains full task objects by id for the taskflow.
  • jobMayById contains full job objects by id for the taskflow.
  • log is the taskflow’s log, we display this as “Workflow log” in the JobMonitor component.
  • simulation is the simulation id the taskflow belongs to
  • primaryJob is the primary job for the taskflow
  • actions contains button objects for the button bar on the simulation view page, they are updated in StateTransitionBehavior.
  • allComplete is a bool which helps StateTransitionBehavior determine actions.
• taskflowMapByTaskId and taskflowMapByJobId are maps of tasks and job id’s which correspond to taskflows. These are used for connecting status and log events to the right objects.
• updateLogs used by StateTransitionBehavior to know if it should update.

Projects

projects: {
  workflowNames: [{ label: 'Saturn', value: 'saturn' }, ...],
  list: [ '2345', '3456345' ],
  active: '2345', // or null
  mapById: {
    '2345' : {
      _id: 2345,
      name: 'sdfsdf',
      description: '',
    }
  },
  simulations: {
    [project id]: {
      list: ['2345', '567657'],
      active: '2345',
    }
  }
},

• workflowNames names of avalable workflows, used for project creation.
• list list of projects by name.
• active the name of the active project.
• mapById a map of projects by id.
• simulations project id’s which correspond to list of simulations that belong to that project.

Simulations

simulations: {
  mapById: {
    [simId]: { ...full simulation object }
  }
},

• mapById map of simulation objects by id.

Files

fs: {
  folderMapById: {
    [id]: {
      open: false,
      folder: { full folder... },
      folderChildren: [ id's... ],
      itemChildren: [ id's... ], 
    }
  },
  itemMapById: {
    [id]: { fullItem },
  }
}

• folderMapById folders by id, which contains:
  • open a bool for if the folder is open.
  • folder the folder object.
  • folderChildren id’s of folders inside the folder.
  • itemChildren id’s of items inside the folder.
• itemMapById a map of item id’s and to which folder othey belong.