Organization of the Repositories¶
The repository contains several types of files.
All example source code, descriptions, test data, and test baselines are stored in the src/ tree.
The major elements of the tree are:
|-- |-- CMakeLists.txt # To build all of the examples |-- _layouts # Contains _default.html |-- custom_theme # tweaks to the material theme |-- mkdocs.yml # config file for MkDocs |-- src # all of the content to create site/ |-- SyncSiteWithRepo.sh # master script to update site/ |-- Admin | |-- ScrapeRepo.py # script to create docs/ in the web site folder | |-- VTKCMakeLists # template for Cxx examples | |-- VTKQtCMakeLists # template for Cxx Qt |-- Artifacts # Additional images, docs, etc. |-- Images # Images used in creating the static web pages. |-- Instructions # Documentation |-- LANGUAGE.md # Summary file for language |-- LANGUAGE # Language examples | |-- TOPIC |-- Testing | |-- Baseline # Baselines for examples | | |-- LANGUAGE | | | |-- TOPIC | |-- Data # Data for examples |-- VTKBook # Markdown version of the VTK Book |-- VTKBookLaTeX # LaTex version of the VTK Book |-- stylesheets |-- extra.css # mkdocs search config
This repository is separate from the vtk examples source files. It contains the generated web pages and other needed files that are copied across from the source repository.
|-- |-- site |-- src # needed content for site generation and site/ operation |-- Artifacts # Additional images, docs, etc. |-- Cache # Stores cache files used by ScrapeRepo.py. |-- Coverage # Stores VTK classes used and classes used by language. |-- Images # Images used in creating the static web pages. |-- Testing | |-- Baseline # Baselines for examples | | |-- LANGUAGE | | | |-- TOPIC | |-- Data # Data for examples |-- Tarballs # Tar files for each Cxx example
Look and Feel¶
A priority in moving from the wiki media VTK Wiki Examples to the Github pages VTK Examples was to provide a modern, familiar look and feel. We also wanted to support tablet and mobile platforms. We chose MkDocs because it generated static HTML pages that can be hosted anywhere.
MkDocs is a python package and can be installed using
The mkdocs.yml file contains the configuration parameters:
A number of markdown extensions are available.
Admonitions are specially marked "topics" that can appear anywhere an ordinary body element can. They contain arbitrary body elements. Typically, an admonition is rendered as an offset block in a document, sometimes outlined or shaded, with a title matching the admonition type. For example:
This is a note or seealso admonition
This is a summary or todo admonition
This is an info or todo admonition
This is a tip, hint or important admonition
This is a success, check or done admonition
This is a question,help faq admonition
This is a warning, caution or attention admonition
This is a failure, fail or missing admonition
This is a danger or error admonition
This is a bug admonition
This is a quote or cite admonition
This is a cite admonition
The codehilite extension highlights code depending on the language.
The toc extension generates a table of contents, like the one at the right of this page.
The Materials theme is built using Google's Material Design guidelines. It offers a unified experience across platforms. The look and feel is a familiar one.
The materials theme is selected in the theme section of the mkdocs.yml file.
Google Analytics tracks the site usage, providing lots of useful statistics. To have Google Analytics track web usage, you need to register the URL with the Google Analytics System. After registration, an HTML snippet is provided to include on every web page to be tracked.
Configuring Google Analytics¶
The google_analytics keyword in the mkdocs.yml file specifies the google analytics unique code for this web site. Currently commented out as this only applies to the old site.
The custom_theme/main.html file defines the metadata for the google site verification:
<meta name="google-site-verification" content="nJqh3TVb_44fMl2gS61agt_bRNFEaI496UaXYL-wLZI" />
The overall look and feel are established at https://cse.google.com/cse/. After setting up the search engine, you can get the code to add to the web pages.
The code is added to custom_theme/main.html.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <gcse:searchbox-only>Search</gcse:searchbox-only>
The search box is added to the web pages by adding the gcse search box html to custom_theme/partials/search.html.
The first implementation had problems loading the Cxx summary. The number of embedded images exceeded the throttle limits of Github. A lazy image load solution solved the problem. Now, images are only loaded if they appear in the browser window.
The web pages have many URL's that link to images and VTK doxygen pages. The long length of these URLs can increase page load times, especially on the language summary pages.
http://tinyurl.com provides a service to map the long URL's into shorted URLs. To speed up the process, we cache the long/short URLs in a file
src/Cache/TestImages.cache in the local checkout of the web repository. ScrapeRepo.py uses this cache to only convert URLs that are not in the cache. ScrapeRepo.py updates the cache after each run.
Minify HTML pages¶
mkdocs generates HTML pages from the markdown files in
doc in the local checkout of the web repository. These pages have lots of white space. We use the python package htmlmin to compress the generated HTML. The compressed pages are about 30% smaller after running htmlmin. We use the command line to process each
site in the local checkout of the web repository.
htmlmin must be installed and can be installed using
- Vet new submissions making sure the guidelines are followed.
- Merge and ...
- Update web pages after new submissions by running
src/SyncSiteWithRepo/pyin the .
In order do do this, administrators need to clone the Web Repository and follow these Web Site Maintenance Instructions.
Some notes about ScrapeRepo.py and SyncSiteWithRepo.sh¶
Given a repo_dir, site_url, web_site_url, web_repo_url, web_repo_dir and a vtk_src_dir,
ScrapeRepo.py proceeds as follows with all files being copied in the the Web Repository:
Adds thumbnails to the Cxx.md, Python.md etc. summary pages. A thumbnail is added if a src/Testing/Baseline/LANG/TOPIC/TestEXAMPLE.png image exists for the example.
Copies non-example .md files like Cxx.md and Python.md files to docs/.
For each example source file create docs/LANG/TOPIC/EXAMPLE.md file that contains:
- If src/Testing/Baseline/LANG/TOPIC/TestEXAMPLE.png exists add the image
- If a src/LANG/TOPIC/EXAMPLE.md description file exists, insert the description
- Insert the source code from src/LANG/TOPIC/EXAMPLE.EXTENSION
For each Cxx source file
- If Cxx/TOPIC/EXAMPLE.cmake exists, add it as the CMakeLists section of the .md file, otherwise, fill in the VTKCMakelists template, including the files listed in the optional src/Cxx/TOPIC/EXAMPLE.extras file.
For each Cxx source file, create a src/Tarballs/EXAMPLE.tar file containing the source and CMakeLists.txt file.
Copies across all needed files such as baseline images and other essential files like
Creates the coverage files
The VTKClassesUsedInExamples.py python script generates two tables for each language. One table lists each class and what classes it uses. The second table lists the classes that are not used in any example.
Wipes the docs and site directory
The docs directory contains all of the md and HTML files for the site. A clean directory prevents old files from being used.
The site directory contains all of the static html files for the site. A clean directory prevents old files from being used.
Runs the ScrapeRepo.py python script.
This script populates the docs directory.
Checks for a successful scrape
This sanity check sees if a reasonable number of files have been updated.
Copies the style sheets and runs mkdocs to build the static website in
Minifies the HTML