Current Projects

Alchemy VR Tools

  • work in progress
  • freelance
  • C++
  • Java
  • Android
  • node.js
  • Oculus Mobile
  • VR

Alchemy VR is a recently formed branch of Atlantic Productions. Their upcoming VR premier "Life on Earth" (narrated by David Attenborough) is to be shown on 80 Gear VRs on the 12th of June (2015).

Alchemy asked me to solve a number of critical problems within a short time frame to meet their June premier deadline.

Oculus's Gear VR software is still in a premature state and a suitable VR player simply didn't exist. To make matters worse, the 80 Samsung S6 phones they'd been sent to drive the Gear VRs were locked down with Oculus's consumer software, this meant all interactions (including loading apps and playing films) had to go through the Oculus Home app. There wasn't enough time between reciving the devices and the premier to request changes from Samsung or Oculus so it was necessary to develop an in-house solution. To top it off, they needed to be able to remotely manage all 80 devices, with the ability to play and pause all devices simultaneously.

I was tasked with developing a custom VR video player, bypassing the oculus home store and developing a remote management system to control all the devices

Project Status

I've completed the custom VR video player (developed using C++, Java Native Interface and Oculus's Mobile SDK).

Alchemy now has full control over what's displayed on their devices. Since rooting the devices was out of the question, I discovered it was possible to steal the context away from the Oculus Home app with a background service.

The remote control system is under development using haxe and node.js.

SuperSL

  • work in progress
  • GLSL
  • C
  • Haxe
  • LALR

SuperSL is an alternative shader programming language designed for the web. It's a superset of GLSL that aims to smooth out the process of writing shaders in browser environments. The key features of SuperSL are:

  • Compile-time type casting and inference
  • Automatic function prototype declarations
  • New directive #include for modular shaders
  • Function type for function reference passing
  • Functions within functions
  • Default function arguments

Motivation

GLSL, the OpenGL shader programming language was introduced in 2004. The necessity for driver implementation simplicity and GPU hardware limitations constrained the language's features and born out of a time where most OpenGL programs were written in C or C++, GLSL was modeled after C.

Fast-forward to 2013 and WebGL reaches its first stable release, using GLSL as the shader language. In the dynamically-typed Javascript environment of the browser, GLSL feels out of place with its strict typing and lack of flexibility which can make shader writing difficult for newcomers and add inertia to shader development. For example, the expression 1.0 * 2 is forbidden in GLSL: 1.0 is a float whereas 2 is an int. Most languages will automatically cast the int to a float but GLSL doesn't, instead it reports a type-mismatch. Another issue is that GLSL can only see functions that have been defined above the current point of execution, to call a function below requires defining a function prototype somewhere above. In addition to its language weaknesses, driver implementations vary from machine to machine and without warning - GLSL can be valid and execute on one machine, but fail on another. With a SuperSL to GLSL compile step, SuperSL can improve GLSL's language features and normalize cross-platform differences.

Details

SuperSL can be used to generate GLSL either at project compile time (like LESS or SASS) or at runtime. To make this possible with a single codebase SuperSL is written in haxe and compiled to Javascript, C++ and Java (and potentially other platforms if the need arises).

SuperSL uses a LALR parser based on the parser in lemon parser generator

Project Status

The project is under heavy development at the moment. The core parser has been completed and work is underway on the validator and auto-typing system.

TL;DR

GLSL isn't ideal for browser based shader development, SuperSL is a superset of GLSL with additional language features to resolve some of GLSL shortcomings and make writing shaders for WebGL easier. In terms of features, SuperSL strikes a balance between GLSL's strictness and Javascript's flexibility without comprising on performance.

AST generation with the core parser
Past Projects

WebGL Volume Renderer

  • freelance
  • WebGL
  • GLSL
  • Javascript
  • haxe
  • three.js

The Met Office's newly formed Informatics Lab is developing a browser-based interactive weather data visualisation tool. The Met's supercomputers generate colossal 3D data fields that cover a volume of atmosphere above the UK. The Aim is to make this data accessible for anyone to view and study in the browser. My involvement in the collaboration is to help solve some of the many technical changes involved in rendering animated volumetric data with WebGL.

Technical Challenges

Achieving high performance with a ray-marched self-shadowing volume renderer is a significant challenge in itself, especially when targeting mobiles, but compounding that challenge is the requirement for the data to be animated and interactive. An easy technique to speed up rendering performance is to cache the lighting occlusion field for the volume in a texture, this would work well assuming our data and light source remain static, however the data will be changing rapidly as the weather fields are animated. To keep the animation smooth we could precompute the occlusion at startup, but this isn't ideal either - we're expecting about 360 frames of animation, if an occlusion pass takes 16ms per frame, then precomputing lighting costs 5.8 seconds at startup! The next option is to compute lighting on the server and bundle it with the weather data, but will the additional download time introduced be really less than the time computing it on the client?

There's a requirement for the user to be able to fly through the data, this is tough - if the user is getting close to the data, the lighting occlusion resolution will have to be increased to create a convincing effect. Additionally, to reduce bandwidth the weather data is fairly coarse so we'll need to layer noise onto the cloud density field - necessitating client-side lighting computation.

Project Status

The Informatics Lab have developed a prototype volume renderer but the performance isn't ideal for public release. My challenge as a contracted developer was to explore techniques to get it running at fast as possible.

An early volume rendering experiment

WebGL Fluid

  • WebGL
  • GLSL
  • Haxe
  • Javascript
  • Fluid Dynamics

The aim of this project was to study using WebGL for high performance simulations and to investigate the performance factors involved.

The simulation solves the Navier-Stokes equation for incompressible flow over a grid with the Jacobi method. Each simulation step is handled as follows:

  • Self-advect the velocity field
  • Handle user interaction with the velocity field
  • Iteratively solve the pressure term of the Navier-Stokes equation to enforces incompressibility
  • Handle user interaction with the dye field
  • Advect dye field using velocity field
  • Advance particles by simulating drag by the fluid

Solving 2D partial differential equations over a grid is a fairly straight forward process in WebGL. Field values are stored in a texture and passed to the GPU, the solver iteration step renders the result to a new texture which is fed back as input in the next step. The old texture then becomes the target for rendering.

Handling particles is less elegant, the particle properties are stored in a texture and a vertex buffer is used to store the texture coordinates of each particle's data. In a simulation step, the particle data is updated by a render-to-texture flip-flop (explained earlier). To render the particles, first a vertex shader uses the coordinate buffer to select particle data from the data texture and uses it to set the position of the rendered vertices, then a fragment shader then draws (and colors) the vertices using GL_Points mode.

Reception

Shortly after posting it front-paged reddit, from there it was picked up by popular content aggregators, including:

And since it was open source, it's even made its way into the landing page of Miller Genuine Draft

To date it's been played with 1.8m times by about 1.5m users.

Hive

  • Objective-C
  • Reverse Engineering
  • UI & UX
  • Javascript
  • Startup

Hive was a team collaboration app I worked on with a small group during university. It was an experiment in developing the ideal collaboration tool. The philosophy was that team cohesion could be improved by reducing the boundaries between computers; the goal was to be able to push content (including running programs) from one device to a teammate’s immediately and intuitively (in a similar manner to moving windows between multiple displays).

'State-sharing' was one of the key technologies we experimented with, this exploited OS X's built in application state saving and restoring capability to transfer an application's state between computers. The result was that you could take any window on your machine and move it to another and it would appear in just the same state on the new machine.

After winning Cisco Open Collaboration Challenge and 3 months of acceleration at dotforge we chose to abandon the project. The primary cause being the reliance on hidden API's within OS X, this meant not only would we be barred from the app store, but that Apple had a significant advantage in developing similar technology - over the course of the acceleration apple released Yosemite with Handoff which behaved similarly to and was based on the same principles as state-sharing.

Hive prototype UI designs

EventSpace

  • UI & UX
  • Javascript
  • Fontent Design
  • Startup

I was asked to do some iPad and browser concept designs for an early stage startup in Sheffield called 'EventSpace'. EventSpace was an event companion managed by the organisers. Its aim was to allow easy communication between attendees and enable instant feedback to organisers.