I have created the official video for this beautiful track by artist Rumpistol:
The animation is a part of a series of different virtual sculptures I made of human bodies and faces. Often in games and 3D art the human body is idealized into sexy or heroic archetypes but in this project the two heads are distorted and glitched in ways that create more unfamiliar expressions that sometimes dissolve into pure absctract shapes.
The animation is created in WebGL using the popular library Three.js and in order to precisely control the progression of the visuals multiple tracks of keyframes are loaded into the application in order to control animation parameters in sync with the music.
The piece is an adaptation of some of the material, that I also use when I perform live as a VJ at concerts. I developed a custom VJ application (Sliider) where the same WebGL code is loaded without preprogrammed keyframes. Instead the VJ application allows all parameters to be controlled via midi or OSC.
See the animation: here
Here’s a WebGL demo I made – when you click the canvas a hand drawn image slowly appears. Try it out:
The effect is achieved using a shader based technique called reaction diffusion as a mask over the image. The reaction effect ‘feeds’ on the line drawing and gradually spreads as far as possible following the black lines. My code builds upon an example by pmneila.
Sliider is a simple yet usable application I’ve created for VJing with webGL and other interactive web based graphics. It works on Mac OSX, Windows and Linux. I am releasing an early preview version that has these features:
- Embedded Chrome browser. 99% of what works in Chrome will work in Sliider.
- full screen output
- A <-> B crossfader
- Load html files from disk or the web
- Syphon output for the OSX version
- Control panel with sliders and buttons
- Drag and click in the scene with the mouse
Download Sliider preview version
Here’s a video where I demonstrate Sliider
What is Sliider
Sliider is open source so everything from my examples to the application itself is open for you to use or modify. The Sliider Windows version, and the Sliider fullscreen OSX version are based on Node-Webkit. Get the source code of Sliider on Github. The Sliider Syphon version for OSX is based on the Chromium embedded framework and the source code is here: CefWithSyphon on Github
Only use Sliider for trusted content. Sliider does not have the security that normally protect you when you browse the web. So don’t use Sliider for browsing cat movie sites please…
Get in touch with me
Sliider preview version is released for Mac, Windows and Linux. It is open source and free. For Mac you can choose between a Syphon and a Fullscreen version. Download from the table below.
Also download a set of example visuals that demonstrate the capabilities of Sliider: Download examples
|App version||Download||Fullscreen output||Syphon output||Preview window||Control scene with mouse|
|Mac: Sliider preview Syphon
(OS X 10.8+)
|Mac: Sliider preview Fullscreen
(OS X 10.7+)
|Windows: Sliider preview||Download||yes||no||no||no|
|Linux: Sliider preview||See info below||yes||no||no||no|
- Download the ‘Examples’ package and the Sliider app for your platform.
- Click the left most ‘Open’ button in the control window
- Click ‘Open file’
- Locate the index.html in one of the Examples folders and click OK
For Linux users
There is no prepackaged version for Linux and I have not tested on Linux. You should be able to run Sliider by downloading Node-Webkit for Linux and the Sliider source. Drop the Sliider source folder on the Node-Webkit application and it should work.
Yes apparently we can have a Syphon input into the web browser. It is a workaround done via Quicktime. So it has some limitations. The biggest limitation being that we can not use the Syphon input as a texture in WebGL. When I did this I prepared a Syphon client in Quartz Composer and converted it to a Quicktime file. Then I put it into my webpage like this:
<embed width="400px" height="300px" name="plugin" src="file:///Syphon%20Client%20QC.mov" type="video/quicktime">
It does seem that we can put CSS filters on the Quicktime element which I hadn’t expected (tested in Safari and Chrome). Like this:
<embed class="filtered" width="400px" height="300px" name="plugin" src="Syphon%20Client%20QC.mov" type="video/quicktime">
I have also tested the awesome up-and-coming CSS custom filters feature but it does not work on videos or embedded content – yet. I have a hope that when it is implemented for the video tag it will also work for the embed tag. That would finally give us some control over a Syphon input transforming the geometry and style of it as we please.
So what is this all good for? Well basically I would like to use the web browser as a VJing tool. And for visualists Syphon is such a nice tool so this discovery is a good step in the right direction for creating a VJ tool using browser technology.
For anyone wondering this has nothing to do with streaming a Syphon source. It’s purely an example that is running on the local machine.
What you see on the right is a screen-shot of the live and navigable application at here.com but interface elements have been removed and the whole maps has received a colored tint.
In other words it’s possible to modify and change the look and functionality of a live website without copying it to another server or hack the server on which the website resides (don’t do that, kids!).
In this video I demonstrate how the modification works:
As a part of my work with Obscura we do visuals for various events. For this one event it would be really appropriate to display some nice slow flyover video on the big screen of 3d maps like the ones you have on Apple maps or Google maps on iOS. I found the maps at here.com and pondered how to get them on the big screen without those navigation buttons while still being able to control camera movements.
Real time browser remix
window.document.getElementById("map-controls").style.display = "none";
And voila the navigation button are gone and same goes for the other UI overlays.
Changing the look of the map was easy with CSS filters that have become available in HTML5.
So using the developer tools I also figured out a bit of the code that lies below the surface on here.com and I found some methods that I could trigger and use to navigate the map even with no visible controls. Keyboard shortcuts are already implemented (arrows, shift, + and -) but I didn’t like the movements – I thought it was going to fast on a big screen.
Problem is I would have to put all this stuff in to the console every time I open the here.com website. And with an accidental reload of the page I would have to do it again.
So I thought about writing a document that opens here.com in a pop up window and then modifies it in the same way as we did above? Well this is called cross site scripting and there are many reasons why browser creators don’t like that.
Here is the final Node-webkit application I made. To run it simply go to Node-webkit site and grab a binary for your platform then download my file and open it with node-webkit: Maps
My small app has plenty of bugs (I had one day before the event to figure it out) but feel free to have a look. Simply rename maps.nw to .zip and unpack it.
Before I end this let me show you how it looked at the event this modification was created for:
I am quite thrilled with the idea of remixing websites and as a visual performer I also find node-webkit brings a lot of potential for writing other tools that can be used for performance. With WebGL in mind this should open up a number of possibilities. And while VJing with a browser has some limitations because of the standard GUI of the browser I could easily see node-webkit applications being made for VJs or other visual artists.
So I will continue exploring this environment and would love to hear if others are doing something similar.
The downside is that it’s not an obvious choice for VJs as it does not integrate with VJ apps. But I’m sure tools will appear over time to overcome this.
Just have a look at some of the mind-blowing visual possibilities WebGL brings for visualists:
Often when delving into a new programming environment the hard part comes right at the beginning when you try to get a grasp of the architecture of things and the tools needed. I have been playing around with webGL for about a week now and I am slowly getting over that first stony hillside I needed to climb before being able to code anything useful.
So I thought I’d share what I found useful along the way in order to get up and running.
1. Programming environment
In theory all you need is a text editor and a browser. But I know from experience that a real code editor is useful to help keep everything readable. So far I am quite pleased with Sublime Text. As for the browser I chose Chrome for development, but it could probably just as well be Firefox. From what I’ve heard Opera and Safari do not quite live up to the others yet and as for Internet Explorer – who knows?. Make sure to have the latest version of the browser.
2. Web server
Yes a web server is needed, loading up pages by simply opening them locally is only going to work for the simplest of coding examples. Fortunately I am on a Mac and got a long way using the built in Python http-server. Here’s a quick how-to:
Open the Terminal app, write ‘cd ‘ and drag the folder with your code files into the Terminal window. Now it says ‘cd /Users/You/Path/To/Your/Folder/’. Make sure there’s a space after cd. Enter. Then write ‘python -mSimpleHTTPServer’. Enter. Now the Terminal should respond ‘Serving HTTP on 0.0.0.0 port 8000′. Go into Chrome and write ‘http://localhost:8000′ as the address. You should now see the contents of your folder as clickable links.
3. WebGL API
webGL is supported by the above mentioned browsers out of the box. However I use an API for accessing webGL as I find the objects and functions of the API hide some of the complexities and provide me with helpful shortcuts to get things done. One of the most popular at the moment seems to be Three.js and so far I am enjoying using it a lot. There are a lot of good resources and examples on Three.js on the net right now.
4. Getting started with three.js
First I followed this tutorial. Then I tried to code something myself. I found out that the documentation is of some use but it’s very incomplete at the moment. So I did what was recommended and dug into the three.js examples (they are in the examples folder of the three.js download). I start out by looking at an example that does something close to what I want to achieve and then I copy the file and start messing with the code. Unfortunately I have often found the examples to be overly complex – especially when I started out – things are starting to make more sense now. I keep peeling away the stuff that I don’t need in the code, while continually checking in the browser what effect my change is having until I have a minimal working example. Then and only then do I start adding my own code. I may also try to combine code from different examples to achieve what I want.
6. Figuring out the error messages
For many things I understand what’s wrong by simply clicking the link in the console and see where in the code the error is. But for the more cryptic error messages Google is my friend. It may seem obvious but simply googling ‘webGL three.js’ + the error message will often bring up something useful. Especially the questions and answers on stackoverflow.com – no wonder as the developer of three.js moved all support to this site. For some browser specific errors searching for ‘chrome’ + error text will bring up an answer.
7. What WebGL isn’t
First of all WebGL builds upon OpenGL but it’s not the same. So reading up OpenGL won’t be directly applicable – especially so if you’re using an API like three.js. OpenGL code examples from other contexts may be portable but not without some translation. If you’re new to the whole realm of realtime 3D graphics including openGL it would probably be useful to read a primer on the general concepts – although one big advantage of three.js lies in the way it uses words and concepts that will be familiar to you if you have played with 3D applications such as 3D Studio, Maya or Cinema4D.
So that’s it from me for now. I don’t offer a complete webGL tutorial just these few tips for now. If you are already up and running with WebGL feel free to add in the comments any other similar tips that helped you when you started out.
That’s what I call my current visual explorations. There’s something endlessly fascinating to me in these 3D humanoid characters and their movements so I will definitely continue to explore this theme through animations, VJ performances etc. Here’s a few animated GIFs (I hope they play in your browser) plus a video from a recent performance in Copenhagen as a part of Obscura – the group of people I’m working with.
Live music performance by ir (aka. Jonas Olesen), visuals by Udart. Copenhagen 2012
Then someone posted a link to this ingenious little app in the Modul8 forums and I have been using it ever since. It picks up the beat after a few seconds and it’s pretty accurate – as long as there is a steady discernible beat to the music of course.
You try it out in demo mode (and click away a dialog box every twenty minutes). It’s an independent app so apart from Modul8 it ought to work with any VJ app that supports midi clock.
1. First you go to the Waveclock preferences and select your microphone or line-in for the input. And some midi channel for the output – IAC Driver for instance.
2. Then in Modul8 or your other VJ app you go to preferences and confirm that IAC Driver is activated as a midi input.
3. Then (in Modul8) you activate the BPM modul and click ‘use midi clock’.