WebGL experiment: Expose line drawing

Screen Shot 2013-12-26 at 21.08.17Here’s a WebGL demo I made – when you click the canvas a hand drawn image slowly appears. Try it out:
http://udart.dk/webgl/expose_line_drawing/

The effect is achieved using a shader based technique called reaction diffusion as a mask over the image. The reaction effect ‘feeds’ on the line drawing and gradually spreads as far as possible following the black lines. My code builds upon an example by pmneila.

Posted in Demo | Tagged , | Leave a comment

Introducing Sliider – VJing with a web browser

Screen Shot 2013-11-01 at 11.26.59

Sliider is a simple yet usable application I’ve created for VJing with webGL and other interactive web based graphics. It works on Mac OSX, Windows and Linux. I am releasing an early preview version that has these features:

  • Embedded Chrome browser. 99% of what works in Chrome will work in Sliider.
  • full screen output
  • A <-> B crossfader
  • Load html files from disk or the web
  • Syphon output for the OSX version
  • Control panel with sliders and buttons
  • Drag and click in the scene with the mouse

Download Sliider preview version

Here’s a video where I demonstrate Sliider

Sliider – Demonstration from Udart (Vibeke Bertelsen) on Vimeo.

What is Sliider

Sliider is basically a Chrome browser with some added features that makes it more suited for live performance. You can create your own html pages with any graphics you like whether it is based on WebGl, canvas or SVG and Sliider will display it. Even pages with Flash content should work. By adding a few lines of code to your page javascript (see slii.inputs), you can have custom buttons and sliders in the control window of Sliider.

For developers

Sliider is open source so everything from my examples to the application itself is open for you to use or modify. The Sliider Windows version, and the Sliider fullscreen OSX version are based on Node-Webkit. Get the source code of Sliider on Github. The Sliider Syphon version for OSX is based on the Chromium embedded framework and the source code is here: CefWithSyphon on Github

Important note

Only use Sliider for trusted content. Sliider does not have the security that normally protect you when you browse the web. So don’t use Sliider for browsing cat movie sites please…

Get in touch with me

Please do get in touch if you’ve created something fun. Get in touch on my Udart Facebook page or submit bugs on the Github Sliider page.

Posted in Cool stuff | Tagged , , , , , , , , , | Leave a comment

Download Sliider preview version

Sliider preview version is released for Mac, Windows and Linux. It is open source and free. For Mac you can choose between a Syphon and a Fullscreen version. Download from the table below.

Also download a set of example visuals that demonstrate the capabilities of Sliider: Download examples

App version Download Fullscreen output Syphon output Preview window Control scene with mouse
Mac: Sliider preview Syphon
(OS X 10.8+)
Download no yes yes yes
Mac: Sliider preview Fullscreen
(OS X 10.7+)
Download yes no no no
Windows: Sliider preview Download yes no no no
Linux: Sliider preview See info below yes no no no

Quick start

  • Download the ‘Examples’ package and the Sliider app for your platform.
  • Click the left most ‘Open’ button in the control window
  • Click ‘Open file’
  • Locate the index.html in one of the Examples folders and click OK

For Linux users

There is no prepackaged version for Linux and I have not tested on Linux. You should be able to run Sliider by downloading Node-Webkit for Linux and the Sliider source. Drop the Sliider source folder on the Node-Webkit application and it should work.

Acknowledgments

Example file “Perlin Blob” derives from Perlin Noise examples by Jaume Sanchez Elias. The Voronoi shader example derives from this shader by Inigo Quilez. The Sketch example uses the osg.js framework

Posted in Software | Tagged , , , , , , | 2 Comments

Syphon into the web browser

Screen Shot 2013-06-03 at 10.20.59Yes apparently we can have a Syphon input into the web browser. It is a workaround done via Quicktime. So it has some limitations. The biggest limitation being that we can not use the Syphon input as a texture in WebGL. When I did this I prepared a Syphon server in Quartz Composer and converted it to a Quicktime file. Then I put it into my webpage like this:

<embed width="400px" height="300px" name="plugin" src="file:///Syphon%20Client%20QC.mov" type="video/quicktime">

It does seem that we can put CSS filters on the Quicktime element which I hadn’t expected (tested in Safari and Chrome). Like this:

<head>
<style>
.filtered {
-webkit-filter: hue-rotate(230deg)
saturate(300%)
blur(6px);
}
</style>
</head>
<body>
<embed class="filtered" width="400px" height="300px" name="plugin" src="Syphon%20Client%20QC.mov" type="video/quicktime">
</body>

I have also tested the awesome up-and-coming CSS custom filters feature but it does not work on videos or embedded content – yet. I have a hope that when it is implemented for the video tag it will also work for the embed tag. That would finally give us some control over a Syphon input transforming the geometry and style of it as we please.

So what is this all good for? Well basically I would like to use the web browser as a VJing tool. And for visualists Syphon is such a nice tool so this discovery is a good step in the right direction for creating a VJ tool using browser technology.

For anyone wondering this has nothing to do with streaming a Syphon source. It’s purely an example that is running on the local machine.

Posted in Cool stuff | Leave a comment

Live online remix of the maps at here.com

A remix could be many things but in this case the word refers to some modifications I made to the maps at here.com in realtime using the Node-Webkit framework. Like this:

Maps_before_after

What you see on the right is a screen-shot of the live and navigable application at here.com but interface elements have been removed and the whole maps has received a colored tint.

In other words it’s possible to modify and change the look and functionality of a live website without copying it to another server or hack the server on which the website resides (don’t do that, kids!).
I made the modifications using simple html and javascript which modifies the website when viewed in my own browser – or in this case a specialized application made using the Node-Webkit framework.

In this video I demonstrate how the modification works:

Background

As a part of my work with Obscura we do visuals for various events. For this one event it would be really appropriate to display some nice slow flyover video on the big screen of 3d maps like the ones you have on Apple maps or Google maps on iOS. I found the maps at here.com and pondered how to get them on the big screen without those navigation buttons while still being able to control camera movements.

Real time browser remix

Via javascript it’s pretty easy to modify the DOM of a web page – thus controlling how the page is displayed in your browser. and hiding an element or a section only requires one line of code. For instance you can navigate to here.com in the Chrome or Firefox browser, choose the 3D map display and bring up the Developer Tools console. There you enter this line:

window.document.getElementById("map-controls").style.display = "none";

And voila the navigation button are gone and same goes for the other UI overlays.
Changing the look of the map was easy with CSS filters that have become available in HTML5.

So using the developer tools I also figured out a bit of the code that lies below the surface on here.com and I found some methods that I could trigger and use to navigate the map even with no visible controls. Keyboard shortcuts are already implemented (arrows, shift, + and -) but I didn’t like the movements – I thought it was going to fast on a big screen.

Security Issues

Problem is I would have to put all this stuff in to the console every time I open the here.com website. And with an accidental reload of the page I would have to do it again.
So I thought about writing a document that opens here.com in a pop up window and then modifies it in the same way as we did above? Well this is called cross site scripting and there are many reasons why browser creators don’t like that.

Node-Webkit

However from other experiments I have become familiar with an application framework called Node-Webkit which provides a handy and quite easy way to create standalone applications using HTML and Javascript. And since node-webkit has a fully functional Chromium (open sourced Chrome) browser embedded I could continue my experiments in Node-webkit. And most importantly – without any security restrictions. Node-webkit allows you to dig into the DOM and javascript of any webpage you load!

Results

Here is the final Node-webkit application I made. To run it simply go to Node-webkit site and grab a binary for your platform then download my file and open it with node-webkit: Maps
My small app has plenty of bugs (I had one day before the event to figure it out) but feel free to have a look. Simply rename maps.nw to .zip and unpack it.

Final Notes

Before I end this let me show you how it looked at the event this modification was created for:

I am quite thrilled with the idea of remixing websites and as a visual performer I also find node-webkit brings a lot of potential for writing other tools that can be used for performance. With WebGL in mind this should open up a number of possibilities. And while VJing with a browser has some limitations because of the standard GUI of the browser I could easily see node-webkit applications being made for VJs or other visual artists.
So I will continue exploring this environment and would love to hear if others are doing something similar.

Posted in Software | Leave a comment

Getting started with WebGL

WebGL Experiment in three.js

WebGL Experiment in three.js

Here’s the thing that takes up my spare time right now: WebGL. For me with my web developer background (PHP, javascript, .NET) and my growing interest in realtime 3D graphics it’s so obvious that I can’t believe I haven’t played around with it before. The good thing is that at this point there are plenty of resources for those who want to learn and that it has been implemented in a stable way in some major browsers.
The downside is that it’s not an obvious choice for VJs as it does not integrate with VJ apps. But I’m sure tools will appear over time to overcome this.

Just have a look at some of the mind-blowing visual possibilities WebGL brings for visualists:
Lights
Ro.me
Blast
Often when delving into a new programming environment the hard part comes right at the beginning when you try to get a grasp of the architecture of things and the tools needed. I have been playing around with webGL for about a week now and I am slowly getting over that first stony hillside I needed to climb before being able to code anything useful.

So I thought I’d share what I found useful along the way in order to get up and running.

1. Programming environment

In theory all you need is a text editor and a browser. But I know from experience that a real code editor is useful to help keep everything readable. So far I am quite pleased with Sublime Text. As for the browser I chose Chrome for development, but it could probably just as well be Firefox. From what I’ve heard Opera and Safari do not quite live up to the others yet and as for Internet Explorer – who knows?. Make sure to have the latest version of the browser.

2. Web server

Yes a web server is needed, loading up pages by simply opening them locally is only going to work for the simplest of coding examples. Fortunately I am on a Mac and got a long way using the built in Python http-server. Here’s a quick how-to:
Open the Terminal app, write ‘cd ‘ and drag the folder with your code files into the Terminal window. Now it says ‘cd /Users/You/Path/To/Your/Folder/’. Make sure there’s a space after cd. Enter. Then write ‘python -mSimpleHTTPServer’. Enter. Now the Terminal should respond ‘Serving HTTP on 0.0.0.0 port 8000′. Go into Chrome and write ‘http://localhost:8000′ as the address. You should now see the contents of your folder as clickable links.

3. WebGL API

webGL is supported by the above mentioned browsers out of the box. However I use an API for accessing webGL as I find the objects and functions of the API hide some of the complexities and provide me with helpful shortcuts to get things done. One of the most popular at the moment seems to be Three.js and so far I am enjoying using it a lot. There are a lot of good resources and examples on Three.js on the net right now.

4. Getting started with three.js

First I followed this tutorial. Then I tried to code something myself. I found out that the documentation is of some use but it’s very incomplete at the moment. So I did what was recommended and dug into the three.js examples (they are in the examples folder of the three.js download). I start out by looking at an example that does something close to what I want to achieve and then I copy the file and start messing with the code. Unfortunately I have often found the examples to be overly complex – especially when I started out – things are starting to make more sense now. I keep peeling away the stuff that I don’t need in the code, while continually checking in the browser what effect my change is having until I have a minimal working example. Then and only then do I start adding my own code. I may also try to combine code from different examples to achieve what I want.

5. Debugging javascript

WebGL is all about javascript and fortunately Chromes developer tools are really good for telling what’s going on and what is going wrong. cmd+option+I brings up the panel. Console shows the error messages but I have also had good use of the Watch Expressions and the Break Points. Here’s a good article to get started with the debugger tools.

6. Figuring out the error messages

For many things I understand what’s wrong by simply clicking the link in the console and see where in the code the error is. But for the more cryptic error messages Google is my friend. It may seem obvious but simply googling ‘webGL three.js’ + the error message will often bring up something useful. Especially the questions and answers on stackoverflow.com – no wonder as the developer of three.js moved all support to this site. For some browser specific errors searching for ‘chrome’ + error text will bring up an answer.

7. What WebGL isn’t

First of all WebGL builds upon OpenGL but it’s not the same. So reading up OpenGL won’t be directly applicable – especially so if you’re using an API like three.js. OpenGL code examples from other contexts may be portable but not without some translation. If you’re new to the whole realm of realtime 3D graphics including openGL it would probably be useful to read a primer on the general concepts – although one big advantage of three.js lies in the way it uses words and concepts that will be familiar to you if you have played with 3D applications such as 3D Studio, Maya or Cinema4D.

So that’s it from me for now. I don’t offer a complete webGL tutorial just these few tips for now. If you are already up and running with WebGL feel free to add in the comments any other similar tips that helped you when you started out.

Posted in Tips and tutorials | Tagged , , , | Leave a comment

Lost In Uncanny Valley

That’s what I call my current visual explorations. There’s something endlessly fascinating to me in these 3D humanoid characters and their movements so I will definitely continue to explore this theme through animations, VJ performances etc. Here’s a few animated GIFs (I hope they play in your browser) plus a video from a recent performance in Copenhagen as a part of Obscura – the group of people I’m working with.

Ben hanging

Ryan head

Alyson aerobics 2

Live music performance by ir (aka. Jonas Olesen), visuals by Udart. Copenhagen 2012

Posted in Portfolio | Tagged , , , , , | Leave a comment

Finding the right beat

Waveclock - BPM detection

Waveclock - BPM detection

At those (admittedly few) occassions when I VJ alongside DJ I have often wished for a tool that would detect the tempo of the music for me. The more I searched for a Mac based solution the more I realized there is some pretty complex music analysis behind beat detection and that few or no freeware tools existed. Vj apps seem to have settled on a tap-the-beat solution and most of the existing BPM detectors are not real time as they only operate on prerecorded audio files.
Then someone posted a link to this ingenious little app in the Modul8 forums and I have been using it ever since. It picks up the beat after a few seconds and it’s pretty accurate – as long as there is a steady discernible beat to the music of course.

Waveclock

You try it out in demo mode (and click away a dialog box every twenty minutes). It’s an independent app so apart from Modul8 it ought to work with any VJ app that supports midi clock.

1. First you go to the Waveclock preferences and select your microphone or line-in for the input. And some midi channel for the output – IAC Driver for instance.

2. Then in Modul8 or your other VJ app you go to preferences and confirm that IAC Driver is activated as a midi input.

3. Then (in Modul8) you activate the BPM modul and click ‘use midi clock’.

Posted in Tips and tutorials | Leave a comment

Using cues, timelines and VJ software for performances

These days as a part of the Obscura crew I am ofting doing visual shows that have more in common with theatre productions than traditional VJing. Often there is some sort of script for a show or a piece of music that is rehearsed. For these types of jobs I have found good use for the application called QLab which is great for triggering a pre planned sequence of events.

Actually the whole point of QLab is to line up a sequence of audio, video files or other types of events and being able to trigger them with a simple ‘Go’ button at the exact right time during the show.
QLab was originally developed with audio in mind and has later been extended with video features so it does not have many features as VJ applications when it comes to live manipulations of images and video. But for those of us who sometimes wish for a timeline in our favorite VJ application, QLab may be an option if you set it up to run alongside your VJ app.
For our shows we typically use Modul8 and I found two ways to achieve this.

1. Capturing the video output of QLab and using it as input in Modul8. This is done with two computers – one with a video capture card such as the Blackmagic Intensity hardware or the Matrox MXO2 mini and the other computer running QLab. With this method you run some images or videos using QLab and you are then able to manipulate them live in Modul8 as you would do with any other source.

2. Setting up QLab to send midi and mapping the buttons in Modul8 so that they respond to the midi commands. This can be done using one computer running both apps or using two computers and a midi connection between them. In this scenario QLab is merely the ‘conductor’ telling Modul8 what to do. Modul8 holds all the images or videos used.

qlab setup

The advantage of these two methods is that you have some way of combining improvised VJing with a sequence of planned cues. For instance – as we did recently you can use the first method to trigger a series of slides containing the lyrics of a song and then use Modul8 to distort and manipulate them. Furthermore as this setup involves two computers two people are able to share this task – one person triggering the lyrics at the right time and one person adding effects. This could be done using any VJ application that is compatible with the video capture card not just Modul8.

Posted in Tips and tutorials | Tagged , , , , , , | 4 Comments

Concrete forest

Concrete Forest – an art project that turned into a party from Obscura on Vimeo.

The video shows the result of a fun little project we did in collaboration with artist Louise Bech Pedersen. We cut out these animal figures of a special acrylic rear projection material and used them for mapping using Modul8 and MadMapper.
Doing the mapping was trivial using these tools – it was the acrylic material itself that proved to be a bit of a challenge. The light projection properties of the material were terrific, a clear and bright image from all angles. For people like us who are used to projecting onto fabric or walls it was a joy to see the brightness of this professional screen material.
However the material was a quite brittle and cracks appeared a few places even when using a saw made for use with acrylics. And the deer lost the tip of the antlers when one of the party goers tried to bring it down with an empty vodka bottle!

Posted in Portfolio | Tagged , , , | Leave a comment
  • Categories

  • Videos

  • Facebook



  • About me

    Vibeke Bertelsen is a visual artist based in Copenhagen who specializes in motion graphics and video projections for live music and other events. Per 2010 I am a part of the group called Obscura working full time doing large scale visuals and video installations. Contact me on