Now that browser-based games running on touch devices are becoming a reality, it’s time to figure out how to implement easy to use and tactile game controls in these new keyboard-less environments.
I’ve long been a fan of how the twin-stick touch controls work in Geometry Wars Touch, so I wasn’t surprised to find that Brighton game UX expert Graham McAllistair also considered this “dynamic display on touch” interface to be the best option for analogue controls.
So last week at my creativeJS training course in Brighton (only four tickets left for the San Francisco event ๐ ) we started experimenting with multi-touch points in JavaScript with the aim of making a game controller that works on iPhone and iPad.
The natural first step : figure out how to get the touch data in JS. It’s actually pretty easy – there are three events that are broadcast by any DOM element that can be touched, touchstart, touchmove and touchend. But before you do that, you probably want to check that you’re running in a touch-able environment. And you can do that by checking that createTouch is a property of the document:
var touchable = 'createTouch' in document;
If this is true, you can add event listeners to your canvas element like this :
if(touchable) { canvas.addEventListener( 'touchstart', onTouchStart, false ); canvas.addEventListener( 'touchmove', onTouchMove, false ); canvas.addEventListener( 'touchend', onTouchEnd, false ); }
And then you define each of the functions specified :
function onTouchStart(event) { //do stuff } function onTouchMove(event) { // Prevent the browser from doing its default thing (scroll, zoom) event.preventDefault(); } function onTouchEnd(event) { //do stuff }
In each touch listener, event is the object that provides all the touch data, and it contains three arrays:
event.touches : all of the current touches
event.targetTouches : all the touches that originated from the DOM element
event.changedTouches : only the touches that triggered the event.
For touchstart and touchend, changedTouches usually only contains one touch event (unless you just happened to touch two fingers at exactly the same time).
Each array contains touch data objects with the following properties :
identifier: An unique number that allows you to track touch events as they move
target: the DOM element that broadcast that event
clientX, clientY: touch position relative to the viewport
screenX, screenY: touch position relative to the screen
pageX, pageY: touch position relative to the full page
Call event.preventDefault(); on a touchmove event to disable the automatic scrolling you normally get on touch devices.
I found this full explanation on SitePen really helpful.
So a quick test for iPad revealed that you could have 11 concurrent touch points (cue infantile jokes about what to use after you’ve run out of fingers…)
Try it here on a touch device or if you don’t have one handy you can watch this video:
So how to turn this into a game controller? Easy! If we pick up a touch down on the left side of the screen, we’ll use that as the centre point of our analogue control, and keep track of that touch ID. I use my 2D vector class to work out the vector between the current touch position and the original touch down position. We can then use that difference vector to dictate the velocity of the ship.
While we have a touch on the left we can ignore other touches on that side of the screen. If we get a touch on the right side, we assume that’s a fire button. It’s a pretty simple system.
Try it here on a touch device.
I’ve started to optimise this for iOS by converting the ship into a single canvas that I’m moving around, but didn’t finish yet. Currently the bullets and the touch circles are being drawn into canvas every frame, so it’s not quite as performant on the iPad1 (pre iOS update) as it could be.
The source is on git hub – I expect I’ll be improving it over the coming months, but please feel free to fork it if you have any suggestions!
33 replies on “Multi-touch game controller in JavaScript/HTML5 for iPad”
Seb…
That’s freaking awesome. I appreciate you taking the time to lay out your train of thought for this project. It’s interesting to see how you approached it.
Looking forward to seeing your sessions at D2WC. Wish I could come early for your JS workshop.
Hi Andy! Such a shame you can’t make the KC course! Glad you like this ๐
Seb
Great stuff, thanks Seb.
I forked your GitHub repo and added a coffeescript ( //jashkenas.github.com/coffee-script/ ) version of your JS.
For the curious here is my branch:
//github.com/mattetti/JSTouchController/tree/coffeescript
Thanks,
– Matt
Dude! this is just perfect for a canvas-based Asteroids game I’ve (almost) finished writing. I was wondering how to get it working in a touch environment, and this looks like a great solution. I will ping you back if it works out.
Very cool! I have also been wondering how to translate my Asteroids game to multitouch. This looks like the way to go.
Looking forward to your talk at FITC!
[…] Multi-touch game controller in JavaScript/HTML5 for iPad (via JavaScript Weekly) Videos from the Day of JS on Mobile back in January have now been posted. […]
nice one, been playing with touch events myself recently in testing opera mobile 11 on android. tiny change suggestion to your touches example: i usually preventDefault also on touchstart, as (in opera mobile 11 at least) otherwise i get the context menu if i long-click / tap-hold without moving, and without it it also seems to have issues with some combinations of touches.
[…] See the rest here: HTML5 Blog on: Mark Bolgiano: Multi-touch game controller in JavaScript/HTML5 for iPad | Seb Lee-Del… […]
[…] spacebar to fire) and I also added some mouse/touch control inspired by Seb Deslisle’s JSTouchController. I don’t think the game is really well suited to touch, but at least I learned how to handle […]
[…] are easy to set up. The code I used for my touch powered buttons I mostly just stole from this post by developer Seb Lee-Delisle. The 3D stuff I mostly learned from this article on 24 Ways by David […]
nice post. just a little typo in the code
in onTouchMove(event), this is event.preventDefault(); instead of e.preventDefault();
Thanks Jerome! *fixed* ๐
And here without canvas … ohnly javascript and css ๐
//www.agentur-obermaier.de/.dev/
Hi Seb. I am currently working on an HTML5 iPhone (or any smartphone, but mainly iPhone) controller for remote controlling games.
I came accross a major problem with the touchEvents on Safari Mobile and would like to know if you had the same problem/noticed it since what you made is the closest from what I’m doing that I could find on the web.
When holding a first finger on the screen, and holding it still (not triggering any touch event with it), and touching the screen with a second finger, it does not trigger a touchstart event. I have to trigger a second touchevent (for example a touchmove with the other finger or a touchend with the second one) for it to trigger the first (touchstart) event. Same goes with the next events, they’re triggered once the next event is made, etc. This is really embarassing because when holding the stick with one finger, and triggering a button with the second, the button sometimes gets stuck or does not trigger at all until I do a touchmove.
If you encountered this problem and found a workaround, please let me know !
I tried sending fake touchmove events at short intervals, but it does not do the trick, I guess it’s a problem at a lower level.
Hi Benjamin,
I haven’t had that problem on iPhones, although many Android phones have very poor touch screen drivers, and Android browser only supports two touch points (from memory). iOS has always seemed really solid though. I just tried what you suggested on this test page //sebleedelisle.com/demos/JSTouchController/Touches.html and it always seemed to get touch starts, but sometimes if the first touch remained still, it wouldn’t always get touch end events. But that’s less of an issue I’m assuming?
I wonder if Mobile Safari has started implemented long touch events – it may be something to do with that. But other than that, I’d double check your code ๐
Cheers!
Seb
I tried your game too, and I get the same problem ๐
What you describe is exactly the problem I have, the touchEnd is not triggered because when you end the touch, it triggers the touchstart event that was on hold before. It IS a big issue, since I have been testing my remote with JSNES (a javascript NES emulator), not getting the touchend is a big issue on a game like Super Mario Bros. for example.
I reported the bug to Apple, I hope they fix this in the near future. In the meantime, if you have an idea for a workaround, please let me know ๐
I really think it might be something to do with the gesture events they’ve implemented. //m14i.wordpress.com/2009/10/25/javascript-touch-and-gesture-events-iphone-and-android/
Hi seb.
Just noticed that in your examples, you’re using an invalid code for the viewport. You’re using “;” instead of “,” : //developer.apple.com/library/ios/#DOCUMENTATION/AppleApplications/Reference/SafariWebContent/UsingtheViewport/UsingtheViewport.html#//apple_ref/doc/uid/TP40006509-SW26
Right now, it works in Safari iOS and some others browsers but it’s not future-proof ๐
Nice catch, thanks Thomas! Should be fixed now.
Could you replace the ship and side controls with pngs?
of course!
This is freaking awesome! I just googled html5 javascript and ipad and found this. Anyone using rails with this? I have some webaps built in ruby on rails. Wonder how to access this using that. Suppose everytime you shoot you have to count your bullets. Would it be possible to store the numer of bullets in a rails model called bullets?
If anybody is using rails and know how to approach this let me know.
Thanks for sharing.
Ok, that just blew my mind. Woow!
[…] demo mainly consists of two open source projects, Bandit racer from Domas Lapinskas, and the multi-touch game controller by Seb Lee-Delisle. With those projects in mind it was just a matter of tying the controls together […]
[…] that Seb Lee-Delisle had already digested this and has created this awesome concept described in Multi-touch game controller in JavaScript/HTML5 for iPad . The code is available on GitHub here: […]
Just wanted to clarify that this works great on Android 2.x+ as well! For most Android 2 devices though, you’ll need a true multitouch-enabled browser that uses //github.com/Philzen/WebView-MultiTouch-Polyfill. As i couldn’t find any browser in the store doing this, i created one for free download: //www.androidpit.de/de/android/market/apps/app/com.changeit.mtbrowser
I took the liberty to hardcode a link to your online example in that browser, as it’s still one of the neatest applications to visualise multitouch in an HTML app.
[…] Multi-touch game controller in JavaScript/HTML5 for iPad El mรฉtodo requestAnimationFrame […]
I ported Seb’s code to work on Windows 8 touch devices: //masaez.com/multi-touch-game-controller-in-javascript-html5-for-windows-8
Is there a way to stop the ships movement dependent on a “collision” with a static image on the canvas?
[…] other custom touch interactions, this is hard to get right. Seb Lee-Delisle has a great write up on Multi-touch game controller in JavaScript/HTML5 for iPad, so I adapted his design to be easily reusable in your WinJS HTML5 […]
[…] Moveable Joystick:ย This example is probably the coolest thing I’ve found, though it may be difficult to add in a hittestobject type of code. Basically you’re working with a joystick and a button, but the joystick is not onscreen and appears wherever you feel most comfortable placing your thumb, so long as it is on the left half of the screen. […]
[…] no gamepad is detected, Joystix next uses Seb Lee-Delisle's JavaScript multi-touch game controller. This works by drawing a thumbstick wherever the user touches (and holds) the screen on the left, […]
[…] Pointer plotter (adapted from Seb) […]