Modern technology has changed many things in our lives, including the way we communicate, travel and entertain ourselves. Electronic instruments and computer simulations have revolutionised science. Mathematics, one of the purest forms of human logic and reasoning, has also been changed by computer approaches. Even art has been undergoing a deep upheaval in the way it is created and appreciated, using the fast processing and graphical output of computers. The boundary between artist, computer programmer, and mathematician is becoming ever more blurred. In this article, Lewis Dartnell leads us through some examples of this exciting new wave of digital art.

New possibilities

Art was once restricted to static creations, such as an oil painting, sculpture or photograph, produced by the artist and then passively regarded by the viewer. Nowadays, however, computer-based visual art allows improved dynamic forms — art that develops or evolves before our eyes. The "artist" may have no more idea of what will finally emerge than anyone else. An even more exciting possibility in computer-aided installations is interactivity — digital art that responds to the viewer's actions, for example through input from movements of the computer mouse, or perhaps even by tracking the motion of different parts of the body. The crucial thing here is that dynamic digital art is not so much about the appearance of the finished product, as about the behaviour generated by the art piece through time. The viewer actually helps to modify the work of art, and becomes the artist him or herself.

Early computer art often involved fractals: A detail of the Mandelbrot set created by Wolfgang Beyer.

Mathematics, as the language of form and pattern, has always played an important role in art, but the rise of computer-generated art blurs the boundaries between art and maths even further. The act of programming — finding ways to store data and then creating logic-based algorithms to generate the desired output — is itself intensely mathematical. But the maths of physics is also used to realistically re-create physical processes, and the area of computer music — getting a computer to find those patterns and features that distinguish music from mere noise — poses a whole new range of mathematical challenges.

Consequently, the early days of computer graphics and digital art were dominated by scientists and research technicians — they were the only people with the specialist knowledge and access to operate the rare and expensive computers of the time. Even in 1969, Jasia Reichardt, who was then a curator at the Institute of Contemporary Arts (ICA) in London, said "So far only three artists that I know have actually produced computer graphics, the rest to date having been made by scientists". Since then, the number of artists using computers in their work has dramatically increased, especially with developments like the emergence of the personal computer and easy-to-learn programming languages. One such new computer language, Processing, is making great strides in enabling artists to become programmers, and programmers to become artists, and we'll take a look at this free software a little later.

Computer games such as Doom 3 and Neverwinter nights feature sophisticated graphics.

One obvious example of an area where computer-generated visuals have become crucial is in the games and cinema industries. The graphics in video games and the computer-generated images (CGI) so common nowadays in films and TV documentaries (TV CGI has come a long way since Walking with Dinosaurs!) rely enormously on mathematics. The models used to realistically animate the physics of a lumbering monster and render this model into a photo-realistic animation — particularly with the way light interacts with tricky materials or surface textures such as rippling water or coats of thick animal hair — would be impossible without computers chugging through the maths of Cartesian coordinates, trigonometry, vector geometries, and so on. It is said that the animators working on the film Shrek really struggled to get a glass of milk to look realistic in the gingerbread man scene. By the time Shrek II was under production, the graphic artists had cracked the problem and were able to greatly improve the realism of the rendering by modelling the fact that milk is not in fact an opaque liquid, but is semi-translucent, letting light inside to scatter about before coming out again. They were so proud of their new skill that the second film slipped in great volumes of milk: whole vats of it being sploshed over the sides of a castle! The realism of computer-generated graphics is continually improving, and in the near future the visuals in a computer game or visual art installation will become indistinguishable from the real thing.

Computers have heralded a revolution not only in the way that visual art can be produced, but also in how the piece interacts with its environment, the performance artist or even the viewers. Imagine not just idly looking at a painting in a stuffy gallery, but dancing in front of an installation and having a projection screen come alive with colour and movement, instantly responding to and interpreting your actions. Kinetic art forms can use different input devices and computer processing to allow immediate feed-back and response. These control inputs extend far beyond the standard computer interfaces of keyboard or mouse, and can include your voice captured by a microphone or body gestures watched by a webcam. This form of interactive kinetic artwork, controlled by movements as seen by a computer, can be very difficult to do well, however. Some very sophisticated problem-solving is often needed to enable a computer program to identify features from a webcam. Picking out the brightest or darkest regions in its visual field are easy, but identifying the head or hand of a performer, especially as it moves and so continually appears to change shape and brightness, is very hard for a computer to do accurately.

Obviously the whole point about dynamic digital art is to be there, experiencing the energetic performance and perhaps even directing it yourself. But you can get a feeling for the scope and imagination involved in digital art in the selection of projects show-cased below. These examples are just a taster of the great variety of digital art being explored around the world; on stage, on television, in gallery installations, and so on. We won't go into the mathematical challenges posed by the pieces here — a great deal more information can be gathered by following the links to the artists' websites, and the reading list below has links to Plus articles that explore relevant pieces of mathematics. But for now, just sit back and enjoy the show.

WithoutTitle

WithoutTitle by LIA.

For a relatively simple computer program (using Processing), this digital art piece by LIA generates strikingly beautiful output. Click on the image above to follow a link to WithoutTitle, and use your mouse to interact with the animation. A dense thicket of lines swarm around the current position of your cursor, and moving around the screen affects how the lines expand and contract. See LIA's website for more information.

R.E.M. Animal

A still from the video for R.E.M.'s Animal.

One recent example of responsive digital art on television was demonstrated within the music video to R.E.M.'s single Animal. A total eclipse of the Sun reveals to the lead singer, Michael Stipe, a whole hidden world of connections and energy lines lying beneath the reality that we are normally restricted to seeing. Figures walking around trail glowing auras, and as Stipe floats in the air singing, his body movements are picked out by the flickering of glowing stars, transiently linking together into constellations.

Shadow Monsters

Shadow Monsters is an interactive installation set up in 2005 that invites viewers to use their hands and arms to project shadow creatures, such as the classic opening-and-closing puppet mouth, onto a white screen. These shadows are monitored in real time by a camera-enabled computer, which then acts to reinterpret the shadow. The silhouettes of monsters are dynamically created from any shadows falling on the screen, complete with rows of teeth, tentacles, and bristling spines along their back. These monstrous creations are then re-projected onto a large viewing screen, just as if the "performers" had actually been casting the shadows themselves. Even better, the computer program generates realistic sound effects of squeaks and roars from the hand gestures as it senses the puppet shadows opening and closing their mouths. It's hard to think of an example of interactive digital art that's more instantly engaging and fun! Check out the video of a shadow monster performance below, or view the still images in the monster zoo on the creator's website.

Messa di Voce

Messa di Voce is an audio-visual performance created by a team of four artists, some sections of which were later recreated as installations for the general public to experience directly. The title, Messa di Voce, is Italian for a vocal technique used by singers, meaning "placing the voice", and is very apt for the unbelievable vocal skills demonstrated by the performance artists here. The speech, singing, and other sound effects created by the two performers are used, along with a video camera following their movements around the stage, as input fed into custom digital art software which generates dynamic visualisations and projects them onto a screen. The stunning effect for the audience is of the artists' voices literally coming alive, the colourful sounds emanating away from their mouths in different magical ways.

The original performances ran in 2003 and 2004, but they have been well-documented in photos and videos that you can view online. This incredible digital art really does need to be seen to be believed!

Get involved!

Much of the digital art show-cased above was created using a new programming language called Processing. This was created at the MIT Media Lab as a collaboration between computer scientists and artists, specifically to make programming as easy and intuitive as possible. This means you'll have absolutely no difficulty in getting hold of this software, learning a little bit about computer programming and getting stuck in creating stunning visual art yourself! Processing can be downloaded for free, and is a very small package to install. It comes with a help file to get you started, but the best source of information for learning the basics, or brushing up on more advanced features if you've programmed before (Processing is based on Java), is to explore the manual and discussion boards on their excellent website. Lots of digital artists have uploaded the computer code from their creations, so you should be able to pick and choose bits to mash up your very own digital art piece in next to no time. There are also a few new books about Processing, aimed at both beginners and more experienced digital artists / computer scientists. You might like to try these two as a first port-of-call.

Processing — A Programming Handbook for Visual Designers and Artists, by Casey Reas and Ben Fry

This handbook, written by the guys that created Processing, is an excellent reference for getting started. Early chapters explain the very basics of computer programming, such as performing calculations, taking inputs, and drawing graphics to the screen, even in 3D. The book demonstrates just how easy it is to get sophisticated-looking animations up and running.

At the end of every section, sample programs demonstrating the techniques you have just learned are provided for you to experiment with, and examples of the more complex projects produced by professional digital artists are shown. The final chapters cover more technical applications, such as integrating your digital art software with electronic circuit boards and even mobile phones. The appendices contain quick-look guides, for example to ASCII codes.

Visualizing Data — Exploring and Explaining Data with the Processing Environment, by Ben Fry

As you will have gathered from this article, much of the use of computers within visual art is in generating some form of data set, perhaps from tracking the movements of a performer or sounds recorded from a microphone, manipulating or transforming the data, and then displaying it for an audience, on a projection screen for example. One of the key steps in this process is the visualisation of the data, which is covered well in this second book.

If you have had your interest in Processing piqued by the Programming Handbook, then this is a good book to continue your learning with. Visualizing Data contains much more on the details of programming with Processing; using colour-coding, accessing databases, and so on. This makes it an excellent resource book for those already comfortable with Processing, but perhaps not as a first book, unless you have had experience with similar languages such as Java before.

Processing — Creative coding and Computational Art by Ira Greenberg

Ira Greenberg has had a diverse career, from painting to programming, and in this book perfectly provides for the needs of both artists and mathematicians learning how to create digital art. The book kicks off with a little history of mathematical and computational art, which helps set the scene for what can be accomplished using Processing. Chapter 3 covers all the standard programming fayre of variables, arrays, loops, if statements, functions, and so on. But of course, the major strength of Processing is in the ease with which stunning images and animations can be generated, and Greenberg wastes no time getting the reader into this more fun material.

Greenberg builds up your confidence from the absolute basics, introducing how to draw points, lines, curves, polygons, control colour, and arrange objects in repeating patterns around the screen. Chapter 10 moves into more advanced techniques for shapes and colours (although a lot of the book's figures lose out in the black-and-white printing), and manipulating loaded-in images for artistic effect. Finally, the book explores in more depth how to build complex animations and control the user interactions with your digital art piece. The book is well illustrated, provides a decent review of useful mathematics in the appendix, and contains much more sample code than either of the two books above.

Further reading on Plus

• The articles Maths goes to the movies and Career interview: Games developer explore some of the maths behind computer generated movies and games;
• The article Matrix: Simulating the world is a hands-on guide on how to create your own computer simulations using Processing;
• The article Perfect buildings: The maths of modern architecture looks at visual modelling in architecture;
• The career interviews Computer music researcher and Audio software engineer explore the mathematical challenges posed by computer music;
• The article Unveiling the Mandelbrot set looks at the maths behind some stunning fractal imagery;
• Look at the Plus archive for more articles on maths in art and music.

About the author

After reading Biological Sciences at Queen's College, Oxford, Lewis Dartnell moved to University College London's new centre for multidisciplinary science, the Centre for Mathematics & Physics in the Life Sciences and Experimental Biology (CoMPLEX). Here he completed his PhD in the field of astrobiology using computer models of the radiation levels on Mars to predict where life could possibly be surviving near the surface. This summer he is organising the Royal Institution Christmas Lectures on Computer Science.

Lewis has won four national communication prizes, including in the Daily Telegraph/BASF Young Science Writer Awards. His popular science book Life in the Universe: A Beginner's Guide has been reviewed in Plus. You can read more of Lewis' work on his website.