Two words: Virtual reality.
Either make it into a fully virtual Oculus Rift game, (or some other VR platform), or make it augmented reality through Google Glass. Either way has some upsides and downsides. Using the Oculus Rift would allow you to have different arenas, lets you explore the world itself, and so on, but has the common issues associated with VR in that you don't know if you're going to run into a table in the process. This could be solved if you had a large enough room, maybe a basement or a local gym, but that would require the Oculus Rift to be set up in that specific area. You would also have to figure out how to get your own arms into the game as well, which would be difficult and likely computer-intensive with current technology. Google Glass would let you retain all your knowledge of what's around you, but still let you interact with and alter your surroundings. The downsides would be in trying to have an online battle where you can see the other person, as that would also require a large area as well as full-body capturing to put them in the picture. In general, though, augmented reality is probably the better option.
Putting these aside, let's see how the game could be put together. First off, the game would come with a piece of chalk. Well, more of a stylus with the shape and feel of a piece of chalk. It connects to the game and allows you to draw in the virtual world, and it will be your main tool throughout the game. Most of the mechanics should be pretty intuitive if you've read the book, though some might be a bit trickier to implement. The main ones are Lines of Forbiddance and Lines of Making. Since the things you draw in the virtual world won't really affect you, there's nothing to stop you from sticking your arm through a Line of Forbiddance. However, it wouldn't be too hard to stop people from drawing beyond them, and perhaps a sound cue, like a ringing in your ears that gets louder as you get closer, would probably solve this issue.
The other issue, trying to animate the Lines of Making, could likely be solved by incorporating a sketchbook. You would be able to draw whatever you want in it, and the game would compare your drawing to its own database. If the database was connected to the internet, you could probably have the option for your sketches to be added to it automatically, or even share them with your friends. The database would have sketches of animals, mythical creatures, knights, and so on already in it that you can view and use. You would have to redraw them in a duel, of course, and your drawings would be compared to the originals to help determine their strength.
If, on the other hand, you want to create your own designs, that's completely possible as well. After all, what's the point of a sketchbook that you can't make your own drawings in? You would draw whatever you want, and the game would check its database for similar drawings. (This shouldn't be too hard for Google Glass, especially since Google already does it for much more complex images.) It would then make a guess as to what your drawing was, (let's say it was a knight), and it would show you other knight designs with attributes of the one you drew added to it. (Say a plumed helmet and a star on the chest.) You could select the one that you like the most, and that would be the ideal version of your drawing, the one that'd have the best capabilities. The last thing would be to identify the joints so that it can be animated in battle, and from then on you can use it whenever you want. When you draw a Chalkling in a duel, it'll get compared to the ideal version, and you have your statistics as well.
As a sidenote, there may be certain drawings that are not allowed, and if the database recognizes them, it might give a suitable response about the reputation of Rithmatics or something similar.
There would probably be books that you can read to learn new techniques, find out about the history of Rithmatics, and practice various defenses. Perhaps a blank folding "book" would come with the chalk, which the game can add text to. This would also make for a good menu system instead of just having things hanging in the air.
Well, that's pretty much all I've got, and although I have no idea how to make a virtual reality video game, if I could it would be like this. Let me know what you think!
PS. If, by any chance, you do know how to make a virtual reality game, please make this a thing. ;-)