A Theoretical Approach to Photorealistic 3D Video: the Future of Film and Gaming?

Generally I am not a big fan of theoretical issues, especially when it comes to something as practical and visual as 3D modelling. This however is something I cannot really experiment with practically (or within an acceptable time frame), so for now it has to remain in the grounds of theory. It is also not strictly archaeological, although I’m sure some applications must come from it.

What I mean with Photorealistic 3D Video is taking the still photogrammetric models you’ve seen before and applying movement to it. This can be done by using either stop motion or a combination of cameras to acquire the original footage, then modelling individual frames and putting the frames in a sequence.


The acquisition of footage depends on what you are trying to achieve. If the object is still itself and it’s going to be moved frame by frame (like traditional stop motion animation) then a single camera is required. Instead of taking a single photo of the scene, like in standard animation, a series of images would be taken to make a Photogrammetric model, as explained previously on this blog. If instead the object is already in motion, as for example an individual acting out a scene, the trick is to use a large number of video-cameras that surround the object, using the same positions described before (a good example of this can be seen here http://www.webpronews.com/get-a-3d-print-of-yourself-in-texas-2013-08). With all the cameras then each frame is isolated as if photographs of the scene were taken at the same time from all angles. With either method, the results should be a large number of frames (24 for second or more depending on format), each of which is made up by 20 to 30 images.

The second step is the creation of the models. This can be done using 123D Catch or any other Photogrammetric software. Each series of images constituting a frame are hence transformed into a single rotatable 3D model.

Then all the frame models are run through other software, at present I am leaning towards gaming software, but video editing software or animation software may be more suitable, so possible options are Unity 3D, Maya or After Effects. Some alignment will have to take place, but by superimposing the models on top of each other and making it so that a single one is visible at a time should create an animation effect, again like stop motion. This is the part I am most unsure about, due to it being quite a demanding task, which may be far too complex for computers at this time. Still, in the future it should be more than possible.


At this stage the result is a series of still models that run through giving the illusion of movement. This can then be combined with technology that is now appearing on the market. In particular it could be used with the Oculus Rift that is soon going to revolutionise how 3D gaming works. By tracking the position of the user, it would be possible to not only see the Photorealistic Video, but also move around it as if it were real. By combining more than one model an entire scene could be created, meaning 3D films in which the user is actually present within the film can become a possibility, as well as uber-realistic videogames.


Basics of Coding for Archaeological Programs: Game Maker Overview


I am not a coder. I am especially not a gaming coder. I am however an archaeologist interested in all aspects of digital archaeology, from 3D modelling to Cad surveys, and I enjoy bending the original use of programs to fit my own needs. The best example I’ve ever seen of this is when some archaeologists used a Kinect to survey a room in 3D.

One of the most interesting programs I’ve ever used is Game Maker. It is not easy, not easy at all, and the main purpose is the creation of small .exe games. However the potential of this program is much more, as it is theoretically possible to do anything using the Game Maker Language, and I’ve seen some people make impressive 3D viewing software. What I am interested in is the ability to create programs to be used in archaeology, and up to now I’ve had success in two projects, one regarding the creation of interactive plans of sites, the other a small game to teach kids the workings of a hill fort.

Given the great potential, I thought it may be worth going through the steps for creating a program for all archaeologists who may be interested in doing so. The first thing to note is that Game Maker is similar to Dreamweaver in that it possesses tools to simplify coding, but allows some coding to be done if needed. The drop boxes used by GM are great to start off, but they get increasingly difficult to use the more complex the program gets. For this reason I use a combination of the two.

GM uses three main elements to create games: rooms, sprites and objects. Rooms are were everything happens. They are were the objects are placed and where they interact with one another. A game can have many rooms or a single room, and we decided what happens in each. Sprites are the images that we see. In the game Tetris, the various shapes can be seen because we have an image file opened as a sprite. What happens to the shapes though is decided by the objects, each of which has the sprite associated to it. The objects are where the coding happens, in which we tell the computer what we want it to do.


So, in Space Invaders we have a single room where the entire game takes place, we have a series of objects with certain commands and represented by image files called sprites. This is the basic working of games, and with programs it is the same. In Word we could say we have a “room” which is everything we see when we open it up, a series of “objects” which are the buttons we can press, each of which is coded to do something, and which are represented by “sprites”, the icons.

The next thing to look at is how to create an object with a sprite in a room. First of all we create a new sprite, and upload the image we want to use. Then we create an object, and in the box that appears we select the sprite we want in the sprite box. Finally we create a room and place the object within the grid that appears. If we then click the green arrow in the toolbar it all opens up, and we have an empty window with the image we selected at the position we told it. Of course nothing happens because we haven’t told the object what to do.


This is the hard part. The commands an object follows are made of two parts: the event and the action. The event tells the object when something happens. It means that if you tell the program that when you click on the object it moves left, it won’t just move left any time, but only when you click it. The action is instead what happens when the event takes place. GM allows you to choose from a large list of events and actions. The main events to remember are the create event, which means that the event happens when the room is opened, and the step event, which means the action happens at any time.


As an example let’s set the event as left key pressed and the action as movement, clicking on the left arrow and the speed as 4 (the speed at which it will move). When we press the green arrow now the room will open up and when we click the left keyboard key the object will move left, without stopping.

I realise this is not strictly archaeological, but I’ll get to that. In the next couple of days I will explain a bit more about this and go into the coding process.