Last week I was doing my usual perusing of the Twitter feed when I came across this update:

http://twitter.com/#!/S0phieH/status/60644300224077824

ok, so starting today I am working on my#1MonthGame, it will be finished on the 18th of may, and released shortly after, wish me luck  

This grabbed my attention, as I needed to jumpstart my game *creation* again. After spending a couple of months on my multiplayer framework, I had a few game ideas for it but found myself procrastinating on actual development.  So I decided to do ‘join’ her in assigning a 4 week deadline to a new game, from scratch. We both used a #1MonthGame hashtag, so next thing you know, 2 others decided to do it too. She made a website for people to post their blogs (1 post a day) called 1MonthGame.com, and more projects joined up. You can join at any time, and give yourself a 4 week deadline from the day you post your project idea. The projects should be from scratch if you can swing it, though it’s not a *rule*. Actually, it’s not a competition, just a peer accountability deadline structure.

It’s neat because it forces you to focus on the small, playable, fun parts of a game. My idea, Monster Meadow, is based off some very simple game mechanics involving attracting critters and monsters, and I am making it for my 3 year old daughter to play on the iPad.  The name of the ‘bad guy’ monsters is Gullygog. This is a word my 1 year old boy says, we don’t really know what it means  I am doing the artwork myself when possible, and I am horrible at art so the shapes will be simple. It’s a simple kind of simulation and interaction game, and of a kind I’ve never tried before.

So I ‘ll be posting over on the 1MonthGame blog for the next 4 weeks This will help me get back into the game creation portion of development and stop focusing on tech so much, and allow me to move forward with my main projects.

http://www.1monthgame.com/blog/?tag=monstermeadow

About a year ago to the day I wrestled with the Facebook API and Shiva. I got something partially working, but it was broken beyond belief in IE. This is because I was relying too heavily on Javascript to do things, and couldn’t safely call Javascript functions from within Shiva when IE was involved.

So every few months I went back to http://developers.facebook.com, grabbed the latest php SDK from github, and tried things out. A pattern emerged. I could usually get a little of the way there (enough to make me feel like continuing) but never enough to work on all the browsers I wanted without breaking.

This time I tried something different! After trying and failing again many times in rapid succession today, I was ready to throw in the towel. But I tried something a little subtler, and then expanded on it. When I got into trouble, I scoured the web for similar solutions I could build on. In particular, I found a nice class structure here: http://johnklingelhoets.com/facebook-api-graph-oauth-class/

I used a good part of this for the wall post and user info stuff.

So.. without further ado, I present to you the Facebook/Shiva Sample Integration. Click to Download the Files

It’s very basic. It will get you authenticated, grab the user’s Facebook info for use inside Shiva (even their profile pic), and give you a way to post to their wall from the app. It uses all php and no Javascript for the Facebook portions. Basically, index.php authenticates the user and asks for certain permissions using the new OAuth format. When permission is granted, the flow returns to this page, and the signed result is stored in a cookie. Control is then passed to the Shiva application page. From within, Shiva makes a call to get the Facebook User Information to fbfunctions.php. It looks at the cookie and uses the token in it to make a call to the Facebook Graph API to grab all the user information we’re interested in. An XML blob is built and sent back to Shiva.

If you hit the wall post button, a simple text message is urlencoded and sent to the fbfunctions.php. You can add a lot more to this, and also use an XML block with lots more data for variety. Please look at http://developers.facebook.com and the Graph API links within to see what else you can do, I can’t help you in that area.

One of the trickiest parts is setting up the new application.

If you go to http://www.facebook.com/developers/ you will probably see a blog post, and on the right hand side is a button which says ‘Set Up New App’. Click it. If you can’t find it, try this link instead: http://www.facebook.com/developers/createapp.php

There are many settings. I pick the defaults for the most part.

At a minimum, you will need two settings in here:

• Facebook Integration->Canvas Page (this is where your app can be found on Facebook)
• Facebook Integration->Canvas URL (this is the full path to your application on your own web server, ie http://mywebsite.com/mygame/fb/)

People get hung up on the canvas url. There are lots of different ways to do things. Facebook requires this url to end with a /

I usually put all my Facebook files in an /fb subfolder with no other subfolders, so I have the main folders to do website stuff with.

So, in my web structure I have:

http://mywebsite.com/mygame/fb

and in that folder are ALL of the files included in the zip file. Facebook is going to be looking for index.php (the default file for the path). I’m not sure if the cert file is needed, I included it anyway.

This should contain:

• index.php (the initial file. Modify this and add your App ID and canvas URL from the Facebook Application Setup)
• gamepage.php (this is really just a basic .html file, but I made it a php file to help it not be cached. It is the normal Shiva web player page)
• fbfunctions.php (this is called from within Shiva, and passed an op parameter to decide what to do. Results are in xml)
• fbclass.php (this is included in fbfunctions.php and has functions for wall posting and other cool things. Not written by me)

You will need to include the FacebookAPI.ste in your Shiva project and change the base site URL in onInit (you’ll get a message about it)

After that, feel free to check out the code. You’ll notice I am not using any of the official Facebook API or SDK stuff from GitHub, feel free to add to this implementation. I am going to be staying away from anything involving lightboxes myself, but be warned that if you want those little boxes to pop up over your Shiva game for things like inviting friends, you’re in for a whole separate ballgame.

I won’t be able to help you too much with problems, everyone’s setup is different and I am NOT a web coder

I promised myself I would begin game design tonight, so I don’t have the chance to create docs yet. Go over the xml files thoroughly. I’ve given you the power to make something incredible here, and there aren’t many bugs in it.

Keep in mind that everything model/data related is driven via the xml files, beginning with DataFiles.xml.

The initial data file location is specified in: RPGData.GetXMLFileList_onEnter

You will also need to set your own server IP/Port in: Network_Connection.onInit

RPGDemo-FullCodeDrop-v1 (315KB)

Usage: You have permission to use this as-is in projects, but not to sell the code. It took me months to get it to this point, don’t steal my work and pass it off as your own in this way. I’d appreciate a shout-out for any error fixes/improvements or just to let me know if you use it to make a game

The entire size of the Engine right now is around 350KB. It loads its data at runtime as needed, based on the XML settings.

Feel free to drop by the IRC Chat with questions. I will send up some documentation as I can.

Quick feature list:

• Full XML Data System
• External asset streaming
• Loading HUD system
• Game Lobby
• Password Protected Game Hosting
• XML Based Entity Definition
• Basic AI Player
• Targetting
• Target Information System
• Multiplayer
• Lag Smoothing
• Chat System
• Chat Commands
• Server Menu
• XML Based Menu System
• Server Object Creation
• Raycast-Based Combat
• Server-Based Rules Lawyering
• Networked Object Template
• Jumping/Charged Jumping
• Orbiting/Zooming Player Camera
• Slow/Fast/Speed Adjustment
• Flying
• Click-Based Spawning
• Auto-Server Swapping
• Shadows/Fake Shadows
• Platform Detection
• Configurable Object Network Affinity
• Configurable Spawners
• Scene Changing/Portalling
• Networked Dynamic Field Manipulation
• Equipping
• Weapon Mounting

Have fun! You have permission to use this as-is, but not to sell the code to anyone. I’d appreciate a shout-out for any error fixes/improvements or just to let me know if you use it to make a game

Here’s a little video of how it all worked out tonight.

www.youtube.com/watch?v=wJFOXTIRoXg

Item Equipping Complete! (Basic version)

I added a <sources> block to my entities file, added a few named remote data sources that contain models/textures, and let er rip. This makes my framework itself around 360kb as a web plugin, and the art of course caches when it downloads.

Here’s what happened!

www.youtube.com/watch?v=Z522Gmm-06M

I’ve given myself a deadline of this Sunday night to have the framework stable enough to be able to begin working on a game project with it. There’s still a ton to do. Before I begin the game project, I will do a quick wrap-up of the codebase as it is then and give it a once-over for new functionality. I may have some time to draw up some short documentation, but it’s not bloody likely right now. I can definitely ‘freeze’ the codebase so that I can go back to it and do more documentation and pattern cleansing after I’ve sent out a game.

So, without further ado, here are my recent vids since the last update.  Oldest first.

3/1 Gremlins. A first test of swapping out the player model and creating an AI entity with it

www.youtube.com/watch?v=NECSYgU7CJs

3/2 iPad Test. A test of the framework on the iPad (I try to do this once a week or so), recorded on the iPhone. Bad quality, but the point is clear

www.youtube.com/watch?v=Em1K2Rm7GTQ

3/4 Server Commands. An XML menu framework for performing commands on the server out of a GUI

www.youtube.com/watch?v=bPRUTcbzLOA

3/5 Gremlin Overdose. Using spawners to make a whole ton of critters

www.youtube.com/watch?v=1phU6kI0Lwg

3/5 Monster Attack Range Test. The beginnings of intelligence

www.youtube.com/watch?v=98avWEDolnc

3/6 Portal/Zoning Test. I love this stuff!

www.youtube.com/watch?v=5_B4TDLmQp8

3/7 Chat Tweaks. Also, I pull a monster and he decides to go burrowing

www.youtube.com/watch?v=GWMiDeu8l1o

3/7 AI Zerg Test. I wonder how many AI I can run in the web player? I find out.. plenty!

www.youtube.com/watch?v=OV_W1wvWeKk

This is a very large update with a lot of refactoring, and many small pieces left undone. It’s been about 10 days since the last code drop, and a lot of new things have gone into the codebase. This will probably be my last code drop for a while, as I am now going game-specific. So I’ve tried to add as much as possible.

Some of the old things have been broken, like the rudimentary combat I put in awhile ago. You’ll need to code your own AIEntity networking events, but you should know how to do that by now, I’ve moved the objects to a generic Net Object framework you can build on. Check the bottom of the post for an example. I also modified the player controller/camera quite heavily from the original style.

Chat Commands Addendum

I forgot to add to the .pdf, I’ve also added in the basics for chat commands you can use ingame.

If you know the tag of an object, you can do this:

\cmd objecttag ainame handler param1 param2 param3

To call a handler on any object with a tag on it.

With the addition of some php/mysql calls, you now have everything you need to build an MMO. Seriously.

The included .pdf describes the major changed areas and some pertinent functions/flow.

• Lobbies- Join/Host
• Targetting
• Auto-Swap Client Server
• Server Objects
• Moving Platforms
• Monster Flocking

RPGDemo-Platforms-Server-AI-Targetting (Full Project Archive)

RPGSeriesMultiplayerFramework-3 (PDF)

Some Videos from the development process:

Initial Server Object Experiment:

www.youtube.com/watch?v=6l-npBVyC6k

Networked Platforms

www.youtube.com/watch?v=-CZHKJsPIkk

Platform Control

www.youtube.com/watch?v=mUBj86SDffc

Initial Zombies

www.youtube.com/watch?v=FPBb3nwkJyA

Test of Server Swapping

www.youtube.com/watch?v=Qc8hnLGbOAM

Creature Flocking/Targetting

www.youtube.com/watch?v=G4Xuyt-Oz9g

[Edit] AIEntity and NetObjectAI Update

After putting out this code drop, I needed to implement the sub AI net events on AIEntity, so here are the updated AI Modules that were affected by the change. Basically, when NetObjectAI is doing its roundup for packNetUpdate, it goes and checks the tPacket table in the child AI (AIEntity). If there’s stuff in there, it adds it to the outgoing queue, forces the update, and empties the child table. All the child AI needs to do is add packets to the table, and handle the unpack. I am reserving event numbers 30+ for child AIs.

AIEntity-NetObjectAI-20110228 (JUST THE UPDATED AIs)

Running Shiva Server on a Cloud (Rackspace)

Thanks to Eric Machala (emachala on the stonetrip.com forums/irc) I know how to setup a game server in the cloud. He posted a great How-To here. There weren’t any code changes, and one Shiva Server license is required per server instance. The great news for this is that the Linux (Ubuntu) server runs many times faster than the dedicated Win2003 server I have on GoDaddy. It can also be scaled up at any time with the click of a button on the rackspace.com website. To be clear, this is just the Shiva Server piece which performs the connectivity for clients and relays packets/establishes channels. There isn’t a way yet to run a dedicated Shiva app itself in Linux, though I’m sure it’s coming.

The cost of running a server is minimal: $0.015 per hour, and a bandwidth allotment of 50GB or so. You pay as you go and can see how much bandwidth you’re using, etc. Minimum cost is $10.95 per month on Rackspace, no setup fees or other costs associated with normal duties. Once you have the server setup the way you want, you can save an image of it and fire up new servers at will (provided you have the additional licenses for the Shiva Server). If you are ‘cursed’ with success and need to scale up rapidly, this is THE WAY to go!

Auto-Transferring Server

I’ve called this a few things in the past, but it’s another of the things which has stopped me from doing what I want to do in the big picture. Without the ability to run a dedicated server for logic which can run things like AI monsters and such, the only real options are to fire up Shiva clients, connect them to channels and scenes, and let them handle the AI duties. You need this if you want to have an authoritative way to make decisions for monsters, NPCs, or anything else which needs to ‘think’ for itself and can communicate its actions to all the connected clients. In addition, you’d need one of these per channel/scene, which could be dozens or hundreds of server programs running.

I’ve had the theory that if I could come up with a way for any single client in a scene to run the AI, I wouldn’t need to necessarily have these dedicated game servers running. The theoretical process requires that if a client that was handling server duties dropped off or disconnected, that we could fail over gracefully to another client and they could resume the server’s responsibilities. If I can accomplish this I don’t really need to worry about spinning up new game servers for logic, only for connection loads.

It’s been a very complex undertaking and I’ve tried it a few times and only managed to make a bunch of spaghetti code which is difficult to follow. I made a point to think about this in different terms this weekend though, and here’s how I went about it.

I asked some simple questions, instead of trying to establish an algorithm in pseudo code.

Q: What is a server?

A: A server runs AI and sends out messages to the clients who are interested.

Q: How can I reliably determine which client will be the server in a way that all other clients can agree with? (DONE)

A: This was a tougher question but ended up having an easier answer than I thought it would. There may be hidden issues but overall the basic precept works. Each client is given (by the Shiva Server) a numeric UserID which is unique. The server increments UserIDs as it runs. So the lowest UserID can be the server. When we get a notification that the previous user who was acting as the server has gone away, we can pick the next lowest UserID to be the server. If it happens to be me, I can immediately begin acting like a server.

Q: How can the new server pick up where the previous one left off? (DONE)

A: This is also tricky, but the basics are conceptually simple. When a server creates a new object that it will control, it sends out a message to all the other clients to create that object. When that happens, the clients tuck a reference to that object into a special table in the Server AI module, as does the real server. When the remote objects are deleted, that is also maintained. In the event that I am made the server, I already have a complete list of all the objects which must be maintained. An additional optimization here would be to know when the server last performed processing on its objects, so that the new server can pick up at about the same interval instead of too quickly.

Q: So how does the server process the objects in a way which doesn’t bog it down? (PARTIAL)

A: This is only a partial answer for now, but it works in initial tests. The server has a single routine which it runs through all its objects and processes them. This routine can be scheduled with a postEvent, or I can run it in a State Loop and examine a time differential in which to perform an update. Right now it is a postEvent.  The objects might have different AI or no AI but need to make certain checks. I’ll need to come up with a way to send the object a message which tells it to run one ‘tick’ of server processing, in which they can pick a new move location, pick a new target, etc.  The objects themselves will be responsible for sending out network updates, but this will probably also be done when the server tells the object to send out the update, so that it doesn’t overbear the client acting as the server.

The test

My initial test was to just verify the changing and transferring of the server as clients joined/dropped. I was pleased to see it happened pretty much instantly, and a notification shows up in chat that we are the new server, or that such and such user has become the server. From there, I worked in creating a remote object (a blue cube). The server runs its processing routine and simply moves the cube by brute force. On the other end, the cube is created but doesn’t move yet (it’s not networked after creation yet). When the original server drops off, the new server immediately begins moving the cube, and so on.

Next steps are to determine the cleanest way to tell objects to perform decisions and send out network updates so their movements and actions can propagate to all players. I’d like to use the existing AIEntity class to do this, and maybe just turn down the net frequency of the net updates selectively. I am in danger of making that class far too complex though, so I need to be careful. I could use an additional AI on the objects to do this stuff, but it feels messy.

One other additional improvement which is necessary now is to work in the client prediction based on previous velocity for objects. I’d thought I would need it and was avoiding it, but if I turn down network frequency on the server objects, the clients are definitely going to need to be able to predict movements in the absence of new move instructions.

Anyway, one by one I am tackling these things which have been holding me back, and this one is a doozy! So far so good, I’m one more step closer to a sensible large scale online game framework (I avoided the term ‘MMO’ there hehe)

When the process is more stable I will do a new code drop and documentation on it. I still haven’t begun to add too many game specific changes, except for a largish change in the move control and camera.

Here they are, for your enjoyment, or dismay!

Zombie300

Networking Breakthrough!

(Updated project file at the bottom)

Tonight saw an amazing new breakthrough in my network technology. I still can’t believe I stumbled on this. My commute is about 2 hours each way when traffic is ‘normal’, and I’ve taken to thinking up ways to add various network optimizations. I do think I had something decent to begin with, and indeed I was very excited about it, but it definitely had lag problems and skipping from a jittery Internet.  The only good way to increase the smoothness was to turn up the fidelity.

This solution came upon me after I had explored many various permutations of network optimization. I was experimenting with ping delay, frame time, scheduling moves into the future, packet ordering, with nothing helping the lag except for some error correction. The packet ordering worked OK actually. I also experimented with various different combinations of SnapShots per second, and network updates per second.

This other solution which I keep mentioning was not difficult to code, and some obvious optimizations showed up for it too.

The simple solution

Instead of scheduling moves into the future (and overlapping them wildly), instead I stored them in a table (analogous to a list/queue) in Shiva. Then, at a regular polling interval, I read the first move from the table and delete it. I do this at the same interval that the move snapshots are captured at, roughly. If the moves in the table get to be too long, I increase the polling interval. If the move table is *really* long (>30 moves which would mean 3 seconds’ worth), I crank up the polling speed dramatically. This accounts for lag spikes.

I begin the polling as soon as we are given a network user ID. Check the Video

www.youtube.com/watch?v=NMYB5XoEzUE

The first change is in RemoteObjectAI.unpackMessage:

 if(nMessageType==1) then
        --this.unpackMove ()
        --We used to unpack the move, but now we are just collecting it in a table
        local t = this.tMoveTable ( )
        table.add ( t, sMessage)
        return
    end

You will also need to modify your original move packet, no need to send out the timestamp difference anymore (saves some space too)

Here’s the ‘magic’ function!

function RemoteObjectAI.onGetNextMove (  )

--------------------------------------------------------------------------------
--Check the move table for existing moves. If more than one exists, we
--grab the top value, turn it into the current target position, and delete it
--This happens 10 times a second for a very smooth effect
local tMoves = this.tMoveTable ( )
local nNumMoves = table.getSize ( tMoves )
--log.message ( nNumMoves )
--No new moves, just schedule the next call into the future
--Also means we're not moving, so turn the speed down to 0
if(nNumMoves ==0) then
this.nCurrentSpeed ( 0)
if(this.hAvatar ( ))    then
object.setAIVariable ( this.hAvatar ( ), "AIEntity","nCurrentSpeed",this.nCurrentSpeed ( ))
end
this.postEvent ( this.nMovePollingInterval ( ),"onGetNextMove" )
return
end
--normal polling interval, at 10 times a second, should be the same rate they were captured at
if(nNumMoves < 10) then
--log.message ( "Setting polling to 0.1" )
this.nMovePollingInterval ( 0.1)
end
--Check to see if we're stockpiling too many. If so, speed up on grabbing them
if(nNumMoves > 15) then
log.message ( "Increasing polling to 0.075" )
this.nMovePollingInterval ( 0.05)
end
--If it's really long, speed up the polling interval
if(nNumMoves > 30) then
log.message ( "Increasing polling to 0.05" )
this.nMovePollingInterval ( 0.033)
end
--Explode the top move from the move table into the 'Message' table so we can get at it's data
local tMoveInfo = this.tMessage ( )
table.empty ( tMoveInfo )
local sMoveInfo = table.getFirst ( tMoves )
table.removeFirst ( tMoves )
--Turn this into a nice neat table
string.explode(sMoveInfo,tMoveInfo,";")
local stringtoNumber = string.toNumber
local tablegetAt = table.getAt
--Should be 1, which is a move event
--local nEventType = stringtoNumber(tablegetAt(tCurrentMessage,0))
local x = stringtoNumber(tablegetAt(tMoveInfo,1))
local y = stringtoNumber(tablegetAt(tMoveInfo,2))
local z = stringtoNumber(tablegetAt(tMoveInfo,3))
local ry = stringtoNumber(tablegetAt(tMoveInfo,4))
local nSpeed = stringtoNumber(tablegetAt(tMoveInfo,5))
--Update the new target coordinates
this.nTargetX (x )
this.nTargetY ( y)
this.nTargetZ ( z)
this.nTargetRY ( ry)
--Original Method
--this.nTimeStep ( 1/this.nFrameCounter ( ))
--FPS Comparison Method
--Figure out how many frames on average we need to interpolate over
--nDiff is the total time between move packets. Find a value to interpolate
local nLastTime = application.getTotalFrameTime ( )
local nDiff =  nLastTime - this.nLastMovePacket ( )
this.nLastMovePacket (nLastTime)
local nMyFrameTime = application.getLastFrameTime ( )
local nNumFrames = nDiff / nMyFrameTime
this.nTimeStep ( 1/nNumFrames)
--Figure out how many frames on average we need to interpolate over
this.nTimeStep ( 1/this.nFrameCounter ( ))
this.nFrameCounter ( 0)
this.nMoveFactor ( 0)
--Speed change? If so, tell our attached avatar how fast we're travelling
if(nSpeed ~= this.nCurrentSpeed ( )) then
if(this.hAvatar ( ))    then
object.setAIVariable ( this.hAvatar ( ), "AIEntity","nCurrentSpeed",this.nCurrentSpeed ( ))
end
end
this.nCurrentSpeed ( nSpeed)
this.postEvent ( this.nMovePollingInterval ( ),"onGetNextMove" )
--------------------------------------------------------------------------------
end

And here is the updated project file. If you come up with improvements, please share!

Full Project Archive: RPGDemo-NetworkBoost-20110118