Showing posts with label appengine. Show all posts
Showing posts with label appengine. Show all posts

Sunday, April 10, 2011

Reviving the Ancient Game of Liubo 六博 as an HTML5 Multiplayer App with Google AppEngine on Python and the Channel API

What is Liubo(六博)?


One of the oldest recorded board games, Liubo, 六博, literally "six sticks" in Chinese, has a mysterious origin and even more mysterious disappearance. Emerging early in Chinese history, it was the most played game during the Han Dynasty and remained so until the emergence of Go, 围棋, in ancient China. In spite of its abundant preservation in the archeological record, and numerous historical writings describing the game, the exact rules of the game remain unknown to this very day.




Liubo: Resurrection


Fortunately, we need not leave this game in perpetual obscurity. Jean-Louis Cazaux, an author of many books on the history of chess and ancient board games, has done an extensive study of the historical and archeological record and posited a reconstruction of the rules. Furthermore, he has subjected these rules to extensive playtesting to discover a game that is quite entertaining to play.
After some exchanges with the author I've been able to implement the game he envisioned on the web with both multiplayer and AI capability.

To play the game, you first sign in using an OpenID such as Google or Yahoo. Then you press "Play" and you will be placed into the game queue for another opponent. If you wish, you may press "Skip to AI" and play the AI instead. Then you will randomly play as either white or black, white goes first as in chess. Two sets of yin-yang sticks are then thrown automatically on the right side of the screen, one mark for yin and two marks for yang. Total the yang plus one for each set of three sticks, and you have your move number, for instance 3-2. This means you move 3 the first move, then 2 the second. To move, you may either enter a stone onto the board at the move number, or if the piece has not moved this turn, you may advance the stone. Advancements go counter-clockwise around the board; press the Moves button to see the move order. At two special positions, numbers 6 and 11, you may move inward towards the center and across the board. If you land directly on the center, your piece is promoted to an "owl" if you don't have one already. This owl can then capture other pieces and put them into your prison; for regular stones, captures only return the piece to the other player. You may have only one owl at a time. Capture all the opponent stones and you win; capture the opponent owl if you have no owl and you also win; have five stones on the board and no owl while your opponent has an owl and you also win; let the other side run out of the 10 minute move clock and you also win. After each game, your elo rank and that of your opponent (including the AI, "SiliconDragon") is adjusted. You may see the top rated players with the Ratings button. You can find the game here:

Click to Play Liubo 六博


HTML5 Multiplayer Design


When starting the design, I first did a static layout of the board in HTML and CSS. I wanted a design that would scale to different screen sizes so I could use it on iPad as well as various desktop sizes. Also I wanted it to work on various browsers including IE8, so I focused on using CSS drawing techniques with borders and screen percentages and tested it on various platforms. Using gradients and fallback CSS I was able to make the board, throwing sticks and pieces without using any images at all.



Once the layout was done, there remained the huge effort of programming. There are many options in creating an interactive HTML5 app today, from WebSockets, to Flash compilation to Canvas, to game-specific libraries such as Impact.js. For speed and compatibility, I chose to use div-based HTML5 javascript with jQuery. Although node.js shows promise, I prefer the established ease of use of Google AppEngine and its python backend, so I went with that. Linking the client and server is the newly released Google Channel API.


Implementing a Google Channel API Client


The trickiest part of the whole project was getting the Channel API to work properly. My first mental reset was realizing it is a unidirectional, not bidirectional, API. That is, the Channel API requires that the server give a channel token to a client and then the client connect to the server, however only the server is allowed to send messages to the client. The client is not allowed to send messages to the server over the channel. Instead, the client must use normal HTTP requests to the server which then must be matched to a client token whereby the server can send a response over the channel.


Illustrating this procedure, first here is a python snippet initiated by client request for a new channel. It runs on the appengine and creates a new channel, stores it to the datastore, and then returns the channel token along with other user data in a JSON:

def put_channel(self, user, profile):
   profile.channeltoken = 
      channel.create_channel(make_channel_id(user.federated_identity()))
   profile.channeldate = datetime.now()
   profile.put()
   profile_json = {
      'screenname': profile.screenname,
      'character': profile.character,
      'rating': profile.rating,
      'rank': compute_rank(profile.rating),
      'numgames': profile.numgames,
      'dateadded': profile.date_added.strftime('%Y-%m-%d'),
      'logoutURL': users.create_logout_url('/'),
      'channeltoken': profile.channeltoken
   }
   self.response.out.write(simplejson.dumps(profile_json))


On the javascript side, we receive this channeltoken, set our channel id to this token, and then create a new channel. We then open a socket and setup socket handlers. The socket open handler requests a new game to join. The socket message handler processes each message sent from the server, such as starting a multiplayer game and receiving moves from an opponent.

self.channel = new goog.appengine.Channel(self.channelid);
self.socket = self.channel.open();
self.socket.onopen = self.socketOnOpened;

self.socketOnOpened = function() {
   // give it a chance to startup
   setTimeout(game.requestJoin, 5000);
};

self.socketOnMessage = function(message) {
   // console.log('received message data:'+message.data);
   var messageArray = $.parseJSON(message.data);
   if (!messageArray) {
      alert('invalid message received: ' + message.data);
      return;
   }
   for (var i=0; i<messageArray.length; i++) {
      var messageline = messageArray[i];
      var command = messageline.shift();
      var data = messageline;
      self.commandProcessor.processCommand(command, data);
   }
};

Going again back to the server-side python code, we create a simple function for sending messages to the client via the channel. When creating a game, we first create a unique gamekey, then send it to the channel. Another important point to note about the Channel API is illustrated here: the server must have a different unique channel to each client it wishes to connect to. There are no broadcast channels or multiple subscribers; it is strictly one-to-one. So the server has created two channel ids, one for the white side and one for the black side, upon client connect. Then as the game progresses, messages are sent from the server to each client via its own respective channel. A unique gamekey, sent by each client during a request, allows the server to connect a request to a particular game, and thus provide a link between two clients in a multiplayer game:


def send_message(self, channelid, json):
   logging.info('sending message to:%s payload:%s',
      channelid, simplejson.dumps(json))
   channel.send_message(channelid, simplejson.dumps(json))

def create_game(self):
   self.create_gamekey()
   game = Game(parent=gamezero)
   game.gamestatus = 'WJ' # waiting for join
   game.gamecreated = datetime.now()
   game.gamekey = newline_encode(self.gamekey)
   game.gamestatus = 'GS' # game started
   game.gamestarted = datetime.now()
   game.put()
   self.send_message(self.black_channelid, [['PB'],
      ['OP',game.white_playername],['GS',self.gamekey]])
   # game started, white's move
   self.send_message(self.white_channelid, [['PW'],
      ['OP',game.black_playername],['GS',self.gamekey],['WM']])


Again returning to the client javascript side, we receive the new gamekey and begin the game. Each time after the client player moves, an HTTP POST request is sent to the server. This request contains the gamekey and the moves for this turn. We don't wait for any reply from the server to our request; instead, our previously established socket listener waits for any commands to be received via the channel API, namely, the other players moves. This methodology is how the Channel API is meant to be used in interactive applications.


self.startGame = function(data){
   $('#gamejoinstatus').html('Starting game...');
   clearTimeout(self.joinTimeout);
   game.gamekey = data[0];
   game.startGame();
}

self.sendMoves = function(moves) {
   var params = {
      gamekey: self.gamekey,
      moves: $.JSON.encode(moves)
   };
   // console.log('POST moves to server:'+params.moves);
   $.post('/games', params)
   .error(function(){
      $('#throwturn').html('Unable to send move');
   });
}

Returning to the python server side, we find our move processor. After receiving the HTTP POST request from the client with the game key and validating moves, we simply relay the move to the other player via the other player's channel id. In addition, we check for a validated game over condition:


def post(self):
   self.user = users.get_current_user()
   self.request.get('gamekey'), self.request.get('moves'))
   self.gamekey = self.request.get('gamekey')
   if not self.user or not self.gamekey:
      self.error(403)
   elif self.request.get('moves'):
      self.moves = simplejson.loads(self.request.get('moves'))
   if self.decode_gamekey():
      self.process_moves()
   else:
      self.error(403)

def process_moves(self):
   # just relay moves to the other side
   self.send_message(self.other_channelid(), self.moves);
   for move in self.moves:
      command = move[0]
      if command == 'GO': # gameover
         status = move[1]
         winner = move[2]
         reason = move[3]
         self.process_gameover(status, winner, reason)
         break

Lastly, let's see how we actually process moves on the client side. I've adopted a simple json-based symmetrical send/receive protocol, so all messages back and forth are sent as json-encoded lists. Each item in the list is itself a list, consisting of a two-character string command code and zero or more data items. The heart of this is the client command processor that receives each different command type from the server and then dispatches it to the appropriate handler:


self.processCommand = function(command, data) {
   // console.log('Command and data received: '+command+' '+data);
   switch (command) {
      case 'WJ': self.waitForJoin(); break;
      case 'OP': self.setOtherPlayer(data); break;
      case 'PW': self.setPlayer('white'); break;
      case 'PB': self.setPlayer('black'); break;
      case 'GS': self.startGame(data); break;
      case 'GO': self.gameOver(data); break;
      case 'WT': self.receiveThrow('white', data); break;
      case 'BT': self.receiveThrow('black', data); break;
      case 'WM': self.takeTurn('white'); break;
      case 'BM': self.takeTurn('black'); break;
      case 'WS': self.moveStone('white', data); break;
      case 'BS': self.moveStone('black', data); break;
      case 'WP': self.promotionMove('white', data); break;
      case 'BP': self.promotionMove('black', data); break;
      case 'WB': self.receiveBonusThrow('white'); break;
      case 'BB': self.receiveBonusThrow('black'); break;
      case 'NR': self.newRating(data); break;
      default:
   }
};




That pretty much covers how to use the Google Channel API in a multiplayer game. The good thing is I found it worked across all the platforms I tested, and at lower volumes it's completely free to use on the Google AppEngine. So you can get something up and running without spending any cash, only if it takes off do you need to front up some mulah. There are many other aspects to the game which I could elaborate if readers desire, but this has provided a good introduction to using the Channel API in your own game. Try it out:



Click to Play Liubo 六博


PS: I recommend the site HTML5Games to see the latest in HTML Indie Gaming.

Sunday, September 19, 2010

Python Startups and the True Entrepreneur

I've got lots of friends that are crazy about Python, about what it can do for development especially with AppEngine, Django and All That.  True enough, delivering on these promises can happen and lead to new ideas in software that weren't possible before.  With this realization comes the thought, then the firm idea, then the inception of a new startup company dedicated to the realization of this concept.  I've got loads of friends thinking of making the jump, and a significant number that have, and python is always at least part of the discussion and sometimes the main topic.


It's not just Python - this kind of thing happens all the time with software entrepreneurs. Of course it seldom turns out the way expected.  Usually after six months or a year of trudging along, a trail of bitter tears and dashed hopes is all that's left.  What sounds fun, exciting, with limitless potential turns into just another way to work for low wages and use up all your savings and the savings of friends, family and investors.  Then it's back to the daily grind, perhaps wiser for it and with Awesome New Coding Skillz, but nonetheless back to workin for Da Man.

But there is such a thing as a True Entrepreneur. I'm not one, but I know them. They are few and far between, and often not astoundingly successful. I'm talking about the people who wouldn't hold down a stable job, even if offered it, not because they don't have skills, but because they can't imagine not running their own business. Someone who would rather work minimum wage, even after years of college and experience, at their own company, rather than make $100 an hour working for The Man. Someone who, far from shying away from the risks of loosing money and skirting close to bankruptcy, actually thrives on this kind of gambling and stress.







In microeconomics, you may have been taught that new businesses have a high rate of failure, between 75-95% depending on the study, which means only 5-25% of startups are still in business after a few years.  Your chance of succeeding, not just surviving, is even less, where me might consider a startup successful if it grows to be worth $1 million, $10 million, $100 million, or more.  We are further taught that the entrepreneur takes on such high risks because of the high chance of reward.  Staying at a major company and becoming a senior executive may get you a high reward, maybe even $500 million, but it will never approach the potential of the most successful entrepreneurs, with multiple billions or even $20 billion at the top level.
 

However success is not the primary motivator of the entrepreneurs I know; it is a factor, but not even I think the primary factor.  The primary factor is self-actualization, which is a combination of ego, arrogance, self-delusion, and wish fullfillment.  If you went simply by the odds, it would be a net loss to be an entrepreneur, like a mutual fund that invested in the lottery.  Sure, you might win, you might even win big, and if you don't play you won't win, true, but statistically you'd be better off putting your money in Treasury bonds.

A True Entrepreneur doesn't found a company just because they think it might succeed; in fact they would found the company even if they knew it would fail.  They simply are compelled to do it, in the way a monk might be compelled to pray in the desert, in the way a Peace Corps volunteer is compelled to help the less fortunate, in the way Guido van Rossum was complled to make life better for developers with Python.  It is a basic, primal, ultimately irrational and unexplainable inner drive.

If you're this kind of person, a True Entrepreneur, you will be compelled into your own business, be it coffee shops or software, and you will over your working life at least be able to eek out a living. If you're not, you either need to get on board with one of these, or stay as an employee or freelancer.  I'm not saying this from the perspective of the entrepreneur, "come join the bandwagon with me!".  Rather, I'm saying this from the outside, from having been on the inside but spending more time out of it, from choosing more often then not to stay out of the game even when many family and friends are in it.  What I'm really trying to do is make sure people understand what they are getting into, so they know if this life - and it is a life - is the one for you.




Wednesday, September 15, 2010

Google AppEngine and Enterprise Support

The Real NORAD in Cheyenne, Wyoming

I just finished watching the Google webinar on their Python AppEngine:

Google Campfire One Python

It's a good talk and the best part is they spend most of the time showing actual source code and running applications.  It always annoys me when developers talk about their programs without showing the source code, or at best showing pseudocode, so it was refreshing to see a talk that was mostly source code.

The premise of the AppEngine is promising: outsourcing admin work and making the whole deployment process very fast and easy.  For small applications and few users it lets you get online with little fuss and no cost.  Where I would be a little concerned is when you start talking about large important applications where you need much more than the 500 MB of storage they offer, where you need 24x7 on-call support when the admin console isn't doing what you think it is, all the big hairy problems that a large enterprise application is known for.  This support aspect concerns me because that's what I've spent most of my professional life doing: supporting massive enterprise applications.  Given that Google is known for their crappy customer service and often non-existent support (as I found when I tried to use Google Earth professionally), I'm worried.

One way to alleviate some of these concerns would be to have a real customer support department at Google.  I'm looking for something like you'd find at Motorola for their public safety radio monitoring, the Big Board room in Schaumburg: I've visited it, or the EDS, now HP, service monitoring room, a NORAD-type facility where you can pay for good monitoring and in-depth support to fix critical issues.  It's relatively difficult to build this kind of organization, especially at Google where the culture of customer service is largely absent, so it would probably be easiest for Google to just buy an existing support company or partner with someone like HP that can provide this level of support.  It will be quite expensive, but the large organizations demanding it have deep pockets.




However even this high support level has the downside of being tied to Google administrative access to sensitive company data.  If I was trying to sell AppEngine to big corporate clients, this is one of the primary concerns I'd have to address, especially in the software and telecommunications field where Google is a primary competitor to many potential users.  Many executives simply won't allow sensitive data to be placed in the hands of a competitor, despite assurances, and there may be strong privacy concerns as well.  In some jurisdictions, storing customer mobile call data on Google servers may not even be legal.

Google could alleviate these concerns by providing infrastructure hardware and software for company-hosted AppEngine instances.  This would allow a company to run their own Google AppEngine in their own datacenter.  This is not without precedent.  Google has taken this approach with search, selling the Google Search Appliance as a stand-alone indexer and search provider for intranets.

Google Search Appliance

A Google AppEngine Appliance would provide the same level of customer control and assurances in the AppEngine space that is already found in the search space.  Furthermore it would build a revenue stream that Google normally wouldn't have access to, the datacenter budgets of enterprises.  Given that dozens or hundreds of AppEngine Appliances might be needed for a large application, or even several large clustered virtual machines hosting thousands of appliances, the potential is huge.

Let's hope we see a Google AppEngine Appliance in the future.  Then we enterprise programmers can start using this great python development tool in our everyday work.