Showing posts with label computer games. Show all posts
Showing posts with label computer games. Show all posts

Tuesday, 15 April 2014

World of Darkness MMO

I worked on the World of Darkness MMO for a year and a half, then decided to return to New Zealand.  It had almost been 10 years since I left New Zealand, and there was some sort of thing where if you hadn't paid taxes in that period of time, you lost rights to something or other.

Living in the USA was great, and I would have loved to continue living there.  My apartment complex had real character.  Double shootings just before I moved in.  Armed police with weapons drawn at my door, as the previous tenant had a warrant out for their arrest.  A friend who was attacked on the footpath between work and the complex, by a short mexican with a small knife.  Walking past a car being hot-wired by some black dude, in the car park outside the movie theatre. Waffle house wednesdays.

It's sad to see it has been cancelled.  We did some really interesting things, and I'd be curious what the years since then resulted in.

Friday, 27 July 2012

BrogueX released

I've published my port of Brogue as an Android paid app.  It's basically playable with a little difficulty, and requires a little work to fix bugs, and a lot of work to enhance the Brogue libtcod-based UI for tablets and mobile devices.

All the details are on the link above.  Buying it supports the work required to make this a first class Android experience, and you get to direct the efforts involved should you choose.

Friday, 9 September 2011

Amiga game playthrough videos

There weren't many Amiga games available in my New Zealand home town. The few that made it there, were too expensive or unappealing. There were many games in the computer magazines (which were readily available) which I wanted to play but was unable to. These days people play through (AKA longplay) the old computer games, record it in their entirety and then upload it to YouTube. This gives me a chance to see what the games had to offer, without having to invest the time myself.

Link: Recorded Amiga Games.
Link: Longplays YouTube channel.

Cadaver:


Castle Master:


Kult:


There's actually a lot of playthroughs for games for other platforms, even recent games. Gives me a chance to experience the best parts of a game without wasting time and money to find out if it was worth it. Not that I would have played them anyway.

Sunday, 15 May 2011

A real Imaginary Realities mirror

Probably over a year or two ago now, Fred Clift replied to a message I sent to the a mailing list and mentioned he had a full mirror of Imaginary Realities. Up to this point, the best mirror was the text only copy hosted at this site. However, the full mirror was not really usable as-is. The index pages were missing and javascript redirected every visitor to the original now defunct URL of the site. So it sat on my hard drive until now.

I've finally had the chance to process the files and put them up for browsing in all their glory. Note that one author does not want their article published any more and requested it be removed from the old mirror, so it has been removed from this one as well. Both the mirror and the downloadable snapshot have had it removed.

Link: http://imaginary-realities.disinterest.org/


Download: imaginary-realities-2011-05-14.zip (13.7 MB)

Unfortunately, it is unlikely the forum threads associated with each article will ever be recovered. Both them and the rest of the forum contents. If someone else mirrored this, and happens to have a copy I'd like to find out otherwise though. But MUD forums come and go all the time, all the time and effort discussing things going to waste. Imaginary Realities, MUDMagic, ... Anyway, enjoy the mirror.

Sunday, 8 May 2011

Roguelike MUD progress #6 - Chinese character sets

Previous post: Roguelike MUD progress #5

Feeling like doing a bit of programming, I decided to run my roguelike MUD code and see if there was anything that I might feel drawn to working on.


This must be the first time I have tested it since I bought a Chinese laptop. I have debug menus in-game which show me all four character sets, and they now show the following.


Windows Telnet looks better than Putty, but if Putty is what I have to use now, then that's the easiest path forward.


It looks like Putty is also not working reliably now. Going back to Windows Telnet, maybe there is some way to get DOS to display CP 437 without having access to the ability to switch operating system languages. And it turns out there is, typing chcp 437 in a DOS console does the trick. But unfortunately, it uses a different font and it also seems to break handling of escape codes.

Unfortunately, this is going to require a lot more research and work. I can't say I am enthused about it, as I would rather be just working on game code. But maybe I can detect Chinese Windows and use unicode characters, as they seem to be available.

Monday, 2 May 2011

Trying Imperian

Since I've spent a little time in the past two days doing some MUD-related work, I felt like trying one out. I decided to give Imperian a go. If I recall correctly, the first choice was from a long list of races. Some had odd names, and I had no idea what the difference was, but making a choice you have to live with for the life of your gameplay is an unimportant one.. so I effectively randomly picked one.

This dumped me in a isolated and staged environment, mostly comprised of filler rooms separating NPCs, each of whom offers a functional service. The filler rooms are not even described. This environment is portrayed as a past version of the city I would enter the real game within. It is certainly better than past newbie experiences I have seen, like "newbie schools" and shared locations where confusing goings-on and killing fields of menial animals exist.

Many of the commands are coloured and hyperlinked, which makes it almost like a click and read experience. This is quite nice, but would be even better if it were done more consistently. The occasional command is simply not linked, and other cases where commands could be linked are not taken advantage of.

The map is nicely coloured, and is clearly matched the the tasks command, making working out where to go to "complete" the introductory area a breeze. However, the overlaying map window seems to serve no purpose. It just features linked rooms, one of which is highlighted as the current location. It is a shame, because with better integration it wouldn't be useless and could be even more useful than the text map displayed below.


Having completed the introductory area, the player is then confined to a large and confusing list of guilds which they have to select.


The description for each of these guilds, is mostly meaningless to me. Another important commitment to a gameplay defining choice which is made into a confusing and arbitrary experience.


Having randomly selected a guild, I am next confined to selection of a council.


The first thing I see when I attempt to read about a council, is a long and meaningless list of names and positions. This means nothing to me and makes me wonder why I am being forced to jump through this hoop. I choose this council, for lack of any idea what else to do.


At this point, I press backspace and.. the browser goes back a page and I no longer have a readily usable game window anymore. I lost interest somewhere along the way anyway, due to all the choices I had to make, and which sapped away my investment in the character.


The padded artificial feeling of the introductory area made it a hollow experience. No rooms were there, or detail added to them, except to provide a list of checks in boxes in things which the player needed to be lead by the nose through. Is it possible to do better? Yes, of course, but it would be much harder and much more work.

Reading selected information and making choices deemed necessary, where the two are not necessary linked, were the points of this experience. It wasn't to understand anything, or to choose things which were interesting to me. I wonder if I could make a game where the player didn't need to make any choices, in order to get into the real game world from a staged introductory area, with understanding of concepts, environment and ability. More story driven, where choices are implicit from chosen actions.

I like the idea of an introductory area. I definitely find it better to keep the player away from the confusion on the game, and the weird announcements and player abilities. The last game I logged into, I saw announcements about "laser tag" games and in another players teleporting in and out willy nilly with messages relating to "orbing" or something.

The web-based client is okay, but the backspace problem is too much for me. It's not the first time that I've had this happen when using a client like this. Really, the game should give me a link that I can drag onto my bookmarks toolbar and when pressed takes me straight back into the game where I left off.

Monday, 3 January 2011

Steam and Dragon Age Awakenings

Buying and downloading the game in Steam, I proceeded to open up the game and start playing. It seemed exactly like Dragon Age Origins, the game it is an expansion of. As a magic user, you start by doing the boring and tedious "Harrowing" quest thing. As a fighter, you start by doing some family drama bollocks including fighting large rats, thinking what a cliche this boring tripe is, and then having a character voice that fighting rats is a boring cliche. Just because you tell the player you know you are making them do your boring linear cliche, doesn't mean it suddenly isn't boring linear cliche. Anyway, the point is that it didn't seem like a new game.

I went back to the main game menu. No sign that it was Origins, and not Awakening. The "Other Campaigns" item was disabled. The Windows system tray icon was the Awakenings one, not the Origins one. How frustrating. So I googled something like "getting dragon age awakenings to actually work in steam". Bingo, some mostly tangential jibber jabber and a hint to run some application to get content registered.

Once you have loaded the game, at the main menu for the game under "Load" will be an entry marked "Other Campaigns". Choose this and select the "Dragon Age: Awakening" campaign.

If you are unable to access the "Other Campaigns" menu, please verify the GCF for both Dragon Age: Origins, and Awakening:
Title: Verifying Game Cache Files (GCF)
Link: http://support.steampowered.com/kb_a...2037-QEUH-3335

After the verification is complete, navigate to the folder where Dragon Age is installed (by default C:\Program Files\Steam\steamapps\common\dragon age origins\)

Find the folder marked "redist", and run the following files:
DAO_UpdateAddinsXml.exe
DAO_UpdateAddinsXml_Steam.exe

Start the game, and re-test the issue.
No dice.. "Error opening file for writing" "Results [-1] [Failed to load XML: ]"


What kind of Mickey Mouse outfit is this? Stuff you bought doesn't work, so you have to go run two applications that you barely understand the purpose of, and then they do not work for inscrutable reasons? I'm a computer games programmer and I barely understand this.

Some googling for the error message, and I stumble across some other unfortunate souls who have had the same problem as myself and without the aid of the shysters who sold them this polished turd, have worked out how to work around this DRM facilitated hoop jumping.
I searched around in the Steam and Documents folders and found all the needed .xml and .exe files. The only thing is, after running the command in command line, the "Other Campaigns" option IS STILL grayed out. The window appeared and said that everything worked fine and that the xml files were updated. It even gave me the Result [0] output, which im pretty sure means it was sucessful. I have no idea what is wrong.

Here is the command I used:
"C:\\\\Program Files\\\\Common Files\\\\BioWare\\\\saferun.exe" "C:\\\\Program Files\\\\Steam\\\\steamapps\\\\common\\\\dragon age origins\\\\redist\\\\DAO_UpdateAddinsXml.exe" /addins="BioWare\\\\Dragon Age\\\\Settings\\\\addins.xml" /manifest="C:\\\\Program Files\\\\Steam\\\\steamapps\\\\common\\\\dragon age origins\\\\addins\\\\dao_prc_ep_1\\\\manifest.xml"
Ah! It was that simple all along :-) In my case, I had to replace "Program Files" with "Program Files (x86)", but that's a minor change.

The Steam sale and Dead Space

I've been buying the odd game on Steam, throughout their Christmas sale. Today is the last day, and Dead Space is half price or something. So I thought, what the heck.

Clicking through from the sale page..


To the actual game page, where amongst the screenshots, system requirements and the like it says the following..


I don't understand what this means. But I think it means that if I were to buy the game, it would be a rental for an indeterminate period of no less than a month. Are they trying to make people to pirate their games just so they get what they paid for in a form that's actually playable?

Thursday, 12 August 2010

Goblin Camp

People rave about Dwarf Fortress, and it all sounds well and good, but I just don't have the time to learn to decipher the arcane interface that immersion into it is given through. However, a new open source project called Goblin Camp has started up that aims to make a similar game.

Given that something like this is a world simulation of sorts, I wonder whether harnessing it to run a MUD world within, is possible or worthwhile.



Screenshot taken from this forum thread.

Saturday, 12 June 2010

Soulfu forums

The guy who hosts the Soulfu forums let the domain name expire because he didn't consider the activity within them to warrant renewing it. However, he still hosts the forums under his own domain name.

Wednesday, 24 March 2010

The MUD development community

If someone is implementing a MUD server from the ground up, the first problem they face is that there is very little documentation telling them what might be involved. Telnet itself is documented through RFCs, with a good number existing mostly covering the different negotiation options. But there is a lot more than that to MUDs. It is strange there is so little MUD-specific documentation, yet MUDs have been around for decades. Coming back to MUD development after over eight years, it surprises me that some topics are not only undocumented, but to a large degree they have not even been explored.

The sharing of information that does tend to happen, happens within forums. Someone will post about a topic and then either people will chime in and discuss that topic and whatever others lead on from it, or they might link to previous discussions on the topic. There is knowledge out there about different topics, but it tends to be shared piecemeal. This is not that much of a surpise, the development of a MUD is a long term effort that may span many years, and it is reasonable to restrict your efforts to that which you actually need to work on.

But what is surprising, is how little value is placed in these discussions. The range of topics that might be discussed are limited, which unsurprisingly leads to them coming up time and time again. Forums where discussion takes place come and go, with all discussion that took place within them more often than not being lost forever. Remember MUD Planet? At one stage considered one of the most popular forums, the discussion that took place there is now lost. MUD-Dev was an extremely popular mailing list and for years after its web site went down, the posts made to it were unavailable except offline in the hands of individuals. MudMagic used to have forums until its administrator decided to take them down, and now the discussions that took place there are effectively inaccessible. At least two wikis have come and gone.

Frankly, beyond the possible immediate benefit of its participants, MUD development discussion is somewhat like pissing into the wind.

Monday, 22 March 2010

In memory of Windows Telnet

There was a time when Windows Telnet came installed with every copy of Windows. Because of this, it was an easily accessible MUD client for anyone who might want to connect to one. However, times have changed. On one hand the most popular MUDs are making use of custom extensions to the telnet protocol that Windows Telnet will never be able to support, and on the other it no longer comes preinstalled with every copy of Windows.

This post is a recognition of Windows Telnet. That despite its problems as a MUD client, it is still an extremely capable and versatile telnet client. Please give less weight to the listed minuses (which are useful to know if you choose to adopt this client) and appreciate its plusses.

Plus: Superior terminal support

Back in the days before Putty was around, having a telnet client available with Windows that implemented ANSI/VT100 terminal support was extremely valuable. And Windows Telnet allows use of a wide range of the escape sequence that make up this terminal support.


The preceding screenshot is of a prototype of a roguelike that I attempted to implement for the recent 7DRL 2010 competition. Windows Telnet was as capable as Putty at displaying the arbitrary screen updates that my game makes.

Minus: Reduced availability

No longer coming preinstalled with every copy of Windows, installing it is now more complicated than just downloading any other MUD or telnet client which you stumble over a link to.

Minus: Negotiation support

If connecting to a server that does not engage in telnet negotiation, Windows Telnet echos what the user types back to them as it sends it. But should the server send any negotiation, this echoing is switched off and the server has no way to switch it back on. The only way it can be reenabled within the lifetime of the current connection is if the user enters the settings screen and explicitly reenables it. This leaves the server in a position of having to do the echoing of the input it receives, as it processes it.


Minus: The NEW-ENVIRON option

One of the options that can be negotiated between the server and the client is NEW-ENVIRON. It allows the server to query environment variables on the client. In practice, there is only one variable that Windows Telnet gives a useful value for and this is SYSTEMTYPE which always has the value WIN32. Now this is useful for identifying that Windows Telnet is in use, but when one of the reasons you might want to know that it is being used is to avoid negotiating and losing its local echo.. it's not really helpful.

Other than SYSTEMTYPE, another of the standard environment variables that is supported for this option is USER. But don't think about querying it if Windows Telnet is in use, because it will crash the user's client.


I guess if someone wanted to discourage use of Windows Telnet, they could detect it through this option's SYSTEMTYPE variable with a warning message about how it should be avoided due to instability. Then do another option request for the user variable and cement the truth of the claim by crashing the client. At this point, it is irrelevant to the user whether the client is echoing locally or not.

Telnet and MUD server development

There's a lot to writing a MUD server these days, and while I have never done so, I imagine even more to writing a MUD client. Between implementing official negotiation options, unofficial negotiation options (like ATCP, MCCP, MSP, MXP and more), terminal support and the game itself, it all adds up.

A MUD server can only be as good as the clients that access it, allow it to be. Those clients may have eccentricities, like a lack of or just incorrect support for negotiation, or a lack of or just incorrect support for terminal escape sequences.. or both. Depending on its developer, the server may have to tiptoe around or avoid making use of client functionality in order to not break those clients that are unable to handle it.

Windows Telnet, bane of negotiation

Back in the day, the advantage of Windows Telnet was that it came installed with every copy of Windows and because of that was a convenient client for a user to take advantage of. No longer coming preinstalled, and a user has to go through a slightly arcane process to install it themselves. As such, as a client it is a lot less relevant to server developers than it once was.

It is still relevant to users however, who perhaps might be using computers in a lab where it is installed, but lack the ability or convenience to use or install another client. And functionality-wise, it provides comprehensive support for ANSI/VT-100 escape codes, something that is lacking in most all of the specialised MUD clients that are available these days.

The problem that it poses, is that if it receives any negotiation whatsoever from the server, it stops locally echoing the text the user enters. It does not handle the negotiation options that the server might send to resume locally echoing, and to resume local echoing the user themselves has to enter the settings screen to toggle the relevant setting. This means that if Windows Telnet is to be supported, no negotiation can be done until the server knows that the client being used is not Windows Telnet.


Negotiation and junk characters

Even if Windows Telnet is not supported, there are clients that have no support, or limited support for negotiating. If negotiation is done, depending on the characters involved, this may result in junk output to the user. This is not something that makes those clients unusable, but it does provide a second rate experience for the user. One client which displays junk characters when it receives selected negotiation options, is GMud.


Terminal input and output

The standard way that terminals work is by having the user enter a command, then displaying the output. The text that comprises these two activities is interleaved and provides a overview of what the user has been doing.


MUD clients have moved away from interleaving input and output and generally provide an input field below the output area, for some clients likely for ease of implementation, but for most, likely because it provides a better experience for the user.


One downside of this approach is that there is no cursor in the output area. This shifts away from the range of escape sequences that allow dynamic update of the output area. In the past, this was used to great effect to allow implementation of console applications like Alpine (an email client), Nano (a text editor) or Rogue (a computer game).

Image from wikipedia
Terminal display generation

Consoles use escape sequences to generate the dynamic displays, like those mentioned above for Alpine, Nano and Rogue. But more often than not, implementing the support for the extent of escape sequences to generate those displays, is not something that is important to implementors of MUD clients. On one hand, it is not a widely understood subject perhaps because it is usually delegated to libraries like curses. And on the other, MUD clients are moving towards providing a more graphical experience that can negate the more common needs the escape sequences are put to. The most commonly supported escape sequences in MUD clients these days merely make use of a wider range of colours, through an implementation of the 256 color mode that xterm first provided.


Unfortunately, the state of escape sequences is comparable to that of negotiation. Clients that can handle the most common sequences, may display some of the less common as junk characters. Even clients that aspire to emulating xterm, as Putty does, may under given circumstances dump sequences to the user.



Two different cases are illustrated by the preceding screenshots.

If the server does not negotiate with Putty otherwise, Putty will operate in line mode where it sends the user's input to the server as the whole lines as the user presses the enter key. The escape sequence \x1b[0x provokes the shown response from Putty, but because the user's input for the current line has not been sent yet, it is displayed as part of that input to be sent when the rest is.

GMud however, does not handle this escape sequence and never even considers sending a response to it. Even worse, it does not make an attempt to process it, and just dumps it at the user.

Conclusion

If the server developer does not mind providing a third-rate experience in Windows Telnet, and junk characters showing in arbitrary clients that a user may have chosen, then this is not that big a deal. Of course, they also have to accept that dynamic displays can not be generated through escape sequences for their users.

But MUDs have been around for over sixteen years, yet there are no resources which document these problems. And no documentation for what constitutes standard telnet and terminal functionality for both new MUD servers and clients. Developers seem to generally read about bits and pieces and cobble together something according to whatever parts of the picture they are aware of.

Personally, I believe it should be possible to define all these things. What different MUD clients handle. The ways in which they behave that create problems for server developers. How to work around all the eccentricities in a way that provides a first class experience to the user, or simply allows the server to say "hey, your client just doesn't do what this MUD needs it to, I see you are using this operating system, here's the name of a suitable client."

Is it possible to identify that Putty is being used without negotiating or sending escape sequences and having their responses dumped to the user? Yes. Is it possible to identify that Windows Telnet is being used without using negotiation and implicitly disabling its local echoing, and also avoiding dumping junk characters to the users of other clients? Yes. But there are a lot of MUD clients and this is an area that needs further exploration and testing. I think it should be entirely possible to define a database of information that allows client fingerprinting, through a careful step by step probing of a client as it connects.

Monday, 14 December 2009

Reverse engineering Amiga software

When I was young I had to walk to school, and it was ten miles both ways, in bare feet, always raining, with snow coming at me sideways on gravel roads. Oh, and everything was a lot more interesting then, because nostalgia hadn't been invented yet.

Living in New Zealand, television shows and movies made their way over here quite a while after they had debuted wherever they originated from. And so did computer games of course. Unfortunately, the only way to get access to software in a manner that didn't require a lifestyle consisting of yacht ownership, smoking jacket wearing or coke sniffing through one hundred dollar bills, was buying pirated disks through the mail.

Reverse engineering

One of the most interesting parts of the pirated software, was the material added by the groups which had originally pirated it. Take for example this video taken from the first disk of Pools of Darkness.


The Amiga console supported an extended range of ANSI commands, some of which allowed moving the cursor an arbitrary number of pixels. By printing a character, then moving the cursor so the next printed character would overwrite some of the last, and repeating this, an image could be rendered in the console. The slow rendering of the image is also a nice effect.

There was one commercial disassembler available for the Amiga, named ReSource. Like IDA Pro, it is an interactive disassembler, an environment where code can be progressively reverse engineered over time.

Here's a screenshot of the SKID_ROW program disassembled in ReSource:

Interestingly, the slow rendering effect is the maximum speed of the console, as the screenshot shows that the complete text to be displayed is written to the console as one lump.

A wiki

In any case, there's a wealth of interesting material preserved from the early days of the Amiga when it was still a contender. If you share my interest in examining it, the most convenient way to do so is by running WinUAE, and perhaps running ReSource within that. But there is also domain knowledge in the tip and tricks to being able to access some of the data, and also what tools can be used in which ways.

I suggested that it would be nice to collect this knowledge on a forum and someone responded by suggesting I start a wiki on one of those sites that are around these days. On a whim, I decided to do so, but in order to make it both easier and more appealing for others to jump in and add their own content, I seeded it with all the information I could find or recall.

It is all very well to start a wiki and throw some information in it. But there must be some level of required information to be present, before others with an interest will consider it a viable place to participate. Or before the effort required on their part to add whatever they may have a whim to, is sufficiently reduced perhaps. I spent a day and a half filling this wiki, before announcing it. It will be interesting to see whether anyone jumps on board.

Link: Reverse engineering for the Commodore Amiga

On a related note

The effect of the text rendering shown above, reminds me of the demo DOS by Andromeda. Embedded Youtube video follows..


Let's face it, AmigaDOS in Kickstart v1.3 just looked plain cool with its blue background and white lines.

Sunday, 22 November 2009

Using Steam in New Zealand

When I buy a game online, I expect to be able to download it as fast as my connection will allow. In New Zealand, living close to some sort of cabinet where proximity is supposed to mean faster connections, this means a standard download speed of ~500k/s.

Steam unfortunately has a standard download speed of ~50k/s.


However, it is a much cheaper way to buy games. I saved $23NZ buying it through Steam instead of through a local store.

Thursday, 2 July 2009

LPMuds.net: A word from Lars

Link: A word from Lars

LPMud (and corresponding the MudOS fork of it) is still a remarkable achievement. It is a virtual machine with a scripting language, code reloading, versioned instances of objects and provides an environment for a game to be developed in.

However, the way LPMud was licensed (covered in passing here) meant that commercial use of the driver was a violation. This didn't stop a furor over MUDs that used it commercially. There was various talk over the years about finding all the contributors and relicensing it, but one problem I recall was that no-one was in contact with Lars Pensjö, the author and namesake.

So, it was interesting to see the above linked post by Lars, and to hear his take on LPmud. This doesn't in anyway affect the licensing situation, I seem to recall other contributors, and perhaps a lack of record of who exactly contributed also stood in the way.

Friday, 1 May 2009

nolimyn: A Stackless-based MUD codebase

Link: nolimyn @ Google Code

It is always nice to see people using code you have written.

nolimyn is a MUD codebase, derived from a simple MUD script I wrote as an example for the Stackless Python examples project.

Monday, 6 April 2009

topmudsites.com: Mud Server Status Protocol

Link: MSSP (Mud Server Status Protocol)

A new development to make MUDs more visible. MUDs which support it can be registered to be scanned automatically, allowing them to have update to date entries in MUD lists.

MSSP Mud Crawler - The only site scanning MSSP supporting MUDs as yet.

The original design is to embed it in the telnet option negotiation, and there's some talk about adding support for it to the maintained version of MudOS, FluffOS.

The telnet option negotiation follows the standard approach:

IAC WILL MSSP    indicates the server supports MSSP.
IAC DO MSSP indicates the client supports MSSP.
IAC DONT MSSP indicates the client doesn't support MSSP.
It is expected that the work involved in getting existing MUDs to implement a telnet option negotiation solution may be prohibitive, and to that end, a easier to add, normal text-based login screen mode is also being fleshed out.
A crawler sends the following text when it connects:
MSSP-REQUEST\r\n
The MUD server then responds with:
MSSP-REPLY-START\r\n
Followed by a sequence of lines in the pattern:
KEY\tVALUE\r\n
Etc..
The MudBytes forum has a dedicated forum for MSSP.

Wednesday, 25 February 2009

Implementing code reloading, part 1

This post continues on from A custom namespacing system for Python, part 2.

Code reloading

There are many situations where it is useful to be able to update the code an application is using while it is running. Some of the most common are when:

  • A service is being provided by the application and interruption to that service for its users is undesirable.
  • Starting up the application takes an excessive amount of time and modifying its behaviour without restarting it can allow for more productive development.
  • The state which the application maintains is built up after it was started, and the ability to modify behaviour of the application without restarting it allows interaction with the accumulated state.
The obvious way to implement the process of code reloading is to take new versions of objects from a changed file and replace the existing versions which were taken from the file before it changed. This approach will be explored first.

Code reloading through overwriting

With the custom namespacing system previously created, a script file can be loaded and its contents placed into a given namespace.

First a script needs to be loaded.
scriptFile = LoadScript("scripts\example.py", "game")
Next the script is run, placing its contents into the given namespace.
RunScript(scriptFile)
This can easily be hooked up to file change events.
def OnFileChanged(filePath):
scriptFile = LoadScript("scripts\example.py", "game")
RunScript(scriptFile)
A simple overwriting code reloading system can be implemented by repeating this process every time a file changes, which is exactly what would happen if there were actually a file change event hooked up in the manner shown. In response to a file change, the previously created objects from before the change are overwritten by the newly created versions from the freshly changed file.

Overwriting functions

If each time a function is called it is accessed via the namespace, the newest version of the function will be the one which is called. This makes sense, as this is the whole point of the overwriting process which happens on a reload. If a programmer wants to keep calling the same version of the function and not start calling new versions as they get put in place, they need to keep their own reference to the function to call, avoiding use of the namespace.

Overwriting classes

Let's try a few things. To start with, assume that "example.py" provides a base class called BaseObject.
class BaseObject:
def PrintClassName(self):
print self.__class__.__name__ +" (original)"
If there a class is defined, it is likely that at some point that class will be instantiated.
import game
baseObjectBeforeReload = game.BaseObject()
At some point "example.py" is changed, specifically the BaseObject class, triggering a reload. This is the new version of the class:
class BaseObject:
def PrintClassName(self):
print self.__class__.__name__ +" (reloaded)"
Now the namespace entry game.BaseObject will be a new class with the old version overwritten. And since the previously created instance baseObjectBeforeReload was created from the old version of the class, it will still be using that old version of the class.

Executing this code:
baseObjectAfterReload = game.BaseObject()
print baseObjectAfterReload.__class__ is not baseObjectBeforeReload.__class__
baseObjectBeforeReload.PrintClassName()
baseObjectAfterReload.PrintClassName()
Gives this output:
False
"BaseObject (original)"
"BaseObject (reloaded)"
For certain applications, this might be a desirable result. It won't update the behaviour of existing instances which are in use within the application, but it will give a different behaviour for new instances created from the newer classes. If a programmer still wants to create instances from older versions of the class, they can do so by taking a reference to the class from the namespace before a reload has happened and the processing of a changed file has overwritten it.

Limitations to naive class overwriting

However, things get more complicated when inheritance is involved. For the purposes of illustration, here are two classes, BaseObject and its subclass Object.

In the file "example1.py", BaseObject is defined:
class BaseObject:
def TokenFunction(self):
print "TokenFunction called"
In the file "example2.py", Object inherits from BaseObject:
import game

class Object(game.BaseObject):
def TokenFunction(self):
game.BaseObject.TokenFunction(self)
There are two things to note about the Object class:
  • Object.__class__ is a direct reference to the whatever game.BaseObject was when the class was loaded.
  • game.BaseObject in TokenFunction is obtained when the function is executed, this means it is not guaranteed to be the same class as Object.__class__.
When these two classes differ, calling TokenFunction will cause the following error:
TypeError: unbound method TokenFunction() must be called with
BaseObject instance as first argument (got Object instance instead)
This is the primary limitation in the overwriting approach. If this approach is used, then a subclass can never refer to its base class through a namespace reference. It needs to do so through a reference to a version of the base class which won't change. self.__class__.__bases__[0] fits the bill, but using it each time is too unwieldy, and it is easier to store a reference to the version of the class used.
import game
BaseObject = game.BaseObject

class Object(BaseObject):
def TokenFunction(self):
BaseObject.TokenFunction(self)
If a programmer can live with this limitation and does not find the need to keep track of the older references too cumbersome, this form of naive overwriting might be more than sufficient. But to be realistic, this requirement is too impractical to work with.

Overwriting classes "intelligently"

By extending the reloading process it may be possible to remove the need to keep track of those older references. When a class is updated, all the subclasses which inherit the old version need to be updated to be compatible with the new version. The base classes of a subclass can be found in the __bases__ attribute of the subclass. If the reference to the old version of a reloaded class in this attribute is replaced with a reference to the new version, then the above error does not occur.

This can be done in the following manner:
import gc, types
for ob1 in gc.get_referrers(oldBaseObjectClass):
# Class '__bases__' references are stored in a tuple.
if type(ob1) is tuple:
for ob2 in gc.get_referrers(ob1):
if type(ob2) in (types.ClassType, types.TypeType):
if ob2.__bases__ is ob1:
__bases__ = list(ob2.__bases__)
idx = __bases__.index(oldBaseObjectClass)
__bases__[idx] = game.BaseObject
ob2.__bases__ = tuple(__bases__)
But the opposite case is now a problem. If existing references to the old version of the class are stored directly before the reload and used after it, the error can still occur.

This approach can be further extended to cover additional cases, like replacing references to the old class in global dictionaries, but there are references which cannot be replaced. This still leaves the burden on the programmer having to know what cases they need to avoid, in order for this code reloading solution to be workable.

Unit tests illustrating the overwriting approach and general limitations in code reloading approaches can be found here.

What next?

It is useful to know the limitations to code reloading, in order to determine how they apply to a given approach, and how they affect the behaviour of any reloaded code whatever the approach taken.

This post is followed by Implementing code reloading, part 2.

Edited: 2009/02/25, 6:00PM. Rewrote the last couple of paragraphs and linked to the next post.

Friday, 30 January 2009

gamedev.net: Why do devs not want customer feedback?

Link: Why do devs not want customer feedback?

A pretty good thread on the value of customer feedback and the reasons it is or isn't as valuable as different factions think it is.

stimarco wrote: [Link]

I've seen behind the scenes of a certain, well-known, very popular MMO based here in the UK. The tools they have to use are just awful; easily the worst scripting language I've ever used, and I've used one which only supported single-bit variables. The company makes a big noise about its utterly irrelevant employee perks and awards for same -- baskets of fruit brought to your desk; games room; etc. -- to attract new people, but the place was like The Stepford Wives. (I was there for a two-day interview. Before the end of the first day, I already knew I didn't want the job; I'd wasted half a day trying to write a trivial game in their scripting language which I could have rattled off in 680x0 assembly language in about 30 minutes! I stuck it out because I was intrigued by the actual game design issues they faced and wanted to know more about it. They have some great people working there, but they're seriously crippled by the tool-chain, IMNSHO.)

Now, this company rakes in six million dollars a *month*, but they didn't want to replace the tool-chain with something that's actually fit for purpose, and pay some interns to convert the old scripts to the new system. The developers were well aware that their tools sucked, but the powers that be weren't willing to take a little short-term pain to radically improve their employees' productivity over the long-term. Bah!
Nice story.

Oluseyi wrote: [Link]
It takes a tremendous amount of work to obtain operable data from user feedback, even when you're sitting there looking at the user and videotaping him/her. Many companies simply don't have the infrastructure in place for this, and for products that do not generate the sorts of revenues that Microsoft had seen for Halo and Halo 2, even a scaled-down version of the test facility may be cost prohibitive.
A post with a reference to a publication detailing previous industry efforts stands out amongst the rest of the opinions.

Anyway, there's a lot of interesting posts, including discussion about the silent majority, the vocal minority and who should have what say and whether their perspective is relevant with a high signal to noise ratio.