Proposal: MUD client identification

If a MUD server developer disregards client problems, then they have little use for what I describe below. But I believe it should be possible to fingerprint the client that is being used to connect to a MUD, and to work with all possible clients in a manner where the user gets the best possible experience.

The problem with negotiation

If a MUD server supports negotiation, it typically tries sends negotiation to any client that connects. If that client is Windows Telnet, then on response to negotiation it will stop echoing the users input and will expect the server to echo it back. This is not standard behaviour, and if the developer of the server wishes to support this client, then it rules out doing negotiation unless users of this client are expected to fend for themselves.

It is possible to identify whether Windows Telnet is being used, by requesting the value of the SYSTEMTYPE variable through the NEW-ENVIRON negotiation option. But engaging in negotiation to know whether to avoid using it, defeats the point. Even if Windows Telnet is disregarded, negotiating may still result in junk characters being displayed to users of clients that do not support it or do not implement it correctly.

The problem with escape sequences

Outside of negotiation, there is another way to get feedback from the client, which can allow it to be identified. This is through the use of escape sequences which a client might support, if it has decent terminal support. Almost any client worth using, whether a telnet client or a MUD client, supports at least the basic set of sequences that allow display of text in different colours. So it should be safe to send them arbitrary sequences that will only result in a response in the clients with more advanced support. But unfortunately it isn't safe, escape sequence support is often haphazard with unsupported sequences resulting in junk characters being displayed to the user. But let's ignore that for now, and assume that we will work around it if possible.

Some clients answer to the ENQ character (5) with an "answerback" string. For Putty, this is "PuTTY". It is not widely supported, but for those clients that do support it, it is an alternative form of feedback to add to negotiation and escape sequences. However, trying it out against a connecting Putty client without any form of server-side negotiation reveals another problem. Putty works in line mode by default, any responses to special server-side characters like this one, or sequences, are included in that line. But they do not magically get sent with the line unbeknownst to the user, instead they are injected as visible text within whatever the user is typing. This is confusing and unacceptable, and it means that we are in a situation where we cannot do anything to identify Putty.

At this stage we know that if we negotiate we clobber Windows Telnet users, and if we send special output that triggers a response we clobber Putty users. Now, if we were able to identify Putty in another way which does not involve doing anything, then we could concentrate on identifying Windows Telnet without doing negotiation through escape sequences, and from there we have a lot more freedom.

Identifying Putty

While we cannot send anything at this point without clobbering the users of some clients, we still have another way to identify Putty. One thing that differentiates Putty from most other clients I have seen, is that it does active negotiation. On connection it will proactively send negotiation to the server, where every other client waits for the server to send negotiation to them.

WILL NAWS
WILL TERMINAL-SPEED
WILL TERMINAL-TYPE
WILL NEW-ENVIRON
DO ECHO
WILL SGA
DO SGA

If we receive negotiation of these commands without doing anything, we know that the client is Putty. This means that while sending negotiation is still off limit due to Windows Telnet, we are now safe to try sending escape sequences in order to identify it.

Identifying Windows Telnet

Windows Telnet has excellent terminal support. It responds to at least two standard sequences.

\x1b[0c
\x1b[0x

Now the most interesting sequence is \x1b[0x. The response to this is supposed to increment the number by two, so it would start with \x1b[2. But Windows Telnet actually sends a response starting with \x1b[3, which gives us a way to specifically identify that it is the client in use. However, we still have to deal with clients that do not properly process this sequence. GMud for instance displays this sequence and does not seem to attempt to process it at all. This is unacceptable, and rules out the use of this sequence as the next line in identification.

The next thing to try is the remaining sequence \x1b[0c. Strangely GMud does not display this and does not respond. Windows Telnet sends a standard response of \x1b[?1;0c. So in theory we can send this sequence, and on receiving a response of \x1b[?1;0c, we can proceed to send \x1b[0c and through its response identify the client in use as Windows Telnet.

The client identification protocol

We now have a protocol for identifying what clients are in use without requiring work on the part of or degrading the experience of the user. It identifies the most problematic clients and knowing that they are not in use, widens the range of techniques that can be used to identify the remaining clients.

1. If negotiation is received from the client, then the client is most likely Putty. The commands received can be compared to the known ones that Putty sends.
2. The escape sequence \x1b[?1;0c should be sent to the client, if is a response then we know the client should be able to handle other escape sequences.
3. The escape sequence \x1b[0x should be sent to the client, if there is a response and it is \x1b[3;1;1;128;128;1;0x then we know the client is Windows Telnet.
4. At this point negotiation can be used to identify what the connected client is. Windows telnet can be used without requiring negotiation.

Now it is not certain that this will work perfectly with all clients. I have limited my experimentation to the clients that I was informed were most problematic. But it is a promising start.

Regarding identification of exactly what MUD client is in use past this point is somewhat easier, given the common approach to using the MUD client name as the terminal type that can be obtained through negotiation.

The MUD development community

If someone is implementing a MUD server from the ground up, the first problem they face is that there is very little documentation telling them what might be involved. Telnet itself is documented through RFCs, with a good number existing mostly covering the different negotiation options. But there is a lot more than that to MUDs. It is strange there is so little MUD-specific documentation, yet MUDs have been around for decades. Coming back to MUD development after over eight years, it surprises me that some topics are not only undocumented, but to a large degree they have not even been explored.

The sharing of information that does tend to happen, happens within forums. Someone will post about a topic and then either people will chime in and discuss that topic and whatever others lead on from it, or they might link to previous discussions on the topic. There is knowledge out there about different topics, but it tends to be shared piecemeal. This is not that much of a surpise, the development of a MUD is a long term effort that may span many years, and it is reasonable to restrict your efforts to that which you actually need to work on.

But what is surprising, is how little value is placed in these discussions. The range of topics that might be discussed are limited, which unsurprisingly leads to them coming up time and time again. Forums where discussion takes place come and go, with all discussion that took place within them more often than not being lost forever. Remember MUD Planet? At one stage considered one of the most popular forums, the discussion that took place there is now lost. MUD-Dev was an extremely popular mailing list and for years after its web site went down, the posts made to it were unavailable except offline in the hands of individuals. MudMagic used to have forums until its administrator decided to take them down, and now the discussions that took place there are effectively inaccessible. At least two wikis have come and gone.

Frankly, beyond the possible immediate benefit of its participants, MUD development discussion is somewhat like pissing into the wind.

In memory of Windows Telnet

There was a time when Windows Telnet came installed with every copy of Windows. Because of this, it was an easily accessible MUD client for anyone who might want to connect to one. However, times have changed. On one hand the most popular MUDs are making use of custom extensions to the telnet protocol that Windows Telnet will never be able to support, and on the other it no longer comes preinstalled with every copy of Windows.

This post is a recognition of Windows Telnet. That despite its problems as a MUD client, it is still an extremely capable and versatile telnet client. Please give less weight to the listed minuses (which are useful to know if you choose to adopt this client) and appreciate its plusses.

Plus: Superior terminal support

Back in the days before Putty was around, having a telnet client available with Windows that implemented ANSI/VT100 terminal support was extremely valuable. And Windows Telnet allows use of a wide range of the escape sequence that make up this terminal support.


The preceding screenshot is of a prototype of a roguelike that I attempted to implement for the recent 7DRL 2010 competition. Windows Telnet was as capable as Putty at displaying the arbitrary screen updates that my game makes.

Minus: Reduced availability

No longer coming preinstalled with every copy of Windows, installing it is now more complicated than just downloading any other MUD or telnet client which you stumble over a link to.

Minus: Negotiation support

If connecting to a server that does not engage in telnet negotiation, Windows Telnet echos what the user types back to them as it sends it. But should the server send any negotiation, this echoing is switched off and the server has no way to switch it back on. The only way it can be reenabled within the lifetime of the current connection is if the user enters the settings screen and explicitly reenables it. This leaves the server in a position of having to do the echoing of the input it receives, as it processes it.


Minus: The NEW-ENVIRON option

One of the options that can be negotiated between the server and the client is NEW-ENVIRON. It allows the server to query environment variables on the client. In practice, there is only one variable that Windows Telnet gives a useful value for and this is SYSTEMTYPE which always has the value WIN32. Now this is useful for identifying that Windows Telnet is in use, but when one of the reasons you might want to know that it is being used is to avoid negotiating and losing its local echo.. it's not really helpful.

Other than SYSTEMTYPE, another of the standard environment variables that is supported for this option is USER. But don't think about querying it if Windows Telnet is in use, because it will crash the user's client.


I guess if someone wanted to discourage use of Windows Telnet, they could detect it through this option's SYSTEMTYPE variable with a warning message about how it should be avoided due to instability. Then do another option request for the user variable and cement the truth of the claim by crashing the client. At this point, it is irrelevant to the user whether the client is echoing locally or not.

Telnet and MUD server development

There's a lot to writing a MUD server these days, and while I have never done so, I imagine even more to writing a MUD client. Between implementing official negotiation options, unofficial negotiation options (like ATCP, MCCP, MSP, MXP and more), terminal support and the game itself, it all adds up.

A MUD server can only be as good as the clients that access it, allow it to be. Those clients may have eccentricities, like a lack of or just incorrect support for negotiation, or a lack of or just incorrect support for terminal escape sequences.. or both. Depending on its developer, the server may have to tiptoe around or avoid making use of client functionality in order to not break those clients that are unable to handle it.

Windows Telnet, bane of negotiation

Back in the day, the advantage of Windows Telnet was that it came installed with every copy of Windows and because of that was a convenient client for a user to take advantage of. No longer coming preinstalled, and a user has to go through a slightly arcane process to install it themselves. As such, as a client it is a lot less relevant to server developers than it once was.

It is still relevant to users however, who perhaps might be using computers in a lab where it is installed, but lack the ability or convenience to use or install another client. And functionality-wise, it provides comprehensive support for ANSI/VT-100 escape codes, something that is lacking in most all of the specialised MUD clients that are available these days.

The problem that it poses, is that if it receives any negotiation whatsoever from the server, it stops locally echoing the text the user enters. It does not handle the negotiation options that the server might send to resume locally echoing, and to resume local echoing the user themselves has to enter the settings screen to toggle the relevant setting. This means that if Windows Telnet is to be supported, no negotiation can be done until the server knows that the client being used is not Windows Telnet.


Negotiation and junk characters

Even if Windows Telnet is not supported, there are clients that have no support, or limited support for negotiating. If negotiation is done, depending on the characters involved, this may result in junk output to the user. This is not something that makes those clients unusable, but it does provide a second rate experience for the user. One client which displays junk characters when it receives selected negotiation options, is GMud.


Terminal input and output

The standard way that terminals work is by having the user enter a command, then displaying the output. The text that comprises these two activities is interleaved and provides a overview of what the user has been doing.


MUD clients have moved away from interleaving input and output and generally provide an input field below the output area, for some clients likely for ease of implementation, but for most, likely because it provides a better experience for the user.


One downside of this approach is that there is no cursor in the output area. This shifts away from the range of escape sequences that allow dynamic update of the output area. In the past, this was used to great effect to allow implementation of console applications like Alpine (an email client), Nano (a text editor) or Rogue (a computer game).


Terminal display generation

Consoles use escape sequences to generate the dynamic displays, like those mentioned above for Alpine, Nano and Rogue. But more often than not, implementing the support for the extent of escape sequences to generate those displays, is not something that is important to implementors of MUD clients. On one hand, it is not a widely understood subject perhaps because it is usually delegated to libraries like curses. And on the other, MUD clients are moving towards providing a more graphical experience that can negate the more common needs the escape sequences are put to. The most commonly supported escape sequences in MUD clients these days merely make use of a wider range of colours, through an implementation of the 256 color mode that xterm first provided.


Unfortunately, the state of escape sequences is comparable to that of negotiation. Clients that can handle the most common sequences, may display some of the less common as junk characters. Even clients that aspire to emulating xterm, as Putty does, may under given circumstances dump sequences to the user.



Two different cases are illustrated by the preceding screenshots.

If the server does not negotiate with Putty otherwise, Putty will operate in line mode where it sends the user's input to the server as the whole lines as the user presses the enter key. The escape sequence \x1b[0x provokes the shown response from Putty, but because the user's input for the current line has not been sent yet, it is displayed as part of that input to be sent when the rest is.

GMud however, does not handle this escape sequence and never even considers sending a response to it. Even worse, it does not make an attempt to process it, and just dumps it at the user.

Conclusion

If the server developer does not mind providing a third-rate experience in Windows Telnet, and junk characters showing in arbitrary clients that a user may have chosen, then this is not that big a deal. Of course, they also have to accept that dynamic displays can not be generated through escape sequences for their users.

But MUDs have been around for over sixteen years, yet there are no resources which document these problems. And no documentation for what constitutes standard telnet and terminal functionality for both new MUD servers and clients. Developers seem to generally read about bits and pieces and cobble together something according to whatever parts of the picture they are aware of.

Personally, I believe it should be possible to define all these things. What different MUD clients handle. The ways in which they behave that create problems for server developers. How to work around all the eccentricities in a way that provides a first class experience to the user, or simply allows the server to say "hey, your client just doesn't do what this MUD needs it to, I see you are using this operating system, here's the name of a suitable client."

Is it possible to identify that Putty is being used without negotiating or sending escape sequences and having their responses dumped to the user? Yes. Is it possible to identify that Windows Telnet is being used without using negotiation and implicitly disabling its local echoing, and also avoiding dumping junk characters to the users of other clients? Yes. But there are a lot of MUD clients and this is an area that needs further exploration and testing. I think it should be entirely possible to define a database of information that allows client fingerprinting, through a careful step by step probing of a client as it connects.