Five years ago, I started PyLink as a challenge: create a multi-network IRC services engine that would be the basis for transparent relaying between networks. The goal of this project was simple: allow networks to loosely federate and share channels while still maintaining their autonomy (opers/services/distinct branding). At the time, this idea really wasn't revolutionary; another project called Janus did pretty much same thing over a decade ago, but suffered from a lot of code rot over the years. I wanted to see if I could make something better.
Implementation wise, PyLink's Relay feature performs "puppeting" over a services link: on each participating network, it creates virtual clients for remote users to send messages and events from. And the initial implementation worked fairly well. While designing the project, I wrote up a pretty lengthy protocol spec for IRCd integrations, and this provided a functional baseline for Relay. In particular, I wanted the IRC integration to be as native as possible, so that moderation tools, modes, and cloaking generally work fine over Relay.
In 2016, I implemented a Clientbot transport. Unlike PyLink's other protocol modules which link as a server, Clientbot uses an IRC bot and forwards users and state info back to all the networks fully connected to Relay (i.e. via the services link). This feature had been part of Janus for quite some time, and many requested the same in PyLink Relay. So I went around and did it, and the end result worked okay. (Clientbot even supports IRCv3 features, woohoo!)
..But, Clientbot quickly became one of many implementation headaches that I wish I thought through more when I first designed it.
Recall that the protocol spec represents what a services server is able to do. This includes things like freely creating virtual users, and keeping track in detail which users are connected and in which channels.* Now, we have the task of providing the same interface given the very limited capabilities of an IRC bot. In short, I did a lot of stubbing to make the Clientbot interface appear reasonable. Effectively the whole process became: create virtual users in the state only, and delegate any attempts to send messages from them to a custom set of handlers. This became the
relay_clientbot plugin, which catches these events and posts them as text into a target channel, much like any other relay bot.
But Clientbot also broke another key expectation:
From the beginning, PyLink tracked users by unique UIDs. Every robust IRC server protocol these days uses them, including UnrealIRCd, InspIRCd, and anything else based on TS6 or P10. (The premise, I'm assuming, is that state tracking is a nightmare if your user keys change all the time.) Unfortunately, IRC C2S does not give you this luxury. Instead, I had to create virtual UIDs to represent every user Clientbot sees, the first time it sees them. And mind you, IRC C2S doesn't explicitly tell you at all when users connect either; you have to just enumerate users as you see them.
And, we need to remember to filter out pseudo UIDs from all outgoing messages. There are many places where these can leak, like in mode arguments or kick targets. (As you can imagine, I screwed this up too on multiple occasions.)
* For those who don't know, keeping track of state is a requirement for all IRC services (unless you don't have any code that needs it). Server protocols are fundamentally different from the client protocol, and client commands like
/who simply do not work here. Servers send each other all the state info that they know about on connect (channels, users, etc.), and expect the other end to keep track of everything themselves.
In 2019, we had the ambitious idea of bridging Discord to IRC using the same protocol specification. Earlier in v2.0, I had already refactored PyLink's core to separate the code most specific to IRC (e.g. connection handling) from the rest of the state tracking tools. So, I thought this would work fine (oh boy, was I wrong).
To summarize, pylink-discord broke even MORE assumptions. I'll write them here in list form to keep things readable:
relay_clientbotto the protocol module itself, to support username spoofing via webhooks..
Plus (and this is really the kicker), we had to monkey-patch all of Python's networking and threading stack for our Discord library to run. And this dependency totally doesn't break in confusing ways whenever new Python versions release.
When Python 3.8 released, pylink-discord broke down completely. For reference, this is the default version shipped with Ubuntu 20.04, so anyone who upgraded early now had a services server that refused to start with no error message whatsoever. And at this point, I was really doubting the feasibility of pylink-discord.
More broadly, I've come to realize that PyLink is simply the wrong tool for bridging together multiple chat platforms. PyLink's APIs were designed to develop IRC services, and had to be tightly coupled to IRC details as a result. Hacking pylink-discord in the way I did was a mistake, and I should've realized it much earlier.
In some ways, this post is a call for help. I don't have nearly as much time these days to work on PyLink, but the project still has way more user interest than developer interest. Just look at this commit breakdown from the last five years:
$ git shortlog -sne 3600 James Lu <firstname.lastname@example.org> 29 Daniel Oaks <mail@hidden> 25 Ken Spencer <mail@hidden> 15 Mitchell Cooper <mail@hidden> 10 Celelibi <mail@hidden> 3 Ian Carpenter <mail@hidden> 3 Ken Spencer <mail@hidden> 2 Austin Ellis <mail@hidden> 2 Celelibi <mail@hidden> 2 Jordy Zomer <mail@hidden>
For something like pylink-discord to be sustainable, a lot of work needs to happen to modernize PyLink and rid it of tech debt.
PyLink changes maintainer 5 times in a row and ends up unmaintained like Janus.
Recently, I've been working on a stateless alternative to all this bridging fuss. It's called RELAYMSG, and it's about as feature complete as say, Discord webhooks. Essentially, messages are sent from virtual users, but those users are not represented in the state at all. Instead, the local IRCd translates commands sent via RELAYMSG into PRIVMSG from a virtual users. (Yes, this method has its ups and downs, but it conveniently makes the code 50 times simpler.)
You can find the IRCv3 proposal for RELAYMSG here: https://github.com/ircv3/ircv3-specifications/pull/417 †
So far the following vendors have implemented it:
† Yes, I know this proposal is controversial. I also feel that there's a non-zero amount of people who think bridging solutions for IRC are an abomination. But I don't think relays are a bad idea in general - after all, it's just a form of interoperability. As long as IRC is an open platform, I will happily hack on it and add features that work for my community. And I hope others feel encouraged to do the same.