Group: Administrator
Location: Scotland
UK modem old-timer, 15 year break, now spends more time coding than playing.
Websites  /  25 Apr 2017, 12:26
Demo/Tournament archive website
I've been working on this for what seems like forever (well, over a year), and after many twists and turns it is starting to take shape, so I've uploaded to, and you guys can take a look.

The site has taken a couple of different forms, and now ends up doing... well, not any of them very well. So for starters you can't even upload yet (explanation below), but I'll keep adding bits over the next few weeks and eventually it will make more sense.
Where the data came from...

Step 1: Took all demo archives I could find, set about removing duplicates, converting .qwz to .qwd etc. Seems simple, but unfortunately compression algorithms (qizmo?) can remove some impulses, appearing to change content. Sigh.
Step 2: For each demo, work out game/match result. Turned out to be tricky due to different mods (kteams, ktpro, ktx, clanring...) using different methods over the years. From weird text strings to handling disconnected players, it still isn't 100%.
Step 3: Identify common games from different demos (think: same game, different viewpoints)
Step 4: Now a big list of games, to give any structure to the archive I tried to find as many tournament archives as were still available, imported from phptourney (website and databases), challonge, eql, nqr, etc etc.
Step 5: Identify common players across teams & games - easier with the tournament information, but again we have different aliases for each player over time, so some mistakes could have been made here too.
Step 6: Race records: Extracting information from demos again, these have also changed format recently (due to me, i was kicking myself here) supporting multi-racing.

There's still heaps to do, and you can't create accounts yet in order to upload. I've kind of screwed up here, as by creating the player records in step 5 above, I can't automatically link the person called Milton to the player called Milton, unless they're

Anyway, things that will be happening soon:

1) Being able to create login accounts (optional logins with social media). In future you will be able to login to quake servers with this account, either to prove identity to others or for uploading new race records/routes etc. Hopefully this could lead to automatic ladder system etc.
2) The demos are all stored as .zip files but uncompressed when you hit download, this is inefficient on the server - will look at converting to uncompressed instead (currently 37GB zipped, 84GB unzipped) and leaving the webserver out of downloading process.
3) Servers automatically uploading games/race-records (and demos) to the website
4) Going through all the tournaments and fixing as many bugs/missing-info as possible (please shout when you see them)

(I'll also try and make everything available to other clients and websites so that this is just a central repository, not taking over from the others)
2017-04-25, 14:14
Sounds like a monumental effort, great job as usual!
2017-04-25, 16:35
2017-04-26, 09:18
Is there some kind of API available for all of this?

Perhaps you could have someone working on frontend bits and pieces while focusing on the backend part of things? There are a lot of third party Angular themes available that one could start developing off of pretty quick to get a lot of stuff "for free". I even bought one to use for a similar idea (take two on a modern tournament website...) but it almost feels like a duplicated effort... :}
2017-05-01, 11:25
Well, one word really WOW. Great effort, nice to follow some old forgotten games/results. I might have some missing demos somewhere on a drive
2017-05-03, 18:34
Just fucking awesome ...... well played indeed.
2017-05-23, 08:45
Wow this looks absolutely amazing! Just what was needed really. There are some good demo archives out there, but it always took ages to find what you looked for! I've been dying to watch some 4on4 games, so will definitely check out some old finals tonight 8)

How about putting this as a news article? I think a lot of people don't notice the blog posts? It took me one month to notice this one!!
2017-06-28, 14:18
I admire your work ethic
You have to be logged in to be able to post a comment.