QuakeWorld.nu Forum

QuakeWorld.nu Forum

You are not logged in.

Announcement

Hi! Please read the rules. Please pay close attention to rules 1 & 5 in particular!

#1 10 Oct 2009 21:19:05

Spirit
Quaddicted Admin
Registered: 14 Jan 2006
Posts: 952
Website

Frag heat maps from MVD demos

Ever since I saw Valve's heat maps for their games I wanted to see some of QuakeWorld. Yesterday that itch returned and I decided to give it another go.

This is what I ended up with (each frag is plotted as transparent dot, so if there are many frags in an area it will show up more bright/solid):

http://www.quaddicted.com/stuff/dm3_heatmap_120000fragsfrom400_4on4games.jpg
400 4on4 games, 120000 frags on DM3.

http://www.quaddicted.com/stuff/dm2_heatmap_70000fragsfrom170_4on4games.png
170 4on4 games, 70000 frags on DM2.

http://www.quaddicted.com/stuff/e1m2_heatmap_60000fragsfrom140_4on4games.jpg
140 4on4 games, 60000 frags on E1M2.

http://www.quaddicted.com/stuff/dm4_heatmap_8000fragsfrom1v1.png
That boring map I hate so much (1on1, 8000 frags).

http://www.quaddicted.com/stuff/dm2_heatmapb.png
DM2 with smaller points, I think it is the same 70000 frags but I am not sure. Did not matter much though when I tried with ~10000. I like how this shows how deaths are likely to happen next to walls.


This is very not scientific. For example some frags from baldm3 and even ztndm3 matches did sneak into the DM3 one because I did not care. Also since the demos came from 2002-2009 there's got to be rule changes involved. Anyways, the majority should be good and thus those good frags easily outnumber the bad ones. Yay for stochastic(?).

If you have a specific request, just provide me with urls to the (mvd) demos.

How to do it yourself (on Linux)
Warning: This is very very ugly. Manual work is needed way too much. If you plan to do more than 4-5 I urge you to write a quick shell script.

Prerequisites: gnuplot 4.3+, imagemagick and some knowledge of your shell.

1. Get and compile Cokeman's amazing MVD Parser. svn co http://qw-dev.net/svn/mvdparser mvdparser

2. Get bsp2bmp (is there a official host of it?). Set maxX, maxY and maxZ to 4096 and minX, minY, minZ to -4096 by uncommenting the for loop after printf("Collecting min/max\n"); and, well, setting those fixed values instead. Compile bsp2bmp then (and I recommend renaming the binary, I called mine bsp2bmp_spiritugh).

3. Get the demos you want to use and put them to mvdparser and its .dat files. Smart people would modify the template.dat now so it only outputs the *events.log files with the coordinates as "xxx yyy zzz" (or even "xxx yyy 15", see below). I noticed that possibility way too late and cannot be arsed to play with it now. Well, run mvdparser for each mvd you want now. I used find . -name "*.mvd" -print0|xargs -0 ./mvdparser

4. cat *events.log > map_events.log

5. Remove all the unneeded stuff and set all Z values to the same (15 is good for the render later). The view is from above so only X and Y are interesting. Gnuplot treats the Z value as actual value of the XY point which is why you want the same value for them all. I used my precious jEdit for this step. Using sed would be much much better...

6. Now create the map outline with your fresh bsp2bmp hack. ./bsp2bmp_spiritugh -p0 -s4 map.bsp map.bmp. p0 means no padding, s4 sets the scale.

7.

gnuplotheatmap.gnu wrote:

set nokey
set notics
set bmargin 0
set tmargin 0
set lmargin 0
set rmargin 0
set view map
set pm3d
set xrange [-4096:4096]
set yrange [-4096:4096]
set terminal pngcairo transparent size 2310,2049 #You cannot set a fixed size of the actual graph, only the whole render. So this is an awkward hack (together with the cropping later). This MUST be pngcairo for transparency
set output "tmp_heatmap.png"
#plot 'map_events.log' with circles linecolor 1 fillstyle transparent solid 0.2 noborder
#plot 'map_events.log' with points

8. This is ugly. First the cropping to remove the clutter on the plot. Then I flip and flip because I couldn't figure out how to do without. Well, it works, so ... party!

gnuplot gnuplotheatmap.gnu && convert tmp_heatmap.png -crop 2048x2048+1+1 -flip tmp_heatmap.png && composite tmp_heatmap.png map.bmp map_heatmap.png && convert map_heatmap.png -flip map_heatmap.png

9. Crop the image and give it a more verbose name.


TODO for smart people:
-Create a tailored template.dat
-Write a bash script that takes a map name (AAA.bsp) and does the rest all by itself....
-Crop the end image automatically.
-Pick two players and a map they frequently play(ed), then paint them in different colours.
-Create those really gorgeous heatmaps by using a raster on the map and counting frags per square (that's how I would do it).
Further links that might be of interest:
http://hlstatsx.hlds.pl/hlstats.php?mod … p;game=dmc
http://forums.interwavestudios.com/topi … #entry1336 (talk to Cokeman if you want to give this ago)

Special thanks for their help to Cokeman and _KaszpiR_.

Last edited by Spirit (16 Oct 2009 18:54:00)

Offline

 

#2 10 Oct 2009 21:51:29

Cokeman
ezQuake developer
From: Sweden
Registered: 29 May 2006
Posts: 74

Re: Frag heat maps from MVD demos

Really nice stuff! Nice to see someone else playing with stuff like this smile Wish I had more time to satisfy the statistic nerd in me... Anyway, as you say, a custom template.dat would be nice (maybe I should do it since I was the one that wrote the idiotic parser for it wink... Rewriting the mvdparser to not be as nasty would be even nicer + you could get heatmaps for a bunch of other stuff (other than kills that is) with it.

Offline

 

#3 11 Oct 2009 01:14:12

_KaszpiR_
Member
From: Warsaw
Registered: 10 Oct 2009
Posts: 70
Website

Re: Frag heat maps from MVD demos

HLStatsX CE is a project that parses logs from Half-Life 1 engine based games (like Deathmatch classic/ Counter-Strike) and Source engine based games (like Counter-Strike:Source, TeamFortress2,Left4Dead).
It consists of mysql database for data backend, log parser in perl and php scripts  that stuff data from mysql and show it on WWW.
Heatmaps are quite 'recent' addition, they are generated by php script ran in cgi cause its time and resource consuming task, so it should be done once in a while (read, in the night).

Maybe if there would be a template.dat with defined kills messages with coordinated in it of the victim and attacker, then hlstatsx could just parse text logs and do the rest.


HLDS.pl - merely Half-Life 1 server stuff.
GFX.Quakeworld.nu - my contribution to QW
My QW stuf created over time - like patched bsp2bmp

Offline

 

#4 11 Oct 2009 12:49:51

mipa
Member
Registered: 05 Feb 2006
Posts: 255

Re: Frag heat maps from MVD demos

No one has ever died in tele-up or gl in dm2??

Offline

 

#5 11 Oct 2009 13:11:17

_KaszpiR_
Member
From: Warsaw
Registered: 10 Oct 2009
Posts: 70
Website

Re: Frag heat maps from MVD demos

if you see precisely, they did
but there are extra fragments of map drawn and nobody actually can reach that area, so it should be removed from view to not confuse wink


HLDS.pl - merely Half-Life 1 server stuff.
GFX.Quakeworld.nu - my contribution to QW
My QW stuf created over time - like patched bsp2bmp

Offline

 

#6 11 Oct 2009 15:05:04

FlePser
Member
From: The Netherlands
Registered: 16 May 2006
Posts: 325
Website

Re: Frag heat maps from MVD demos

I would like to know where I've been dying the most in all my 2on2's tongue


Custom maps for the show, episodes for the pro.
There is no theory of evolution. Just a list of animals Chuck Norris allows to live.

Offline

 

#7 11 Oct 2009 15:22:54

time!
Member
From: Spain
Registered: 08 Nov 2008
Posts: 346

Re: Frag heat maps from MVD demos

Spirit wrote:

each frag is plotted as transparent dot, so if there are many frags in an area it will show up more bright/solid

This is kinda confusing for me, maybe is for my lack of english, but when i read this i understand that the less red dots, the more frags, because the map looks more transparent, more solid, less painted. I think will be better to say "each frag is plotted as RED dot" instead.

And, that last dm2 image looks much more better, imho should be all like that one.

Anyway, really cool stuff ^^


"the quieter you become, the more you are able to hear"

Offline

 

#8 11 Oct 2009 15:34:48

Milton
Member
From: Finland
Registered: 12 Sep 2006
Posts: 19
Website

Re: Frag heat maps from MVD demos

time! wrote:

Spirit wrote:

each frag is plotted as transparent dot, so if there are many frags in an area it will show up more bright/solid

This is kinda confusing for me, maybe is for my lack of english, but when i read this i understand that the less red dots, the more frags, because the map looks more transparent, more solid, less painted. I think will be better to say "each frag is plotted as RED dot" instead.

And, that last dm2 image looks much more better, imho should be all like that one.

Anyway, really cool stuff ^^

The dots are transparent and red. The more of these red dots you have on top of each other the higher the opacity gets, resulting in solid red areas.

Offline

 

#9 11 Oct 2009 15:53:21

time!
Member
From: Spain
Registered: 08 Nov 2008
Posts: 346

Re: Frag heat maps from MVD demos

I know, but i had to look at the images, just got confused when i 1st read that. But anyway as i said, maybe is my lack of english.


"the quieter you become, the more you are able to hear"

Offline

 

#10 12 Oct 2009 09:51:23

_KaszpiR_
Member
From: Warsaw
Registered: 10 Oct 2009
Posts: 70
Website

Re: Frag heat maps from MVD demos

for Windows users i've compiled bsp2bmp and mvdparser:
http://kaszpir.hlds.pl/index.php?path=quake%2Fbsp2bmp/
http://kaszpir.hlds.pl/index.php?path=q … mvdparser/

get the zips seen in directory listing, there is packed source files and .exe

notice - i do not have antivirus at home so keep in mind to scan the files!


HLDS.pl - merely Half-Life 1 server stuff.
GFX.Quakeworld.nu - my contribution to QW
My QW stuf created over time - like patched bsp2bmp

Offline

 

#11 12 Oct 2009 11:34:08

Jjonez
Project Admin
From: Belgium
Registered: 31 Oct 2006
Posts: 66
Website

Re: Frag heat maps from MVD demos

hm. What about the periodicity that is apparent in some of the first set of images. If you look at DM2 RL room, I would expect that to be uniformly red (naive), or at least transition gradually from the spawn (or walls) at high density towards the RL (high densitity again) across a lower density zone. But the first DM2 image shows concentrations within the low density zone between the spawn and the RL. Something (maybe) similar can be seen getting away from the RA teleport in DM4. I wonder if the distance between the hotspots might be correlated to RL or SG reload time or something like that...

Then again, if you look at the 2nd DM2 image, the effect seems to have vanished, so it might just as well be an artifact of the sampling/colouring itself?

Offline

 

#12 14 Oct 2009 23:09:47

Arkage
Member
Registered: 14 Oct 2009
Posts: 1

Re: Frag heat maps from MVD demos

Hey.
Thanks for figuring this out as this is great a addition.

but I have a problem i am using windows so I downloaded the two windows exes above and tryed that way.
I got the .bmp of the map no problem but I cant get the events log.
But I cant get mvd parser to work, I keep getting cant load template.dat even tho the template.dat is in the same folder as the .exe.

Any ideas what im doing wrong?

Thanks for reading.

Offline

 

#13 15 Oct 2009 00:52:10

HangTime
Admin
Registered: 10 Dec 2006
Posts: 173

Re: Frag heat maps from MVD demos

Some observations I found 'interesting' (although probably not that surprising if I give it some thought):

-A lot of frags in dm3 water (perhaps because weak players go there fishing for weapons)
-As mentioned frags tend to be concentrated around walls.
-SNG on dm3 has a relative low volume of frags
-Because the map is 2d, you may get slightly misleading concentrations in some places i.e. people died at different heights.  This partly explains the concentrated blotches on the dm3 lifts.

Offline

 

#14 15 Oct 2009 01:26:07

dmt
Member
Registered: 13 Jan 2009
Posts: 32

Re: Frag heat maps from MVD demos

To state the obvious; frags are concentrated around walls cause players get hit by rockets or shaft and fly against them. Just saying this so we hopefully won't be seeing players avoiding walls at all costs around servers soon smile

ps. I want bigger pictures!

Last edited by dmt (15 Oct 2009 01:30:50)

Offline

 

#15 15 Oct 2009 01:38:31

dEus
Member
From: Lipsia
Registered: 27 Feb 2006
Posts: 671

Re: Frag heat maps from MVD demos

i don't think it records the place where the body lands but the place where 'death set in' ie where the frag occured. so the many frags near walls are likely to be related to the splash damage of the rocket launcher.


your awesome sucks

Offline

 

#16 15 Oct 2009 02:29:06

Runamok.foe
Member
From: Australia
Registered: 30 Dec 2006
Posts: 242
Website

Re: Frag heat maps from MVD demos

I'd like to see a 3d version of this, you add demos to the playlist and it gruns mvdparser, or loads death locations from a file already created, and you could represent the deaths with a zombie gib or a nail or something tiny and maybe (if fps feasible) you could make red light spheres with r_rocketlight .05 gl_flashblend 2 r_rocketlightcolor (red) or something. Or maybe a semitransparent sprite.

Last edited by Runamok.foe (15 Oct 2009 02:30:13)


'on 120 ping i have beaten mortuary dirtbox and reload' (tm) mz adrenalin
'i watched sting once very boring and not good at all' (tm) mz adrenalin
'i shoulda won all three imo' (tm) ego aka krohm
'dont need to learn a thing' (tm) ego aka krohm

Offline

 

#17 15 Oct 2009 12:52:24

Hooraytio
EQL Admin
From: Skövde
Registered: 21 Jun 2007
Posts: 408

Re: Frag heat maps from MVD demos

This is pretty nice but the concentration of the dots was kinda expected after having played dm2, e1m2 and dm3 for over 10 years big_smile

That sng part in dm3 is kinda low on frags and that can be experienced in games, there is no spawn in the actual room for fraghunters to guard and there is no major weapon or powerup. Instead the room is used to stock up on health and ammo, hide from strong enemies and to some extent sneak attack players coming from lifts or ra.

The many frags in dm3 water is not that unexpected either just think about the massive spam of quadrockets and pines and add all the discharges and lower tier weapon battles in there smile

Offline

 

#18 16 Oct 2009 18:46:42

Spirit
Quaddicted Admin
Registered: 14 Jan 2006
Posts: 952
Website

Re: Frag heat maps from MVD demos

I just changed the plot line in the gnuplot file. I think it did not work previously. wink

Offline

 

#19 08 Nov 2009 18:09:25

Spirit
Quaddicted Admin
Registered: 14 Jan 2006
Posts: 952
Website

Re: Frag heat maps from MVD demos

More things to try:

Use the coordinates for spawning items. I created an mapname.ent file with a couple of thousand deaths.

"classname" "item_key1"
"scale" "0.1"

scale works in Darkplaces. Sadly the items are dropped to the floor, I could not find a workaround. Use r_showbboxes to see some fancy miasma.

http://imgur.com/i8PURl.jpg
http://imgur.com/NQt3m.jpg

Try hacking the engine to make the bboxes less opaque so you can use more kills. You can use about 15000 maximum.

Offline

 

#20 08 Nov 2009 18:52:27

Spirit
Quaddicted Admin
Registered: 14 Jan 2006
Posts: 952
Website

Re: Frag heat maps from MVD demos

I also played with http://code.google.com/p/image-tempest/ today but this is the best I came up with. Meh.

http://imgur.com/WTt6Ol.png
Argh, screwed up when saving. This is 10000 frags. Took 35 minutes to render...

http://imgur.com/jDeSC.png
1000 kills. Ugly grey tint.


I used the test/pil.py and commented the weird test functions. You have to convert Quake coordinates to a 0,8192 range (from -4096,4096). And you better scale down. Rendering 4092x4092 was impossible for me, it ate RAM like crazy. The size is determined from the input image you give it.

Last edited by Spirit (08 Nov 2009 18:57:15)

Offline

 

#21 08 Nov 2009 19:13:52

Spirit
Quaddicted Admin
Registered: 14 Jan 2006
Posts: 952
Website

Re: Frag heat maps from MVD demos

.ent file again with a mod that stops the keys from dropping (thanks Supa!):
http://imgur.com/i49UM&WPJ8A&4bc2V&xHJ5E (click the First, Second etc Image links)

edit: meh, second image was during flicker. Dunno what causes it.

edit2: http://www.quaddicted.com/stuff/killmiasma.zip (noclip, r_showbboxes 1 (or 0.x)).

Last edited by Spirit (08 Nov 2009 19:17:01)

Offline

 

#22 08 Nov 2009 19:22:17

dEus
Member
From: Lipsia
Registered: 27 Feb 2006
Posts: 671

Re: Frag heat maps from MVD demos

that dm3 for some reason looks like it needs a horizontal flip badly.


your awesome sucks

Offline

 

#23 08 Nov 2009 22:23:41

Spirit
Quaddicted Admin
Registered: 14 Jan 2006
Posts: 952
Website

Re: Frag heat maps from MVD demos

ach gottdammit!

Offline

 

#24 08 Nov 2009 22:55:49

dEus
Member
From: Lipsia
Registered: 27 Feb 2006
Posts: 671

Re: Frag heat maps from MVD demos

nice heatmap still. looks like something straight from valve!


your awesome sucks

Offline

 

#25 12 Nov 2009 01:49:26

_KaszpiR_
Member
From: Warsaw
Registered: 10 Oct 2009
Posts: 70
Website

Re: Frag heat maps from MVD demos

3d boxes look amazing smile
maybe instead of boxes it could be possible to render flat semi transparent sprite?


HLDS.pl - merely Half-Life 1 server stuff.
GFX.Quakeworld.nu - my contribution to QW
My QW stuf created over time - like patched bsp2bmp

Offline

 

Board footer

Powered by PunBB
© Copyright 2002–2005 Rickard Andersson