Jump to content

Photo

Results: The Hawken Community's Collective Opinions

* * * * * 2 votes

  • Please log in to reply
110 replies to this topic

#1
JeffMagnum

JeffMagnum

    Advanced Member

  • Members
  • PipPipPip
  • 781 posts

*
POPULAR

As some of you may know, I recently posted a thread asking for volunteers to give me their ideas on certain issues the community currently disagrees about. After hours and hours of data entry and crunching numbers, I’ve finally ended up with something useful and insightful. If you’d like to view all of the data I have in Excel and Word, here’s a download link to a .zip. Feel free to use it however you want.

 

MMR ended up being the only meaningful statistic (except for region on a question or two), so I’ve ignored everything else to save myself a lot of pointless work. Here are the demographics of everyone who responded:

 

N=42

n 0-1600 MMR=2

n 1600-2000 MMR=13

n 2000-2400 MMR=15

n 2400-2600 MMR=8

n 2600-3000 MMR=3

Average MMR=2153.756098

Median MMR=2168

Average Ping=74.292

US Players=32

EU Players=7

UK Players=2

AS Players=1

 

Pie charts make it easier to visualize:

 

ehMLvXC.png

uh2H6hu.png

 

This is the list of statements with relevant information about the responses to each of them:

Spoiler

 

What We Agree On

 

 

There’s clear consensus on some of those statements, and I think we should discuss those first. 76% (32/42) of the people surveyed believe air speed is in a good spot, and agreement isn’t limited to MMR ranges in particular—the mean and median are only 21.3 and 33 MMR below the mean and median, respectively.  However, wanting air speeds reduced strongly correlated with region. Despite making up only 24% (n=10) of the total sample group, players outside of the United States accounted for 75% (6/8) of the votes for decreasing air speed.

 

74% (31/42) of the respondents think the level restriction on AC should be lowered or removed. A lot of those voters also voted for universal AC, so there’s a clear desire for AC to be more accessible. I’ve wanted this change ever since AC was introduced, and I’m glad to see widespread support for the idea. It also has roughly equal support among all MMRs, with the mean being 19.7 and the median being 32 above the overall average

 

People definitely don’t want boosting to require more fuel. 90% (38/42) favor leaving it alone, making this an almost unanimous opinion.

 

88% (37/42) want to see A-Classes keep the same interval between dodging, and a staggering 95% (40/42) desire the same for B-Classes. Frequency of dodges with C-Classes isn’t agreed upon to quite the same extent, but 74% (31/42) is more than enough to say the timer should probably be left as it is now.

 

76% (33/42) would like a nerf to the Incinerator, though there’s some disagreement on how severe it should be. 25/32 favor a slight one. Nerfing Incin has pretty good support among all MMR ranges, though the median is somewhat high at 82 above the overall median.

 

Scanner in its current form is almost universally hated. 93% (40/43) of the votes there are either for it to be reworked or removed completely. Interestingly, the median and mean MMRs of the players (n=11) who want it removed are 140.9 and 132 above the overall values respectively, while the median and mean MMRs of players who just want a rework (n=29) are 27.2 and 18 below the respective overall stats.  

 

87% (33/38) of the people who responded don’t mind sustained weapons in their current iteration. I’m not too surprised here.

 

Bunker is the map with the most support (74%) for a rework, while Bazaar comes in at second with 67%. Almost everyone seems to hate Bunker equally, while players a little above the average MMR tend to find more issues with Bazaar.

 

There’s very clear support (95%) for MMR to be visible in-game to players in some capacity. Now that Scrimbot and hawken.herokuapp.com exist, hiding MMR does nothing more than slightly inconvenience players.  

 

71% of players would like to see some form of leaderboards implemented. With the new devs saying they want to encourage competitive play in a greater capacity than the old ones did, this seems like an obvious thing to add.

 

 

What We Don’t Agree On

 

 

 

Let’s get to the main thing everyone fights over first: Air Compressor. Keep in mind when looking at the chart that a lot of people voted for AC to both be left as is and made universal, so there ended up being 54 votes total. I didn’t intend for people to vote twice here, so it’s harder to draw a conclusion from the data I have.

 

x4hKBlh.png

 

What we can tell is that people who want it gone completely (n=3) and people who want it nerfed (n=8) are a small but vocal minority. The average MMR for those options is decidedly below the other two as well.

 

2PfOmHh.png

 

So what does the wider community think? I’m personally for universal AC, as it seems unfair to keep something that drastically alters gameplay behind an HC and XP wall even if that internal isn’t necessarily the best option for the slots it takes.

 

A majority of people want fuel use in the air to stay the same, but there’s a sizable minority that’d like to see it increase.

 

JayF1CL.png

 

To be honest, I was surprised by the MMR distribution here. The mean and median MMRs of people who’d like to see fuel use increase (n=14) are 43.7 and 112 above the respective overall values, while players who’d like it to stay the same (n=26) are only above by 7.3 and 56. Only one person voted for less fuel use.

 

8WpVEyr.png

 

Opinions on the current ground speeds mostly tend toward leaving them as they are, but the percentage of people who want to do that only hits 48%. Most of the other people would like to see them increased to some extent.

 

KwAmMdW.png

 

People of all MMRs tend to support the majority view of leaving speeds the same or increasing them. One person wanted a decrease, but I didn't include them on the graph since their MMR was 529 below the average and it made the info a lot harder to interpret.

 

WaKVA51.png

 

What should we do to resolve this? I personally voted for increasing them for some mechs in particular, but I’m open to keeping them as they are. Nerfing them would hurt the game pretty badly in my opinion, so I’m glad there’s virtually no support for that.

 

The Technician has always been a controversial mech, and that shows here.

 

39uVvUU.png

 

It’s clear that a lot of people aren’t a fan. 26 people (67%) are in favor of some sort of nerf or for it to be removed completely, while only 13 (33%) want it to stay as it is or be buffed. This subject had one of the most interesting distributions of votes by MMR.

 

shRpXvf.png

 

Why is this? I personally think Technician is balanced in high-level play, but in lower-MMR games where people tend to die less quickly when they fuzzy bunny up, does it completely throw off the balance if one team has one when the other doesn’t?

 

Something else came up here too. Many of the posters in my thread wanted a choice to completely rework Tech, so what are some ideas for doing that?

 

I’m glad to see so much support for getting rid of the weapon raise delay.

 

MRUy7uS.png

 

It’s pretty clear that people want it either shortened or gotten rid of entirely. I’d ideally like to see it go, but reducing it would be fine too. The first two responses predictably have a higher average MMR than the third one, but it’s honestly not as large of a gap as I expected.

 

vkug4QO.png

 

52% of the respondents voted for Last Eco and Uptown to be changed. What’s surprising is how mean MMR varies with each map, especially Uptown.

 

0FHop3p.png

 

My theory is that it’s because high-MMR players tend to hold the central Uptown chokepoints much more effectively, causing fights to become boring very quickly.

 

Another contentious issue is TTK: lots of people are happy with how it is now, and lots of other people are nostalgic for the longer TTK we used to have. Here’s how it breaks down:

 

oYUFv18.png

 

The last option wasn’t really supposed to be grouped with the first two, but I’m including it since people voted as if it was. The data is almost impossible to interpret because of that, since people who would’ve voted for increasing the TTK might have chosen the last option instead.

 

The MMR differences are odd if that’s the only reason, though.

 

Lh404PZ.png

 

I suspect it’s a combination of what I said above and people wanting to bring A-Classes in line with the other two. This is a fairly popular idea in high-MMR circles, mostly because of mobility becoming less important as the aim of your enemies gets better.  

 

bBphY5O.png

 

And that seems to be exactly what happened.

 

 

Why I Bothered to Do This

 

 

This was partially inspired by one of capnjosh’s posts in a thread of Omega’s about getting something together that shows the directions people want the game to go. I hope it helps the new devs get an idea of the community’s wants in general, plus I was curious myself about actual data.

 

Good job for getting through all that.


Edited by JeffMagnum, 28 March 2015 - 01:45 AM.

  • capnjosh, LadyTiggs, OmegaNull and 58 others like this

d1eZeG3.png


#2
Amidatelion

Amidatelion

    Advanced Member

  • Members
  • PipPipPip
  • 2474 posts

Why can't I like this multiple tiems.



#3
Garuda

Garuda

    Newbie

  • Members
  • Pip
  • 4 posts

simply amazing on the post and the info provided. you don't get the opportunity to see what is the percentages on what and how something should be changed. props for going the extra mile on mmr as well.

 

i'm really hoping this will help the new dev team on choosing a good path, not so much as a "perfect path".

Cheers to the future of Hawken!


  • Dekar likes this

#4
TheVulong

TheVulong

    Advanced Member

  • Members
  • PipPipPip
  • 99 posts

-First of all, thanks for all the hard work you put into gathering the info and making all the charts - trurly an amazing job.

-Second, i would like to point out that overall 1/4 or 1/3 of the people who voted has an opinon that opposes the one of a majority. And 1/4 and 1/3 is a big chunk of people which should not be completely ignored IMO.

-Third, i'm not sure how relevent these results are considering that there are about 700 people(according to Steam Charts) still playing Hawken and only 42 people who provided their opinion on the stated issues. 


  • axxoe and DieselCat like this

#5
CrimsonKaim

CrimsonKaim

    Advanced Member

  • Members
  • PipPipPip
  • 1235 posts

Maybe we should make all the changes for US players only (in this case) as the clear majority here are US players :D

 

 

 

(nice data btw.)


  • Dancing likes this

- Sitting next to the sound box in Last Eco -


#6
Elite_is_salty

Elite_is_salty

    Advanced Member

  • Members
  • PipPipPip
  • 1144 posts

The weapon raise delay hurts the asses of godly scout players. it should stay.

And 4k6k8k ftw


  • TheVulong, CrimsonKaim and DieselCat like this

The PC community is the red headed step child Reloaded never wanted but got saddled with when they married the PC community's mother.

---------------------------------------------------------------

Shoutout to mah real Africans out there.


#7
JeffMagnum

JeffMagnum

    Advanced Member

  • Members
  • PipPipPip
  • 781 posts

-First of all, thanks for all the hard work you put into gathering the info and making all the charts - trurly an amazing job.

-Second, i would like to point out that overall 1/4 or 1/3 of the people who voted has an opinon that opposes the one of a majority. And 1/4 and 1/3 is a big chunk of people which should not be completely ignored IMO.

-Third, i'm not sure how relevent these results are considering that there are about 700 people(according to Steam Charts) still playing Hawken and only 42 people who provided their opinion on the stated issues. 

 

I don't think they should be ignored, but a company can't cater to 25% of their customer base (assuming everyone spends an equal amount and all) while going against the 75% that wants something different. But it's not a straight either/or thing in a lot of the scenarios here, and I'd love to have constructive discussions about compromises and whatever else in this thread.

 

And it's definitely not a perfect survey or data analysis by any means, but I had to be realistic about time and effort constraints. Doing this with just 42 people probably took close to 24 actual hours, all things considered. I would've loved to do this with a proper sample size, but I didn't want to spend five solid days on it. Even if everyone active on the forums had responded to my thread, I still would've had a relatively awful confidence interval since Hawken's total population is so small. I think it represents what the forums want decently well, but not what players in general do. It might, but I can't accurately say if it does. 


Edited by JeffMagnum, 28 March 2015 - 12:06 AM.

  • DerMax, TheVulong, OdinTheWise and 1 other like this

d1eZeG3.png


#8
Anichkov3

Anichkov3

    Advanced Member

  • Members
  • PipPipPip
  • 246 posts

42 people participated in the survey. 

This is roughly speaking ~ 17% of the players (if Ashfire908 use average statistics for the month), or ~ 11% of the players (to use Steam average statistics for the last 30 days) (I say it is blurry largest average number of players - the actual number of players known only to developers and may be close to 1000 and it is then only 4%)

This is reminiscent of the street social surveys - a presidential candidate do you support  :sleep:

To know the general opinion of the players - it is useful ....

 

As I said before: 

 

 

Developers should be careful with this kind of voting. Opinion of the crowd far from always true.....

Remember that only you - the developers to decide what to do, and not always be around to listen to people who do not know what they want.

Edited by Anichkov3, 28 March 2015 - 12:14 AM.

  • FRX23, Dancing, axxoe and 1 other like this

http://www.hawken.ru- ???? ??????????? ?????????? ??????? (Website of the Russian community)

http://ru.hawken.wikia.com - ??????? ????????? ?? ???? (Russian Wikipedia HAWKEN)

http://vk.com/hawken - ???????? ??????????? ?????????? ??????? ? ?? (social network VK)


#9
TheVulong

TheVulong

    Advanced Member

  • Members
  • PipPipPip
  • 99 posts

I don't think they should be ignored, but a company can't cater to 25% of their customer base (assuming everyone spends an equal amount and all) while going against the 75% that wants something different. But it's not a straight either/or thing in a lot of the scenarios here, and I'd love to have constructive discussions about compromises and whatever else in this thread.

True.


  • Dancing likes this

#10
JeffMagnum

JeffMagnum

    Advanced Member

  • Members
  • PipPipPip
  • 781 posts

Developers should be careful with this kind of voting. Opinion of the crowd far from always true.....

Remember that only you - the developers to decide what to do, and not always be around to listen to people who do not know what they want.

 

To be fair, ADH would've been far better off if they did. HAB1 almost universally agreed the direction the game was going with Ascension was a bad idea, yet the devs pretty much changed nothing except bugs (RIP death laser AR) and a few balance things. Apparently other semi-secret groups that had early access to builds and information did similarly little to change their minds. Nept went into why Hawke probably wouldn't have been that successful anyway, but breaking the entire game in a huge way and then having to spend months undoing it probably didn't help.


Edited by JeffMagnum, 28 March 2015 - 12:38 AM.

  • LoC_TR likes this

d1eZeG3.png


#11
AsianJoyKiller

AsianJoyKiller

    Advanced Member

  • Members
  • PipPipPip
  • 343 posts

God damn this is sexy.


  • LoC_TR likes this

#12
Anichkov3

Anichkov3

    Advanced Member

  • Members
  • PipPipPip
  • 246 posts

That is why the: Remember that only you - the developers to decide what to do, and not always be around to listen to people who do not know what they want. In my previous developers had some top-secret group of players / testers who and recommended balance of weapons in the game.


Edited by Anichkov3, 28 March 2015 - 12:25 AM.

http://www.hawken.ru- ???? ??????????? ?????????? ??????? (Website of the Russian community)

http://ru.hawken.wikia.com - ??????? ????????? ?? ???? (Russian Wikipedia HAWKEN)

http://vk.com/hawken - ???????? ??????????? ?????????? ??????? ? ?? (social network VK)


#13
JeffMagnum

JeffMagnum

    Advanced Member

  • Members
  • PipPipPip
  • 781 posts

The devs are responsible for developing the game; playing it isn't their job. Without a lot of firsthand experience with the meta, balancing based on spreadsheets doesn't go very far. Besides, they ended up reversing a ton of changes people hated from the start with the Steam update. 


  • -Tj- and comic_sans like this

d1eZeG3.png


#14
DerMax

DerMax

    Advanced Member

  • Members
  • PipPipPip
  • 908 posts

This is about the single most useful thread the new forums have seen. Bravo, Jeff.

 

The survey was obviously not perfect, as it was statistically biased towards NA and had relatively few correspondents, but it still gives a good idea of what direction the game should go.

 

P.S. Get an avatar willya


Edited by DerMax, 28 March 2015 - 12:42 AM.

  • TheVulong and LaurenEmily like this

#15
ThirdEyE

ThirdEyE

    Advanced Member

  • Members
  • PipPipPip
  • 163 posts

HV3fRD8.gif


  • LoC_TR likes this

oSpBaPA.png


#16
Anichkov3

Anichkov3

    Advanced Member

  • Members
  • PipPipPip
  • 246 posts

In my opinion it is theoretically possible to add such surveys in the game itself. When Laila asked to immediately after entering the game "Help improve the game, honestly answer a few questions."


  • Luaq likes this

http://www.hawken.ru- ???? ??????????? ?????????? ??????? (Website of the Russian community)

http://ru.hawken.wikia.com - ??????? ????????? ?? ???? (Russian Wikipedia HAWKEN)

http://vk.com/hawken - ???????? ??????????? ?????????? ??????? ? ?? (social network VK)


#17
Lioot

Lioot

    Advanced Member

  • Members
  • PipPipPip
  • 309 posts

This information is amazing, as well as the amount of effort you put into this. Just wish I saw the thread earlier. Does AS mean Asia? Does this mean their is no Australian input? We want kangaroo mechs you know


  • gopherAnime likes this

#18
JeffMagnum

JeffMagnum

    Advanced Member

  • Members
  • PipPipPip
  • 781 posts

Yeah, it's Asia. Tbh I'd be open to doing another writeup and getting new stats from the aggregate data if someone else did the initial data entry from the forums to Excel. I have a tenkeyless keyboard, and it was hell to enter all those numbers on it.

 

I don't think it's necessary though, as I'm not going to be able to get an accurate sample from the game's playerbase regardless of how many people submit forms. It's always going to skew toward high-MMR viewpoints, since the players who are on the forums plus willing to participate in this are likely to be fairly dedicated.

 

Assuming a population of 2000, the confidence interval is ~15 at 50%, ~13 at 75%, and ~9 at 90%, so it's accurate enough to get rough trends. Even if every single person who's posted at least once on the forums (all 329 of them) submitted data to me and they actually represented who plays Hawken overall, it still wouldn't be that great of a survey just because of how few people are here. 


Edited by JeffMagnum, 28 March 2015 - 01:36 AM.

d1eZeG3.png


#19
Nov8tr

Nov8tr

    Advanced Member

  • Members
  • PipPipPip
  • 679 posts

Good job mate. Thanks for the hard work. Nice looking at the numbers. Hopefully it will help the Devs too.


"Nov8tr" is pronounced "INNOVATOR"

aEGHJsh.gif?1

Yes I'm really 64 yrs old. July 6, 1953


#20
Nept

Nept

    Advanced Member

  • Members
  • PipPipPip
  • 939 posts

93% of respondents think that Jeff is a nerd.


Edited by Nept, 28 March 2015 - 02:32 AM.

  • Elite_is_salty, Merl61, -Tj- and 6 others like this

#21
JeffMagnum

JeffMagnum

    Advanced Member

  • Members
  • PipPipPip
  • 781 posts

:(


  • DerMax, Nept, Elite_is_salty and 3 others like this

d1eZeG3.png


#22
defekt

defekt

    Advanced Member

  • Members
  • PipPipPip
  • 95 posts

Superb work, Jeff. 

 

You've stated yourself several times in your replies that you accept that the sample size is small, not to mentioned skewed, i.e., in the majority the people responding tend to be broadly happy with the way things are simply by virtue of them still being here, but there really isn't anything that you could have done to mitigate that.  At least some of the old folks who were put off have shoved their noses through the doors and, hopefully, participated in your survey.  (Even tho I didn't; not for any nefarious reasons, I just didn't have the time.)

 

Once again: good job, sir.


  • (Unknown)1590d2c747fabd likes this

defekt_inv.jpg


#23
MomOw

MomOw

    Advanced Member

  • Members
  • PipPipPip
  • 1039 posts

Great job !


  • (Unknown)1590d2c747fabd likes this

IRZUTYo.png

gXO9Nfd.pngmXasTsY.pngft4VqcE.png

 

KDR Member | Streamer | Priority Target II

Spoiler

#24
Elite_is_salty

Elite_is_salty

    Advanced Member

  • Members
  • PipPipPip
  • 1144 posts

You know guys. The HAWKEN official steam group has over 22k members. When a post is made there, everyone of those will get a notification. If a survey was posted on the steam group, expect a much larger number of people to voice their opinions about those important aspects of the game, and then you can get a wider view of things.


The PC community is the red headed step child Reloaded never wanted but got saddled with when they married the PC community's mother.

---------------------------------------------------------------

Shoutout to mah real Africans out there.


#25
DerMax

DerMax

    Advanced Member

  • Members
  • PipPipPip
  • 908 posts

You know guys. The HAWKEN official steam group has over 22k members. When a post is made there, everyone of those will get a notification. If a survey was posted on the steam group, expect a much larger number of people to voice their opinions about those important aspects of the game, and then you can get a wider view of things.

No one is going to spend weeks and weeks of their precious time collecting the data and analyzing it.



#26
n3onfx

n3onfx

    Advanced Member

  • Members
  • PipPipPip
  • 511 posts

 

I don't think it's necessary though, as I'm not going to be able to get an accurate sample from the game's playerbase regardless of how many people submit forms. It's always going to skew toward high-MMR viewpoints, since the players who are on the forums plus willing to participate in this are likely to be fairly dedicated.

 

 

There is one danger in this and it's leaning toward changes that would mess up the experience for lower mmr and new users. In some cases it's beneficial (making air compressor accessible earlier for example levels the playing field a bit more for new players). In other cases changes that make sense for the higher mmr/ competitive range would break the game at lower stages.

 

The biggest example of this here is giving A classes more health. I'd love for them to have a bit more since in those higher tiers people aim better and play better as a team resulting in A classes being a bit hopeless when faced with a wall of C class. But I can't even imagine what would happen for new players that can't aim well, A class would rule those matches and we'd have another 20 posts a week on the Steam forums crying about Scout.

 

I think it's a good example of a small change at the top that would make a big impact at the bottom.


Edited by n3onfx, 28 March 2015 - 04:08 AM.

  • TheVulong and gopherAnime like this

t

t

DWEH3ZP.png   CRITICAL  RqKpxHn.png    ASSIST   VDNrFxD.png

t

t


#27
Vdragon

Vdragon

    Advanced Member

  • Members
  • PipPipPip
  • 65 posts

Good and sort of useful work, but:

42 people is not really representative, but that have been said already, also high MMR != better game understanding (especially since the MMR is at best a correlation between a skill level and a number).

Another thing is , in the AC poll, the difference between the averages is 90, but your lower and higher bound are 1600 an 3000, which makes for 1400... As there's no standard deviation, no min, no max, this graphic is pretty much meaningless and only tend to push the point you may want to make seen. tldr this graph is useless because the difference is negligeable considering the sample size and the lack of crucial information needed to give a meaning to it. And look at the tech one, it's a 450 difference between lower and higher...


Edited by Vdragon, 28 March 2015 - 04:39 AM.

0FAtmIq.png


#28
(Unknown)1590d2c747fabd

(Unknown)1590d2c747fabd

    Advanced Member

  • Members
  • PipPipPip
  • 145 posts

<SNIP> but a company can't cater to 25% of their customer base (assuming everyone spends an equal amount and all) while going against the 75% that wants something different. <SNIP>

*THIS* is a very important point... particularly the bit about "assuming everyone spends an equal amount".

 

Clearly you cannot, as non-company employee, have controlled for this but the reality is that as a business the decisions made are going to be tied to revenue numbers. This is the cold hard reality of business that we all as players with desires need to remember. Its what will keep Reloaded and Hawken alive.

 

So lets say that, in a given topic, 50% think A is the answer, 35% think B is the answer, and 15% think C is the answer. That seems fairly straight forward right? But here is the kicker... what if the 15% actually represents a larger active revenue stream? Or the 50% is a medium revenue stream but costs you the most in resources/overhead to maintain? Or the 35% represent a larger overall available market to capture? Then things get REALLY sticky!

 

To be highly clear, the work and effort and results you provided are FUZZY BUNNY AMAZING and give a quantitative perspective on a a traditionally qualitative discussion. And I am not saying Reloaded and @capnjosh are not going to do cool things for the community/players/etc either!

 

Its just that we, as players, just need to keep in mind that if Reloaded doesn't always follow "the math" we can see available to us, that there is likely other "financial math" that we can't see that is modifying the answer.


A little link from the "old days" because it seems I lost all my street cred with the forum reboot:

http://hawken.mirror...primary-weapon/

#29
JeffMagnum

JeffMagnum

    Advanced Member

  • Members
  • PipPipPip
  • 781 posts

Another thing is , in the AC poll, the difference between the averages is 90, but your lower and higher bound are 1600 an 3000, which makes for 1400... As there's no standard deviation, no min, no max, this graphic is pretty much meaningless and only tend to push the point you may want to make seen. tldr this graph is uselesss because the difference is negligeable considering the sampel size and the lack of crucial information needed to give a meaning to it.


What? I'm not trying to be a fuzzy bunny, but I honestly don't understand half of what you're trying to get across here.

Yeah, I know 42 isn't the best sample size, but with such a small population, I would have to sample a stupid percentage of it to get truly accurate data. That's not easy when I have to do everything manually and am relying on people to report info themselves on forums that only a few percent of the players actively use. If this game had a population of 500k then I could get a reasonably accurate sample of 500 (0.1% of the total) without any issues. To get the same CI here (assuming a playerbase of 2000), I'd need to get results from 400 people, or 20% of the total. That's logistically impossible for me or probably even the devs to get.
  • (Unknown)1590d2c747fabd likes this

d1eZeG3.png


#30
Hijinks_The_Turtle

Hijinks_The_Turtle

    Advanced Member

  • Members
  • PipPipPip
  • 342 posts

I ran out of likes. -_-  Good work Jeff.



#31
LRod

LRod

    Advanced Member

  • Members
  • PipPipPip
  • 84 posts

Your data is highly biased, and is likely due to your sampling plan.  You are sampling a subset of the population that would be classified as having behaviour that would indicate that they are 'enthusiasts' and is indicated by your average MMR being far beyond the normal distribution.  In other words, you are sampling in the tail of the distribution, and that tail is biased towards behaviour that is not indicative of the majority of the population.  You are drawing conclusions that would be characterized in statistics parlance as a Type II Error.  You believe there is statistial value to your analysis, but in fact you do not have a sample population that is indicative of the entire spectrum.  This is known as the Central Limits Theorem...this is not really an issue with the sample size (40/700-ish is about 5% and is plenty for this analysis) but rather how the sample was gathered.  Specifically, it is not a random sample across all users; including casual users that do not actively engage the community.  I think the questions were great, and the response level (including the details provided) was suitable for what you were trying to do, and I think you did a nice job with the analysis, but you really need the email distribution list in order to see all of the variables in play.


  • TheVulong, gopherAnime and (Unknown)1590d2c747fabd like this

"The supreme art of war is to subdue the enemy without fighting"  - Sun Tzu


#32
JeffMagnum

JeffMagnum

    Advanced Member

  • Members
  • PipPipPip
  • 781 posts
I acknowledged that it was biased toward the opinions of high-MMR/dedicated players exactly due to the reasons you gave multiple times already in previous comments. The data represents the viewpoint of the average active forum user far more than it does the average player, which is why I specified it was the collective opinion of the Hawken community in the title. The sample size itself isn't that big of a deal compared to the sampling bias, but it's still not close to ideal.
  • AsianJoyKiller and (Unknown)1590d2c747fabd like this

d1eZeG3.png


#33
(Unknown)1590d2c747fabd

(Unknown)1590d2c747fabd

    Advanced Member

  • Members
  • PipPipPip
  • 145 posts

Your data is highly biased, and is likely due to your sampling plan.  You are sampling a subset of the population that would be classified as having behaviour that would indicate that they are 'enthusiasts' and is indicated by your average MMR being far beyond the normal distribution.  In other words, you are sampling in the tail of the distribution, and that tail is biased towards behaviour that is not indicative of the majority of the population.  You are drawing conclusions that would be characterized in statistics parlance as a Type II Error.  You believe there is statistial value to your analysis, but in fact you do not have a sample population that is indicative of the entire spectrum.  This is known as the Central Limits Theorem...this is not really an issue with the sample size (40/700-ish is about 5% and is plenty for this analysis) but rather how the sample was gathered.  Specifically, it is not a random sample across all users; including casual users that do not actively engage the community.  I think the questions were great, and the response level (including the details provided) was suitable for what you were trying to do, and I think you did a nice job with the analysis, but you really need the email distribution list in order to see all of the variables in play.

 

I agree that the group sampled is a bit... limited... and I realize your response is intended to be a thoughtful academic critique, not an attack but I still think would think there is some validity & applicability:

 

1) The questions that were being analyzed are arguably being posed by the forum users, not necessarily the entire Hawken user base. So while the exact answers can't be attributed to the entire player base, we can say they apply to the group posing the question.

 

2) Including the MMR and correlating that with the responses does give us some expectations that we can extrapolate to the casual users especially if the correlation is strong. (I am not saying this is perfect but is possibly "close enough for government work")

 

3) At absolute worst its a "study to justify a study" by indicating the issue possibly isn't exactly as black and white or as "easy"  as possibly perceived.

 

Disclaimer: I am fully aware that my "opinion" plus $5.00 USD will get you a cup of coffee at Starbucks.


Edited by oSPANNERo, 28 March 2015 - 06:30 AM.

  • LRod likes this
A little link from the "old days" because it seems I lost all my street cred with the forum reboot:

http://hawken.mirror...primary-weapon/

#34
Mergaz

Mergaz

    Advanced Member

  • Members
  • PipPipPip
  • 53 posts

MMR average of the research is not the same as the average population ... It has lost credibility and can not be accepted as a method of analysis. The right would be a number equal or similar of players in each MMR tier.


Edited by Mergaz, 28 March 2015 - 06:31 AM.


#35
CraftyDus

CraftyDus

    Advanced Member

  • Members
  • PipPipPip
  • 1354 posts

s8JlpMN.gif

 

 

thats some sexy sciencing mr doctor jeffmagnum,

 

best thread poll, world


Edited by craftydus, 28 March 2015 - 06:32 AM.

  • dorobo and (Unknown)1590d2c747fabd like this

EOC Raider, Bolt Pred, Rev Gl Gren, EOC Infil, All the Reapers, Father, Expert in Guitar Kung Fu, and Founder of TPG Hawken

I4U54qx.jpg     bQCgI0k.png   zd30MxR.png   vP7JiOe.png     uq0awfp.gif

lwY3QRd.jpg


#36
(Unknown)1590d2c747fabd

(Unknown)1590d2c747fabd

    Advanced Member

  • Members
  • PipPipPip
  • 145 posts

MMR average of the research is not the same as the average population ... It has lost credibility and can not be accepted as a method of analysis.

 

I am not 100% certain if you are responding to my post but in case you are, let me clarify my position/thoughts:

 

I was suggesting that if there is a statistically significant correlation between MMR and Response to a question then it could be used to extrapolate the data and make a hypothesis that could be tested further.

 

For an over simplified example:

 

Lets say the that the current data set STRONGLY suggested that in choosing between playing as a Class A and Class C, as MMR increases players preference moves from A to C. (Correlation approaching 1.0)

 

I would also assert that "casual players" have a lower MMR. (This could be flawed and I am presenting no evidence but lets go with it for the sake of the example.)

 

Then we could infer that "casual players prefer playing as Class A" despite them not having been surveyed.

 

 

Now, this theory would need to be validated with a different study (Both MMR correlating with "commitment" to the game as well as the overall extrapolation) but at least we have theories and initial findings to start working with and refining.


Edited by oSPANNERo, 28 March 2015 - 06:51 AM.

A little link from the "old days" because it seems I lost all my street cred with the forum reboot:

http://hawken.mirror...primary-weapon/

#37
Interrobang87

Interrobang87

    Advanced Member

  • Members
  • PipPipPip
  • 172 posts

guh math


  • LoC_TR and (Unknown)1590d2c747fabd like this

#38
Call_Me_Ishmael

Call_Me_Ishmael

    Advanced Member

  • Members
  • PipPipPip
  • 1114 posts

Jeff,

 

I'll pull your data and do a regression analysis tonight (full day of errands, first).

 

LRod... for 95% confidence (5% alpha/type-I error), 20% box (user choice of 5 choices, the sample size is smaller for fewer choices), you need about 47 responses.  Jeff has enough for most of his poll to be statistically-significant.  Regression will tell us what isn't significant.


  • comic_sans and (Unknown)1590d2c747fabd like this

Did I say Call Me Ishmael?

 

You should call me Luna.


#39
sabrehawk90

sabrehawk90

    Advanced Member

  • Members
  • PipPipPip
  • 130 posts

Yeah, nice job on all this data. Should be helpful sometime later on.

 

I'm gonna elaborate a bit on the posts regarding MMR and sample size/accuracy though...

 

MMR average of the research is not the same as the average population ... It has lost credibility and can not be accepted as a method of analysis. The right would be a number equal or similar of players in each MMR tier.

 

I've been compiling this thing to get an idea of our current average MMR. It's not free of flaws given the resources I have to work with, but it's giving a consistent idea of what our current active player base is hovering at. I'll keep it updated since I'll need more information to get a more accurate estimate (Especially data from weekends, I'd expect more people being able to play during these times).

 

Going from the data I already have though, the current average MMR of the active players should be around 1618.

 

We're given that the average MMR of the respondents is 2153.

 

We can also notice that only 5% of the respondents are below the average.

 

Just spitting out statistics. I'm more of a fence sitter, so I'll let other people to decide what this means.


Edited by sabrehawk90, 28 March 2015 - 07:27 AM.

  • (Unknown)1590d2c747fabd likes this

2XhpJes.png


#40
Vdragon

Vdragon

    Advanced Member

  • Members
  • PipPipPip
  • 65 posts

fuzzy bunny this i wanted to answer you Jeff to epxlain what i said, but chrome deleted it 3 times so that'll wait until i'm less pissed.


0FAtmIq.png





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users