Richard Bartle recently posted a first go at a reputation system over on his blog.
Some people are prats. Some of these prats play virtual worlds. Non-prats would prefer not to play with them, but have no way of identifying them; this is because whenever systems are implemented to identify the prats, the prats use them to make non-prats seem like they’re the prats.
In other words, there isn’t a reputation system yet invented that griefers can’t use as an instrument of griefing.
His proposed solution, basically, is a shared friends/reputation list that works similarly to “You might also like…” systems on some music sites. If you like the same sort of folks that Mary likes, maybe you’ll like the folks Mary likes that you haven’t met yet. That sort of thing.
About the only thing I’d add to it is the ability to actually denote “I like this person and would like to add their stored reputation marks to my list” somehow. This is solely to get new players seeded into the mix; since they’re the ones who most need a ‘this guy is a jerk’ marks, and yet won’t actually have any opinions yet to share. There’s also some minor anti-gaming (gaming of the system, not gaming in general) things that can be done, but overall it just seems like a fairly standard database problem.
So, you can’t tear it apart there, but since I think it’s generally a good idea, tear it apart here! As an added benefit, Dr. Bartle will probably see your comments here and mark me down on his personal reputation system. Bonus for all!
#1 by SirBruce on June 18th, 2006
As long as the reputation system is positive-only, it sounds like it could work.
Bruce
#2 by Sachant on June 18th, 2006
Second Life from what I see has something like that where you can basically give people points for things they do. Like you can say they are a great builder, good dresser, good person and mark it all down so that it shows up on the reputation. I didn’t see any ability to put negatives when I was poking around.
Of course Ebay you can mark people down based on doing business with them as well so the only chance you have to say something positive or negative is when you’ve had a direct dealing. Could be interesting as a part of the auction systems in games though there isn’t enough variety in crafted items or quality levels to be able to comment on. At most fair business practices would be about it.
I could see the system being abused even if you could only put positives down. A positive based system though could work but it would have to be done carefully. Again, it would be too easy for a large guild base to be able to give all their own members glowing recommendations when they are in fact a guild of ninja-looters and griefers. So perhaps positive marks would only be allowed to come from people outside your own guild. Even then there is room for some abuse but it’s an interesting concept and one that has been needed for a long while.
Shadowbane has a guild history list that people can see so that people know if someone is a guild hopper as well. That might help if it was implemented in other games as far as being able to identify the fair weather guild members that just join for ph@t l3wtz then leave. It doesn’t give the whole story though of whether the person isn’t finding the right guild or really is a ninja-looting guildhopper though.
Any reputation system is going to be prone to some abuses in some way unless what can be said is either very limited or very explicit about what someone is being rated on and who can do the rating.
#3 by SirBruce on June 18th, 2006
No, you’re right, SL is positive only as well.
The cool part that Bartle is proposing is an Friendster-like system where your friend’s friends are also flagged for you.
Bruce
#4 by TPRJones on June 18th, 2006
The key is never displaying absolute results. As long as all results are displayed in relationship to your own rankings (i.e. “if you like you might also like…”) then there’s not much griefers can do. They can pump up their own ratings all day long, but unless you’ve ranked them positively you’ll never see the results of that pumping because you’ll never get into the mix of people that like who they’ve liked.
I don’t see how negatives could hurt too much as long as that’s the way it goes. Any negatives the griefers farm onto people would only be shown to the friends of those griefers, and never to the general server population.
#5 by SirBruce on June 18th, 2006
Griefers can have a lot of friends. Especially if they’re in a guild. This is a long-studied issue and basically, negative ratings tend to cause too many problems for too little gain.
#6 by Sachant on June 18th, 2006
XFire has the friends of friends too. It makes it really easy to connect to other people through friends that you may have missed so I can see where that is going.
Like SirBruce said though, there’s a lot of crossover with guilds.
I’d like to see something like that developed though.
#7 by Fraxas on June 18th, 2006
Even something as simple as sorting the LFG list by friends then friends-of-friends would be helpful. I think another essential self-policing point is to have the mutual friend known to both parties at all times; when a friend of a friend /tells me to join a group, I want to know who it is we both know.
Another interesting point is that I don’t actually need software to keep track of how much I like various people; I’m a human, I’m actually REALLY GOOD AT THAT THANKS. What I need the software for is good searching and indexing, which I do suck at.
#8 by Freakazoid on June 18th, 2006
I look forward to charging griefers to be on my friends list.
Not just a flat-out join my list for cheap, but after a month or two of gaining a lot of people’s trust. Then charge griefers up the ass to get on my list and ruin everyone’s day. I’ll make millions! (of in-game money, that is)
#9 by Patrick McKenzie on June 18th, 2006
Lets say “X trusts Y”. You have no way of knowing, unless you personally trust X, that X is not Y (we all know forcing one human = one account never works because there is no way to verify offline identity), is not Y’s accomplice in crime, etc. The same is true if A,B,C,D and X all trust Y. So I’m thinking we exclude all information from outside your own social network.
Within your own social network, I’m conflicted on whether you should be able to just give a simple “thumbs up” or whether there should be a few different granularities of “thumbs up”. Perhaps “thumbs up and I trust his recommendations” and “thumbs up but applies to him only” would work. Then if I run an instance with Smellyorc I can thumbs up him and if we do it on a regular basis I could say “You know, I really do trust Smellyorc, his friends seem good, I’m guessing if you’re good enough for Smellyorc you’re good enough for me” and go the second step. To prevent everyone from just doing the highest-trust-level in some sort of perversion of trust like eBay I think you should obscure what level people trust each other at.
Then its just a matter of integrating this into your interface. I think putting it in your /who result and /lfg tool are good enough, although I suppose you could color names differently or something if you really wanted to. The sole result I would return is “X is a friend of y friends of yours, including A, B, and C”.
#10 by Patrick McKenzie on June 18th, 2006
(Warning: Math ahead)
Incidentally, the problem that strikes me with Bartle’s solution is that you’re going to end up with incredibly sparse arrays for most people, so that the dynamic range of your dot products is going to be really, really small. I’m guessing it will look something like a zipf distribution — you have absolutely nothing in common with the overwhelming majority of the server population, share one friend in common with some sliver, two friends with a slightly smaller sliver, three, four, … perfect alignment with one person. Here’s the rub: the problem with reading recommendations from the folks at the right end of the curve is that they *don’t tell you anything you don’t already know* (if your friends and my friends are co-extent then what useful information does that get you?). As you move towards the left trying to find useful information you’re going to rapidly get lots of opinions, potentially drowning the utility of the tool (Tell me how a non-mathematically-inclined end user is suppose to interpret this output: “Of 47 people who share exactly one friend with you, 4 of them trust Bob.”).
#11 by Evangolis on June 18th, 2006
What do you, as a player, do when you are not a jerk, but your friend X is? Oh, X treats you wel, and in fact he is more availble and helpful to you than any of your other friends are, but you have wide reports of him being an absolute jerk. When he is with you, it is easy to keep his jerkiness in hand, but in your absence, he’s a raging pain. And he plays a lot more than you do.
This is an actual situation that occurred when I first started playing EQ. I’m not sure how I would have marked him up, but if he could see that I didn’t mark him well, he would be hurt. Worse, he probably would have treated me even better. This system might resist the griefer, but how about the neurotic?
Beyond that, I’m not sure this adds much beyond the social feedback that already exists.
I wonder if there is something more in the nature of networks that might inform a ratings system better? Alas, I’m only on chapter one of that book.
#12 by SirBruce on June 18th, 2006
Ummm, you either accept being friends with that person, and the shit that comes with it, or you don’t. Just because you’re a friend with him doesn’t mean you can’t get in the good graces of someone who doesn’t like him. They’ll rate you as a friend. To others in their network, you’re going to come off somewhat higher than neutral. (Their friend’s positive is closer to them than your negative that comes from once removed.) That’s assuming, of course, that there are negatives, which I don’t like anyway. With a positve rating system, you’ll be seen as a mild positive, and the jerk won’t get any positive.
Bruce
#13 by Mandella on June 18th, 2006
Just to make a clarifying point, until fairly recently you *could* give negative points in Second Life. The ability to do so was taken out, probably for the very reasons discussed above…
#14 by Apache on June 18th, 2006
A mmog isn’t a forum. Can’t use slashdot style karma system in it.
#15 by Richard Bartle on June 19th, 2006
Patrick McKenzie>the problem that strikes me with Bartle\’e2\’80\’99s solution is that you\’e2\’80\’99re going to end up with incredibly sparse arrays for most people, so that the dynamic range of your dot products is going to be really, really small.
Yes, this could be an issue: players group with so few people that their lists barely intersect. I don’t know what the critical mass would have to be on a 10,000-player server before it became viable: at minimum you’d need to have rated a player the same way that someone else rated them, and for them to have rated the player you’re querying. It could be that this happens so infrequently to start with that it never gets off the ground.
Richard
#16 by Richard Bartle on June 19th, 2006
TPRJones>I don\’e2\’80\’99t see how negatives could hurt too much as long as that\’e2\’80\’99s the way it goes. Any negatives the griefers farm onto people would only be shown to the friends of those griefers, and never to the general server population.
Better yet, the griefers won’t get access to the opinions of non-griefers. If a gang of griefers all rate player X as being poor, when player X is actually good, then the opinions of all those other players who rated X as good will be ignored when the griefers want to check someone out. It’s a double-edged sword.
This doesn’t apply so much if you can rate people without having grouped with them, because then griefers would just use low-level throwaway alts to sour a reputation. It also avoids a lot of trouble if you can’t import lists of friends (except maybe from your own alts) because otherwise people can be effectively blacklisted by what might have been a mistake.
Richard
#17 by Richard Bartle on June 19th, 2006
Freakazoid>I look forward to charging griefers to be on my friends list.
You can grief them by not actually doing it – they’ve no way of telling.
Actually, you’d be smart to do it. If you rate a griefer as being good, that would mean your list wouldn’t intersect with those of people who rated them as being bad. You’d be sacrificing your own ability to judge the worthiness of other players because you’d only be getting responses from people who thought the prat wasn’t a prat.
You could get round this, of course, by asking your friends to run the check on your behalf. I’m sure they won’t mind.
Richard
#18 by Evangolis on June 19th, 2006
Bruce, I don’t think I expressed my concern properly. I wasn’t concerned with how I would be perceived, I was concerned with having to choose between hurting someone’s feelings by not giving them a favorable rating or misleading others. As I said, when he was in my presence, he was well behaved, because he would ask me before doing something jerky, it was only second hand, from RL friends, that I knew what he did when I wasn’t around. As an extension, I’d note that this person generally tended to treat the people he grouped with overly well, it was everyone else he was a jerk to. MMOs are a more nuanced setting than something like ebay, with a wider range of expressive options. My concern is for people who have multiple behavior modes, rather than simply being entirely jerks all the time.
Alternatively, with a strong external social structure, such as an out of game guild, I can see poor players being endorsed by good players for reasons other than in game behavior. This wouldn’t be explict gaming of the system, just ordinary social pressure to say nice things about your ‘cool friends’, even if they are really neither.
#19 by Mandlar on June 19th, 2006
Hire Facebook to your social networking. Then everyone is friends with everyone!
#20 by Tide on June 19th, 2006
Sorry to be obtuse, but I’ve built a service for this and it’s about to enter Beta. If anyone is interested in participating they can email me (tidemail@gmail.com) to be rostered. Appreciate the feedback. Thanks.
#21 by Jason Booth on June 19th, 2006
My bet is that the 6th degree of seperation theory kicks in fast; that within one or two jumps, your linked to the biggest jerk you’ve ever encountered on the server. That isn’t all bad though, as that form of “hey, it’s a small world” perspective might act to collectively cool the environment of the game and make it more friendly as a whole.
#22 by JHB on June 19th, 2006
I’ve often thought that players should be given one “goodguy” medal and one “buthead” medal. Players would get them at the start of each calendar day – they would be non-accumulating. Players could award them as they saw fit.
The medals would contribute to a “character” value in the recipient\’e2\’80\’99s viewable stats.
#23 by harl on June 19th, 2006
This is all based on “the enemy of my enemy is my friend” This statement is not reliable and thus any system based on it will be flawed.
#24 by TPRJones on June 19th, 2006
Actually it’s more like “the friend of my friend is my friend” and “the friend of my enemy is my enemy”. There shouldn’t be too much info crossing in from “the enemy of my enemy” side of things given how this has been so far discribed.
Admittedly, that’s also not always true, but it’s got better odds of being so than the old platitude does.
#25 by Brask Mumei on June 20th, 2006
As usual, we look for ways to improve magically ranking other players while still leaving the players with pen & paper to record their own opinions of others.
As already mentioned, I’m good at deciding if I like someone or not. The trick is remembering the next time I meet them that I don’t like them. What I want is a client-side database that tracks my notes/descriptions of each player based upon the UID of the player. (NOT based on character name, as that is prone to error & typos)
Rather than investing enormous effort on building automated schemes to propogate this meta information (meta information which, incidentally, is a lot more nuanced than Like/Dislike!), leave it to the players to share rep as they wish. If Bob has an opinion of Jane, I’d rather judge myself how that affects my opinion of Jane. This, again, is what humans are good at.
The resulting system is still griefable – nothing stops would be griefers from spreading lies while pretending to be good honourable people. The difference is that there is no computer algorithm deciding to spread the lies – it is instead only a result of each individual having chosen to accept that griefer as authoratative. Again, this is something which humans have had thousands of years of experience with, for good or for bad, so can mostly deal with.
In conclusion, I’d like to dismiss all the fancy “ranking” schemes in exchange for a simple way to:
1) Make notes about other players from within the game. Notes are plain text, “Stole my kill”, “Owe 50 gold for favour”, etc.
2) Flag other players and have the flags visible whenever the character is. Flags are *all* user defineable, not a mere “good”/”bad” flag.
3) Ability to talk to another player about a third party with the UUIDs in a relatively error free way.
#3 could be as easy as being able to click on someone in your database and have their name + UUID pasted to your text as a uneditable coloured string, allowing the receiver to bring up their corresponding entry and add their own notes. This allows people to alert others about those that they like or don’t like. The people receiving the name can act as they wish on the info.
Please note that character names should not be unique ids! It is just too gameable to rely on unique character names. It should be impossible/difficult for users to type in the unique ids – they should have to acquire them in game somehow to avoid accidental references.
#26 by Xilren on June 20th, 2006
Well said Brask; that WOULD be rather useful.
But that could be combined with any LFG system to corss reference automatically. I.e. You get a group invite and the system automatically cross referrences the people in that group with your self updated contacts lists (friends, ignores, general info) based on account UID, not character name.
#27 by harl on June 20th, 2006
TPRJones> No it’s explicitly an “the enemy of my enemy is my friend” system. He explicitly states this, look at the section about griefers giving someone negative rep and it showing as positive to me. Say two pirates are fighting each other or competing over the same star system. They give each other negative rep. I’ve been killed in the past by one of them and so I’ve given him negative rep. Now when I encounter the second pirate they will show up as friendly/nice/good people and proceed to blow me out of the sky. That’s an example of it happening randomly. Obviously it’s highly exploitable. Create two characters. Go be an ass with one and become hated. Then have that char give bad rep to the other one. Now the second char shows up as really good and you use her to scam.
It’s a decent idea but the web/friend of a friend aspect is borked.
#28 by Matt on June 20th, 2006
This idea was great when I posted about it last year.
http://whatwouldmattdo.com/2006/06/20/bartle-stole-my-idea-ok-not-really/
I’m just kidding, but it’s along the same lines. And as I said on my blog, the real reason I bring it up is because I want someone to implement it already! Hurry! WoW alone would be ten times as good if I could easily find good people to group with…
#29 by Darniaq on June 20th, 2006
I prefer eBay’s system:
http://www.darniaq.com/wordpress/2006/05/general-gaming/character-accountability-idea/
#30 by Richard Bartle on June 21st, 2006
harl>No it\’e2\’80\’99s explicitly an \’e2\’80\’9cthe enemy of my enemy is my friend\’e2\’80\’9d system.
Then I didn’t explain it very well. It’s not like that at all.
>Say two pirates are fighting each other or competing over the same star system. They give each other negative rep. I\’e2\’80\’99ve been killed in the past by one of them and so I\’e2\’80\’99ve given him negative rep. Now when I encounter the second pirate they will show up as friendly/nice/good people
No, they won’t. People don’t get to rate their own characters.
If you encounter someone (let’s say X) with a rep unknown to you, then this is what happens:
1) Every player’s opinion list is scanned, and those that have an opinion concerning X are long-listed.
2) Each long-listed opinion list is compared against your own opinion list. If there’s no intersection, they’re removed; otherwise, they’re put in the short-list.
3) The short-listed opinion lists are ordered based on how much they agree with your own opinions. Ideally, there would be a perfect match (positive or negative) concerning those individuals you’ve grouped with, but if that’s not the case then you’d go with the closest match instead.
4) The opinion list matching your own opinion list the closest is taken as the reference. You are told how this opinion list rates X. For extra fidelity, you could be told how close a match the list was; you could also ask what the second-closest and third-closest opinions were.
Now from this, you can see that the opinion list used for rating any individual could vary every time. It’s also entirely anonymous – I may hate the person whose list is being used, but neither of us know the other is involved.
In your pirate example, I want to know what people think about X. Y has rated X as bad, but the fact I’ve rated Y as bad doesn’t mean I’ll take the inverse of Y’s opinions. If it happens that out of all the players in the game, Y’s opinions match mine the closest then I’ll be told that X is bad. I’ll never be told that X is good based on Y’s opinions; I may be told that X is good if the best-matching opinion set to mine is by someone who thinks X is good (and Y is bad), but if they do think that then the chances are that I’d actually agree with them anyway.
Richard
#31 by Pluto's Dad on June 21st, 2006
The only other thing this would need is it would have to be player based, rather than character. That way, creating alts won’t allow you to “reset” your ranking.
The hiding of absolutes, so you only see relative “if you like this person you’ll like that person” or “if you don’t like this person you won’t like that person” as someone suggested is a good idea and solves the griefers giving each other good points.
For people worried about RPing evil, there is no problem. People can RP evil and still be non-griefers and not jerks.