Code of Botduct

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Code of Botduct

Nicolás Tamargo de Eguren-2
Hi!

The number of bots running on MusicBrainz has grown quite fast in the
last months, so we've decided it's a good time to get us some basic
rules for botting. The current draft can be read at:

http://wiki.musicbrainz.org/User:Reosarevok/Code_of_Botduct

Please discuss, whether you're a voter or a bot owner - does any of it
seem wrong to you?

We'll freeze discussion at the beginning of August and then publish a
final version.

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

Philip Jägenstedt
On Mon, Jul 23, 2012 at 8:48 AM, Nicolás Tamargo de Eguren
<[hidden email]> wrote:

> Hi!
>
> The number of bots running on MusicBrainz has grown quite fast in the
> last months, so we've decided it's a good time to get us some basic
> rules for botting. The current draft can be read at:
>
> http://wiki.musicbrainz.org/User:Reosarevok/Code_of_Botduct
>
> Please discuss, whether you're a voter or a bot owner - does any of it
> seem wrong to you?
>
> We'll freeze discussion at the beginning of August and then publish a
> final version.

This seems a bit weird to me: "Yes-voting on bot edits is discouraged
unless the voter can 100% confirm they're correct, since it helps them
to go through with less eyes on them."

Why is it a problem if 3 votes of 99% confidence (higher than most
non-bot edits) takes the edit out of the queue? I vote yes to bot
edits where things seems to line up, just as with human edits.

--
Philip Jägenstedt

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

Ian McEwen
On Mon, Jul 23, 2012 at 07:53:24PM +0200, Philip Jägenstedt wrote:

> On Mon, Jul 23, 2012 at 8:48 AM, Nicolás Tamargo de Eguren
> <[hidden email]> wrote:
> > Hi!
> >
> > The number of bots running on MusicBrainz has grown quite fast in the
> > last months, so we've decided it's a good time to get us some basic
> > rules for botting. The current draft can be read at:
> >
> > http://wiki.musicbrainz.org/User:Reosarevok/Code_of_Botduct
> >
> > Please discuss, whether you're a voter or a bot owner - does any of it
> > seem wrong to you?
> >
> > We'll freeze discussion at the beginning of August and then publish a
> > final version.
>
> This seems a bit weird to me: "Yes-voting on bot edits is discouraged
> unless the voter can 100% confirm they're correct, since it helps them
> to go through with less eyes on them."
>
> Why is it a problem if 3 votes of 99% confidence (higher than most
> non-bot edits) takes the edit out of the queue? I vote yes to bot
> edits where things seems to line up, just as with human edits.
>
The issue here is that there's an implied baseline (i.e. intelligence) for human editors that you can't necessarily extend to bots. While most bot edits are correct, there's often edge cases which the code hasn't accounted for, which this sort of "structural voting" ("where things seem to line up") won't catch in many cases. It's exacerbated by the fact bots usually have a very regular edit note structure, which makes it very easy to glaze over probable mistakes.

In fact, this very problem is what spurred this code-of-conduct initiative in the first place. In the particular case I saw, it was a mismatch between Discogs' conventions for listing countries and MB's (as I recall), which is obviously not a case the bot author had written for -- it didn't come up very many times! It was also a removal -- so, destructive edit. This particular edit got up to two yes votes before anyone noticed it wasn't correct, presumably by exactly this voting methodology.

Yes votes are noted specifically because they don't do anything *except* accelerate the removal of the edit from the edit queue. Unless you're 100% sure on a bot Yes vote, it therefore just increases the chances of an error by limiting the number of people who see the edit. It only takes three people voting in this sort of fashion to quickly accept problematic edits. As the document goes on to say, re-entering edits is also a lot easier than reverting edits.

This same premise should potentially apply to human edits as well, of course, but in this case bots are the topic under discussion.

The other changes in the document should also help with this problem (smaller number of edits usually means less eyes-glazing-over while voting, for example), but voting on bot edits does still merit care.

--
Ian McEwen <[hidden email]> <[hidden email]>
A262 D5C4 40CB 0E1C 5F24 C3A1 ABED 1ABD 7131 A76F
http://ianmcorvidae.net/

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users

attachment0 (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

Philip Jägenstedt
On Mon, Jul 23, 2012 at 8:14 PM, Ian McEwen
<[hidden email]> wrote:

> On Mon, Jul 23, 2012 at 07:53:24PM +0200, Philip Jägenstedt wrote:
>> On Mon, Jul 23, 2012 at 8:48 AM, Nicolás Tamargo de Eguren
>> <[hidden email]> wrote:
>> > Hi!
>> >
>> > The number of bots running on MusicBrainz has grown quite fast in the
>> > last months, so we've decided it's a good time to get us some basic
>> > rules for botting. The current draft can be read at:
>> >
>> > http://wiki.musicbrainz.org/User:Reosarevok/Code_of_Botduct
>> >
>> > Please discuss, whether you're a voter or a bot owner - does any of it
>> > seem wrong to you?
>> >
>> > We'll freeze discussion at the beginning of August and then publish a
>> > final version.
>>
>> This seems a bit weird to me: "Yes-voting on bot edits is discouraged
>> unless the voter can 100% confirm they're correct, since it helps them
>> to go through with less eyes on them."
>>
>> Why is it a problem if 3 votes of 99% confidence (higher than most
>> non-bot edits) takes the edit out of the queue? I vote yes to bot
>> edits where things seems to line up, just as with human edits.
>>
>
> The issue here is that there's an implied baseline (i.e. intelligence) for human editors that you can't necessarily extend to bots. While most bot edits are correct, there's often edge cases which the code hasn't accounted for, which this sort of "structural voting" ("where things seem to line up") won't catch in many cases. It's exacerbated by the fact bots usually have a very regular edit note structure, which makes it very easy to glaze over probable mistakes.
>
> In fact, this very problem is what spurred this code-of-conduct initiative in the first place. In the particular case I saw, it was a mismatch between Discogs' conventions for listing countries and MB's (as I recall), which is obviously not a case the bot author had written for -- it didn't come up very many times! It was also a removal -- so, destructive edit. This particular edit got up to two yes votes before anyone noticed it wasn't correct, presumably by exactly this voting methodology.
>
> Yes votes are noted specifically because they don't do anything *except* accelerate the removal of the edit from the edit queue. Unless you're 100% sure on a bot Yes vote, it therefore just increases the chances of an error by limiting the number of people who see the edit. It only takes three people voting in this sort of fashion to quickly accept problematic edits. As the document goes on to say, re-entering edits is also a lot easier than reverting edits.
>
> This same premise should potentially apply to human edits as well, of course, but in this case bots are the topic under discussion.
>
> The other changes in the document should also help with this problem (smaller number of edits usually means less eyes-glazing-over while voting, for example), but voting on bot edits does still merit care.

As long as our edit queue is so big that there's no chance of all
edits being checked, removing good edits from the queue is a good
thing, regardless of who made them. I've voted no to bot edits where
tracklists or bonus tracks don't line up, but see no reason to abstain
in situations where I'd vote yes to a human edit. After all, the error
rate of a bot is certainly going to be lower than humans.

I suggest restating the guideline as: "Voting yes on bot edits is only
allowed if the voter has carefully verified that they are correct,
since bots lack the common sense of a human editor and may fail on
corner cases."

--
Philip Jägenstedt

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

Per Starbäck
In reply to this post by Nicolás Tamargo de Eguren-2
> http://wiki.musicbrainz.org/User:Reosarevok/Code_of_Botduct
>
> Please discuss, whether you're a voter or a bot owner - does any of it
> seem wrong to you?

I think it should start with saying that bots should make clear that
they *are* bots, both in their user pages and their names. I guess
that is stated elsewhere now, but better collect all about how to make
bots together on one page.

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

Nicolás Tamargo de Eguren
On Tue, Jul 24, 2012 at 2:12 PM, Per Starbäck <[hidden email]> wrote:
>> http://wiki.musicbrainz.org/User:Reosarevok/Code_of_Botduct
>>
>> Please discuss, whether you're a voter or a bot owner - does any of it
>> seem wrong to you?
>
> I think it should start with saying that bots should make clear that
> they *are* bots, both in their user pages and their names. I guess
> that is stated elsewhere now, but better collect all about how to make
> bots together on one page.

It's not necessary in their names because we do have a "Bot" flag (and
that will show in all edits made by bots, see
http://musicbrainz.org/edit/18442057 for example).

But you're right that we forgot to say bot owners need to let us know
so we can set them to Bot :) I've added it - will find out exactly
*how* they should contact us to be set to bot and clarify that when I
have an answer :)

> _______________________________________________
> MusicBrainz-users mailing list
> [hidden email]
> http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users



--
Nicolás Tamargo de Eguren

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

lixobix
In reply to this post by Ian McEwen
Ian McEwen wrote
The issue here is that there's an implied baseline (i.e. intelligence) for human editors that you can't necessarily extend to bots. While most bot edits are correct, there's often edge cases which the code hasn't accounted for, which this sort of "structural voting" ("where things seem to line up") won't catch in many cases. It's exacerbated by the fact bots usually have a very regular edit note structure, which makes it very easy to glaze over probable mistakes.

In fact, this very problem is what spurred this code-of-conduct initiative in the first place. In the particular case I saw, it was a mismatch between Discogs' conventions for listing countries and MB's (as I recall), which is obviously not a case the bot author had written for -- it didn't come up very many times! It was also a removal -- so, destructive edit. This particular edit got up to two yes votes before anyone noticed it wasn't correct, presumably by exactly this voting methodology.

Yes votes are noted specifically because they don't do anything *except* accelerate the removal of the edit from the edit queue. Unless you're 100% sure on a bot Yes vote, it therefore just increases the chances of an error by limiting the number of people who see the edit. It only takes three people voting in this sort of fashion to quickly accept problematic edits. As the document goes on to say, re-entering edits is also a lot easier than reverting edits.
I take a different view. Bot editing is about formalities, not whether each case is right or wrong in a more general sense.

I say this in reference to the issue you mentioned regarding the bot removal of discogs links. However, I maintain that the so called 'errors' in that case were not so at the time the bot entered the edits. The objections were that the mismatch between the countries on MB and discogs releases was not a good reason to remove the links.

As far as I'm aware, this is contrary to the official guidelines concerning releases, which state that near-identical releases released in different countries are in fact separate releases. The same appears to be true on discogs.

The problem in my eyes was that users were voting against removals on the basis that the country was wrong on one site or the other. This is IMO an example of informal voting, i.e. voting on the basis of another issue that is separate to the specific edit. So while a human may 'recognise' that a country or other piece of data is wrong (which, in some of the above cases, was a mistake) voting on such a suspicion rather than the information provided directly seems to me an odd voting behaviour.

Therefore, in those cases the issue was not whether the bot was right or wrong in each particular edit; on a formal level, it was 100% correct. The real issue was that due to a lack of guidelines regarding the conditions for linking an MB to a discogs release, many users were deciding that a 100% match was not required. This is fine in terms of voting, but saying that the bot is making mistakes is simply not true in such cases. What's more, those concerned by the destructive nature of the edits didn't seem to balance the apparent risk against the benefits:

1) The number of 'errors' was very small in proportion to the correct removals
2) The same bot (and others) automatically adds discogs links where the data does match. Any data subsequently changed on either site will result in the automatic addition of the link, eventually.
3) This was not a case of losing 'core' data. These were links that were dubious due to the fact that data did not match.


In relation to the proposed guidelines, I think the issue is not the amount of edits a bot is making. The issue is whether any particular bot task has been scrutinised for a sufficient period of time to establish whether it is working correctly WITHIN THE CURRENT GUIDELINES; NOT whether it is making the right or wrong decision, in any particular user's opinion, on a point that the guidelines cannot decide.


My proposal instead is that each new task by any bot must be approved by a number or users before it can begin. This would involve using x number of trial cases, a small number that could be manually reverted fairly easily if there were problems. (I believe there is a beta version of MB, perhaps that could be used?).

Following initial approval, those users involved are automatically subscribed to edits for that particular task, for a period of time or further number of edits. This 'probation' period would highlight any more remote issues. After a successful probation, the bot is allowed unrestricted editing rights for that task, although any subsequent issue could result in suspension.


I completely disagree with the 'no "yes" voting unless 100% correct' proposal because in some cases there is no 100% correct. In reality there is often conflicting data from different sources, or a lack of data, meaning that it is impossible in many edits to be 100% correct. Moreover, I don't think a separation between voting for humans and voting for bots will be sustainable. I much prefer the attitude of 'if it looks correct, vote yes' because it keeps things moving, even if there are occasional mistakes. Also, being able to blanket 'yes' vote a particular bot task is hugely beneficial in that it formalises large numbers of edits that would take longer to verify if undertaken by a range of users, with a range of understanding of MB rules.

I actually think that automated edits are far superior to manual edits when combined with strong guidelines. They can apply standards quickly and with more precessions than manual edits ever could. Users spending more time establishing guidelines and increasing the the number of potential bot tasks is far more efficient for the database as a whole than focusing on individual edits with no overarching guidelines.
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

Per Starbäck
> I much prefer the attitude of 'if it looks correct, vote yes'
> because it keeps things moving, even if there are occasional mistakes.

I agree that we shouldn't be that afraid of having occasional
mistakes, but I think the point is
that some kind of "it looks correct" check has already been done by the bot.
If you don't know anything about the issue beforehand and just sees
that yes, this seems correct,
then your vote really isn't contributing much.

Say there was a bot that just added birthdates from Wikipedia. If the
artist is a person without birthday set,
and has one or more Wikipedia links, and all of those Wikipedia pages
agree on the same birthdate it sets
the birthdate. Maybe the bot even checks to see if the Wikipedia pages
have mentioned that same birthdate
for some time to avoid recent errors or vandalism that hopefully are
corrected soon.
It makes an edit with an edit note

  "Setting birthdate to the same as on http://en.wikipedia.org/wiki/John_Doe."

What are you gonna do when voting?

A) Only vote yes if can 100% confirm they're correct, because you know
this about John Doe from
elsewhere, or you check the Wikipedia page and see if there is a
source to the birthdate there, and
then check that (perhaps with a trip to your library)

B) Vote yes because "it looks correct" (why wouldn't it be?)

I think that A is overly cautious, but that B is unnecessary work. You
almost don't add anything at all.
Of course it looks legit! You're only confirming that the bot doesn't
have a big bug so it's not doing what it's
saying that it's doing or something like that.

> I actually think that automated edits are far superior to manual edits when
> combined with strong guidelines. They can apply standards quickly and with
> more precessions than manual edits ever could.

Some bots can be really trustworthy. If a particular bot is really
trustworthy I see no reason not to
give it autoeditor status. It would a big waste of human efforts if it
would have to be confirmed hundreds
or thousands of times that that bot doesn't have a big obvious bug.

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

lixobix
Per Starbäck wrote
A) Only vote yes if can 100% confirm they're correct, because you know
this about John Doe from
elsewhere, or you check the Wikipedia page and see if there is a
source to the birthdate there, and
then check that (perhaps with a trip to your library)

B) Vote yes because "it looks correct" (why wouldn't it be?)

I think that A is overly cautious, but that B is unnecessary work. You
almost don't add anything at all.
Of course it looks legit! You're only confirming that the bot doesn't
have a big bug so it's not doing what it's
saying that it's doing or something like that.

Some bots can be really trustworthy. If a particular bot is really
trustworthy I see no reason not to
give it autoeditor status. It would a big waste of human efforts if it
would have to be confirmed hundreds
or thousands of times that that bot doesn't have a big obvious bug.
The only problem with giving bots auto-editor status is that if there somehow is a complication, it would be very hard to see. At least having the edits appear in the queue increases the chance of such a complication being spotted, even if in reality most of the edits are glossed over. On the other hand, I agree that once a bot has been assessed any such complications should be so few and far between that they could be an acceptable loss in exchange for the general benefit.

I understand what you mean by yes votes that don't add anything, but my attitude has been that yes in the default vote. I abstain if there is a lack of information for adding edits, and vote no if the same is true for a removal or change edit, if the editor fails to provide evidence when asked, or if it just appears plain wrong. So, a yes vote doesn't really do much except say that an edit looks correct based upon the info provided.

I imagine around 95% o edits are yes voted. It's also quite possible that not all voters thoroughly cross-check listed sources either; I confess that I don't for every single edit. Practically, if you subscribe to a fair number of artists and editors, it simply takes far too much time to completely check every edit. In reality, you make judgements based upon:

1) Who is doing the edit: Certain editors you get used to seeing, and you know that they know what they're doing.

2) The information provided: Anything marked "Imported from discogs" has probably been done using the script, so there's little chance of error unless the data is incorrect on discogs. In other cases, there may simply be an amazon, wikipedia or label link. If so, it will be far more likely than not that the data is correct, irrespective of whether you actually check the link or not.

My general viewpoint is that voting is not the be all and end all: It does not in every case decide for certain whether a piece of data is correct. I just don't think it's practical to treat it as such, unless you can commit a good chunk of time every day. Instead, I see it as an overview to the recent changes that concern each user, a way to register what has been changed, what is being questioned and sometimes what is wrong.

Also, note that the system by default accepts all addition edits that have 0 no votes. Essentially, it is designed so that a yes vote doesn't really do much except remove the edit from your queue. The case is different for removals, so it is more important to consider them.
jw
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

jw
On Thu, Jul 26, 2012 at 06:39:44AM -0700, lixobix wrote:
> I understand what you mean by yes votes that don't add anything, but my
> attitude has been that yes in the default vote.
[...]
> I imagine around 95% o edits are yes voted. It's also quite possible that
> not all voters thoroughly cross-check listed sources either; I confess that
> I don't for every single edit. Practically, if you subscribe to a fair
> number of artists and editors, it simply takes far too much time to
> completely check every edit.

I don't think this is a good thing, especially not for
merges/removals/splits. Why not abstain by default, and only vote "yes"
if you are 100% sure? "Yes" votes on (possibly) wrong edits are very bad:
- They remove the edit faster from the queue. I try to vote at least on
  all merges/removals/splits for my subscriptions, but I can't do it
  every day. So a "look right" yes vote deprives me of the opportunity
  to thoroughly check the edit if I want to. Same for other edit
  types: Voting yes on a "looks ok" edit removes the chance for
  other editors to check the edit (if they vote queue-based, not
  subscription-mail based). It is just not as bad as for
  merges/removals/splits because they can be easier reverted.
- It gives the edit a false credibility in the history. Of course it is
  possible to ask questions later, but the editors who voted are often
  not responsive anymore.

So I would encourage all editors to default-vote abstain, especially for
edits that are hard to revert. Independent if a bot or a human made that
edit.


Johannes

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

Per Starbäck
In reply to this post by lixobix
> The only problem with giving bots auto-editor status is that if there
> somehow is a complication, it would be very hard to see.

If it says that Mozart played the ukulele on Sgt Pepper it will be
found out, even
if no one checked the edit where it happened!
And certainly I'm not the only one who often takes a look at the applied changes
to my subscribed artists (and not only the open edits)!

> At least having the
> edits appear in the queue increases the chance of such a complication being
> spotted, even if in reality most of the edits are glossed over.

Yes, but all fhose edits being glossed over is a(n increasing) problem I think,
since it takes away eyes from edits that really could use some.

> I understand what you mean by yes votes that don't add anything, but my
> attitude has been that yes in the default vote. I abstain if there is a lack
> of information for adding edits, and vote no if the same is true for a
> removal or change edit, if the editor fails to provide evidence when asked,
> or if it just appears plain wrong. So, a yes vote doesn't really do much
> except say that an edit looks correct based upon the info provided.

Most edits *should* go through, and *will* go through, and the voting
just changes how
much time it takes. For the database at large it doesn't matter that
much how quickly it goes
through though. The person most affected is often the editor. When I
was ripping my cds before
I was an autoeditor I had a special stack of cds by the computer that
I had to continue work
on later or that I should re-rip later, after some of my edits had
gone through. Then I really
appreciated it when people voted yes to my edits even if they just
thought that "yes, this looks
alright", since that meant I could continue doing what I was in the
middle of instead of
having to wait for long time and meanwhile starting on something else,
sometimes getting
confused because I forgot what I was doing. (That still happens, but
not that often anymore.
I have a cd box in front of me right now where I'm waiting for an edit
from July 17 to go through
so I can continue -- just 5 days left now!)

I see not much reason to give that kind of courtesy to bots. What's
the hurry? Yes, it can also be
in the middle of something (... and then when I've connected this
release with that discogs release
I'll compare all the recording credits with the discogs credits ...),
but really, a bot should be able to
be really patient.

> I imagine around 95% o edits are yes voted.

Really? I would have thought most edits didn't get any votes at all,
or at least not
enough votes for them to matter. I have 8 open votes right now, 2 of
which have one
yes vote and 6 of which have no votes, so that's what I'm seeing at least.
(I'm not complaining though! I'm a lazy voter myself and mostly vote
when I find an error.)

Maybe some of this discussion will be a moot point when/if the new
edit system will
group edits together. A bot edit that adds a couple of hundred discogs
links (for example)
in one go is worth voting about!

> My general viewpoint is that voting is not the be all and end all: It does
> not in every case decide for certain whether a piece of data is correct.

I'm tempted to go into my ideas about Musicbrainz voting in general (many more
edits should be autoapplied without voting!), but keeping it to bots,
yes, it isn't the
end of the world if an incorrect edit goes through, but if it really
needs voting on at all
that would be because there can be errors and we hope to find them in
time not always,
but at least rather often. Voting yes by default doesn't scale well as
a strategy. If everyone was
doing it errors would almost never be found in time!

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

lixobix
In reply to this post by jw
jw wrote
I don't think this is a good thing, especially not for
merges/removals/splits. Why not abstain by default, and only vote "yes"
if you are 100% sure? "Yes" votes on (possibly) wrong edits are very bad:
- They remove the edit faster from the queue. I try to vote at least on
  all merges/removals/splits for my subscriptions, but I can't do it
  every day. So a "look right" yes vote deprives me of the opportunity
  to thoroughly check the edit if I want to. Same for other edit
  types: Voting yes on a "looks ok" edit removes the chance for
  other editors to check the edit (if they vote queue-based, not
  subscription-mail based). It is just not as bad as for
  merges/removals/splits because they can be easier reverted.
- It gives the edit a false credibility in the history. Of course it is
  possible to ask questions later, but the editors who voted are often
  not responsive anymore.

So I would encourage all editors to default-vote abstain, especially for
edits that are hard to revert. Independent if a bot or a human made that
edit.
Yes, I suppose it is problematic if some users are prevented from seeing the edit. I think I was presuming too much from my own voting experiences: I subscribe to a fairly large number of artists, so my queue builds up quickly. I've not been on for a few days, my queue has 644 edits. Just out of interest, is this a lot compared to what you usually have?

I suppose the situation is different if you subscribe to a smaller number of artists but check more thoroughly. For artists with many subscribers that would work out well, but for less popular artists often no-one votes, which is frustrating as an editor if your edit requires at least one yes vote.

With regards to false credibility, I think that if there has been a mistake it will become apparent in time. Later edits may appear to conflict, in which case users have to looks at the editing history side by side with the new information to clarify where the error is. Usually in such cases, I look at the sources in the original edit (if there are any) and contrast those with the newer ones. There are many cases of data that was added incorrectly but went through due to a lack of 'no' votes, so as a rule accept all data to be dubious to some extent, but it's not really an issue until someone notices an error or inconsistency.

More to the point though, I don't consider the votes of others, past or present, to be any indication of correctness: You can never tell how carefully any other user has voted on a particular edit, so you can only ever vote on the information provided (or your own research). So for me, the fact that something has been yes voted doesn't affect my own judgement as to the correctness of the edit.
Per Starbäck wrote
Yes, but all fhose edits being glossed over is a(n increasing) problem I think,
since it takes away eyes from edits that really could use some.

Most edits *should* go through, and *will* go through, and the voting
just changes how
much time it takes. For the database at large it doesn't matter that
much how quickly it goes
through though. The person most affected is often the editor. When I
was ripping my cds before
I was an autoeditor I had a special stack of cds by the computer that
I had to continue work
on later or that I should re-rip later, after some of my edits had
gone through. Then I really
appreciated it when people voted yes to my edits even if they just
thought that "yes, this looks
alright", since that meant I could continue doing what I was in the
middle of instead of
having to wait for long time and meanwhile starting on something else,
sometimes getting
confused because I forgot what I was doing. (That still happens, but
not that often anymore.
I have a cd box in front of me right now where I'm waiting for an edit
from July 17 to go through
so I can continue -- just 5 days left now!)

I see not much reason to give that kind of courtesy to bots. What's
the hurry? Yes, it can also be
in the middle of something (... and then when I've connected this
release with that discogs release
I'll compare all the recording credits with the discogs credits ...),
but really, a bot should be able to
be really patient.

> I imagine around 95% o edits are yes voted.

Really? I would have thought most edits didn't get any votes at all,
or at least not
enough votes for them to matter. I have 8 open votes right now, 2 of
which have one
yes vote and 6 of which have no votes, so that's what I'm seeing at least.
(I'm not complaining though! I'm a lazy voter myself and mostly vote
when I find an error.)

Maybe some of this discussion will be a moot point when/if the new
edit system will
group edits together. A bot edit that adds a couple of hundred discogs
links (for example)
in one go is worth voting about!

> My general viewpoint is that voting is not the be all and end all: It does
> not in every case decide for certain whether a piece of data is correct.

I'm tempted to go into my ideas about Musicbrainz voting in general (many more
edits should be autoapplied without voting!), but keeping it to bots,
yes, it isn't the
end of the world if an incorrect edit goes through, but if it really
needs voting on at all
that would be because there can be errors and we hope to find them in
time not always,
but at least rather often. Voting yes by default doesn't scale well as
a strategy. If everyone was
doing it errors would almost never be found in time!
To clarify, I actually meant 95% of edits that are voted on at all seem to be yes voted.

The frustration issue for editors is one reason to yes vote rather than abstain; it allows editors to work faster, more focused. Obviously it's not such an issue for bots, although allowing them to work on subsequent edits is a good thing.

Even if hypothetically everyone did yes vote by default, and errors were made, I don't see how these would be absolute. As I mentioned above, considering that the database is cross referenced with other sources and users' own knowledge, errors will eventually be spotted: Someone has to own each release, so it's highly unlikely that nobody would ever notice, particularly when other edits are founded on that data.

Removing edits from the queue does remove the opportunity for wider scrutiny in one sense, but it gives users more time to create new edits or look over artist pages (possibly finding other errors). So, even if some errors pass, the data has been filtered to some degree or other. Data that is un-sourced or incorrect will eventually be removed, but more users spending their time checking edits that are probably right is less productive than using that time finding existing data that is questionable or adding new data.

Bot auto editing is probably more beneficial on balance than non-auto, but the question is how we decide upon the granting of privileges.
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

Nicolás Tamargo de Eguren
There has been no strong opposition except from the "yes voting"
angle, which I see as no big problem - it says "discouraged", not
"prohibited" or something, and quite good arguments have been given
about why it is useful. People who were going to vote yes anyway will
keep doing it, I assume, but we can live with that.

I've moved the page to http://wiki.musicbrainz.org/Bots/Code_of_Conduct

--
Nicolás Tamargo de Eguren

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

Frederic Da Vitoria
Different subject: I sometimes wish the bots would explain why they inserted edits. Some bots do, some don't, especially cover art bots. I think the advice to add an edit note is as valid for a bot as it is for a human user.

Shouldn't we at least advise to do this?

--
Frederic Da Vitoria
(davitof)

Membre de l'April - « promouvoir et défendre le logiciel libre » - http://www.april.org


_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
jw
Reply | Threaded
Open this post in threaded view
|

Re: Code of Botduct

jw
Hi Frederic,

On Sun, Aug 12, 2012 at 12:30:28AM +0200, Frederic Da Vitoria wrote:
> Different subject: I sometimes wish the bots would explain why they
> inserted edits. Some bots do, some don't, especially cover art bots. I
> think the advice to add an edit note is as valid for a bot as it is for a
> human user.
>
> Shouldn't we at least advise to do this?

Yes, it should be added as a requirement! I also noticed this mainly
with the cover art bots... even stuff like "from hard disk, origin
unknown" is better than nothing.


Johannes

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users