where to find ratings in db dump

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

where to find ratings in db dump

Marcel Ackermann
Hi,

I've downloaded and extracted the latest mbdump.tar.bz2, derived, edit, editor, stats - but wasn't able to locate the ratings of (track, album, artist).
In which archive and which file within this archive do I find those?

Thank you very much,
Marcel

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
Reply | Threaded
Open this post in threaded view
|

Re: where to find ratings in db dump

Calvin Walton-2
On Fri, 2014-06-13 at 01:22 +0100, Marcel Ackermann wrote:
> Hi,
>
> I've downloaded and extracted the latest mbdump.tar.bz2, derived,
> edit, editor, stats - but wasn't able to locate the ratings of (track,
> album, artist).
> In which archive and which file within this archive do I find those?

I believe that the recording, release group, and artist ratings overall
ratings can be found in the "recording_meta", "release_group_meta" and
"artist_meta" tables, which should be included in the dump - these
tables are probably in the -derived tar file along with other CC-NC
licensed data.

Note that we don't store track ratings, and 'album' is just a release
type. Depending on what you want to use the data for, either of the
"release" or "release group" ratings might be appropriate.

The rating data in MusicBrainz itself is, of course, notoriously
incomplete. You have to be a registered editor to use ratings, and not
many active editors do anything with them.

I'm not sure about whether we include per-user information about ratings
in the dumps - this is probably stripped out for privacy issues.

--
Calvin Walton <[hidden email]>


_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
Reply | Threaded
Open this post in threaded view
|

Re: where to find ratings in db dump

Ian McEwen
On Fri, Jun 13, 2014 at 01:18:08PM -0400, Calvin Walton wrote:

> On Fri, 2014-06-13 at 01:22 +0100, Marcel Ackermann wrote:
> > Hi,
> >
> > I've downloaded and extracted the latest mbdump.tar.bz2, derived,
> > edit, editor, stats - but wasn't able to locate the ratings of (track,
> > album, artist).
> > In which archive and which file within this archive do I find those?
>
> I believe that the recording, release group, and artist ratings overall
> ratings can be found in the "recording_meta", "release_group_meta" and
> "artist_meta" tables, which should be included in the dump - these
> tables are probably in the -derived tar file along with other CC-NC
> licensed data.
>
> Note that we don't store track ratings, and 'album' is just a release
> type. Depending on what you want to use the data for, either of the
> "release" or "release group" ratings might be appropriate.
>
Releases don't have ratings, only release groups
(http://tickets.musicbrainz.org/browse/MBS-4816)

Otherwise all of this is correct.

> The rating data in MusicBrainz itself is, of course, notoriously
> incomplete. You have to be a registered editor to use ratings, and not
> many active editors do anything with them.
>
> I'm not sure about whether we include per-user information about ratings
> in the dumps - this is probably stripped out for privacy issues.
>

Yes; the *_rating_raw tables are not included in data dumps, only the
aggregate information in the *_meta tables, as noted above.

> --
> Calvin Walton <[hidden email]>
>
>
> _______________________________________________
> MusicBrainz-users mailing list
> [hidden email]
> http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users

_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users

attachment0 (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: where to find ratings in db dump

Marcel Ackermann
Thank you very much Ian and Calvin!
So I was looking for ratings of users of tracks (explicit feedback) as I am writing my masters thesis about music recommendation and the million song contest is only playcounts (implicit feedback), so I was hopeful when I read "user submitted annotations, tags and ratings" - but I understand that this is not part of what you release. 
Are you aware of anything like this out there? (There is the yahoo kdd data but they don't include titles of anything, just IDs)
Thank you very much,
Marcel


Ian McEwen <[hidden email]> schrieb am 19:24 Freitag, 13.Juni 2014:


On Fri, Jun 13, 2014 at 01:18:08PM -0400, Calvin Walton wrote:

> On Fri, 2014-06-13 at 01:22 +0100, Marcel Ackermann wrote:
> > Hi,
> >
> > I've downloaded and extracted the latest mbdump.tar.bz2, derived,
> > edit, editor, stats - but wasn't able to locate the ratings of (track,
> > album, artist).
> > In which archive and which file within this archive do I find those?
>
> I believe that the recording, release group, and artist ratings overall
> ratings can be found in the "recording_meta", "release_group_meta" and
> "artist_meta" tables, which should be included in the dump - these
> tables are probably in the -derived tar file along with other CC-NC
> licensed data.
>
> Note that we don't store track ratings, and 'album' is just a release
> type. Depending on what you want to use the data for, either of the
> "release" or "release group" ratings might be appropriate.
>

Releases don't have ratings, only release groups
(http://tickets.musicbrainz.org/browse/MBS-4816)

Otherwise all of this is correct.

> The rating data in MusicBrainz itself is, of course, notoriously
> incomplete. You have to be a registered editor to use ratings, and not
> many active editors do anything with them.
>
> I'm not sure about whether we include per-user information about ratings
> in the dumps - this is probably stripped out for privacy issues.
>

Yes; the *_rating_raw tables are not included in data dumps, only the
aggregate information in the *_meta tables, as noted above.


> --
> Calvin Walton <[hidden email]
>
>
>
> _______________________________________________
> MusicBrainz-users mailing list
> [hidden email]
> http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users


_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users


_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users
Reply | Threaded
Open this post in threaded view
|

Re: where to find ratings in db dump

Calvin Walton-2
On Fri, 2014-06-13 at 22:22 +0100, Marcel Ackermann wrote:

> Thank you very much Ian and Calvin!
> So I was looking for ratings of users of tracks (explicit feedback) as
> I am writing my masters thesis about music recommendation and the
> million song contest is only playcounts (implicit feedback), so I was
> hopeful when I read "user submitted annotations, tags and ratings" -
> but I understand that this is not part of what you release.
> Are you aware of anything like this out there? (There is the yahoo kdd
> data but they don't include titles of anything, just IDs)
> Thank you very much,
> Marcel

Unfortunately, for this type of data, privacy is a really big concern.
Even with user data anonymized, given the song data (recording titles,
artist names, tags, ratings) it's sometimes possible to profile
particular individuals and find out who they are.

Your best bet might be to look for data sets where individual users are
explicitly making their listening habits and rated/tagged tracks public.

A few places to consider:
* It might be worth contacting the folks at http://libre.fm and asking
them if there's a possibility of getting a dump of public user data for
your project.

* Last.fm also has a lot of this sort of data, but you'd be limited to
extracting it via their API, e.g.
http://www.last.fm/api/show/user.getPersonalTags - which will be subject
to rate limits (5 req/s).

* That said, we do at MusicBrainz have user preferences for "Allow other
users to see my subscriptions / tags / ratings" - I wonder if we could
filter data based on this preference to only include public information,
and make that available.

--
Calvin Walton <[hidden email]>


_______________________________________________
MusicBrainz-users mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-users