musicbrainz database setup

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

musicbrainz database setup

Prakash Rewani

Hi ..

I am having issues installing musicbrainz database ..below is what I am doing and having issues..

 

As I am now following the below procedure..

[0] https://github.com/lalinsky/mbslave

 

I got stuck while importing data to the tables..i was able to install python and pscopg2 and configure it . Also was able to run the python scripts as described in the instructions. But while running the ./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2  encountered error  as below..
 
[postgres@uweb1 mbadmin]$ ./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2
Importing data from mbdump.tar.bz2
 - Loading artist
Traceback (most recent call last):
  File "./mbslave-import.py", line 41, in <module>
    load_tar(filename, db,ignored_tables)
  File "./mbslave-import.py", line 27, in load_tar
    cursor.copy_from(tar.extractfile(member), fulltable)
psycopg2.DataError: date/time field value out of range: "0"
HINT:  Perhaps you need a different "datestyle" setting.
CONTEXT:  COPY artist, line 1, column last_updated: "0"

 

I am very new to postgres but it looks like somesort of mismatch of timezone set in my env with yours,

For your information I have ran the below command in my env.

 

postgres=# show datestyle;

 DateStyle

-----------

 ISO, MDY

(1 row)      

postgres=# select current_timestamp;

             now             

------------------------------

 2012-05-18 13:18:11.42695-04

(1 row)

postgres=# show timezone;

  TimeZone 

------------

 US/Eastern

(1 row)

postgres=# SELECT * FROM pg_timezone_names where name like 'US/Eastern';

    name    | abbrev | utc_offset | is_dst

------------+--------+------------+--------

 US/Eastern | EDT    | -04:00:00  | t

(1 row)

 

 

I am really stuck now on this …please let me know..

 

Thanks

 

Prakash Rewani

 

 


This transmittal may be a confidential communication or may otherwise be privileged or confidential. If it is not clear that you are the intended recipient, you are hereby notified that you have received this transmittal in error; any review, dissemination, distribution or copying of this transmittal is strictly prohibited. If you suspect that you have received this communication in error, please notify us immediately by telephone at 1-203-967-1800 x 2004, or e-mail at [hidden email] and immediately delete this message and all its attachments.
_______________________________________________
MusicBrainz-devel mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel
Reply | Threaded
Open this post in threaded view
|

Re: musicbrainz database setup

Lukáš Lalinský
Sorry, I missed this email. You've probably already solved the problem, but in case you didn't, it looks like a mismatch between the database dump format and the local table structure. Are you using the latest version of mbslave?

Lukas


On Sat, May 19, 2012 at 10:52 AM, Prakash Rewani <[hidden email]> wrote:

Hi ..

I am having issues installing musicbrainz database ..below is what I am doing and having issues..

 

As I am now following the below procedure..

[0] https://github.com/lalinsky/mbslave

 

I got stuck while importing data to the tables..i was able to install python and pscopg2 and configure it . Also was able to run the python scripts as described in the instructions. But while running the ./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2  encountered error  as below..
 
[postgres@uweb1 mbadmin]$ ./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2
Importing data from mbdump.tar.bz2
 - Loading artist
Traceback (most recent call last):
  File "./mbslave-import.py", line 41, in <module>
    load_tar(filename, db,ignored_tables)
  File "./mbslave-import.py", line 27, in load_tar
    cursor.copy_from(tar.extractfile(member), fulltable)
psycopg2.DataError: date/time field value out of range: "0"
HINT:  Perhaps you need a different "datestyle" setting.
CONTEXT:  COPY artist, line 1, column last_updated: "0"

 

I am very new to postgres but it looks like somesort of mismatch of timezone set in my env with yours,

For your information I have ran the below command in my env.

 

postgres=# show datestyle;

 DateStyle

-----------

 ISO, MDY

(1 row)      

postgres=# select current_timestamp;

             now             

------------------------------

 2012-05-18 13:18:11.42695-04

(1 row)

postgres=# show timezone;

  TimeZone 

------------

 US/Eastern

(1 row)

postgres=# SELECT * FROM pg_timezone_names where name like 'US/Eastern';

    name    | abbrev | utc_offset | is_dst

------------+--------+------------+--------

 US/Eastern | EDT    | -04:00:00  | t

(1 row)

 

 

I am really stuck now on this …please let me know..

 

Thanks

 

Prakash Rewani

 

 


This transmittal may be a confidential communication or may otherwise be privileged or confidential. If it is not clear that you are the intended recipient, you are hereby notified that you have received this transmittal in error; any review, dissemination, distribution or copying of this transmittal is strictly prohibited. If you suspect that you have received this communication in error, please notify us immediately by telephone at 1-203-967-1800 x 2004, or e-mail at [hidden email] and immediately delete this message and all its attachments.

_______________________________________________
MusicBrainz-devel mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel


_______________________________________________
MusicBrainz-devel mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel
Reply | Threaded
Open this post in threaded view
|

Re: musicbrainz database setup

Prakash Rewani

Hi..

Thanks..

 

I tried to .. but was of no luck..below is what I am using..

 

https://github.com/lalinsky/mbslave..

 

Even I tried to load the tables separately still it failed while loading the artist table giving the same errors as posted in my previous message .. although couple of other tables I was able to load.

 

Please let me know..

 

Thanks

 

Prakash Rewani

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Lukáš Lalinský
Sent: Friday, May 25, 2012 1:28 PM
To: MusicBrainz Developer Discussion
Subject: Re: [mb-devel] musicbrainz database setup

 

Sorry, I missed this email. You've probably already solved the problem, but in case you didn't, it looks like a mismatch between the database dump format and the local table structure. Are you using the latest version of mbslave?

 

Lukas

 

On Sat, May 19, 2012 at 10:52 AM, Prakash Rewani <[hidden email]> wrote:

Hi ..

I am having issues installing musicbrainz database ..below is what I am doing and having issues..

 

As I am now following the below procedure..

[0] https://github.com/lalinsky/mbslave

 

I got stuck while importing data to the tables..i was able to install python and pscopg2 and configure it . Also was able to run the python scripts as described in the instructions. But while running the ./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2  encountered error  as below..
 
[postgres@uweb1 mbadmin]$ ./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2
Importing data from mbdump.tar.bz2
 - Loading artist
Traceback (most recent call last):
  File "./mbslave-import.py", line 41, in <module>
    load_tar(filename, db,ignored_tables)
  File "./mbslave-import.py", line 27, in load_tar
    cursor.copy_from(tar.extractfile(member), fulltable)
psycopg2.DataError: date/time field value out of range: "0"
HINT:  Perhaps you need a different "datestyle" setting.
CONTEXT:  COPY artist, line 1, column last_updated: "0"

 

I am very new to postgres but it looks like somesort of mismatch of timezone set in my env with yours,

For your information I have ran the below command in my env.

 

postgres=# show datestyle;

 DateStyle

-----------

 ISO, MDY

(1 row)      

postgres=# select current_timestamp;

             now             

------------------------------

 2012-05-18 13:18:11.42695-04

(1 row)

postgres=# show timezone;

  TimeZone 

------------

 US/Eastern

(1 row)

postgres=# SELECT * FROM pg_timezone_names where name like 'US/Eastern';

    name    | abbrev | utc_offset | is_dst

------------+--------+------------+--------

 US/Eastern | EDT    | -04:00:00  | t

(1 row)

 

 

I am really stuck now on this …please let me know..

 

Thanks

 

Prakash Rewani

 

 


This transmittal may be a confidential communication or may otherwise be privileged or confidential. If it is not clear that you are the intended recipient, you are hereby notified that you have received this transmittal in error; any review, dissemination, distribution or copying of this transmittal is strictly prohibited. If you suspect that you have received this communication in error, please notify us immediately by telephone at 1-203-967-1800 x 2004, or e-mail at [hidden email] and immediately delete this message and all its attachments.


_______________________________________________
MusicBrainz-devel mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel

 


This transmittal may be a confidential communication or may otherwise be privileged or confidential. If it is not clear that you are the intended recipient, you are hereby notified that you have received this transmittal in error; any review, dissemination, distribution or copying of this transmittal is strictly prohibited. If you suspect that you have received this communication in error, please notify us immediately by telephone at 1-203-967-1800 x 2004, or e-mail at [hidden email] and immediately delete this message and all its attachments.
_______________________________________________
MusicBrainz-devel mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel
Reply | Threaded
Open this post in threaded view
|

Re: musicbrainz database setup

Lukáš Lalinský
On Thu, May 31, 2012 at 1:35 PM, Prakash Rewani <[hidden email]> wrote:

Hi..

Thanks..

 

I tried to .. but was of no luck..below is what I am using..

 

https://github.com/lalinsky/mbslave..

 

Even I tried to load the tables separately still it failed while loading the artist table giving the same errors as posted in my previous message .. although couple of other tables I was able to load.

 

Please let me know..


I just tried to install a fresh database and it worked for me. Can you please try running this in the mbslave directory:

git rev-parse HEAD

It should print d5403d56a7c507b017e28bbfd26d7781612cd851. If it returns anything else, you need to update the sources.

Lukas


_______________________________________________
MusicBrainz-devel mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel
Reply | Threaded
Open this post in threaded view
|

Re: musicbrainz database setup

Prakash Rewani

Hi..

 

Thank you.. I was able to install the musicbraiz database now properly.

But as I went ahead to implement replication below is what I got an error..

 

[root@uweb1 musicbrainz]# ./mbslave-sync.py

Downloading http://ftp.musicbrainz.org/pub/musicbrainz/data/replication//replication-59467.tar.bz2

Processing /tmp/tmpiD05_f.tar.bz2

Traceback (most recent call last):

  File "./mbslave-sync.py", line 189, in <module>

    process_tar(tmp, db, schema, ignored_tables, schema_seq, replication_seq, hook)

  File "./mbslave-sync.py", line 128, in process_tar

    tar = tarfile.open(fileobj=fileobj, mode='r:bz2')

  File "/usr/lib/python2.5/tarfile.py", line 1157, in open

    return func(name, filemode, fileobj)

  File "/usr/lib/python2.5/tarfile.py", line 1229, in bz2open

    t = cls.taropen(name, mode, fileobj)

  File "/usr/lib/python2.5/tarfile.py", line 1183, in taropen

    return cls(name, mode, fileobj)

  File "/usr/lib/python2.5/tarfile.py", line 1047, in __init__

    self.name = os.path.abspath(name)

  File "/usr/lib/python2.5/posixpath.py", line 402, in abspath

    if not isabs(path):

  File "/usr/lib/python2.5/posixpath.py", line 49, in isabs

    return s.startswith('/')

AttributeError: 'NoneType' object has no attribute 'startswith'

 

Please let me know, how do I resolve it..

 

Waiting for your reply..thank u again

 

Prakash Rewani

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Lukáš Lalinský
Sent: Friday, June 01, 2012 11:25 AM
To: MusicBrainz Developer Discussion
Subject: Re: [mb-devel] musicbrainz database setup

 

On Thu, May 31, 2012 at 1:35 PM, Prakash Rewani <[hidden email]> wrote:

Hi..

Thanks..

 

I tried to .. but was of no luck..below is what I am using..

 

https://github.com/lalinsky/mbslave..

 

Even I tried to load the tables separately still it failed while loading the artist table giving the same errors as posted in my previous message .. although couple of other tables I was able to load.

 

Please let me know..

 

I just tried to install a fresh database and it worked for me. Can you please try running this in the mbslave directory:

 

git rev-parse HEAD

 

It should print d5403d56a7c507b017e28bbfd26d7781612cd851. If it returns anything else, you need to update the sources.

 

Lukas

 


This transmittal may be a confidential communication or may otherwise be privileged or confidential. If it is not clear that you are the intended recipient, you are hereby notified that you have received this transmittal in error; any review, dissemination, distribution or copying of this transmittal is strictly prohibited. If you suspect that you have received this communication in error, please notify us immediately by telephone at 1-203-967-1800 x 2004, or e-mail at [hidden email] and immediately delete this message and all its attachments.
_______________________________________________
MusicBrainz-devel mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel