Weeksio's Progress this week

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Weeksio's Progress this week

Jeff Weeks
Progress this week:

-Boosted Solr schema.xml version from 1.2 to 1.5.
-Edited sir to reflect
-I changed direction on how I'm porting the analysis from the old server to the new. Instead of using the old java ported as is to Solr, I'm determining the steps each analysis takes and defining those steps directly in schema.xml.  This should make future maintenance much easier. 
-I've been making my way through each core deciding which of the new field types to use for each attribute, importing each core's data from the database, fixing errors as they pop up, trying to improve the indexing and think into the future a bit as to how we will utilize some of Solr's features.  I'm though Annotation, Area, Artist, CDStub, Editor...and the further I go the quicker it goes as each one covers more cases likely to appear in future fields as I go.

TODOs:
-Porting the Area, Artist, Label Boost configurations
-Add remaining entities to sir
-Continue working through cores

Questions:

-MusicBrainzKeepAccentsAnalyzer:
I'm still not sure why we need this...maybe I'm overlooking something.  It looks like tokens aren't stored, only analyzed/indexed; but if we use the same analysis at query time as at index time (which it appears we do) the indexed tokens retaining their accents will never be accessed.  ...correct?

-_store fields:
I really don't understand the purpose of these...could someone explain?

Would an ICUTransformFilter using Greek-Latin and Cyrillic-Latin rule sets do the trick? ...or are resources the primary concern?

-I noticed our analyzers don't use stop words?  Is this something to continue?  Seems like a good conservative list of English stop words would be useful.

Personal Criticism:
-I've been really bad about getting my code up on github...hope to get that up today.  I just haven't gotten into the habit of it....I'm so used to working by myself on school projects.

Personal Observations:
-Solr is awesome!  It's super powerful...and it's really easy to get bogged down thinking about adding extras.


Best,
Jeff


_______________________________________________
MusicBrainz-devel mailing list
[hidden email]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel